2026-09-24 14:38:07 -04:00
|
|
|
|
<!doctype html>
|
|
|
|
|
|
<html lang="en">
|
|
|
|
|
|
<head>
|
|
|
|
|
|
<meta charset="utf-8">
|
|
|
|
|
|
<meta name="viewport" content="width=device-width, initial-scale=1">
|
|
|
|
|
|
<title>roto — take builder</title>
|
|
|
|
|
|
<style>
|
2026-09-24 14:51:15 -04:00
|
|
|
|
:root { --bg:#0b0d13; --panel:#12151f; --line:#232836; --fg:#e8eaf0;
|
|
|
|
|
|
--dim:#8891a5; --accent:#fbbf24; --ok:#4ade80; --err:#f87171; }
|
2026-09-24 14:38:07 -04:00
|
|
|
|
* { box-sizing: border-box; }
|
2026-09-24 14:51:15 -04:00
|
|
|
|
body { margin:0; background:var(--bg); color:var(--fg);
|
|
|
|
|
|
font:13px/1.5 ui-monospace,SFMono-Regular,Menlo,monospace; }
|
|
|
|
|
|
header { padding:12px 18px; border-bottom:1px solid var(--line);
|
|
|
|
|
|
display:flex; gap:10px; align-items:center; flex-wrap:wrap; }
|
|
|
|
|
|
h1 { font-size:14px; margin:0 8px 0 0; letter-spacing:.04em; }
|
|
|
|
|
|
h1 span { color:var(--dim); font-weight:400; }
|
|
|
|
|
|
main { padding:16px 18px 40px; display:flex; flex-direction:column; gap:16px; }
|
|
|
|
|
|
.row { display:flex; gap:16px; flex-wrap:wrap; }
|
|
|
|
|
|
.panel { background:var(--panel); border:1px solid var(--line); border-radius:4px; padding:12px; }
|
|
|
|
|
|
.panel h2 { font-size:11px; text-transform:uppercase; letter-spacing:.08em;
|
|
|
|
|
|
color:var(--dim); margin:0 0 8px; font-weight:500; }
|
|
|
|
|
|
canvas { display:block; image-rendering:pixelated; max-width:100%; border-radius:2px; }
|
|
|
|
|
|
button,input[type=text],select { font:inherit; background:#1c2130; color:var(--fg);
|
|
|
|
|
|
border:1px solid var(--line); border-radius:3px; padding:5px 10px; cursor:pointer; }
|
|
|
|
|
|
button:hover { border-color:var(--accent); }
|
|
|
|
|
|
input[type=text] { cursor:text; }
|
|
|
|
|
|
label.ctl { display:grid; grid-template-columns:132px 1fr 46px; gap:10px;
|
|
|
|
|
|
align-items:center; margin-bottom:6px; }
|
|
|
|
|
|
label.ctl span:first-child { color:var(--dim); }
|
|
|
|
|
|
label.ctl output { text-align:right; color:var(--accent); }
|
|
|
|
|
|
input[type=range] { width:100%; accent-color:var(--accent); }
|
|
|
|
|
|
#status { color:var(--dim); } #status.ok{color:var(--ok)} #status.err{color:var(--err)} #status.warn{color:var(--accent)}
|
|
|
|
|
|
#readout { color:var(--dim); font-size:12px; }
|
|
|
|
|
|
|
|
|
|
|
|
/* the frame strip is the editing surface */
|
|
|
|
|
|
#strip { display:flex; flex-wrap:wrap; gap:4px; }
|
|
|
|
|
|
.fr { position:relative; cursor:pointer; border:2px solid transparent; border-radius:3px; line-height:0; }
|
|
|
|
|
|
.fr canvas { border-radius:1px; }
|
|
|
|
|
|
.fr span { position:absolute; left:2px; bottom:2px; font-size:10px; line-height:1.2;
|
|
|
|
|
|
padding:0 3px; border-radius:2px; background:#000a; color:#fff; }
|
|
|
|
|
|
.fr.keep { border-color:var(--ok); }
|
|
|
|
|
|
.fr.drop { border-color:#2a2f3e; }
|
|
|
|
|
|
.fr.drop canvas { opacity:.26; filter:grayscale(1); }
|
|
|
|
|
|
.fr.cur { border-color:var(--accent); }
|
|
|
|
|
|
.fr.keep::after { content:'●'; position:absolute; right:3px; top:1px; color:var(--ok); font-size:10px; }
|
|
|
|
|
|
|
|
|
|
|
|
#sheet { display:flex; flex-wrap:wrap; gap:10px; }
|
|
|
|
|
|
#sheet .cell { display:flex; flex-direction:column; gap:3px; cursor:pointer; }
|
|
|
|
|
|
#sheet .cell span { color:var(--dim); font-size:11px; }
|
|
|
|
|
|
#sheet .cell:hover span { color:var(--accent); }
|
|
|
|
|
|
#palette { display:flex; gap:12px; flex-wrap:wrap; }
|
|
|
|
|
|
.sw { display:flex; align-items:center; gap:5px; color:var(--dim); font-size:11px; }
|
|
|
|
|
|
.sw input { width:26px; height:20px; padding:0; border:1px solid var(--line); background:none; }
|
|
|
|
|
|
.legend { color:var(--dim); font-size:11px; margin-top:6px; }
|
|
|
|
|
|
kbd { background:#1c2130; border:1px solid var(--line); border-radius:2px;
|
|
|
|
|
|
padding:0 4px; color:var(--fg); font-size:11px; }
|
2026-09-24 14:38:07 -04:00
|
|
|
|
</style>
|
|
|
|
|
|
</head>
|
|
|
|
|
|
<body>
|
|
|
|
|
|
<header>
|
2026-09-24 14:51:15 -04:00
|
|
|
|
<h1>roto <span>— frame removal + take builder</span></h1>
|
|
|
|
|
|
<button id="btn-synth">Synthetic</button>
|
|
|
|
|
|
<input type="text" id="framedir" value="frames" size="7" title="frame directory">
|
2026-09-24 14:38:07 -04:00
|
|
|
|
<button id="btn-frames">Load frames</button>
|
|
|
|
|
|
<button id="btn-play">Play</button>
|
2026-09-24 14:53:09 -04:00
|
|
|
|
<select id="speed" title="playback speed"><option value="1">1x</option><option value="0.5">½x</option><option value="0.25">¼x</option></select>
|
|
|
|
|
|
<audio id="audio" controls hidden style="height:28px;vertical-align:middle"></audio>
|
2026-09-24 14:51:15 -04:00
|
|
|
|
<button id="btn-keepall">Keep all</button>
|
|
|
|
|
|
<button id="btn-suggest">Suggest</button>
|
|
|
|
|
|
<input type="text" id="takename" value="line_01" size="9" title="take name">
|
Registered photo underlay as the plate reference
The generated face oval was never going to be good enough to draw from:
MediaPipe's face oval is the FACE boundary, cut at the hairline and excluding
hair, ears, jaw and neck, so it is an egg by construction. Segmentation would
give a real head outline but costs a 16MB model and per-frame inference for a
shape that gets replaced by a drawing anyway.
So the plate layer becomes switchable, and the useful modes are photographic:
the source frame mapped into raster space through the same transform chain the
contours go through. Registration is the whole point - the head sits still and
a drawing traced from the underlay is already aligned to the mouth. An
unregistered underlay would be decoration.
- underlay.js: pixel->raster affine (a general affine, since MediaPipe
normalises x by width and y by height), registered draw, palette posterise
- plate modes: photo / photo dim / posterized / oval / oval+photo / none, B cycles
- worksheet cells are registered composites rather than raw crops
- Save frame 4x writes a 1280x800 PNG to draw on
- selftest: FACE_OVAL simplicity, which was never asserted; a wrong ordering
there reads as a lumpy plate rather than an obvious bowtie
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-24 14:57:59 -04:00
|
|
|
|
<select id="plateMode" title="plate representation (B to cycle)">
|
|
|
|
|
|
<option value="photo-dim">photo dim</option>
|
|
|
|
|
|
<option value="photo">photo</option>
|
|
|
|
|
|
<option value="posterize">posterized</option>
|
|
|
|
|
|
<option value="oval" selected>oval</option>
|
|
|
|
|
|
<option value="oval+photo">oval + photo</option>
|
|
|
|
|
|
<option value="off">none</option>
|
|
|
|
|
|
</select>
|
|
|
|
|
|
<button id="btn-saveframe">Save frame 4x</button>
|
2026-09-24 14:38:07 -04:00
|
|
|
|
<button id="btn-export">Export .take</button>
|
|
|
|
|
|
<span id="status"></span>
|
|
|
|
|
|
</header>
|
|
|
|
|
|
|
|
|
|
|
|
<main>
|
|
|
|
|
|
<div class="row">
|
|
|
|
|
|
<div class="panel">
|
|
|
|
|
|
<h2>source + landmarks</h2>
|
|
|
|
|
|
<canvas id="cv-source"></canvas>
|
|
|
|
|
|
<div class="legend">outer lip <b style="color:#4ade80">—</b> · inner lip <b style="color:#f87171">—</b></div>
|
|
|
|
|
|
</div>
|
|
|
|
|
|
<div class="panel">
|
|
|
|
|
|
<h2>stabilised (head-local)</h2>
|
|
|
|
|
|
<canvas id="cv-stab"></canvas>
|
2026-09-24 14:51:15 -04:00
|
|
|
|
<div class="legend">should sit still except the mouth · grey = held plate outline</div>
|
2026-09-24 14:38:07 -04:00
|
|
|
|
</div>
|
|
|
|
|
|
<div class="panel">
|
|
|
|
|
|
<h2>flat render — 320×200 indexed</h2>
|
|
|
|
|
|
<canvas id="cv-render"></canvas>
|
|
|
|
|
|
<div class="legend" id="framelabel"></div>
|
Registered photo underlay as the plate reference
The generated face oval was never going to be good enough to draw from:
MediaPipe's face oval is the FACE boundary, cut at the hairline and excluding
hair, ears, jaw and neck, so it is an egg by construction. Segmentation would
give a real head outline but costs a 16MB model and per-frame inference for a
shape that gets replaced by a drawing anyway.
So the plate layer becomes switchable, and the useful modes are photographic:
the source frame mapped into raster space through the same transform chain the
contours go through. Registration is the whole point - the head sits still and
a drawing traced from the underlay is already aligned to the mouth. An
unregistered underlay would be decoration.
- underlay.js: pixel->raster affine (a general affine, since MediaPipe
normalises x by width and y by height), registered draw, palette posterise
- plate modes: photo / photo dim / posterized / oval / oval+photo / none, B cycles
- worksheet cells are registered composites rather than raw crops
- Save frame 4x writes a 1280x800 PNG to draw on
- selftest: FACE_OVAL simplicity, which was never asserted; a wrong ordering
there reads as a lumpy plate rather than an obvious bowtie
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-24 14:57:59 -04:00
|
|
|
|
<div class="legend">audio drives the clock — dropped frames, never drift<br>
|
|
|
|
|
|
plate representation: <kbd>B</kbd> cycles · photo modes are <b>registered</b>
|
|
|
|
|
|
into raster space, so tracing them lands on the mouth</div>
|
2026-09-24 14:38:07 -04:00
|
|
|
|
</div>
|
|
|
|
|
|
</div>
|
|
|
|
|
|
|
2026-09-24 14:51:15 -04:00
|
|
|
|
<div class="panel">
|
|
|
|
|
|
<h2>frames — green kept (gets its own drawing) · dim held from the last kept frame</h2>
|
|
|
|
|
|
<div id="strip"></div>
|
|
|
|
|
|
<input type="range" id="scrub" min="0" max="0" value="0" style="width:100%;margin-top:10px">
|
|
|
|
|
|
<div class="legend">
|
Registered photo underlay as the plate reference
The generated face oval was never going to be good enough to draw from:
MediaPipe's face oval is the FACE boundary, cut at the hairline and excluding
hair, ears, jaw and neck, so it is an egg by construction. Segmentation would
give a real head outline but costs a 16MB model and per-frame inference for a
shape that gets replaced by a drawing anyway.
So the plate layer becomes switchable, and the useful modes are photographic:
the source frame mapped into raster space through the same transform chain the
contours go through. Registration is the whole point - the head sits still and
a drawing traced from the underlay is already aligned to the mouth. An
unregistered underlay would be decoration.
- underlay.js: pixel->raster affine (a general affine, since MediaPipe
normalises x by width and y by height), registered draw, palette posterise
- plate modes: photo / photo dim / posterized / oval / oval+photo / none, B cycles
- worksheet cells are registered composites rather than raw crops
- Save frame 4x writes a 1280x800 PNG to draw on
- selftest: FACE_OVAL simplicity, which was never asserted; a wrong ordering
there reads as a lumpy plate rather than an obvious bowtie
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-24 14:57:59 -04:00
|
|
|
|
<kbd>←</kbd> <kbd>→</kbd> step · <kbd>X</kbd> delete · <kbd>K</kbd> keep · <kbd>B</kbd> background ·
|
2026-09-24 14:51:15 -04:00
|
|
|
|
click to select · double-click or shift-click to toggle ·
|
|
|
|
|
|
the mouth keeps <b>every</b> frame regardless
|
|
|
|
|
|
</div>
|
|
|
|
|
|
<div class="legend" id="readout"></div>
|
|
|
|
|
|
</div>
|
|
|
|
|
|
|
2026-09-24 14:38:07 -04:00
|
|
|
|
<div class="row">
|
2026-09-24 14:51:15 -04:00
|
|
|
|
<div class="panel" style="flex:1 1 400px">
|
2026-09-24 14:38:07 -04:00
|
|
|
|
<h2>knobs</h2>
|
|
|
|
|
|
<label class="ctl"><span>vertices</span><input type="range" id="verts" min="4" max="16" step="2" value="8"><output id="vertsv"></output></label>
|
2026-09-24 14:51:15 -04:00
|
|
|
|
<label class="ctl"><span>contour avg ±f</span><input type="range" id="contourSmooth" min="0" max="4" value="1"><output id="contourSmoothv"></output></label>
|
|
|
|
|
|
<label class="ctl"><span>anchor avg ±f</span><input type="range" id="smoothWin" min="0" max="8" value="2"><output id="smoothWinv"></output></label>
|
2026-09-24 14:38:07 -04:00
|
|
|
|
<label class="ctl"><span>closed-mouth cut</span><input type="range" id="apertureThresh" min="0" max="400" value="120"><output id="apertureThreshv"></output></label>
|
Fix teeth band filling the whole mouth
Three causes, all of them mine:
Otsu always returns a split, including on a homogeneous region - given a dark
cavity with no teeth it invents a threshold and calls half the pixels bright.
The gate is now the separation between the two class means, which is the only
thing that says whether the split means anything. Coverage was the wrong
signal: it is high both when the mouth is full of teeth and when the region is
uniformly dark and Otsu has split noise.
The row scan tracked the last qualifying row anywhere rather than where the run
from the top stops, so one bright row near the bottom - a lit lower lip inside
the ring - pushed the line to full height. It now breaks at the first failing
row once the run has started.
MediaPipe's inner lip landmarks sit slightly outside the real opening, so the
sampled region included lip pixels, which are bright and sit exactly at the
boundary where they do most damage. The ring is now eroded toward its centroid
before sampling, with the amount exposed as a knob.
Adds a diagnostic panel showing the sampled crop, pixels above threshold, and
the resolved line, because tuning this from numbers alone does not tell you
whether the region being measured is even the right region.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-24 15:14:49 -04:00
|
|
|
|
<label class="ctl"><span>teeth contrast</span><input type="range" id="teethOn" min="1" max="60" value="16"><output id="teethOnv"></output></label>
|
|
|
|
|
|
<label class="ctl"><span>cavity erode</span><input type="range" id="teethErode" min="0" max="40" value="18"><output id="teethErodev"></output></label>
|
2026-09-24 15:12:09 -04:00
|
|
|
|
<label class="ctl"><span>teeth dwell</span><input type="range" id="teethDwell" min="0" max="6" value="1"><output id="teethDwellv"></output></label>
|
2026-09-24 14:51:15 -04:00
|
|
|
|
<label class="ctl"><span>suggest tolerance</span><input type="range" id="tol" min="2" max="60" value="14"><output id="tolv"></output></label>
|
|
|
|
|
|
<div class="legend">
|
|
|
|
|
|
<b>contour avg</b> 0 = off, 1 = ±1 frame. Removes per-frame landmark
|
|
|
|
|
|
jitter. Push past 2 and it starts eating articulation.<br>
|
|
|
|
|
|
<b>anchor avg</b> smooths the head transform only — never the contour.<br>
|
Fix teeth band filling the whole mouth
Three causes, all of them mine:
Otsu always returns a split, including on a homogeneous region - given a dark
cavity with no teeth it invents a threshold and calls half the pixels bright.
The gate is now the separation between the two class means, which is the only
thing that says whether the split means anything. Coverage was the wrong
signal: it is high both when the mouth is full of teeth and when the region is
uniformly dark and Otsu has split noise.
The row scan tracked the last qualifying row anywhere rather than where the run
from the top stops, so one bright row near the bottom - a lit lower lip inside
the ring - pushed the line to full height. It now breaks at the first failing
row once the run has started.
MediaPipe's inner lip landmarks sit slightly outside the real opening, so the
sampled region included lip pixels, which are bright and sit exactly at the
boundary where they do most damage. The ring is now eroded toward its centroid
before sampling, with the amount exposed as a knob.
Adds a diagnostic panel showing the sampled crop, pixels above threshold, and
the resolved line, because tuning this from numbers alone does not tell you
whether the region being measured is even the right region.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-24 15:14:49 -04:00
|
|
|
|
<b>teeth contrast</b> how far apart the dark and bright halves of the
|
|
|
|
|
|
cavity must be before the split means anything — Otsu always returns
|
|
|
|
|
|
<i>some</i> threshold, so this is what stops it inventing teeth in a dark
|
|
|
|
|
|
mouth. <b>erode</b> shrinks the sampled region inward, away from the lip
|
|
|
|
|
|
edge. <b>dwell</b> is how many frames a change must persist.<br>
|
2026-09-24 14:51:15 -04:00
|
|
|
|
<b>suggest tolerance</b> only affects the Suggest button: max head movement
|
|
|
|
|
|
allowed before a new drawing is required.
|
|
|
|
|
|
</div>
|
2026-09-24 14:38:07 -04:00
|
|
|
|
</div>
|
Fix teeth band filling the whole mouth
Three causes, all of them mine:
Otsu always returns a split, including on a homogeneous region - given a dark
cavity with no teeth it invents a threshold and calls half the pixels bright.
The gate is now the separation between the two class means, which is the only
thing that says whether the split means anything. Coverage was the wrong
signal: it is high both when the mouth is full of teeth and when the region is
uniformly dark and Otsu has split noise.
The row scan tracked the last qualifying row anywhere rather than where the run
from the top stops, so one bright row near the bottom - a lit lower lip inside
the ring - pushed the line to full height. It now breaks at the first failing
row once the run has started.
MediaPipe's inner lip landmarks sit slightly outside the real opening, so the
sampled region included lip pixels, which are bright and sit exactly at the
boundary where they do most damage. The ring is now eroded toward its centroid
before sampling, with the amount exposed as a knob.
Adds a diagnostic panel showing the sampled crop, pixels above threshold, and
the resolved line, because tuning this from numbers alone does not tell you
whether the region being measured is even the right region.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-24 15:14:49 -04:00
|
|
|
|
<div class="panel" style="flex:0 1 190px">
|
|
|
|
|
|
<h2>teeth measurement</h2>
|
|
|
|
|
|
<div id="cv-teeth"></div>
|
|
|
|
|
|
<div class="legend" id="teethinfo"></div>
|
|
|
|
|
|
<div class="legend">green = above threshold · amber = resolved line</div>
|
|
|
|
|
|
</div>
|
|
|
|
|
|
<div class="panel" style="flex:1 1 240px">
|
2026-09-24 14:38:07 -04:00
|
|
|
|
<h2>palette</h2>
|
|
|
|
|
|
<div id="palette"></div>
|
|
|
|
|
|
<div class="legend" style="margin-top:12px">
|
|
|
|
|
|
Flat indexed fills, no antialiasing — the rasteriser writes palette
|
|
|
|
|
|
indices, the way <code>csd_render_poly</code> does.
|
|
|
|
|
|
</div>
|
|
|
|
|
|
</div>
|
|
|
|
|
|
</div>
|
|
|
|
|
|
|
|
|
|
|
|
<div class="panel">
|
Registered photo underlay as the plate reference
The generated face oval was never going to be good enough to draw from:
MediaPipe's face oval is the FACE boundary, cut at the hairline and excluding
hair, ears, jaw and neck, so it is an egg by construction. Segmentation would
give a real head outline but costs a 16MB model and per-frame inference for a
shape that gets replaced by a drawing anyway.
So the plate layer becomes switchable, and the useful modes are photographic:
the source frame mapped into raster space through the same transform chain the
contours go through. Registration is the whole point - the head sits still and
a drawing traced from the underlay is already aligned to the mouth. An
unregistered underlay would be decoration.
- underlay.js: pixel->raster affine (a general affine, since MediaPipe
normalises x by width and y by height), registered draw, palette posterise
- plate modes: photo / photo dim / posterized / oval / oval+photo / none, B cycles
- worksheet cells are registered composites rather than raw crops
- Save frame 4x writes a 1280x800 PNG to draw on
- selftest: FACE_OVAL simplicity, which was never asserted; a wrong ordering
there reads as a lumpy plate rather than an obvious bowtie
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-24 14:57:59 -04:00
|
|
|
|
<h2>drawings needed — registered reference per kept frame, with the range it holds</h2>
|
2026-09-24 14:38:07 -04:00
|
|
|
|
<div id="sheet"></div>
|
|
|
|
|
|
</div>
|
|
|
|
|
|
</main>
|
|
|
|
|
|
|
|
|
|
|
|
<script type="module" src="./js/app.js"></script>
|
|
|
|
|
|
</body>
|
|
|
|
|
|
</html>
|