Frame removal for plates; mouth keeps every frame
Sparseness was being applied for two different reasons at once. Aesthetic sparseness is set by the extraction rate; labour sparseness only binds on the plate, because a human draws each one. The mouth is traced and therefore free, and in limited animation lip sync is routinely the densest element - on 1s while heads hold on 2s and 3s. So: the mouth gets a key on every frame, and the frame strip is now the editing surface for deciding which frames need their own plate drawing. All frames start kept; delete the ones you don't want. - strip of face-cropped thumbnails, keep/drop per frame, keyboard driven - worksheet panel lists the drawings needed and the range each one holds - Suggest runs error-tolerance decimation on head pose as a starting point - export writes sparse plate keys + dense mouth keys, with a hold manifest - smoothContours: bounded exception to "never smooth the contour", which held only while keys were sparse enough to reject detector noise by sampling - averages are now a RADIUS in frames: 0 is off, 1 is +-1 - GPU delegate falls back to CPU instead of failing - #synth / #frames autorun for headless smoke tests Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
7a6bdea157
commit
a082208ad5
7 changed files with 489 additions and 265 deletions
146
index.html
146
index.html
|
|
@ -5,57 +5,68 @@
|
|||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title>roto — take builder</title>
|
||||
<style>
|
||||
:root {
|
||||
--bg: #0b0d13; --panel: #12151f; --line: #232836; --fg: #e8eaf0;
|
||||
--dim: #8891a5; --accent: #fbbf24; --ok: #4ade80; --err: #f87171;
|
||||
}
|
||||
:root { --bg:#0b0d13; --panel:#12151f; --line:#232836; --fg:#e8eaf0;
|
||||
--dim:#8891a5; --accent:#fbbf24; --ok:#4ade80; --err:#f87171; }
|
||||
* { box-sizing: border-box; }
|
||||
body { margin: 0; background: var(--bg); color: var(--fg);
|
||||
font: 13px/1.5 ui-monospace, SFMono-Regular, Menlo, monospace; }
|
||||
header { padding: 14px 18px; border-bottom: 1px solid var(--line);
|
||||
display: flex; gap: 14px; align-items: baseline; flex-wrap: wrap; }
|
||||
h1 { font-size: 14px; margin: 0; letter-spacing: .04em; }
|
||||
h1 span { color: var(--dim); font-weight: 400; }
|
||||
main { padding: 18px; display: flex; flex-direction: column; gap: 18px; }
|
||||
.row { display: flex; gap: 18px; flex-wrap: wrap; }
|
||||
.panel { background: var(--panel); border: 1px solid var(--line); border-radius: 4px; padding: 12px; }
|
||||
.panel h2 { font-size: 11px; text-transform: uppercase; letter-spacing: .08em;
|
||||
color: var(--dim); margin: 0 0 8px; font-weight: 500; }
|
||||
canvas { display: block; image-rendering: pixelated; max-width: 100%; border-radius: 2px; }
|
||||
#cv-timeline { width: 100%; height: 90px; }
|
||||
button, input[type=text] {
|
||||
font: inherit; background: #1c2130; color: var(--fg);
|
||||
border: 1px solid var(--line); border-radius: 3px; padding: 5px 11px; cursor: pointer; }
|
||||
button:hover { border-color: var(--accent); }
|
||||
input[type=text] { cursor: text; }
|
||||
label.ctl { display: grid; grid-template-columns: 116px 1fr 44px;
|
||||
gap: 10px; align-items: center; margin-bottom: 6px; }
|
||||
label.ctl span:first-child { color: var(--dim); }
|
||||
label.ctl output { text-align: right; color: var(--accent); }
|
||||
input[type=range] { width: 100%; accent-color: var(--accent); }
|
||||
#status { color: var(--dim); }
|
||||
#status.ok { color: var(--ok); } #status.err { color: var(--err); } #status.warn { color: var(--accent); }
|
||||
#readout { color: var(--dim); font-size: 12px; }
|
||||
#sheet { display: flex; flex-wrap: wrap; gap: 8px; }
|
||||
#sheet .cell { display: flex; flex-direction: column; gap: 3px; cursor: pointer; }
|
||||
#sheet .cell span { color: var(--dim); font-size: 11px; }
|
||||
#sheet .cell:hover span { color: var(--accent); }
|
||||
#palette { display: flex; gap: 12px; flex-wrap: wrap; }
|
||||
.sw { display: flex; align-items: center; gap: 5px; color: var(--dim); font-size: 11px; }
|
||||
.sw input { width: 26px; height: 20px; padding: 0; border: 1px solid var(--line); background: none; }
|
||||
.legend { color: var(--dim); font-size: 11px; margin-top: 6px; }
|
||||
.legend b { font-weight: 400; }
|
||||
.k { color: var(--accent); } .s { color: var(--err); } .c { color: #5b6478; }
|
||||
body { margin:0; background:var(--bg); color:var(--fg);
|
||||
font:13px/1.5 ui-monospace,SFMono-Regular,Menlo,monospace; }
|
||||
header { padding:12px 18px; border-bottom:1px solid var(--line);
|
||||
display:flex; gap:10px; align-items:center; flex-wrap:wrap; }
|
||||
h1 { font-size:14px; margin:0 8px 0 0; letter-spacing:.04em; }
|
||||
h1 span { color:var(--dim); font-weight:400; }
|
||||
main { padding:16px 18px 40px; display:flex; flex-direction:column; gap:16px; }
|
||||
.row { display:flex; gap:16px; flex-wrap:wrap; }
|
||||
.panel { background:var(--panel); border:1px solid var(--line); border-radius:4px; padding:12px; }
|
||||
.panel h2 { font-size:11px; text-transform:uppercase; letter-spacing:.08em;
|
||||
color:var(--dim); margin:0 0 8px; font-weight:500; }
|
||||
canvas { display:block; image-rendering:pixelated; max-width:100%; border-radius:2px; }
|
||||
button,input[type=text],select { font:inherit; background:#1c2130; color:var(--fg);
|
||||
border:1px solid var(--line); border-radius:3px; padding:5px 10px; cursor:pointer; }
|
||||
button:hover { border-color:var(--accent); }
|
||||
input[type=text] { cursor:text; }
|
||||
label.ctl { display:grid; grid-template-columns:132px 1fr 46px; gap:10px;
|
||||
align-items:center; margin-bottom:6px; }
|
||||
label.ctl span:first-child { color:var(--dim); }
|
||||
label.ctl output { text-align:right; color:var(--accent); }
|
||||
input[type=range] { width:100%; accent-color:var(--accent); }
|
||||
#status { color:var(--dim); } #status.ok{color:var(--ok)} #status.err{color:var(--err)} #status.warn{color:var(--accent)}
|
||||
#readout { color:var(--dim); font-size:12px; }
|
||||
|
||||
/* the frame strip is the editing surface */
|
||||
#strip { display:flex; flex-wrap:wrap; gap:4px; }
|
||||
.fr { position:relative; cursor:pointer; border:2px solid transparent; border-radius:3px; line-height:0; }
|
||||
.fr canvas { border-radius:1px; }
|
||||
.fr span { position:absolute; left:2px; bottom:2px; font-size:10px; line-height:1.2;
|
||||
padding:0 3px; border-radius:2px; background:#000a; color:#fff; }
|
||||
.fr.keep { border-color:var(--ok); }
|
||||
.fr.drop { border-color:#2a2f3e; }
|
||||
.fr.drop canvas { opacity:.26; filter:grayscale(1); }
|
||||
.fr.cur { border-color:var(--accent); }
|
||||
.fr.keep::after { content:'●'; position:absolute; right:3px; top:1px; color:var(--ok); font-size:10px; }
|
||||
|
||||
#sheet { display:flex; flex-wrap:wrap; gap:10px; }
|
||||
#sheet .cell { display:flex; flex-direction:column; gap:3px; cursor:pointer; }
|
||||
#sheet .cell span { color:var(--dim); font-size:11px; }
|
||||
#sheet .cell:hover span { color:var(--accent); }
|
||||
#palette { display:flex; gap:12px; flex-wrap:wrap; }
|
||||
.sw { display:flex; align-items:center; gap:5px; color:var(--dim); font-size:11px; }
|
||||
.sw input { width:26px; height:20px; padding:0; border:1px solid var(--line); background:none; }
|
||||
.legend { color:var(--dim); font-size:11px; margin-top:6px; }
|
||||
kbd { background:#1c2130; border:1px solid var(--line); border-radius:2px;
|
||||
padding:0 4px; color:var(--fg); font-size:11px; }
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<header>
|
||||
<h1>roto <span>— video → take builder</span></h1>
|
||||
<button id="btn-synth">Synthetic take</button>
|
||||
<input type="text" id="framedir" value="frames" size="8" title="frame directory">
|
||||
<h1>roto <span>— frame removal + take builder</span></h1>
|
||||
<button id="btn-synth">Synthetic</button>
|
||||
<input type="text" id="framedir" value="frames" size="7" title="frame directory">
|
||||
<button id="btn-frames">Load frames</button>
|
||||
<button id="btn-play">Play</button>
|
||||
<input type="text" id="takename" value="line_01" size="10" title="take name">
|
||||
<select id="playfps" title="playback fps"><option>6</option><option selected>12</option><option>24</option></select>
|
||||
<button id="btn-keepall">Keep all</button>
|
||||
<button id="btn-suggest">Suggest</button>
|
||||
<input type="text" id="takename" value="line_01" size="9" title="take name">
|
||||
<button id="btn-export">Export .take</button>
|
||||
<span id="status"></span>
|
||||
</header>
|
||||
|
|
@ -70,7 +81,7 @@
|
|||
<div class="panel">
|
||||
<h2>stabilised (head-local)</h2>
|
||||
<canvas id="cv-stab"></canvas>
|
||||
<div class="legend">live contour · <b class="k">—</b> active key pose · should sit still except the mouth</div>
|
||||
<div class="legend">should sit still except the mouth · grey = held plate outline</div>
|
||||
</div>
|
||||
<div class="panel">
|
||||
<h2>flat render — 320×200 indexed</h2>
|
||||
|
|
@ -79,19 +90,35 @@
|
|||
</div>
|
||||
</div>
|
||||
|
||||
<div class="panel">
|
||||
<h2>frames — green kept (gets its own drawing) · dim held from the last kept frame</h2>
|
||||
<div id="strip"></div>
|
||||
<input type="range" id="scrub" min="0" max="0" value="0" style="width:100%;margin-top:10px">
|
||||
<div class="legend">
|
||||
<kbd>←</kbd> <kbd>→</kbd> step · <kbd>X</kbd> delete · <kbd>K</kbd> keep ·
|
||||
click to select · double-click or shift-click to toggle ·
|
||||
the mouth keeps <b>every</b> frame regardless
|
||||
</div>
|
||||
<div class="legend" id="readout"></div>
|
||||
</div>
|
||||
|
||||
<div class="row">
|
||||
<div class="panel" style="flex:1 1 380px">
|
||||
<div class="panel" style="flex:1 1 400px">
|
||||
<h2>knobs</h2>
|
||||
<label class="ctl"><span>vertices</span><input type="range" id="verts" min="4" max="16" step="2" value="8"><output id="vertsv"></output></label>
|
||||
<label class="ctl"><span>min hold</span><input type="range" id="minHold" min="1" max="8" value="2"><output id="minHoldv"></output></label>
|
||||
<label class="ctl"><span>change gate</span><input type="range" id="distThresh" min="0" max="40" value="6"><output id="distThreshv"></output></label>
|
||||
<label class="ctl"><span>vel smoothing</span><input type="range" id="velSmooth" min="1" max="11" step="2" value="3"><output id="velSmoothv"></output></label>
|
||||
<label class="ctl"><span>anchor smoothing</span><input type="range" id="smoothWin" min="1" max="21" step="2" value="5"><output id="smoothWinv"></output></label>
|
||||
<label class="ctl"><span>exposure</span><input type="range" id="exposure" min="1" max="4" value="2"><output id="exposurev"></output></label>
|
||||
<label class="ctl"><span>contour avg ±f</span><input type="range" id="contourSmooth" min="0" max="4" value="1"><output id="contourSmoothv"></output></label>
|
||||
<label class="ctl"><span>anchor avg ±f</span><input type="range" id="smoothWin" min="0" max="8" value="2"><output id="smoothWinv"></output></label>
|
||||
<label class="ctl"><span>closed-mouth cut</span><input type="range" id="apertureThresh" min="0" max="400" value="120"><output id="apertureThreshv"></output></label>
|
||||
<div class="legend" id="readout"></div>
|
||||
<label class="ctl"><span>suggest tolerance</span><input type="range" id="tol" min="2" max="60" value="14"><output id="tolv"></output></label>
|
||||
<div class="legend">
|
||||
<b>contour avg</b> 0 = off, 1 = ±1 frame. Removes per-frame landmark
|
||||
jitter. Push past 2 and it starts eating articulation.<br>
|
||||
<b>anchor avg</b> smooths the head transform only — never the contour.<br>
|
||||
<b>suggest tolerance</b> only affects the Suggest button: max head movement
|
||||
allowed before a new drawing is required.
|
||||
</div>
|
||||
</div>
|
||||
<div class="panel" style="flex:1 1 300px">
|
||||
<div class="panel" style="flex:1 1 280px">
|
||||
<h2>palette</h2>
|
||||
<div id="palette"></div>
|
||||
<div class="legend" style="margin-top:12px">
|
||||
|
|
@ -102,18 +129,7 @@
|
|||
</div>
|
||||
|
||||
<div class="panel">
|
||||
<h2>timeline</h2>
|
||||
<canvas id="cv-timeline"></canvas>
|
||||
<input type="range" id="scrub" min="0" max="0" value="0" style="width:100%;margin-top:8px">
|
||||
<div class="legend">
|
||||
velocity curve · <b class="c">|</b> candidate minima ·
|
||||
<b class="k">|</b> key as rendered (<code>f</code>) ·
|
||||
<b class="s">|</b> true extreme (<code>src</code>) — a large gap means min-hold and exposure are fighting
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="panel">
|
||||
<h2>selected poses — the vocabulary the take actually contains</h2>
|
||||
<h2>drawings needed — one per kept frame, with the range it holds</h2>
|
||||
<div id="sheet"></div>
|
||||
</div>
|
||||
</main>
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue