Measure the frame rate rather than believing the container

`probe` took `r_frame_rate` whenever it was at or under the cap, on the
grounds that it is the rate that keeps every distinct source frame. It is
not a claim about frames at all: ordinary iPhone footage declares 120 over
a stream whose timestamps are 1/30s apart, and resampling it up turned an
11-second clip into 1293 proxy frames instead of 323 — four times the
encode, four times the tracing stills (91MB against 23MB), four times the
blobs and the rows, for 970 frames that are copies of their neighbours.

So `_measured_rate` reads the timestamps and `_choose_rate` keeps whichever
declared rate they bear out. Two details carry it: the times are sorted
before differencing, because an HEVC stream arrives in decode order and
differencing that measures the reordering delay instead of the rate; and
the statistic is the MEDIAN interval, which is what keeps the property the
nominal rate was being taken for — a take held on one frame still reports
the rate of the parts that move, so no distinct frame is dropped. A genuine
120fps capture still extracts at 120, and there is a test on that
specifically.

`Source.probe` also stopped being the place a reading goes to be preserved.
The facts are a pure function of bytes that are the row's own identity, so
a re-upload re-reads them: otherwise every already-uploaded source would
have gone on resampling to four times the frames with no way to correct it
short of deleting the row.

Already-extracted footage is untouched — `extraction_key` still says
scheme 3, so those jobs stay done and reachable. Bumping it re-extracts
everything at the corrected rate.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Your Name 2026-10-04 23:22:50 -04:00
parent 8d20097e61
commit e26ad723fa
3 changed files with 210 additions and 7 deletions

View file

@ -159,6 +159,16 @@ def _extract_stills(job, proxy_path, frames_dir, frames, root):
MAX_RATE = 120 # a capture rate; past this the container is describing something else
# How many packet timestamps `_measured_rate` reads, and the fewest intervals it
# will draw a conclusion from. 300 is a flat cost on a long take and still a
# wide enough sample for a median; below 8 intervals there is not enough of a
# stream to outvote one odd timestamp, so the metadata is left to speak.
RATE_SAMPLE = 300
RATE_MINIMUM = 8
# How far a declared rate may sit from the measured one and still be taken as
# what the stream is: 2% covers 30 against 30000/1001 and nothing like 120
# against 30.
RATE_TOLERANCE = 0.02
def probe_image(path):
@ -184,6 +194,75 @@ def probe_audio(path):
return duration
def _measured_rate(path):
"""The rate the stream's own packet timestamps imply, or None.
THE CONTAINER'S SUMMARY OF ITSELF IS NOT EVIDENCE, and this is the function
that goes and looks. An iPhone's `r_frame_rate` is 120 on footage whose
timestamps are 1/30s apart, which is the difference between 323 frames and
1293 — four times the encode, four times the tracing stills, four times the
blobs, for 970 frames that are copies of their neighbours.
It reads TIMESTAMPS, not frames: `-show_entries packet=pts_time` demuxes
without decoding, so this costs a file read and no pixels. The times are
SORTED before differencing because a stream with B-frames arrives in decode
order — an HEVC clip's first packets come out 0, 0.133, 0.067, 0.033 — and
differencing that order measures the reordering rather than the rate.
THE MEDIAN INTERVAL, which is what makes this safe on genuinely variable
input. It answers "how far apart are two frames normally", so a take held on
one frame for a second still reports the rate of the parts that move, and
choosing it keeps every distinct frame — the property `probe` used to reach
for by taking the nominal rate. Only the last few intervals of the sample are
unreliable (a frame whose turn comes after the window is missing from it), and
a median does not care.
Returning None is the honest answer for a clip too short to sample, and this
also swallows a probe that fails outright: the rate the metadata declares is
the documented fallback, so an optimisation must not be able to refuse an
upload that would otherwise have been accepted.
"""
try:
text = _command(["ffprobe", "-v", "error", "-select_streams", "v:0",
"-show_entries", "packet=pts_time", "-of", "json",
"-read_intervals", f"%+#{RATE_SAMPLE}", str(path)])
packets = json.loads(text).get("packets") or []
times = sorted(float(packet["pts_time"]) for packet in packets
if (packet.get("pts_time") or "N/A") != "N/A")
except (ValueError, OSError):
return None
intervals = sorted(b - a for a, b in zip(times, times[1:]) if b > a)
if len(intervals) < RATE_MINIMUM:
return None
median = intervals[len(intervals) // 2]
return 1.0 / median if median > 0 else None
def _choose_rate(nominal, average, measured):
"""The rate to resample onto, as an exact Fraction.
A DECLARED RATE IS PREFERRED WHEN IT AGREES WITH THE TIMESTAMPS, because it is
the exact rational the stream was authored at — 30000/1001 is not a float, and
`limit_denominator` on a measured 29.97 is a guess at a number the container
already states. So the measured rate is used to CHOOSE between what the
container declares, and only stands in itself when neither declaration
describes the stream.
"""
candidates = [rate for rate in (nominal, average) if 0 < rate <= MAX_RATE]
if measured:
agreeing = [rate for rate in candidates
if abs(float(rate) - measured) <= RATE_TOLERANCE * measured]
if agreeing:
return min(agreeing, key=lambda rate: abs(float(rate) - measured))
from_timestamps = Fraction(measured).limit_denominator(1001)
if 0 < from_timestamps <= MAX_RATE:
return from_timestamps
# Nothing to go on but the metadata, and nominal first keeps the rate that
# drops no distinct frame. An unusable pair falls through to the refusal
# below, which names the rate the file claimed rather than one of these.
return nominal if 0 < nominal <= MAX_RATE else average
def probe(path):
"""What the upload is, as far as choosing a proxy rate goes.
@ -201,12 +280,21 @@ def probe(path):
of itself disagreed with the container's own contents, so the guard rejected
CFR video for being variable.
THE RATE IS THE NOMINAL ONE. `r_frame_rate` is the rate every timestamp in the
stream can be expressed at, which is the rate that keeps every distinct source
frame; resampling to the average would drop some. Duration is preserved either
way — ffmpeg's CFR conversion is driven by timestamps, so the audio stays in
sync at any rate — so this trades a possible duplicated frame against a
certainly lost one.
THE RATE IS MEASURED AND THE DECLARATIONS ARE VOTED ON, which is the same
distrust applied to the one number that still comes from here. This used to
take `r_frame_rate` outright — the rate every timestamp in the stream can be
expressed at, and so the rate that keeps every distinct source frame. The
trouble is that it is not a claim about frames at all: the file above declares
120 and holds 30, and resampling it up cost four times the encode, four times
the tracing stills and four times the blobs for 970 duplicated frames. So
`_measured_rate` reads the timestamps, `_choose_rate` keeps whichever declared
rate they bear out, and the nominal rate is believed when it is true rather
than because it is nominal.
Duration is preserved either way — ffmpeg's CFR conversion is driven by
timestamps, so the audio stays in sync at any rate — and the median interval
keeps the no-distinct-frame-dropped property that taking the nominal rate was
reaching for. See `_measured_rate`.
"""
data = json.loads(_command(["ffprobe", "-v", "error", "-show_streams",
"-show_format", "-of", "json", str(path)]))
@ -217,7 +305,8 @@ def probe(path):
average = Fraction(video.get("avg_frame_rate") or "0")
if nominal <= 0 and average <= 0:
raise ValueError("the video's frame rate is unknown")
rate = nominal if 0 < nominal <= MAX_RATE else average
measured = _measured_rate(path)
rate = _choose_rate(nominal, average, measured)
if not 0 < rate <= MAX_RATE:
raise ValueError(f"the video reports a frame rate of {float(rate):g}, which is "
"not a rate footage can be measured at")
@ -230,6 +319,11 @@ def probe(path):
# `-r` a rounded one is how a long take drifts out of sync.
"rate": f"{rate.numerator}/{rate.denominator}",
"nominal_fps": float(nominal), "average_fps": float(average),
# What the timestamps said, and null when there were too few to ask.
# Recorded because it is the input to a decision this file used not to
# make, and the one number that explains a chosen rate matching
# neither declaration.
"measured_fps": measured,
"width": int(video["width"]), "height": int(video["height"]),
"duration": duration,
# KEPT, AND NO LONGER TRUSTED AS A COUNT. See the docstring: this is