Short answer: ffprobe -show_entries format=duration reports the container's duration, which is the maximum over its streams, and in a generated clip the silent audio track always runs the full promised length. The picture can end over a second early and that number does not move, so any test comparing it against the planned duration is green by construction. Ask the streams instead, add an assertion that the picture is not shorter than the sound, and derive the tolerance from the file's own frame rate.
The pipeline builds a vertical clip from still images, a voice track and burned-in subtitles. There was a test asserting that the finished file lasts as long as the build promised. To check that the test worked, I shortened every image segment by 10%: the picture now ends before the voice does, so the end of a spoken sentence plays over nothing. Several hundred tests stayed green, including that one.
TL;DR
-
Fix: assert on
stream=durationforv:0, and add a second assertion that the video stream is not shorter than the audio stream in the same file. Two numbers, both from streams, both able to go red. -
Alternative: count frames with
-count_framesif you want to be certain the picture is really there. Slower, and for this defect the stream duration was enough. - Trap: a tolerance picked by eye. Half a second looks modest and swallows a 10% break whole on any clip shorter than five seconds, because 0.1 × 5 = 0.5.
Why the test could not fail
Because it was reading the container, and the container's duration is the longest stream in it. Two facts have to line up for this to become a blind spot, and in a generated clip they always do.
The first is what the field means. ffprobe's own documentation does not hide it, and the word is right there in the option description for -show_format:
Show information about the container format of the input multimedia stream. All the container format information is printed within a section with name "FORMAT".
The second is how the audio track gets its length. The clip is assembled in one ffmpeg invocation, with images laid out by their shares of the timeline and the voice placed at its own offsets. To hold a continuous audio timeline, the build stretches a silence source across the whole declared length, which is anullsrc, documented as "The null audio source, return unprocessed audio frames." Whatever the picture does, that track is exactly as long as the build said the clip would be.
Put those together and the number the test trusted is structurally unbreakable. The audio holds the full duration, the container reports the maximum, and the test compares that maximum against the planned duration. It will agree forever.
I did not see this immediately. My first suspicion was the tolerance, because a passing test with a deliberately broken input looks like a threshold problem, and I spent a while adjusting a number that had nothing to do with it. The tolerance was innocent. The measurement was pointed at the wrong object.
Six lines that show it on any machine
This is the whole mechanism, reproduced from scratch on ffmpeg 7.1.5. Build a file whose picture is two seconds and whose silence is three, which is exactly the shape the broken pipeline produced:
ffmpeg -v error -y \
-f lavfi -i "color=c=black:s=320x240:r=30:d=2" \
-f lavfi -i "anullsrc=r=44100:cl=stereo:d=3" \
-map 0:v -map 1:a -c:v libx264 -pix_fmt yuv420p -c:a aac short.mp4
Then ask the same file three different questions:
ffprobe -v error -show_entries format=duration -of csv=p=0 short.mp4
# 3.000000 <- the container, and what a naive test reads
ffprobe -v error -select_streams v:0 -show_entries stream=duration -of csv=p=0 short.mp4
# 2.000000 <- the picture, a third shorter than the file claims
ffprobe -v error -select_streams a:0 -show_entries stream=duration -of csv=p=0 short.mp4
# 3.000000 <- the sound, holding the container's number up by itself
A one-third shortfall in the picture, and the duration the test reads does not budge. If the first command and the second disagree on a file of yours, some stream is carrying the length for all the others.
| What you ask for | What it reports | What it hides |
|---|---|---|
format=duration |
the container, the maximum over all streams | any stream that is shorter than the longest one |
stream=duration on v:0
|
how long the picture lasts | nothing about whether the sound matches it |
stream=duration on a:0
|
how long the sound lasts | nothing about whether the picture matches it |
stream=r_frame_rate |
frames per second as written in the file | variable frame rate, where avg_frame_rate is the honest field |
-count_frames |
frames actually present | costs a full decode pass |
On the real clip the gap was 13.139 s reported against 11.833 s of actual picture, a shortfall of 1.306 s that no assertion noticed.
What the FFmpeg source says, on both sides of the file
The maximum-over-streams rule is not an inference from behaviour. It is written twice in the source, independently, once for reading a file and once for writing one, which is worth knowing because it means the property is not going to be fixed out from under you.
On the reading side, update_stream_timings() in libavformat/demux.c folds each stream's duration into a running maximum:
if (is_text)
duration_text = FFMAX(duration_text, duration1);
else
duration = FFMAX(duration, duration1);
The is_text branch is the interesting part. Subtitle and data streams are kept in a separate accumulator, and when one of them is an outlier the function logs Ignoring outlier non primary stream duration and discards it. So FFmpeg does have a guard against one stream dragging the container's length around, and that guard covers text streams only. Audio and video are equal participants in the same maximum with nothing between them. That is precisely why a short picture is invisible in the container number while a long subtitle track would not be.
On the writing side, mov_write_mvhd_tag() in libavformat/movenc.c builds the movie header duration the same way, walking the tracks and keeping max_track_len. The MP4 you just wrote records the longest track as the movie's length.
The header documentation agrees, and says something useful about who owns the value. From the AVFormatContext.duration comment in libavformat/avformat.h:
Duration of the stream, in AV_TIME_BASE fractional seconds. Only set this value if you know none of the individual stream durations and also do not set any of them. This is deduced from the AVStream values if not set.
"Deduced from the AVStream values" is the container duration's whole nature. It is a derived figure, and asking it about the picture is a category error.
The second blind spot: two numbers from one measurement
The same test had a second defect that would have kept it green even with the right ffprobe call. It compared the measured duration against a duration recorded in the artifact's metadata, and that metadata had been filled in by the same ffprobe invocation during the build.
A check like that is not a comparison. It is one number being shown to itself, and it passes under every possible breakage, because whatever moves one side moves the other identically. It looks like verification in a diff, it reads like verification in a test name, and it has no power to fail.
There is only one way to tell this class of test from a real one, and it is the thing I should have done before trusting any of them: break the source on purpose and watch the colour. A test that stays green when you have deliberately damaged its subject is not covering the subject. The fix is to source the second number independently: measure the voice from the voice file, not from the build's note about the voice file.
Three assertions that actually hold
Three separate statements about one assembled clip, all of them about streams:
- The video stream's duration against what the build promised.
- The video stream against the audio stream in the same file, asserting that the picture does not end before the sound.
- The sum of the scene's frame shares against the duration of that scene's voice track, measured again from the voice file rather than read from metadata.
The third is not redundant, and I confirmed that by breaking both things separately. A layout defect, where the frame shares are computed wrongly, sails past the first two: ffmpeg faithfully assembles the distorted plan, and the clip comes out exactly as long as it was told to be. Under a 10% layout break, the scene's voice measured 3.295 s while its frames were given 2.966 s, the third assertion went red, and the first two stayed green. Two classes of defect, two different failures. Either assertion alone would have shipped the other class.
The tolerance has to come from the format
Video is cut into segments and each one encodes to a whole number of frames, so a small shortfall is structural rather than a bug, and it grows with the number of segments. That makes the tolerance derivable instead of arbitrary:
tolerance = number of segments × frame duration
Six segments at 30 fps gives 6 × (1/30) = 0.2 s. The real clip came in 39 ms under its promise, a little over one frame across the whole thing, comfortably inside that. The deliberate break produced 1.306 s, which is 6.5 times the tolerance.
Take the frame rate from the file rather than from a constant in the builder:
ffprobe -v error -select_streams v:0 -show_entries stream=r_frame_rate -of csv=p=0 short.mp4
# 30/1
Asking the file has a side benefit: the assertion now also proves the builder wrote the frame rate it was configured with. One caveat from the AVStream.r_frame_rate documentation, which is unusually candid:
Real base framerate of the stream. This is the lowest framerate with which all timestamps can be represented accurately (it is the least common multiple of all framerates in the stream). Note, this value is just a guess!
For constant-frame-rate output, which is what this pipeline produces, it is the right field. If your clips are variable frame rate, use avg_frame_rate instead and accept that a per-segment frame tolerance means less.
Now the part that convinced me the derivation was worth the effort. Write the break as a fraction p of the clip length, with n segments of average length s at f frames per second. The shortfall is p × n × s and the tolerance is n / f, so their ratio is:
break / tolerance = p × s × f
The segment count cancels. The safety factor does not depend on how long the clip is, only on how long a segment is and how many frames there are per second. For this pipeline that is 0.1 × 2.19 × 30 = 6.57, and the figure actually measured was 1.306 / 0.2 = 6.53.
A fixed tolerance has no such property. Half a second is comfortable on a long clip and useless on a short one, because the break scales with the material while the constant does not: at 10%, a clip of five seconds or less hides its entire breakage inside 0.5 s. That is the same test passing or failing depending on the length of the input, which is not a tolerance, it is a coin flip.
For the sum-of-shares assertion the tolerance is different and tight, a millisecond per frame share, because the shares are rounded to milliseconds one at a time and there is nowhere else for error to accumulate.
The dead end that turned out useful
Wanting the frame rate in the test, my first move was to import the constant from the generator module so the value would not be duplicated. That produced:
ImportError: cannot import name ... from partially initialized module ...
(most likely due to a circular import)
The generator module is brought up by a registry, so reaching into it by name from a test closes a loop. The accident was lucky: being unable to import the constant is what pushed me to read the frame rate off the finished file, which is the better assertion anyway for the reason above.
The message itself is worth a second look, because it diagnoses which of two different failures happened. In CPython's ceval.c the wording with "partially initialized module" is selected by a call to _PyModuleSpec_IsInitializing(spec). Without that phrase, the name genuinely does not exist in the module. With it, the name may well exist and the module simply has not finished executing. Same ImportError, two different problems, and the text tells you which.
Step by step
- Break the subject of the test on purpose before trusting the test. Shorten the output, skew the layout, whatever damage the test claims to catch.
- If it stays green, check what object the measurement points at before touching the tolerance.
- Replace
format=durationwith-select_streams v:0 -show_entries stream=durationeverywhere a test means "how long is the picture". - Add an assertion comparing the video stream against the audio stream in the same file.
- Find any check that compares a measurement against metadata written by the same measurement, and re-source one side of it.
- Derive the tolerance from the container format: segments times frame duration, with the frame rate read from the file.
- Verify that each class of defect lands on a different assertion, by breaking each class separately.
Bottom line
A test named after the thing you care about does not necessarily measure it. In any format with several streams, the container's duration is the maximum across them rather than what a viewer sees, so a clip can lose a second of picture and still report its promised length. Two numbers that came from one measurement will agree under every breakage, and the only way to tell such a test from a working one is to damage the input and watch for red. The tolerance belongs to the check: derived from the frame rate it keeps a constant safety margin at any clip length, and picked by eye it fails both ways at once.
No real defect turned up in the builder in the end. The clip was the length it promised, and the 39 ms came from frame quantisation. What changed is that the tests can now go red if that stops being true.
Originally published at bitpage.me — BitPage, a technical blog on backend, databases, infrastructure and incident post-mortems.
Top comments (0)