FFmpeg
This benchmark measures how much audio and video a server can transcode with
FFmpegFFmpegA free, open-source collection of libraries and command-line tools to decode, encode, transcode, filter and stream audio and video. Its ffmpeg command converts between nearly all media formats, and its libraries, such as libavcodec, power many media players and services.`ffmpeg` docsGlossary when many jobs are run at once. It starts independent, single-threaded
ffmpeg processes from a shared start signal: audio encoding runs on the CPUCPU: Central Processing UnitThe general-purpose processor of a computer, which runs the operating system and programs. A CPU has one or more physical cores, and with SMT each core runs two or more hardware threads.WikipediaGlossary,
and H.264H.264A lossy video codec standardized in 2003, also called AVC. It is the most widely supported video format, used in Blu-ray, broadcast, streaming, and video calls. Common encoders are x264 and hardware encoders such as NVENC.SpecificationGlossary and H.265H.265Another name for HEVC, from the number of its standard.SpecificationGlossary video encoding and decoding run on the CPU and, when
available, on NVIDIA GPUs.
This benchmark reports the aggregate throughput of each scenario at several worker counts: frames per second for video, and audio seconds processed per second for audio. It emits every measured repetition as raw data, without a headline score.
Purpose
Spare Cores Navigator publishes empirical performance measurements for more than 5,000 cloud server types. Media businesses often run video and audio transcoding and similar tasks in the cloud, because their compute needs change over time. The Navigator previously covered this kind of work only through proxies from the Geekbench suite: photo editing, ray tracing, and document rendering.
This benchmark measures transcoding directly. It runs many single-threaded jobs in parallel and times the whole group, so the result shows how much media a server type processes at once, rather than how fast a single job finishes.
Limitations
These results compare transcoding throughput across server types. Note the following limitations:
- Output Quality: This benchmark measures speed, not output quality, and computes no quality metric. GPUGPU: Graphics Processing UnitA processor with thousands of simple cores that run the same operation on many data elements in parallel. It was built to render graphics, and is now also used for general-purpose parallel work such as machine learning training and inference, scientific simulation, and video encoding. Cloud providers rent GPUs attached to virtual machines, with their own dedicated GPU memory.WikipediaGlossary encode settings are calibrated to approximately match the CPU encode quality, but the match is not exact. See also CPU and GPU Encode Quality.
- Storage: Encoded output is
discarded through FFmpegFFmpegA free, open-source collection of libraries and command-line tools to decode, encode, transcode, filter and stream audio and video. Its
ffmpegcommand converts between nearly all media formats, and its libraries, such as libavcodec, power many media players and services.`ffmpeg` docsGlossary's null muxernull muxerAn FFmpeg output format, selected with-f null, that discards everything it receives instead of writing a file. It is used to measure decoding, filtering or encoding speed, or to run analysis filters, without the cost of writing to disk.FFmpeg formatsGlossary, so file, disk, and object-storage writes are not measured. - Fixed Media and Settings: Every job uses the same short video clip or music sample with fixed codec settings. Other resolutions, settings, and codecs are not covered, for example AV1AV1: AOMedia Video 1A royalty-free video codec by the Alliance for Open Media, the successor of VP9. It compresses better than H.264 and HEVC at the same quality, at the cost of slower encoding. Common encoders are libaom, SVT-AV1 and rav1e.SpecificationGlossary (AOMedia Video 1).
- GPUs: Only NVIDIA GPUs are used, for video only; audio always runs on the CPU. A GPU scenario that cannot start on a given GPU is skipped.
Usage
Run the image via Docker. With no options, it runs the CPU and audio scenarios and skips the GPU scenarios:
docker run --rm ghcr.io/sparecores/benchmark-ffmpeg:main > results.json
The image contains its test media, so it needs no network access at run time.
To include the NVIDIA GPU scenarios, pass the GPUs to the container. This needs the NVIDIA driverNVIDIA driverThe kernel module and libraries that let the operating system and programs use NVIDIA GPUs. Each driver version supports CUDA up to a certain version. Containers use the host's driver through the NVIDIA Container Toolkit, so it is installed on the host rather than in container images.NVIDIA docsGlossary and the NVIDIA Container Toolkit on the host:
docker run --rm --gpus all ghcr.io/sparecores/benchmark-ffmpeg:main > results.json
See GPU Run for the GPU requirements.
Production runs the container privileged and with host networking; a plain
docker run needs neither. See
Production Setup before comparing
results with production runs.
A production run on an AWSAWS: Amazon Web ServicesThe cloud computing platform of Amazon, launched in 2006 and the largest public cloud by market share. It offers hundreds of services, from virtual machines (EC2) and object storage (S3) to managed databases and machine learning.AWSGlossary (Amazon Web Services)
m9g.16xlarge took about 10 minutes. Run time grows with each worker count
measured, and with GPU scenarios. See
Run Duration for the steps and timeouts.
Settings
The following settings are the most common; see the full environment-variable reference for all options and defaults:
FFMPEG_BENCH_TARGET_SECONDSsets the wall time of each measured repetition (default: 5s).FFMPEG_BENCH_REPETITIONSandFFMPEG_BENCH_MAX_REPETITIONSset how many repetitions run at each worker count (default: three, up to five).FFMPEG_BENCH_WORKERSreplaces the automatic worker counts with an explicit, comma-separated list.FFMPEG_BENCH_MAX_WORKERScaps the number of parallel workers.FFMPEG_BENCH_TIMEOUT_SECONDSsets the deadline for the whole run (default: 7,200s).
Results
The process prints one compact JSONJSON: JavaScript Object NotationA text format for structured data, built from objects (key-value pairs), arrays, strings, numbers, booleans, and null. Most languages and tools read and write it, which makes it a common format for API responses and benchmark output.JSONGlossary line to stdoutstdoutThe standard output stream of a process, one of the three standard streams with stdin and stderr. Programs write their results to it; a shell can redirect it to a file or another program, such as > results.jsonl, while logs on stderr still reach the terminal.WikipediaGlossary when the run ends; logs go to
stderrstderrThe standard error stream of a process, one of the three standard streams with stdin and stdout. Programs write logs, progress, and error messages to it, so that they stay separate from the output on stdout when the output is redirected to a file or another program.WikipediaGlossary. The main fields of the JSON line are the following:
scenarios[]has one entry per scenario, with itsname,backend(cpuorgpu),operation(encodeordecode), andcodec. A scenario that could not run hasskipped: trueand askip_reason.- Each
scaling_steps[]entry is one worker count:workersparallel jobs, each processingmedia_duration_secseconds of media. - Each
repetitions[]entry is one measurement at that worker count. Video repetitions reportaggregate_fps(frames per second across all workers) andprocessed_frames; audio repetitions reportaudio_seconds_per_secandprocessed_audio_seconds. Thewall_time_secfield is the time from the shared start to the last worker's exit. successful_workers,failed_workers, andtimed_out_workersdescribe each repetition. A failed worker contributes no media, but its time still counts.- There is no score, peak, or median: aggregate the repetitions yourself, for example by taking the median rate at each worker count.
host,ffmpeg,sources, andmethodologyrecord the server, the FFmpegFFmpegA free, open-source collection of libraries and command-line tools to decode, encode, transcode, filter and stream audio and video. Itsffmpegcommand converts between nearly all media formats, and its libraries, such as libavcodec, power many media players and services.`ffmpeg` docsGlossary build, the test media, and the measurement settings.
If benchmark.py itself fails, the process exits with code 1 and prints a JSON
object with an error field; if ffmpeg is not on the PATH, it exits with
code 1 and prints no JSON. Exit code 0 does not mean that every scenario ran:
check skipped, skip_reason, and failed_workers in the output.
Workloads
All scenarios run by default, in the following groups:
- Audio Encoding: VorbisVorbisA royalty-free, lossy audio codec by the Xiph.Org Foundation, usually stored in Ogg files. It was designed as a free alternative to MP3 and AAC, and is used in games and streaming; Opus has largely replaced it for new uses.VorbisGlossary, OpusOpus: audio codecA royalty-free, lossy audio codec standardized by the IETF in 2012. It covers speech and music from low to high bitrates with low delay, which makes it common in voice and video calls and web streaming.OpusGlossary (audio codec), AACAAC: Advanced Audio CodingA lossy audio codec standardized in MPEG-2 and MPEG-4 as the successor of MP3. It sounds better than MP3 at the same bitrate, and it is the usual audio track of MP4 files and streaming video.FFmpeg AAC encoderGlossary (Advanced Audio Coding), MP3MP3: MPEG-1 Audio Layer IIIA lossy audio codec and file format from the 1990s, still the most widely supported format for compressed music.WikipediaGlossary, and FLACFLAC: Free Lossless Audio CodecA lossless audio codec by the Xiph.Org Foundation. It compresses audio without discarding any information, so decoding restores the original samples bit for bit.FLACGlossary (Free Lossless Audio Codec) encodes of one music sample, on the CPU.
- CPU Video: H.264H.264A lossy video codec standardized in 2003, also called AVC. It is the most widely supported video format, used in Blu-ray, broadcast, streaming, and video calls. Common encoders are x264 and hardware encoders such as NVENC.SpecificationGlossary and H.265H.265Another name for HEVC, from the number of its standard.SpecificationGlossary encoding and decoding of one film clip, with one codec thread per job.
- NVIDIA Video: The same H.264 and H.265
encoding and decoding on NVIDIA GPUs, with NVENCNVENC: NVIDIA Video EncoderThe hardware video encoder of NVIDIA GPUs, a dedicated unit next to the CUDA cores. It encodes H.264, HEVC and, on newer GPUs, AV1, much faster than software encoders, though usually at a somewhat lower quality for the same bitrate. FFmpeg exposes it as encoders such as
h264_nvenc.NVIDIA GPUs in FFmpegGlossary (NVIDIA Video Encoder) and NVDECNVDEC: NVIDIA Video DecoderThe hardware video decoder of NVIDIA GPUs, a dedicated unit next to the CUDA cores. It decodes codecs such as H.264, HEVC, VP9 and, on newer GPUs, AV1, leaving the CPU and the CUDA cores free.NVIDIA GPUs in FFmpegGlossary (NVIDIA Video Decoder).
See How Measurement Works for the worker counts, timing, and repetitions.
Design History
We designed this benchmark to measure aggregate capacity. The harnessharnessThe code that runs a benchmark tool. It prepares the inputs, starts the tool with fixed settings, often several times or in parallel, checks and collects the tool's output, and prints the results in one format. Its settings, not the tool's defaults, decide what the benchmark measures.WikipediaGlossary
(benchmark.py) times all ffmpeg processes together, from
their shared start until the last worker exits. Unlike a sum of the speeds
reported by each process, this measurement includes process startup and
teardown.
Worker counts follow physical cores rather than logical CPUs. Two single-threaded workers on sibling hyperthreads compete for the same core, which can lower throughput, especially for decoding.
We later calibrated GPU encodes against CPU encodes. NVENC has no direct equivalent to the CPU encoders' quality setting, so we compared several NVENC quality levels on the test clip and chose the closest match. In harness version 3.3.0, the image also began requesting the NVIDIA driverNVIDIA driverThe kernel module and libraries that let the operating system and programs use NVIDIA GPUs. Each driver version supports CUDA up to a certain version. Containers use the host's driver through the NVIDIA Container Toolkit, so it is installed on the host rather than in container images.NVIDIA docsGlossary's video libraries directly. Many earlier GPU runs could not load those libraries.
This benchmark does not check output quality at run time, CPU and GPU encode quality only match approximately, and GPU results depend on the features each GPU and driver support. See the changelog for the calibration data and the measurement-length study.
References
See the full list of references used in this documentation. To change, rebuild, or test this image, see the contributor guide.