Workloads
Audio Encoding
Every audio job decodes the bundled FLACFLAC: Free Lossless Audio CodecA lossless audio codec by the Xiph.Org Foundation. It compresses audio without discarding any information, so decoding restores the original samples bit for bit.FLACGlossary (Free Lossless Audio Codec) music sample, resamples it when needed, and encodes stereo output with one codec thread, using VorbisVorbisA royalty-free, lossy audio codec by the Xiph.Org Foundation, usually stored in Ogg files. It was designed as a free alternative to MP3 and AAC, and is used in games and streaming; Opus has largely replaced it for new uses.VorbisGlossary, OpusOpus: audio codecA royalty-free, lossy audio codec standardized by the IETF in 2012. It covers speech and music from low to high bitrates with low delay, which makes it common in voice and video calls and web streaming.OpusGlossary (audio codec), AACAAC: Advanced Audio CodingA lossy audio codec standardized in MPEG-2 and MPEG-4 as the successor of MP3. It sounds better than MP3 at the same bitrate, and it is the usual audio track of MP4 files and streaming video.FFmpeg AAC encoderGlossary (Advanced Audio Coding), MP3MP3: MPEG-1 Audio Layer IIIA lossy audio codec and file format from the 1990s, still the most widely supported format for compressed music.WikipediaGlossary, or FLAC. The audio scenarios run on the CPUCPU: Central Processing UnitThe general-purpose processor of a computer, which runs the operating system and programs. A CPU has one or more physical cores, and with SMT each core runs two or more hardware threads.WikipediaGlossary only; see Audio on the CPU Only.
| Scenario | Encoder | Settings | Sample rate |
|---|---|---|---|
ogg_vorbis_160k | libvorbis | 160 kbps | 44.1 kHz |
opus_128k | libopus | 128 kbps | 48 kHz |
aac_128k | aac (FFmpegFFmpegA free, open-source collection of libraries and command-line tools to decode, encode, transcode, filter and stream audio and video. Its ffmpeg command converts between nearly all media formats, and its libraries, such as libavcodec, power many media players and services.`ffmpeg` docsGlossary's native AAC encoder) | 128 kbps | 44.1 kHz |
mp3_192k | libmp3lame | 192 kbps | 44.1 kHz |
flac_lossless | flac (FFmpeg's native FLAC encoder) | Compression level 5 | 44.1 kHz |
The opus_128k workload uses 48 kHz because libopus accepts only 8, 12, 16,
24, or 48 kHz, not the sample's 44.1 kHz.
Audio results are reported as audio_seconds_per_sec: seconds of audio
processed per second of wall time, across all workers.
CPU Video
The CPU video scenarios use FFmpegFFmpegA free, open-source collection of libraries and command-line tools to decode, encode, transcode, filter and stream audio and video. Its ffmpeg command converts between nearly all media formats, and its libraries, such as libavcodec, power many media players and services.`ffmpeg` docsGlossary's software codecs, with one codec thread per
job for both decoding and encoding:
| Scenario | Operation | Input | Decoder | Encoder and settings |
|---|---|---|---|---|
cpu_h264_encode | Encode | H.264H.264A lossy video codec standardized in 2003, also called AVC. It is the most widely supported video format, used in Blu-ray, broadcast, streaming, and video calls. Common encoders are x264 and hardware encoders such as NVENC.SpecificationGlossary clip | h264 | libx264 -crf 18 |
cpu_h265_encode | Encode | H.264 clip | h264 | libx265 -crf 18, one frame thread, no thread pool |
cpu_h264_decode | Decode | H.264 clip | h264 | None |
cpu_h265_decode | Decode | H.265H.265Another name for HEVC, from the number of its standard.SpecificationGlossary clip | hevc | None |
The encode scenarios also decode their H.264 input on the CPU, so they measure
a full transcode. CRFCRF: Constant Rate FactorThe quality-based rate control of the x264 and x265 encoders, which varies the bitrate to keep the perceived quality constant across scenes. Lower values mean higher quality and larger files; x264's default is 23 on a scale of 0 to 51.FFmpeg wikiGlossary
(Constant Rate Factor) 18 is a
quality target; no -preset is set, so libx264 and libx265 use their
default preset (medium).
Video results are reported as aggregate_fps: frames processed per second of
wall time, across all workers.
NVIDIA Video
The NVIDIA video scenarios decode on the GPUGPU: Graphics Processing UnitA processor with thousands of simple cores that run the same operation on many data elements in parallel. It was built to render graphics, and is now also used for general-purpose parallel work such as machine learning training and inference, scientific simulation, and video encoding. Cloud providers rent GPUs attached to virtual machines, with their own dedicated GPU memory.WikipediaGlossary and, for encodes, keep the frames
in GPU memory for NVENCNVENC: NVIDIA Video EncoderThe hardware video encoder of NVIDIA GPUs, a dedicated unit next to the CUDA cores. It encodes H.264, HEVC and, on newer GPUs, AV1, much faster than software encoders, though usually at a somewhat lower quality for the same bitrate. FFmpeg exposes it as encoders such as h264_nvenc.NVIDIA GPUs in FFmpegGlossary
(NVIDIA Video Encoder):
| Scenario | Operation | Input | Decoder | Encoder and settings |
|---|---|---|---|---|
gpu_h264_encode | Encode | H.264H.264A lossy video codec standardized in 2003, also called AVC. It is the most widely supported video format, used in Blu-ray, broadcast, streaming, and video calls. Common encoders are x264 and hardware encoders such as NVENC.SpecificationGlossary clip | h264_cuvid | h264_nvenc -preset p4 -rc vbr -cq 22 -b:v 0 |
gpu_h264_decode | Decode | H.264 clip | h264_cuvid | None |
gpu_h265_encode | Encode | H.264 clip | h264_cuvid | hevc_nvenc -preset p4 -rc vbr -cq 22 -b:v 0 |
gpu_h265_decode | Decode | H.265H.265Another name for HEVC, from the number of its standard.SpecificationGlossary clip | hevc_cuvid | None |
Each job runs with -hwaccel cuda on one GPU. The -cq 22 constant-quality
setting was chosen to approximately match the CPU encodes at CRF 18; see
CPU and GPU Encode Quality.
Test Media
The image build downloads the test media from the Spare Cores CDNCDN: Content Delivery NetworkA network of distributed servers that cache content, such as web pages, images or video, and serve it from locations close to its users. It lowers latency for the users and offloads traffic from the origin server.WikipediaGlossary (Content Delivery Network) and checks each file's SHA-256SHA-256: Secure Hash Algorithm 256A cryptographic hash function of the SHA-2 family that maps any data to a 256-bit digest. Even a one-bit change in the data gives a completely different digest, so it is used to verify that a downloaded file is intact, and in digital signatures and certificates.NIST FIPS 180-4Glossary (Secure Hash Algorithm 256) checksum. At run time, the H.264H.264A lossy video codec standardized in 2003, also called AVC. It is the most widely supported video format, used in Blu-ray, broadcast, streaming, and video calls. Common encoders are x264 and hardware encoders such as NVENC.SpecificationGlossary and H.265H.265Another name for HEVC, from the number of its standard.SpecificationGlossary clips are checked again; the audio file is not. The files are the following:
- H.264 clip (
source.mp4): an excerpt of 30 seconds from the Blender Foundation's open movie Tears of Steel, cut at 60 seconds from the 1080p release with stream copy. It is 1920x800 pixels at 24 fpsfps: Frames Per SecondThe number of frames, the individual images of an animation or simulation, that a program produces each second. A higher rate means smoother motion or faster processing.WikipediaGlossary (Frames Per Second). License: CC BY 3.0CC BY 3.0: Creative Commons Attribution 3.0A Creative Commons license that allows anyone to copy, adapt and use a work, also commercially, as long as they credit its author.LicenseGlossary (Creative Commons Attribution 3.0), as stated on the Tears of Steel sharing page. - H.265 clip (
source-hevc.mp4): the same excerpt, transcoded withlibx265 -crf 18 -preset medium, so the H.265 decode scenarios get the same frame size, frame rate, and duration. - Music sample (
source.flac): the "Entre dos Aguas" sample performed by Michael Laucke, about 85 seconds at 44.1 kHz. License: CC0 1.0CC0 1.0: Creative Commons Zero 1.0A Creative Commons dedication with which authors waive all their copyright in a work, placing it as close to the public domain as the law allows. Anyone may use the work for any purpose, without attribution.LicenseGlossary (Creative Commons Zero 1.0).
Each job loops its input (-stream_loop -1) and stops after the media length
set for that worker count (-t), so the length of a job does not depend on the
length of the file. The harnessharnessThe code that runs a benchmark tool. It prepares the inputs, starts the tool with fixed settings, often several times or in parallel, checks and collects the tool's output, and prints the results in one format. Its settings, not the tool's defaults, decide what the benchmark measures.WikipediaGlossary (benchmark.py) probes each
file's resolution, frame rate, and duration at run time and reports them in
sources.
How Measurement Works
This benchmark runs its 13 scenarios in a fixed order, as defined in
benchmark.py: the eight video scenarios first, then the
five audio scenarios. Each repetition measures one group of parallel workers in
the following steps:
- The harness starts every worker as a separate process. Each worker first
waits to read from one shared pipe, so
ffmpeghas not started yet when the last worker is ready. - The harness records the start time, then closes the pipe. Closing it
ends every worker's wait at the same moment, and each worker starts its
ffmpegcommand. The start time comes from a monotonic clock, which only moves forward and is not changed by system time adjustments such as NTPNTP: Network Time ProtocolA protocol that keeps computer clocks in sync with reference time servers over a network. A client adjusts its system clock in small steps or, when it is far off, in one jump, so wall-clock time on a server can move backward or skip ahead.NTPGlossary (Network Time Protocol) corrections, so the measured wall time cannot jump. - It records each worker's exit time and exit code. The wall time is the time from the release to the last exit, so process startup and teardown are included.
- It computes the rate: successful workers × media length per worker, divided by the wall time. For video, the media length is converted to frames with the clip's frame rate.
The harness never adds up the speed that each ffmpeg process reports for
itself. Workers that time out are stopped, recorded with exit code 124, and
count as failed. Encoded output goes to the null muxernull muxerAn FFmpeg output format, selected with -f null, that discards everything it receives instead of writing a file. It is used to measure decoding, filtering or encoding speed, or to run analysis filters, without the cost of writing to disk.FFmpeg formatsGlossary and is discarded; see
Storage and Output.
See Worker Counts for how many workers each group has, and Run Duration for the repetitions and timeouts.