Skip to main content

Contributor Guide

Local Development​

Run the unit tests and a syntax check from the sc-images root:

python3 images/benchmark-ffmpeg/test_benchmark.py
python3 -m py_compile images/benchmark-ffmpeg/benchmark.py

The tests in test_benchmark.py are not copied into the image, and CICI: Continuous IntegrationThe practice of building and testing code automatically on every change, usually on a hosted service such as GitHub Actions, so problems show up before the change is merged.WikipediaGlossary does not run them, so run them before you open a pull request.

Build the image locally from the sc-images root:

docker build -t benchmark-ffmpeg:local images/benchmark-ffmpeg

FFmpeg Build​

The image compiles FFmpegFFmpegA free, open-source collection of libraries and command-line tools to decode, encode, transcode, filter and stream audio and video. Its ffmpeg command converts between nearly all media formats, and its libraries, such as libavcodec, power many media players and services.`ffmpeg` docsGlossary from the upstream release tarball rather than using the Ubuntu package. build-ffmpeg.sh enables only what this benchmark needs:

  • CPUCPU: Central Processing UnitThe general-purpose processor of a computer, which runs the operating system and programs. A CPU has one or more physical cores, and with SMT each core runs two or more hardware threads.WikipediaGlossary codecs: libx264, libx265, libvorbis, libopus, libmp3lame, and FFmpeg's native AACAAC: Advanced Audio CodingA lossy audio codec standardized in MPEG-2 and MPEG-4 as the successor of MP3. It sounds better than MP3 at the same bitrate, and it is the usual audio track of MP4 files and streaming video.FFmpeg AAC encoderGlossary (Advanced Audio Coding) and FLACFLAC: Free Lossless Audio CodecA lossless audio codec by the Xiph.Org Foundation. It compresses audio without discarding any information, so decoding restores the original samples bit for bit.FLACGlossary (Free Lossless Audio Codec) encoders.
  • NVIDIA video: --enable-cuvid --enable-nvenc --enable-ffnvcodec, built against nv-codec-headers. The NVIDIA libraries are loaded from the host driver at run time.

The build supports amd64 and arm64 only, and both include the NVIDIA support. It does not use --enable-nonfree: the NVIDIA options do not need it in FFmpeg 9 and later, and it would make the binary non-redistributable.

The versions are pinned in BUILD_ARGS and can be overridden with docker build --build-arg when you test a newer upstream release:

  • FFMPEG_VERSION: 9.0.1
  • NV_CODEC_HEADERS_TAG: n13.0.19.1

The build downloads from ffmpeg.org, git.videolan.org, and the Spare Cores CDNCDN: Content Delivery NetworkA network of distributed servers that cache content, such as web pages, images or video, and serve it from locations close to its users. It lowers latency for the users and offloads traffic from the origin server.WikipediaGlossary (Content Delivery Network). CICI: Continuous IntegrationThe practice of building and testing code automatically on every change, usually on a hosted service such as GitHub Actions, so problems show up before the change is merged.WikipediaGlossary enables compressed swap (zram) on the builder for the FFmpeg compile, set by the ZRAM file.

Image Metadata​

CICI: Continuous IntegrationThe practice of building and testing code automatically on every change, usually on a hosted service such as GitHub Actions, so problems show up before the change is merged.WikipediaGlossary publishes the following two files as image labels on the GitHub Container Registry:

  • DESCRIPTION: the one-sentence summary, shown as the image description. Keep it consistent with the opening of the README.
  • CATEGORY: benchmark.

Test Media​

The image bundles three test files, served from the Spare Cores CDN and described in SOURCE.md. The Dockerfile copies SOURCE.md into the image at /opt/benchmark-ffmpeg/SOURCE.md, so keep the file.

To regenerate the files, complete the following steps:

  1. prepare-video-fixture.sh downloads the Tears of SteelTears of SteelA 2012 science-fiction short film by the Blender Foundation, made with open-source tools and released under CC BY 3.0. Its high-resolution footage is freely available, which makes it common test material for video codecs.Project pageGlossary 1080p release and cuts the H.264H.264A lossy video codec standardized in 2003, also called AVC. It is the most widely supported video format, used in Blu-ray, broadcast, streaming, and video calls. Common encoders are x264 and hardware encoders such as NVENC.SpecificationGlossary excerpt of 30 seconds, starting at 60 seconds, with stream copy, into source.mp4.
  2. prepare-hevc-fixture.sh transcodes source.mp4 into source-hevc.mp4 with libx265 -crf 18 -preset medium.
  3. Download the original FLAC music sample from Wikimedia Commons and save it as source.flac; no script fetches it.
  4. upload-fixtures.sh uploads all three files to the S3S3: Simple Storage ServiceThe object storage service of AWS, which stores files as objects in buckets and serves them over HTTP. Its API has become a de facto standard that many other storage services and tools support.Amazon S3Glossary (Simple Storage Service) bucket sc-cdn-cae3awai, under the prefix sc-inspector/benchmark-ffmpeg, with the AWSAWS: Amazon Web ServicesThe cloud computing platform of Amazon, launched in 2006 and the largest public cloud by market share. It offers hundreds of services, from virtual machines (EC2) and object storage (S3) to managed databases and machine learning.AWSGlossary (Amazon Web Services) profile sc.

The prepare scripts check their output against the pinned SHA-256SHA-256: Secure Hash Algorithm 256A cryptographic hash function of the SHA-2 family that maps any data to a 256-bit digest. Even a one-bit change in the data gives a completely different digest, so it is used to verify that a downloaded file is intact, and in digital signatures and certificates.NIST FIPS 180-4Glossary (Secure Hash Algorithm 256) hashes. If you change a file's content, update its hash in every place it is pinned:

  • BUILD_ARGS and the ARG defaults in the Dockerfile, which verify the downloads at build time.
  • benchmark.py, which verifies the video files again at run time and reports the audio hash in its output.
  • The defaults in prepare-video-fixture.sh and prepare-hevc-fixture.sh.
  • The SHA-256 table in SOURCE.md.

The fixture files themselves are ignored by Git.