Skip to main content

Container images

Spare Cores container images collect details about cloud server hardware and benchmark server performance for the Spare Cores Navigator. The Navigator's Inspector runs these images on cloud servers and publishes the results in the Navigator, which lists more than 5,000 cloud server types.

Each image is built for amd64 and arm64 unless its docs say otherwise. The docs of each image can be found linked from the image list, published at https://docs.sparecores.com/images/<image>/.

Usage​

Run an image via Docker:

docker run --rm ghcr.io/sparecores/<image>:main

Some images have no default command. Append a command after the image name. For example, run benchmark with a script such as openssl.sh. The stress-ngstress-ngAn open-source tool that loads a computer in selectable ways, with hundreds of stress tests (stressors) that exercise the CPU, memory, file systems, and kernel interfaces. With --metrics, it reports each stressor's throughput in bogo ops/s.stress-ng on GitHubGlossary image uses sh as its entrypoint, so pass the stressor with -c 'stress-ng --cpu 1'.

Other images need extra parameters, such as environment variables or Docker options. See the production commands and parameters in inspector/tasks.py in the sc-inspector repository. Each image's main page lists its settings, output, and any additional Docker options. For images without docs, see the image's Dockerfile.

Some images need the following extra Docker options:

  • GPUs: --gpus all for benchmark-vllm-gpu, nvbandwidthnvbandwidthAn open-source NVIDIA tool that measures memory copy bandwidth and latency between the CPU and NVIDIA GPUs, between GPUs, and within one GPU. It copies with the GPU's copy engines or with kernels on its streaming multiprocessors, and reports one matrix per test, with a row and a column for each CPU or GPU taking part. It prints plain text or JSON, and an optional multinode build measures GPUs across several servers.GitHubGlossary, and llama.cpp, and for the GPUGPU: Graphics Processing UnitA processor with thousands of simple cores that run the same operation on many data elements in parallel. It was built to render graphics, and is now also used for general-purpose parallel work such as machine learning training and inference, scientific simulation, and video encoding. Cloud providers rent GPUs attached to virtual machines, with their own dedicated GPU memory.WikipediaGlossary runs of benchmark-llm and benchmark-ffmpeg.
  • Shared memory: --shm-size=4g for benchmarks of the vLLM family, which fail with Docker's default 64 MiB /dev/shm.
  • Privileged mode: --privileged for dmidecode, hwinfo, and virtualization, and for the images that raise their priority with nice -n -20niceA Unix command that runs a program with a changed scheduling priority, from -20 (highest) to 19 (lowest). Raising a process's priority above the default needs the CAP_SYS_NICE capability, which root has.man pageGlossary: benchmark, benchmark-passmark, benchmark-pgbench-postgres, benchmark-postgres-server, benchmark-redis, and benchmark-web. Without it, the nice images run at normal priority.
  • Disk access: --privileged --pid=host for benchmark-storage. It unmounts and overwrites every disk it may write to, so do not run it on a machine with data you need.

Benchmark images run their workload under resource-tracker, which samples CPUCPU: Central Processing UnitThe general-purpose processor of a computer, which runs the operating system and programs. A CPU has one or more physical cores, and with SMT each core runs two or more hardware threads.WikipediaGlossary, memory, disk, network, and GPU usage once per second.

Images​

The following tables list the images by their CATEGORY. Where an image has docs, its main page covers what the image measures, how to run it, and how to read its output.

Benchmarks​

ImageDescriptionDocs
benchmarkGeneral CPU, memory, and compression benchmarks: GeekbenchGeekbenchA cross-platform CPU and GPU compute benchmark by Primate Labs. Its CPU benchmark runs workloads modeled on real applications, such as file compression, image processing, and web browsing, first on one core and then on all cores, and reports a Single-Core and a Multi-Core score. Geekbench 6 scores are calibrated so that a baseline system scores 2500.GeekbenchGlossary 6 scores, OpenSSL speed throughput, LMbenchLMbenchA suite of portable micro-benchmarks for Unix-like systems, written by Larry McVoy and Carl Staelin. Its tools each measure one basic operation, such as memory bandwidth and latency, context switches, or system call overhead.LMbenchGlossary bw_membw_memThe memory bandwidth tool of LMbench. It moves data through a buffer of a given size in one of several modes, such as rd (read and sum an array), wr (write a constant to each element), rdwr (read, then write the same location), and the copy modes cp, fcp, and bcopy. It reports the buffer size in MB and the bandwidth in MB/s; with -P, it runs several processes, each with its own buffer, and reports their total bandwidth.`bw_mem` manualGlossary memory bandwidth, and compression ratiocompression ratioA measure of how much a compressor shrinks data, from the sizes of the uncompressed and the compressed data. It is most often given as uncompressed size divided by compressed size, where higher is better, but some tools report compressed size divided by uncompressed size as a percentage, where lower is better.WikipediaGlossary and throughput on a Silesia corpusSilesia corpusA set of 12 files, ~211 MB in total, compiled by Sebastian Deorowicz at the Silesian University of Technology to test lossless compression on a mix of data types: text, executables, databases, images, and XML. Its dickens file holds the collected works of Charles Dickens, ~10 MB of English text.Silesia corpusGlossary text file.Docs
benchmark-ffmpegMeasures aggregate FFmpeg transcoding throughput in frames per second for H.264H.264A lossy video codec standardized in 2003, also called AVC. It is the most widely supported video format, used in Blu-ray, broadcast, streaming, and video calls. Common encoders are x264 and hardware encoders such as NVENC.SpecificationGlossary and H.265H.265Another name for HEVC, from the number of its standard.SpecificationGlossary video encoding and decoding on CPUs and NVIDIA GPUs, and in audio seconds processed per second for CPU audio encoding.Docs
benchmark-llmSingle-stream LLM inference speedLLM inference speedHow fast hardware runs an LLM, usually measured in tokens per second. It is measured separately for prompt processing, in which the model reads the input prompt, and text generation, in which it produces output tokens one at a time, because the first is usually limited by compute and the second by memory bandwidth. Benchmarks of LLM servers also measure latency, such as the time to first token, and the throughput of many concurrent requests.Glossary benchmark with llama.cppllama.cppAn open-source C/C++ library and set of tools for LLM inference, started by Georgi Gerganov in 2023 and built on the ggml tensor library. It runs models in the GGUF format, usually quantized, on CPUs of many architectures, with code optimized for their SIMD instruction sets, and on GPUs through backends such as CUDA, Metal, and Vulkan. Its tools include a command-line client, an HTTP server, and a benchmark tool, llama-bench.llama.cpp on GitHubGlossary's llama-benchllama-benchThe benchmark tool of llama.cpp. It measures the speed of prompt processing and text generation in tokens per second, for given models, token counts, and settings. Each test is repeated several times, and the tool reports the average and standard deviation, as a table or in machine-readable formats such as JSON Lines.llama-bench READMEGlossary, measuring prompt processingprompt processingThe first phase of LLM inference, also called prefill, in which the model reads all tokens of the input prompt and stores their intermediate results in its key-value cache. The prompt's tokens are processed together in batches, so this phase is usually limited by compute and runs much faster per token than text generation. Its speed decides how long a long input takes before the first output token appears.Glossary and text generationtext generationThe second phase of LLM inference, also called decoding, in which the model produces output tokens one at a time, each depending on the tokens before it. Every step reads all model weights and the key-value cache, so this phase is usually limited by memory bandwidth rather than compute, and runs much slower per token than prompt processing.Glossary in tokenstokenThe unit of text that an LLM reads and writes, as defined by the model's tokenizer, such as a word, part of a word, or a punctuation mark. Prompt and output lengths are counted in tokens, and LLM inference speed is usually measured in tokens per second.Glossary per second for six models at several token counts, on CPUs and NVIDIA GPUs.Docs
benchmark-passmarkPassMarkPassMarkThe PerformanceTest benchmark suite of PassMark Software, for Windows, Linux, macOS, and mobile devices. It runs fixed CPU, memory, disk, and graphics tests, such as math workloads involving compression, encryption, and physics simulations, and combines the results into an overall PassMark Rating.PerformanceTestGlossary PerformanceTest benchmark of CPU and memory performance, reporting the CPU MarkCPU MarkThe overall CPU score of PassMark PerformanceTest. It combines the results of the CPU tests, such as integer and floating-point math, prime numbers, sorting, encryption, compression, physics, and a single-threaded test, into one unitless number, where higher is better. PassMark Software publishes CPU Mark results submitted by users for thousands of processors.PassMark CPU BenchmarksGlossary and Memory MarkMemory MarkThe overall memory score of PassMark PerformanceTest. It combines the results of the memory tests, such as database operations, cached and uncached reads, writes, and latency, into one unitless number, where higher is better.PassMark Memory BenchmarksGlossary composite scores and the individual test results behind them.Docs
benchmark-pgbench-postgresPostgreSQLPostgreSQLAn open-source object-relational database system, developed by a global community and descended from the POSTGRES project at UC Berkeley in 1986. It uses and extends SQL, with transactions, many index types, extensions, and JSON data.PostgreSQL docsGlossary database benchmark focusing on CPU and memory performance, measuring transactions per minute and latency against a local or remote server.Docs
benchmark-redisRedisRedisAn open-source, in-memory key-value data store, used as a database, cache, and message broker. It keeps its data in RAM, supports data structures such as strings, hashes, lists, and sets, and can persist data to disk with periodic snapshots or an append-only log.Redis docsGlossary 7.4 SET throughput and latency benchmark with memtier_benchmarkmemtier_benchmarkAn open-source load generator for Redis and Memcached, developed by Redis Ltd. It runs many concurrent connections over one or more threads, sends a configurable mix of SET and GET requests, and reports operations per second, latency averages, and latency percentiles per request type.memtier_benchmark on GitHubGlossary, one server and client pair per CPU.Docs
benchmark-storageMeasures raw block device performance with fio, reporting IOPSIOPS: Input/Output Operations Per SecondThe number of read and write operations a storage device completes per second. It is the main measure of random-access performance, and depends on the block size, the queue depth and the mix of reads and writes.WikipediaGlossary (Input/Output Operations Per Second), bandwidth, and latency for one disk per group and for groups of identical disks.Docs
benchmark-vllm-cpuLLMLLM: Large Language ModelA neural network, typically with billions of parameters, trained on large amounts of text to predict the next token. It can generate, summarize and translate text, answer questions and write code.WikipediaGlossary serving benchmark with vLLMvLLM: virtual Large Language ModelAn open-source inference and serving engine for LLMs, started at UC Berkeley in 2023. Its PagedAttention technique manages the key-value cache in fixed-size blocks, as an OS pages virtual memory, and it batches concurrent requests continuously, so one server can answer many requests at once. It runs on GPUs and CPUs and serves models through an OpenAI-compatible HTTP API.vLLM docsvLLM on GitHubOrigin of the nameGlossary and GuideLLMGuideLLMAn open-source tool of the vLLM project for evaluating LLM deployments. It sends synthetic or real requests to an inference server over an OpenAI-compatible HTTP API, at a fixed concurrency, a fixed request rate, or a sweep of rates, and measures latency and throughput, such as the time to first token and output tokens per second. It writes its results as JSON, CSV, or HTML reports.GuideLLM on GitHubGlossary on AMD64AMD64The 64-bit version of the x86 instruction set architecture, introduced by AMD in 2003 and also implemented by Intel, which runs most cloud servers. Linux distributions and Docker name it amd64 or x86_64.WikipediaGlossary CPUs with AVX-512AVX-512: Advanced Vector Extensions 512A set of x86 SIMD instructions that operate on 512-bit registers, so that one instruction processes, for example, 16 single-precision floating-point numbers at once. Intel introduced it in its server CPUs with the Skylake generation, and AMD supports it from the Zen 4 generation. It is made of several subsets, such as the foundation, avx512f, and avx512_bf16 for bfloat16 arithmetic, which Linux lists as flags in /proc/cpuinfo.WikipediaGlossary (Advanced Vector Extensions 512) and Arm64Arm64The 64-bit execution state of the Arm architecture, also called AArch64, used by server CPUs such as AWS Graviton, Ampere Altra, and Google Axion. Linux distributions and Docker name it arm64 or aarch64.WikipediaGlossary CPUs, measuring time to first token, time per output token, inter-token latency, end-to-end request latency, and token and request throughput at loads from one request at a time to peak.Docs
benchmark-vllm-cpu-avx2LLM serving benchmark with vLLM and GuideLLM on AMD64 CPUs with AVX2AVX2: Advanced Vector Extensions 2A set of x86 SIMD instructions that operate on 256-bit registers, introduced by Intel with the Haswell generation in 2013 and supported by AMD since 2015. It extends most integer vector instructions of the earlier AVX set to 256 bits. Nearly all current x86 CPUs support it, and AVX-512 is its wider successor.WikipediaGlossary (Advanced Vector Extensions 2) but without AVX-512 (Advanced Vector Extensions 512), measuring time to first token, time per output token, inter-token latency, end-to-end request latency, and token and request throughput at loads from one request at a time to peak.Docs
benchmark-vllm-gpuLLM serving benchmark with vLLM and GuideLLM on NVIDIA GPUs, measuring time to first token, time per output token, inter-token latency, end-to-end request latency, and token and request throughput at loads from one request at a time to peak.Docs
benchmark-webStatic web server benchmark, serving files with binservebinserveAn open-source static web server written in Rust, distributed as a single binary and configured with one JSON file. It maps URL routes to files and directories, can keep files cached in memory, and can also serve encrypted connections and list directory contents.binserve on GitHubGlossary and measuring throughput and latency with wrkwrkAn open-source HTTP benchmarking tool written in C. It generates load from a few threads, each handling many open connections through an event loop, and reports latency and requests-per-second statistics. Lua scripts can customize the requests it sends and process the responses.wrk on GitHubGlossary.Docs
nvbandwidthNVIDIA GPU memory copy bandwidth and latency benchmark with nvbandwidthnvbandwidthAn open-source NVIDIA tool that measures memory copy bandwidth and latency between the CPU and NVIDIA GPUs, between GPUs, and within one GPU. It copies with the GPU's copy engines or with kernels on its streaming multiprocessors, and reports one matrix per test, with a row and a column for each CPU or GPU taking part. It prints plain text or JSON, and an optional multinode build measures GPUs across several servers.GitHubGlossary, measuring copies between the host CPU and each GPU, between GPUs, and within one GPU.Docs
stress-ngMeasures bogo ops/s for 85 CPU, memory, and kernel stressors with stress-ng, in single-worker and all-CPU runs, and measures CPU scaling as worker counts increase.Docs
stress-ng-longrunPinned stress-ng version for the 24-hour long-running benchmark.Docs

Hardware Discovery​

ImageDescriptionDocs
dmidecodeReads the server's DMIDMI: Desktop Management InterfaceA standard from the 1990s for describing a computer's hardware to management software. Its data model lives on in SMBIOS, which is why the firmware tables that describe a server's hardware are often called DMI tables, and Linux exposes some of their values under /sys/class/dmi/id/.WikipediaGlossary (Desktop Management Interface) and SMBIOSSMBIOS: System Management BIOSA standard for the tables in which a computer's firmware describes its hardware to the operating system, such as the system's manufacturer and model, its CPUs and memory modules, and the firmware version. It grew out of DMI, and on Linux, tools such as dmidecode read these tables. In a VM, the virtualization platform sets the values.DMTF SMBIOS standardGlossary (System Management BIOS) tables with dmidecode to report hardware details such as memory modules and firmware.Docs
hwinfoCollects CPU, memory, and topology details with hwloc, likwid, and lshw.Docs
virtualizationChecks whether the server supports KVMKVM: Kernel-based Virtual MachineThe virtualization module of the Linux kernel, which lets Linux run VMs with the hardware virtualization extensions of Intel and AMD CPUs. A server can run VMs with it only if its CPU exposes these extensions to the operating system, which many cloud VMs do not.WikipediaGlossary (Kernel-based Virtual Machine) hardware virtualization and reports the result as JSONJSON: JavaScript Object NotationA text format for structured data, built from objects (key-value pairs), arrays, strings, numbers, booleans, and null. Most languages and tools read and write it, which makes it a common format for API responses and benchmark output.JSONGlossary.Docs

Services​

ImageDescriptionDocs
benchmark-postgres-serverPostgreSQL 18 server running at high priority, used as the remote target of database benchmarks.Docs
llama.cppllama.cpp server (llama-server) with CUDACUDA: Compute Unified Device ArchitectureThe parallel computing platform and programming model of NVIDIA GPUs. It runs general-purpose code on the GPU, and most machine learning frameworks rely on its libraries, such as cuBLAS and cuDNN.NVIDIA docsGlossary (Compute Unified Device Architecture) support, built from a pinned commit.Docs

Base Images​

ImageDescriptionDocs
resource-trackerThe resource-trackerresource-trackerAn open-source Spare Cores tool that runs a command and samples resource usage while it runs, such as CPU, memory, disk, network, GPU, and GPU memory use. It can optionally stream the samples to the Spare Cores Sentinel platform, so that a run can be checked to see its aggregated and detailed resource utilization, e.g., whether it used all CPU or GPU cores as intended while running a Navigator benchmark workload.resource-tracker on GitHubGlossary binary of Spare Cores Navigator, copied into the other images to record resource usage while benchmarks run.Docs
vllm-cpu-base-avx2vLLM CPU build for x86_64 hosts with AVX2 (Advanced Vector Extensions 2) but no AVX-512 (Advanced Vector Extensions 512), used as the base of benchmark-vllm-cpu-avx2.Docs

The following docs cover more than one image. Where a covered image has a main page of its own, that page describes only what differs:

  • The vLLM benchmarks manual in vllm-common/, which is not a published image, covers the vLLM family's benchmarks.
  • The benchmark-pgbench-postgres docs cover benchmark-postgres-server.
  • The stress-ng docs cover stress-ng-longrun.

Contributing​

See the following pages for maintaining and extending the sc-images repository:

  • Adding an Image: the folder layout, the metadata files that control each image's build, and how dependencies are ordered.
  • CI and Build Internals: how CICI: Continuous IntegrationThe practice of building and testing code automatically on every change, usually on a hosted service such as GitHub Actions, so problems show up before the change is merged.WikipediaGlossary builds, caches, tracks, and publishes the images.

License​

The sc-images repository is licensed under the Mozilla Public License 2.0MPL 2.0: Mozilla Public License 2.0The open-source license of the Navigator's software components. Anyone may use, modify and distribute the code, also commercially and combined with closed-source code, but changes to the licensed files must be published under the same license.Spare Cores licenseMozilla FAQGlossary.