Skip to main content

Limitations

Disclaimer: What This Benchmark Does Not Deliver​

Cross-Stressor Comparison​

Bogo ops/s (bogus operations per second) results are not comparable across stressors. Each stress-ng stressorstress-ng stressorOne stress test of stress-ng, such as cpu, memcpy, or fork, selected with a command-line option of the same name and run by one or more worker processes.stress-ng man pageGlossary counts how many times it completes its own unit of work per second, and defines that unit differently. Compare a stressor only with itself, on the same stress-ngstress-ngAn open-source tool that loads a computer in selectable ways, with hundreds of stress tests (stressors) that exercise the CPU, memory, file systems, and kernel interfaces. With --metrics, it reports each stressor's throughput in bogo ops/s.stress-ng on GitHubGlossary version. The image pins stress-ng 0.20.01 through the tag of its base image.

A Single Score​

The stressor suite reports one value per stressor and worker count, and does not combine them into a category or overall score. It also does not attribute the results of the hypervisor stressors to a specific cause, such as the hypervisor or the CPUCPU: Central Processing UnitThe general-purpose processor of a computer, which runs the operating system and programs. A CPU has one or more physical cores, and with SMT each core runs two or more hardware threads.WikipediaGlossary.

Application Performance​

The stressors are synthetic loops. Results show how fast a server runs each loop, not how fast a given application runs. This benchmark does not measure disk or network performance. For disk performance, see the benchmark-storage image.

Scope Decisions​

Run Length​

Each stressor runs once, for 5 seconds (15 seconds for stream), with no warm-up and no repetitions. A full run took about 15 minutes at the median in production. A single short run does not show variance between runs or effects that build up over time, such as CPU-credit throttlingCPU-credit throttlingA limit on burstable cloud server types, which earn CPU credits while they run below a baseline CPU usage and spend them to run above it. When the credits run out, the provider throttles the server to its baseline, so sustained load runs slower than a short burst.AWS burstable instancesGlossary on shared-coreshared-coreDescribes a cloud server type whose vCPUs share physical cores with other servers, often with a guaranteed baseline of only a fraction of a core. Such server types are cheaper, but their CPU performance can vary with the load of neighboring servers or be limited by CPU credits.Google Cloud shared-core machine typesGlossary server types. The stress-ng-longrun image covers sustained load in an experimental 24-hour run.

Worker Count​

The multi-worker run uses one worker per logical CPU available to the container, not one per physical core. The script would use the physical core count only if the psutilpsutilA cross-platform Python library for retrieving information on running processes and system utilization, such as CPU, memory, disks, and network. It can count both logical CPUs and physical cores.psutil docsGlossary Python package were installed, and the image does not install it. On a server with SMTSMT: Simultaneous MultithreadingA CPU technique that runs two (or more) hardware threads on one physical core, sharing its execution units, e.g. Intel Hyper-Threading. It raises throughput when a thread stalls, but the threads compete for the core. Cloud vendors usually sell each thread as a vCPU.WikipediaGlossary (Simultaneous Multithreading), the worker count therefore includes both threads of each core, even though the output field is named physical_cpus.

Design Constraints​

Missing Results​

A stressor run that fails, takes 30 seconds longer than its duration, or prints no result is recorded as null, and the script still exits with code 0. The following stressors are null in some or all production runs:

  • ipsec-mb has been null in every successful production run so far.
  • tsc, usersyscall, and x86syscall are x86-only. On Arm servers, stress-ng still lists them, so they run and return null.
  • l1cache, vm, zlib, and cpu were null in a few production runs.

Because these stressors are still listed, meta.stressor_count counts them. Check each entry for null instead of relying on the count.