Workloads
Stressor Suite
run_stressng_benchmarks.py runs the
stressor suite as follows:
- It gets the list of stressors from
stress-ng --stressorsstress-ngAn open-source tool that loads a computer in selectable ways, with hundreds of stress tests (stressors) that exercise the CPU, memory, file systems, and kernel interfaces. With--metrics, it reports each stressor's throughput in bogo ops/s.stress-ng on GitHubGlossary, and keeps the curated stressors on that list, in alphabetical order. - It runs each stress-ng stressorstress-ng stressorOne stress test of
stress-ng, such ascpu,memcpy, orfork, selected with a command-line option of the same name and run by one or more worker processes.stress-ng man pageGlossary with one worker, then with one worker per logical CPUCPU: Central Processing UnitThe general-purpose processor of a computer, which runs the operating system and programs. A CPU has one or more physical cores, and with SMT each core runs two or more hardware threads.WikipediaGlossary available to the container, if there is more than one. Stressors run one at a time. - Each run is
stress-ng --metrics --metrics-brief --<stressor> <workers> -t <seconds>, for 5 seconds, or 15 seconds forstream. - It reads stress-ng bogo ops/sstress-ng bogo ops/s: bogus operations per secondThe throughput metric of
stress-ng: how many times a stressor completes its own unit of work per second. Each stressor defines that unit differently, so the values compare only within one stressor and onestress-ngversion.stress-ng man pageGlossary (bogus operations per second) in real time from the stress-ng metrc linestress-ng metrc lineA line in whichstress-ng --metricsprints a stressor's results when the run ends, taggedmetrc: the bogo ops count, the real, user, and system time, and the bogo ops/s, among other columns.stress-ng man pageGlossary of the run's output.
There is no warm-up and no repetition. A run that exits with an error, takes
30 seconds longer than its duration, or prints no metrc line for the stressor
is recorded as null.
Stressor Categories
The 85 stressors fall into the following seven categories:
| Category | Stressors | Exercises |
|---|---|---|
cpu_integer | 13 | Integer arithmetic, branching, and string handling |
cpu_float_simd | 18 | Floating-point math and vector operations |
crypto | 2 | Password hashing and encryption |
compression | 2 | Data and image compression |
memory | 11 | Caches and memory bandwidth |
algorithms | 13 | Searching, sorting, and memory allocation |
hypervisor | 26 | System calls, process creation, page tables, timers, and scheduling |
The following tables summarize what each stressor does. Each stressor name
links to its entry in the
stress-ng manual,
which describes a newer stress-ng version than the 0.20.01 this image pins.
Integer Stressors
| Stressor | What it does |
|---|---|
cpu | Works through all methods of the cpu stressor in turn |
branch | Branches to random locations, loading the branch predictor |
atomic | Runs atomic operations on integers shared among workers |
funccall | Calls functions with 1 to 9 arguments of various types |
funcret | Passes and returns small to large data types by value |
hash | Hashes random short strings with various hash functions |
rotate | Rotates unsigned integers of 8 to 128 bits |
str | Runs C library string functions on random strings |
wcs | Runs C library wide-character string functions on random strings |
nop | Issues no-op instructions |
bitops | Runs 32-bit bit-manipulation tricks |
intmath | Runs the basic arithmetic operations on signed integers of 8 to 128 bits |
prime | Finds ever larger primes, so its bogo ops/s drop over the run |
Floating-Point and SIMD Stressors
These stressors cover floating-point math and SIMDSIMD: Single Instruction, Multiple DataA form of parallel processing in which one CPU instruction applies the same operation to several data elements at once. CPUs implement it with vector instruction set extensions, which x86 and Arm CPUs both provide.WikipediaGlossary (Single Instruction, Multiple Data) vector operations:
| Stressor | What it does |
|---|---|
fma | Runs multiply-add operations on floating-point arrays, as fused instructions where available |
fp | Adds, multiplies, and divides a range of floating-point types |
trig | Computes trigonometric functions at three precisions |
matrix | Runs floating-point operations on a 2D matrix, mixing memory, cache, and math work |
matrix-3d | Runs the same kind of operations on a 3D matrix |
eigen | Runs matrix arithmetic with the Eigen C++ library |
vecmath | Runs integer math on 128-bit vectors |
vecfp | Runs floating-point arithmetic on vectors |
vecshuf | Shuffles and repacks the elements of vectors |
vecwide | Runs 8-bit math on vectors of 4 to 2,048 bytes, showing how well the compiler vectorizes it |
monte-carlo | Estimates pi and e with Monte Carlo experiments |
mpfr | Runs multi-precision floating-point math with a GNU library |
expmath | Computes exponential functions |
logmath | Computes logarithms |
powmath | Computes power functions |
hyperbolic | Computes hyperbolic functions at three precisions |
fractal | Computes a Mandelbrot set in double precision |
veccmp | Compares integers in 128-bit vectors |
Crypto and Compression Stressors
| Stressor | What it does |
|---|---|
crypt | Hashes a random password with many crypt(3) methods, such as bcrypt and yescrypt |
ipsec-mb | Runs cryptographic routines with Intel's multi-buffer crypto library; supported only on modern Intel x86_64AMD64The 64-bit version of the x86 instruction set architecture, introduced by AMD in 2003 and also implemented by Intel, which runs most cloud servers. Linux distributions and Docker name it amd64 or x86_64.WikipediaGlossary CPUs |
zlib | Compresses random data in one process and decompresses it in another |
jpeg | Compresses a generated image with lossy image compression |
Memory Stressors
| Stressor | What it does |
|---|---|
cache | Reads and writes memory at random, widespread addresses to thrash the CPU cache |
cacheline | Has several processes read and write the bytes of one shared cache line |
l1cache | Reads and writes a buffer twice the size of the L1 cache, forcing evictions |
memcpy | Copies and moves memory at three alignments |
memrate | Reads and writes a buffer with 8- to 1024-bit accesses at full speed |
memthrash | Thrashes a 16 MB buffer with at least one thread per CPU |
stream | Measures memory bandwidth on buffers the size of the L3 cache, loosely based on the STREAM benchmark |
vm | Repeatedly maps, writes, and unmaps memory |
cachehammer | Mixes random reads, writes, prefetches, and cache flushes |
ptr-chase | Follows random pointers across thousands of memory pages |
spinmem | Keeps two processes in sync through shared memory with busy spin loops |
Algorithm Stressors
| Stressor | What it does |
|---|---|
bsearch | Binary-searches a sorted array of integers |
hsearch | Searches a hash table that is 80% full |
qsort | Sorts integers with quicksort |
heapsort | Sorts integers with heapsort |
mergesort | Sorts integers with merge sort |
bitonicsort | Sorts integers with bitonic sort |
skiplist | Stores and searches integers in a skip list |
tree | Adds, finds, and removes integers in several kinds of trees |
malloc | Allocates, resizes, and frees memory at random |
mcontend | Has several threads contend for one physical memory page while moving between CPUs |
insertionsort | Sorts integers with insertion sort |
bubblesort | Sorts integers with bubble sort |
fibsearch | Searches a sorted array of integers with Fibonacci search |
Hypervisor Stressors
The hypervisor stressors are the ones whose throughput on a VMVM: virtual machineA computer emulated in software on a physical host, with its own OS and a share of the host's CPUs, memory, and storage. Most cloud servers are VMs.WikipediaGlossary is dominated
by hypervisor overhead, such as the cost of each VM exitVM exitA transition from a virtual machine's guest code to the hypervisor, triggered by an operation the hypervisor has to handle, such as some privileged instructions, I/O, or interrupts. Each exit and return costs CPU time, so workloads that cause many exits run slower in a VM.Glossary, the extra
page-table lookups of nested pagingnested pagingHardware support for memory virtualization, in which the CPU translates a guest's virtual addresses through the guest's page tables and then through a second set of page tables kept by the hypervisor. It removes the need for the hypervisor to maintain shadow page tables, but a TLB miss then needs more memory lookups. Intel calls it Extended Page Tables and AMD Rapid Virtualization Indexing.WikipediaGlossary, and the emulation of timers and traps:
| Stressor | What it does |
|---|---|
context | Switches between threads in user space |
switch | Forces context switches between two processes that block each other |
clone | Rapidly creates and reaps clones |
fork | Forks children that exit immediately |
vfork | Vforks children that exit immediately |
fault | Generates minor and major page faults |
tlb-shootdown | Changes shared memory mappings, forcing TLBTLB: Translation Lookaside BufferA CPU cache of recent translations from virtual to physical memory addresses. A TLB shootdown invalidates entries on other cores after a mapping changes, which needs an interrupt to each of them.WikipediaGlossary (Translation Lookaside Buffer) flushes on other CPUs |
mprotect | Changes the page protection of shared memory, causing TLB flushes |
munmap | Unmaps the shared memory of child processes page by page |
null | Calls I/OI/O: Input/OutputThe transfer of data between a computer and the outside world, such as reading and writing disks or sending and receiving network packets. A workload limited by its speed rather than by the CPU is called I/O-bound.WikipediaGlossary (Input/Output) system calls on /dev/null |
zero | Calls I/O system calls on /dev/zero |
getrandom | Reads random bytes from the kernel |
pipe | Writes and reads large blocks through a pipe between two processes |
eventfd | Passes short event messages between two processes through the kernel |
hrtimers | Runs many high-resolution timers with short, random intervals |
clock | Reads clocks and waits on timers, causing frequent context switches |
yield | Has at least two processes per CPU yield the CPU to each other |
resched | Moves processes through priority levels and scheduling policies |
vdso | Calls the kernel functions mapped into each process that avoid a full system call |
usersyscall | Handles system calls in user space with a signal handler |
priv-instr | Executes privileged instructions that the kernel traps |
tsc | Reads the CPU's time-stamp counter |
x86syscall | Makes system calls with the x86_64 syscall instruction |
cpu-sched | Moves processes between CPUs and scheduling policies |
timermix | Generates timer signals at a high rate |
mtx | Locks and unlocks mutexes |
The tsc, usersyscall, and x86syscall stressors return null on Arm
servers. See Missing Results.
CPU Scaling Loop
The CPU scaling loop runs the cpu stressor with the stress-ng div16 methodstress-ng div16 methodA method of the stress-ng cpu stressor, selected with --cpu-method div16, in which each operation runs 50,000 16-bit unsigned integer divisions. Integer division takes many CPU cycles and depends little on memory, so it loads the core's arithmetic units directly.stress-ng man pageGlossary
(--cpu-method div16) for 10 seconds at each worker count that
count.awk prints, and prints bogo ops/s for each. The loop itself is a shell command in the Navigator's
Inspector; Production Setup shows it in full.
The count.awk script takes the CPU count N and prints the following worker
counts:
- every count from 1 to N, if N is 32 or less
- otherwise, 1, N, and the even counts below N that are divisible by N / 32 (rounded down)
The Inspector uses this loop to determine a server type's SCoreSCore: Spare Cores scoreA CPU speed score of a server type in Spare Cores Navigator, used to compare the speed of server types. It is measured with the cpu stressor of stress-ng at increasing worker counts. The single-core SCore is the result with one worker; the multi-core SCore, the raw multi-core CPU speed that SCore means on its own, is the highest result.Glossary (Spare Cores
score): its raw multi-core CPU speed, for comparing the speed of server types.
The div16 method was chosen because it scaled as expected across x86_64 and
Arm servers, with and without SMTSMT: Simultaneous MultithreadingA CPU technique that runs two (or more) hardware threads on one physical core, sharing its execution units, e.g. Intel Hyper-Threading. It raises throughput when a thread stalls, but the threads compete for the core. Cloud vendors usually sell each thread as a vCPU.WikipediaGlossary (Simultaneous Multithreading). See
Design History.