Skip to main content

Workloads

Stressor Suite​

run_stressng_benchmarks.py runs the stressor suite as follows:

  1. It gets the list of stressors from stress-ng --stressorsstress-ngAn open-source tool that loads a computer in selectable ways, with hundreds of stress tests (stressors) that exercise the CPU, memory, file systems, and kernel interfaces. With --metrics, it reports each stressor's throughput in bogo ops/s.stress-ng on GitHubGlossary, and keeps the curated stressors on that list, in alphabetical order.
  2. It runs each stress-ng stressorstress-ng stressorOne stress test of stress-ng, such as cpu, memcpy, or fork, selected with a command-line option of the same name and run by one or more worker processes.stress-ng man pageGlossary with one worker, then with one worker per logical CPUCPU: Central Processing UnitThe general-purpose processor of a computer, which runs the operating system and programs. A CPU has one or more physical cores, and with SMT each core runs two or more hardware threads.WikipediaGlossary available to the container, if there is more than one. Stressors run one at a time.
  3. Each run is stress-ng --metrics --metrics-brief --<stressor> <workers> -t <seconds>, for 5 seconds, or 15 seconds for stream.
  4. It reads stress-ng bogo ops/sstress-ng bogo ops/s: bogus operations per secondThe throughput metric of stress-ng: how many times a stressor completes its own unit of work per second. Each stressor defines that unit differently, so the values compare only within one stressor and one stress-ng version.stress-ng man pageGlossary (bogus operations per second) in real time from the stress-ng metrc linestress-ng metrc lineA line in which stress-ng --metrics prints a stressor's results when the run ends, tagged metrc: the bogo ops count, the real, user, and system time, and the bogo ops/s, among other columns.stress-ng man pageGlossary of the run's output.

There is no warm-up and no repetition. A run that exits with an error, takes 30 seconds longer than its duration, or prints no metrc line for the stressor is recorded as null.

Stressor Categories​

The 85 stressors fall into the following seven categories:

CategoryStressorsExercises
cpu_integer13Integer arithmetic, branching, and string handling
cpu_float_simd18Floating-point math and vector operations
crypto2Password hashing and encryption
compression2Data and image compression
memory11Caches and memory bandwidth
algorithms13Searching, sorting, and memory allocation
hypervisor26System calls, process creation, page tables, timers, and scheduling

The following tables summarize what each stressor does. Each stressor name links to its entry in the stress-ng manual, which describes a newer stress-ng version than the 0.20.01 this image pins.

Integer Stressors​

StressorWhat it does
cpuWorks through all methods of the cpu stressor in turn
branchBranches to random locations, loading the branch predictor
atomicRuns atomic operations on integers shared among workers
funccallCalls functions with 1 to 9 arguments of various types
funcretPasses and returns small to large data types by value
hashHashes random short strings with various hash functions
rotateRotates unsigned integers of 8 to 128 bits
strRuns C library string functions on random strings
wcsRuns C library wide-character string functions on random strings
nopIssues no-op instructions
bitopsRuns 32-bit bit-manipulation tricks
intmathRuns the basic arithmetic operations on signed integers of 8 to 128 bits
primeFinds ever larger primes, so its bogo ops/s drop over the run

Floating-Point and SIMD Stressors​

These stressors cover floating-point math and SIMDSIMD: Single Instruction, Multiple DataA form of parallel processing in which one CPU instruction applies the same operation to several data elements at once. CPUs implement it with vector instruction set extensions, which x86 and Arm CPUs both provide.WikipediaGlossary (Single Instruction, Multiple Data) vector operations:

StressorWhat it does
fmaRuns multiply-add operations on floating-point arrays, as fused instructions where available
fpAdds, multiplies, and divides a range of floating-point types
trigComputes trigonometric functions at three precisions
matrixRuns floating-point operations on a 2D matrix, mixing memory, cache, and math work
matrix-3dRuns the same kind of operations on a 3D matrix
eigenRuns matrix arithmetic with the Eigen C++ library
vecmathRuns integer math on 128-bit vectors
vecfpRuns floating-point arithmetic on vectors
vecshufShuffles and repacks the elements of vectors
vecwideRuns 8-bit math on vectors of 4 to 2,048 bytes, showing how well the compiler vectorizes it
monte-carloEstimates pi and e with Monte Carlo experiments
mpfrRuns multi-precision floating-point math with a GNU library
expmathComputes exponential functions
logmathComputes logarithms
powmathComputes power functions
hyperbolicComputes hyperbolic functions at three precisions
fractalComputes a Mandelbrot set in double precision
veccmpCompares integers in 128-bit vectors

Crypto and Compression Stressors​

StressorWhat it does
cryptHashes a random password with many crypt(3) methods, such as bcrypt and yescrypt
ipsec-mbRuns cryptographic routines with Intel's multi-buffer crypto library; supported only on modern Intel x86_64AMD64The 64-bit version of the x86 instruction set architecture, introduced by AMD in 2003 and also implemented by Intel, which runs most cloud servers. Linux distributions and Docker name it amd64 or x86_64.WikipediaGlossary CPUs
zlibCompresses random data in one process and decompresses it in another
jpegCompresses a generated image with lossy image compression

Memory Stressors​

StressorWhat it does
cacheReads and writes memory at random, widespread addresses to thrash the CPU cache
cachelineHas several processes read and write the bytes of one shared cache line
l1cacheReads and writes a buffer twice the size of the L1 cache, forcing evictions
memcpyCopies and moves memory at three alignments
memrateReads and writes a buffer with 8- to 1024-bit accesses at full speed
memthrashThrashes a 16 MB buffer with at least one thread per CPU
streamMeasures memory bandwidth on buffers the size of the L3 cache, loosely based on the STREAM benchmark
vmRepeatedly maps, writes, and unmaps memory
cachehammerMixes random reads, writes, prefetches, and cache flushes
ptr-chaseFollows random pointers across thousands of memory pages
spinmemKeeps two processes in sync through shared memory with busy spin loops

Algorithm Stressors​

StressorWhat it does
bsearchBinary-searches a sorted array of integers
hsearchSearches a hash table that is 80% full
qsortSorts integers with quicksort
heapsortSorts integers with heapsort
mergesortSorts integers with merge sort
bitonicsortSorts integers with bitonic sort
skiplistStores and searches integers in a skip list
treeAdds, finds, and removes integers in several kinds of trees
mallocAllocates, resizes, and frees memory at random
mcontendHas several threads contend for one physical memory page while moving between CPUs
insertionsortSorts integers with insertion sort
bubblesortSorts integers with bubble sort
fibsearchSearches a sorted array of integers with Fibonacci search

Hypervisor Stressors​

The hypervisor stressors are the ones whose throughput on a VMVM: virtual machineA computer emulated in software on a physical host, with its own OS and a share of the host's CPUs, memory, and storage. Most cloud servers are VMs.WikipediaGlossary is dominated by hypervisor overhead, such as the cost of each VM exitVM exitA transition from a virtual machine's guest code to the hypervisor, triggered by an operation the hypervisor has to handle, such as some privileged instructions, I/O, or interrupts. Each exit and return costs CPU time, so workloads that cause many exits run slower in a VM.Glossary, the extra page-table lookups of nested pagingnested pagingHardware support for memory virtualization, in which the CPU translates a guest's virtual addresses through the guest's page tables and then through a second set of page tables kept by the hypervisor. It removes the need for the hypervisor to maintain shadow page tables, but a TLB miss then needs more memory lookups. Intel calls it Extended Page Tables and AMD Rapid Virtualization Indexing.WikipediaGlossary, and the emulation of timers and traps:

StressorWhat it does
contextSwitches between threads in user space
switchForces context switches between two processes that block each other
cloneRapidly creates and reaps clones
forkForks children that exit immediately
vforkVforks children that exit immediately
faultGenerates minor and major page faults
tlb-shootdownChanges shared memory mappings, forcing TLBTLB: Translation Lookaside BufferA CPU cache of recent translations from virtual to physical memory addresses. A TLB shootdown invalidates entries on other cores after a mapping changes, which needs an interrupt to each of them.WikipediaGlossary (Translation Lookaside Buffer) flushes on other CPUs
mprotectChanges the page protection of shared memory, causing TLB flushes
munmapUnmaps the shared memory of child processes page by page
nullCalls I/OI/O: Input/OutputThe transfer of data between a computer and the outside world, such as reading and writing disks or sending and receiving network packets. A workload limited by its speed rather than by the CPU is called I/O-bound.WikipediaGlossary (Input/Output) system calls on /dev/null
zeroCalls I/O system calls on /dev/zero
getrandomReads random bytes from the kernel
pipeWrites and reads large blocks through a pipe between two processes
eventfdPasses short event messages between two processes through the kernel
hrtimersRuns many high-resolution timers with short, random intervals
clockReads clocks and waits on timers, causing frequent context switches
yieldHas at least two processes per CPU yield the CPU to each other
reschedMoves processes through priority levels and scheduling policies
vdsoCalls the kernel functions mapped into each process that avoid a full system call
usersyscallHandles system calls in user space with a signal handler
priv-instrExecutes privileged instructions that the kernel traps
tscReads the CPU's time-stamp counter
x86syscallMakes system calls with the x86_64 syscall instruction
cpu-schedMoves processes between CPUs and scheduling policies
timermixGenerates timer signals at a high rate
mtxLocks and unlocks mutexes

The tsc, usersyscall, and x86syscall stressors return null on Arm servers. See Missing Results.

CPU Scaling Loop​

The CPU scaling loop runs the cpu stressor with the stress-ng div16 methodstress-ng div16 methodA method of the stress-ng cpu stressor, selected with --cpu-method div16, in which each operation runs 50,000 16-bit unsigned integer divisions. Integer division takes many CPU cycles and depends little on memory, so it loads the core's arithmetic units directly.stress-ng man pageGlossary (--cpu-method div16) for 10 seconds at each worker count that count.awk prints, and prints bogo ops/s for each. The loop itself is a shell command in the Navigator's Inspector; Production Setup shows it in full.

The count.awk script takes the CPU count N and prints the following worker counts:

  • every count from 1 to N, if N is 32 or less
  • otherwise, 1, N, and the even counts below N that are divisible by N / 32 (rounded down)

The Inspector uses this loop to determine a server type's SCoreSCore: Spare Cores scoreA CPU speed score of a server type in Spare Cores Navigator, used to compare the speed of server types. It is measured with the cpu stressor of stress-ng at increasing worker counts. The single-core SCore is the result with one worker; the multi-core SCore, the raw multi-core CPU speed that SCore means on its own, is the highest result.Glossary (Spare Cores score): its raw multi-core CPU speed, for comparing the speed of server types. The div16 method was chosen because it scaled as expected across x86_64 and Arm servers, with and without SMTSMT: Simultaneous MultithreadingA CPU technique that runs two (or more) hardware threads on one physical core, sharing its execution units, e.g. Intel Hyper-Threading. It raises throughput when a thread stalls, but the threads compete for the core. Cloud vendors usually sell each thread as a vCPU.WikipediaGlossary (Simultaneous Multithreading). See Design History.