# stress-ng

> Measures bogo ops/s for 85 CPU, memory, and kernel stressors with stress-ng, in single-worker and all-CPU runs, and measures CPU scaling as worker counts increase.

Source: [`images/stress-ng/`](https://github.com/SpareCores/sc-images/tree/main/images/stress-ng) in the `sc-images` repository. Image: [`ghcr.io/sparecores/stress-ng:main`](https://github.com/orgs/SpareCores/packages/container/package/stress-ng).

This benchmark measures the throughput of CPU, memory, and kernel-heavy
workloads with `stress-ng`. It runs 85 curated stressors in seven categories,
each for 5 seconds (15 seconds for `stream`), first with one worker and then
with one worker per available CPU. A second mode runs the `cpu` stressor at
increasing worker counts to trace how throughput scales across cores.

This benchmark reports stress-ng bogo ops/s (bogus operations per second) for
each stress-ng stressor and worker count, as one JSON document. The scaling
mode prints one `<workers>,<bogo ops/s>` line per worker count.

## Purpose

[Spare Cores Navigator](https://docs.sparecores.com/overview) publishes
empirical performance measurements for over 5,000 cloud server types, such as
raw CPU speed, [memory bandwidth](https://docs.sparecores.com/images/benchmark/bw-mem.md), and
[compression throughput](https://docs.sparecores.com/images/benchmark/compression.md). This benchmark
covers a wide range of CPU and memory work in one run, from integer and
floating-point math to sorting, hashing, and memory bandwidth.

It also includes a `hypervisor` category: stressors such as context switches,
page faults, system calls, and timers, whose throughput on a VM is dominated by
hypervisor overhead. These stressors are meant to expose differences in
virtualization overhead across providers.

The Navigator's Inspector uses the scaling mode to determine a server type's
SCore (Spare Cores score): its raw multi-core CPU speed, for comparing the
speed of server types. See
[CPU Scaling Loop](https://docs.sparecores.com/images/stress-ng/workloads.md#cpu-scaling-loop).

The [`stress-ng-longrun`](https://docs.sparecores.com/images/stress-ng-longrun.md) image pins an older
`stress-ng` version for an experimental 24-hour run on a few server types. The
run shows the following:

- sustained CPU throughput over time
- throttling on shared-core server types
- noisy-neighbor variance by time of day

## Limitations

These results are synthetic `stress-ng` throughput figures, not application
performance. The following are the main limitations:

- **[Not Comparable Across Stressors](https://docs.sparecores.com/images/stress-ng/limitations.md#cross-stressor-comparison):**
  Each stressor counts its own kind of operation, so compare bogo ops/s only
  for the same stressor and the same `stress-ng` version.
- **[No Single Score](https://docs.sparecores.com/images/stress-ng/limitations.md#a-single-score):** The stressor
  suite reports one value per stressor and worker count, and does not combine
  them.
- **[Short, Single Runs](https://docs.sparecores.com/images/stress-ng/limitations.md#run-length):** Each stressor
  runs once for 5s (15s for `stream`), with no warm-up and no repetitions.
- **[Logical CPUs, Not Physical Cores](https://docs.sparecores.com/images/stress-ng/limitations.md#worker-count):**
  The multi-worker run uses one worker per logical CPU available to the
  container, even though the output field is named `physical_cpus`.
- **[Missing Results Are `null`](https://docs.sparecores.com/images/stress-ng/limitations.md#missing-results):** A
  stressor that fails, times out, or is not supported on the CPU architecture
  is recorded as `null`, and the run still succeeds.

## Usage

The image has no default command. Append the stressor suite's command after
the image name:

```bash
docker run --rm ghcr.io/sparecores/stress-ng:main \
  -c 'python3 /usr/local/bin/run_stressng_benchmarks.py' > stress-ng.json
```

Production also runs the command under `nice -n -20`, which needs
`--privileged`. See [Production Setup](https://docs.sparecores.com/images/stress-ng/usage.md#production-setup) for
the production commands, including the CPU scaling loop.

The stressor suite took about 7 to 66 minutes in production, about 15 minutes
at the median. See [Run Duration](https://docs.sparecores.com/images/stress-ng/usage.md#run-duration).

### Settings

The following settings shorten a run, for example for a quick check; see the
[full environment-variable reference](https://docs.sparecores.com/images/stress-ng/usage.md#environment-variables)
for details:

- `STRESSNG_QUICK=1` runs each stressor for 2 seconds instead of 5 seconds
  (`stream` still runs for 15 seconds).
- `STRESSNG_LIMIT=<N>` runs only the first N stressors in alphabetical order.

### Results

The stressor suite prints one JSON document to stdout when the run ends.
The JSON document's main fields are the following:

- `stressors.<name>` holds one entry per worker count: `"1"`, and the CPU
  count for the multi-worker run. Each entry is
  `{"bogo_ops_per_sec": <value>}`, the total across all workers, or `null` if
  the run failed.
- `categories` lists the stressors in each of the seven categories.
- `meta.physical_cpus` is the worker count of the multi-worker run, and
  `meta.stressor_count` the number of stressors run.

See [Output](https://docs.sparecores.com/images/stress-ng/usage.md#output) for every field.

## Workloads

This benchmark runs the following workloads:

- **[Stressor Suite](https://docs.sparecores.com/images/stress-ng/workloads.md#stressor-suite):**
  [`run_stressng_benchmarks.py`](https://github.com/SpareCores/sc-images/blob/main/images/stress-ng/run_stressng_benchmarks.py) runs 85
  stressors in seven categories, each with one worker and then one worker per
  CPU.
- **[CPU Scaling Loop](https://docs.sparecores.com/images/stress-ng/workloads.md#cpu-scaling-loop):** The `cpu` stressor
  with the stress-ng div16 method runs for 10 seconds at each worker count from
  1 up to the CPU count, with [`count.awk`](https://github.com/SpareCores/sc-images/blob/main/images/stress-ng/count.awk) picking the counts.

For more information on stressor categories and run order, see
[Workloads](https://docs.sparecores.com/images/stress-ng/workloads.md).

## Design History

The CPU scaling loop uses the `div16` method of the `cpu` stressor. We ran
every `--cpu-method` on selected x86_64 and Arm servers, with and without SMT
(Simultaneous Multithreading). Other methods scaled more or less than
expected: on a server with 2 vCPUs and SMT, some reached up to 1.8 times the
single-worker throughput with two workers, while `div16` stayed at about the
single-worker throughput.

The stressor suite came with an upgrade from `stress-ng` 0.17.08 to 0.20.01.
At first, only a subset of 23 stressors also ran with all CPUs; now every
stressor does.

See the [changelog](https://docs.sparecores.com/images/stress-ng/design-history.md) for the changes to this benchmark.

## References

See the full list of [references](https://docs.sparecores.com/images/stress-ng/glossary.md) used in this
documentation.
