# nvbandwidth

> GPU bandwidth, GPU latency benchmark workloads: Node baseline directional bandwidth (nvbandwidth) and 11 more.

GPU bandwidth, GPU latency

## Node baseline directional bandwidth (nvbandwidth) {#nvbandwidth-all-duplex}

Conservative per-direction bandwidth (GB/s) under simultaneous bi-directional saturation utilizing CE (Copy Engine).

Benchmark details<dl><dt>Benchmark ID</dt><dd><code>nvbandwidth:all:duplex</code></dd><dt>Unit of measurement</dt><dd>GB/s</dd><dt>Server types evaluated</dt><dd>180</dd><dt>Total measurements</dt><dd>180</dd><dt>Range of measurements</dt><dd>5.3 – 178.6 <small>GB/s</small></dd><dt>Better</dt><dd>↑ Higher is better</dd></dl>

Score distribution <small>(180 measurements)</small>

![Score distribution of Node baseline directional bandwidth (nvbandwidth)](/benchmarks/histograms/nvbandwidth-all-duplex.svg)

## Node GPU-to-host bandwidth (nvbandwidth) {#nvbandwidth-all-gpu_to_host}

Total node egress rate (GB/s) when ALL GPUs stream to CPU concurrently utilizing CE (Copy Engine).

Benchmark details<dl><dt>Benchmark ID</dt><dd><code>nvbandwidth:all:gpu_to_host</code></dd><dt>Unit of measurement</dt><dd>GB/s</dd><dt>Server types evaluated</dt><dd>180</dd><dt>Total measurements</dt><dd>180</dd><dt>Range of measurements</dt><dd>6.6 – 229.3 <small>GB/s</small></dd><dt>Better</dt><dd>↑ Higher is better</dd></dl>

Score distribution <small>(180 measurements)</small>

![Score distribution of Node GPU-to-host bandwidth (nvbandwidth)](/benchmarks/histograms/nvbandwidth-all-gpu_to_host.svg)

## Node host-to-GPU bandwidth (nvbandwidth) {#nvbandwidth-all-host_to_gpu}

Total node ingest rate (GB/s) when CPU streams to ALL GPUs concurrently utilizing CE (Copy Engine).

Benchmark details<dl><dt>Benchmark ID</dt><dd><code>nvbandwidth:all:host_to_gpu</code></dd><dt>Unit of measurement</dt><dd>GB/s</dd><dt>Server types evaluated</dt><dd>180</dd><dt>Total measurements</dt><dd>180</dd><dt>Range of measurements</dt><dd>6.2 – 228.8 <small>GB/s</small></dd><dt>Better</dt><dd>↑ Higher is better</dd></dl>

Score distribution <small>(180 measurements)</small>

![Score distribution of Node host-to-GPU bandwidth (nvbandwidth)](/benchmarks/histograms/nvbandwidth-all-host_to_gpu.svg)

## SM/CE copy efficiency (nvbandwidth) {#nvbandwidth-efficiency-sm_ce_ratio}

Ratio (between 0.0 and 1.0) showing compute-core copy efficiency vs. raw DMA Copy Engine capacity, or in other terms, the ratio between CE (Copy Engine) and Streaming Multiprocessor (SM).

Benchmark details<dl><dt>Benchmark ID</dt><dd><code>nvbandwidth:efficiency:sm_ce_ratio</code></dd><dt>Unit of measurement</dt><dd>ratio</dd><dt>Server types evaluated</dt><dd>180</dd><dt>Total measurements</dt><dd>180</dd><dt>Range of measurements</dt><dd>0.0717 – 1 <small>ratio</small></dd><dt>Better</dt><dd>↑ Higher is better</dd></dl>

Score distribution <small>(180 measurements)</small>

![Score distribution of SM/CE copy efficiency (nvbandwidth)](/benchmarks/histograms/nvbandwidth-efficiency-sm_ce_ratio.svg)

## GPU-to-GPU P2P duplex bandwidth (nvbandwidth) {#nvbandwidth-p2p-duplex}

Per-direction NVLink/P2P lane speed (GB/s) during simultaneous transfers in both directions between two GPUs utilizing CE (Copy Engine).

Benchmark details<dl><dt>Benchmark ID</dt><dd><code>nvbandwidth:p2p:duplex</code></dd><dt>Unit of measurement</dt><dd>GB/s</dd><dt>Server types evaluated</dt><dd>33</dd><dt>Total measurements</dt><dd>33</dd><dt>Range of measurements</dt><dd>4.9 – 770.4 <small>GB/s</small></dd><dt>Better</dt><dd>↑ Higher is better</dd></dl>

Score distribution <small>(33 measurements)</small>

![Score distribution of GPU-to-GPU P2P duplex bandwidth (nvbandwidth)](/benchmarks/histograms/nvbandwidth-p2p-duplex.svg)

## GPU-to-GPU P2P gather bandwidth (nvbandwidth) {#nvbandwidth-p2p-gather}

Intake bandwidth (GB/s) on a single target GPU when all peer GPUs send to it at once utilizing CE (Copy Engine).

Benchmark details<dl><dt>Benchmark ID</dt><dd><code>nvbandwidth:p2p:gather</code></dd><dt>Unit of measurement</dt><dd>GB/s</dd><dt>Server types evaluated</dt><dd>33</dd><dt>Total measurements</dt><dd>33</dd><dt>Range of measurements</dt><dd>9.4 – 795.3 <small>GB/s</small></dd><dt>Better</dt><dd>↑ Higher is better</dd></dl>

Score distribution <small>(33 measurements)</small>

![Score distribution of GPU-to-GPU P2P gather bandwidth (nvbandwidth)](/benchmarks/histograms/nvbandwidth-p2p-gather.svg)

## GPU-to-GPU P2P latency (nvbandwidth) {#nvbandwidth-p2p-latency}

Direct transit latency (ns) between GPU pairs over NVLink/P2P utilizing SM (Streaming Multiprocessor).

Benchmark details<dl><dt>Benchmark ID</dt><dd><code>nvbandwidth:p2p:latency</code></dd><dt>Unit of measurement</dt><dd>ns</dd><dt>Server types evaluated</dt><dd>33</dd><dt>Total measurements</dt><dd>33</dd><dt>Range of measurements</dt><dd>394.7 – 1.6K <small>ns</small></dd><dt>Better</dt><dd>↓ Lower is better</dd></dl>

Score distribution <small>(33 measurements)</small>

![Score distribution of GPU-to-GPU P2P latency (nvbandwidth)](/benchmarks/histograms/nvbandwidth-p2p-latency.svg)

## GPU-to-GPU P2P bandwidth (nvbandwidth) {#nvbandwidth-p2p-single}

Single NVLink/P2P lane write speed (GB/s) between any two GPUs utilizing CE (Copy Engine).

Benchmark details<dl><dt>Benchmark ID</dt><dd><code>nvbandwidth:p2p:single</code></dd><dt>Unit of measurement</dt><dd>GB/s</dd><dt>Server types evaluated</dt><dd>33</dd><dt>Total measurements</dt><dd>33</dd><dt>Range of measurements</dt><dd>9.9 – 774.7 <small>GB/s</small></dd><dt>Better</dt><dd>↑ Higher is better</dd></dl>

Score distribution <small>(33 measurements)</small>

![Score distribution of GPU-to-GPU P2P bandwidth (nvbandwidth)](/benchmarks/histograms/nvbandwidth-p2p-single.svg)

## Per-GPU GPU-to-host bandwidth (nvbandwidth) {#nvbandwidth-slot-gpu_to_host}

Average physical PCIe slot egress speed (GB/s) per single GPU utilizing CE (Copy Engine).

Benchmark details<dl><dt>Benchmark ID</dt><dd><code>nvbandwidth:slot:gpu_to_host</code></dd><dt>Unit of measurement</dt><dd>GB/s</dd><dt>Server types evaluated</dt><dd>180</dd><dt>Total measurements</dt><dd>180</dd><dt>Range of measurements</dt><dd>6.6 – 57.4 <small>GB/s</small></dd><dt>Better</dt><dd>↑ Higher is better</dd></dl>

Score distribution <small>(180 measurements)</small>

![Score distribution of Per-GPU GPU-to-host bandwidth (nvbandwidth)](/benchmarks/histograms/nvbandwidth-slot-gpu_to_host.svg)

## Per-GPU host-to-GPU bandwidth (nvbandwidth) {#nvbandwidth-slot-host_to_gpu}

Average physical PCIe slot ingest speed (GB/s) per single GPU utilizing CE (Copy Engine).

Benchmark details<dl><dt>Benchmark ID</dt><dd><code>nvbandwidth:slot:host_to_gpu</code></dd><dt>Unit of measurement</dt><dd>GB/s</dd><dt>Server types evaluated</dt><dd>180</dd><dt>Total measurements</dt><dd>180</dd><dt>Range of measurements</dt><dd>6.1 – 57.9 <small>GB/s</small></dd><dt>Better</dt><dd>↑ Higher is better</dd></dl>

Score distribution <small>(180 measurements)</small>

![Score distribution of Per-GPU host-to-GPU bandwidth (nvbandwidth)](/benchmarks/histograms/nvbandwidth-slot-host_to_gpu.svg)

## Per-GPU host-device latency (nvbandwidth) {#nvbandwidth-slot-latency}

Average round-trip transit time (ns) between CPU and individual GPUs utilizing SM (Streaming Multiprocessor).

Benchmark details<dl><dt>Benchmark ID</dt><dd><code>nvbandwidth:slot:latency</code></dd><dt>Unit of measurement</dt><dd>ns</dd><dt>Server types evaluated</dt><dd>180</dd><dt>Total measurements</dt><dd>180</dd><dt>Range of measurements</dt><dd>558.8 – 8.6K <small>ns</small></dd><dt>Better</dt><dd>↓ Lower is better</dd></dl>

Score distribution <small>(180 measurements)</small>

![Score distribution of Per-GPU host-device latency (nvbandwidth)](/benchmarks/histograms/nvbandwidth-slot-latency.svg)

## Per-GPU device memory copy bandwidth (nvbandwidth) {#nvbandwidth-slot-local_copy}

Average on-GPU memory copy bandwidth (GB/s) per single GPU (device-to-device copy within the same GPU's memory) utilizing CE (Copy Engine).

Benchmark details<dl><dt>Benchmark ID</dt><dd><code>nvbandwidth:slot:local_copy</code></dd><dt>Unit of measurement</dt><dd>GB/s</dd><dt>Server types evaluated</dt><dd>82</dd><dt>Total measurements</dt><dd>82</dd><dt>Range of measurements</dt><dd>13.1 – 3.2K <small>GB/s</small></dd><dt>Better</dt><dd>↑ Higher is better</dd></dl>

:::warning

A copy both reads and writes every byte, so the scores are roughly 35-45% of the memory bandwidth listed on GPU datasheets (e.g. ~1,490 vs 3,350 GB/s on H100 SXM), and should not be compared to those figures directly. Time-sliced virtual GPU profiles (e.g. T4-4Q) can report proportionally lower values than the full GPU. Only recorded with nvbandwidth v0.7 and later.

:::

Score distribution <small>(82 measurements)</small>

![Score distribution of Per-GPU device memory copy bandwidth (nvbandwidth)](/benchmarks/histograms/nvbandwidth-slot-local_copy.svg)
