nvbandwidth
GPU bandwidth, GPU latency
Node baseline directional bandwidth (nvbandwidth)
Conservative per-direction bandwidth (GB/s) under simultaneous bi-directional saturation utilizing CECE: Copy EngineA DMA (Direct Memory Access) engine of an NVIDIA GPU that copies data between GPU memory, host memory, and the memory of other GPUs without using the GPU's compute cores. A GPU has one or more copy engines, which CUDA uses for memory copies, so data transfers can run at the same time as compute kernels.CUDA programming guideGlossary (Copy Engine).
Benchmark details
- Benchmark ID
nvbandwidth:all:duplex- Unit of measurement
- GB/s
- Server types evaluated
- 180
- Total measurements
- 180
- Range of measurements
- 5.3 – 178.6 GB/s
- Better
- ↑ Higher is better
Score distribution (180 measurements)
Node GPU-to-host bandwidth (nvbandwidth)
Total node egress rate (GB/s) when ALL GPUs stream to CPUCPU: Central Processing UnitThe general-purpose processor of a computer, which runs the operating system and programs. A CPU has one or more physical cores, and with SMT each core runs two or more hardware threads.WikipediaGlossary concurrently utilizing CE (Copy Engine).
Benchmark details
- Benchmark ID
nvbandwidth:all:gpu_to_host- Unit of measurement
- GB/s
- Server types evaluated
- 180
- Total measurements
- 180
- Range of measurements
- 6.6 – 229.3 GB/s
- Better
- ↑ Higher is better
Score distribution (180 measurements)
Node host-to-GPU bandwidth (nvbandwidth)
Total node ingest rate (GB/s) when CPU streams to ALL GPUs concurrently utilizing CE (Copy Engine).
Benchmark details
- Benchmark ID
nvbandwidth:all:host_to_gpu- Unit of measurement
- GB/s
- Server types evaluated
- 180
- Total measurements
- 180
- Range of measurements
- 6.2 – 228.8 GB/s
- Better
- ↑ Higher is better
Score distribution (180 measurements)
SM/CE copy efficiency (nvbandwidth)
Ratio (between 0.0 and 1.0) showing compute-core copy efficiency vs. raw DMA Copy Engine capacity, or in other terms, the ratio between CE (Copy Engine) and Streaming Multiprocessor (SMSM: Streaming MultiprocessorThe basic processing unit of an NVIDIA GPU, which runs the threads of a kernel in groups, in parallel. Each SM has its own cores, registers, shared memory, and caches, and a GPU has from a few to more than a hundred SMs, so its compute throughput scales with their number.CUDA programming guideGlossary).
Benchmark details
- Benchmark ID
nvbandwidth:efficiency:sm_ce_ratio- Unit of measurement
- ratio
- Server types evaluated
- 180
- Total measurements
- 180
- Range of measurements
- 0.0717 – 1 ratio
- Better
- ↑ Higher is better
Score distribution (180 measurements)
GPU-to-GPU P2P duplex bandwidth (nvbandwidth)
Per-direction NVLink/P2P lane speed (GB/s) during simultaneous transfers in both directions between two GPUs utilizing CE (Copy Engine).
Benchmark details
- Benchmark ID
nvbandwidth:p2p:duplex- Unit of measurement
- GB/s
- Server types evaluated
- 33
- Total measurements
- 33
- Range of measurements
- 4.9 – 770.4 GB/s
- Better
- ↑ Higher is better
Score distribution (33 measurements)
GPU-to-GPU P2P gather bandwidth (nvbandwidth)
Intake bandwidth (GB/s) on a single target GPUGPU: Graphics Processing UnitA processor with thousands of simple cores that run the same operation on many data elements in parallel. It was built to render graphics, and is now also used for general-purpose parallel work such as machine learning training and inference, scientific simulation, and video encoding. Cloud providers rent GPUs attached to virtual machines, with their own dedicated GPU memory.WikipediaGlossary when all peer GPUs send to it at once utilizing CE (Copy Engine).
Benchmark details
- Benchmark ID
nvbandwidth:p2p:gather- Unit of measurement
- GB/s
- Server types evaluated
- 33
- Total measurements
- 33
- Range of measurements
- 9.4 – 795.3 GB/s
- Better
- ↑ Higher is better
Score distribution (33 measurements)
GPU-to-GPU P2P latency (nvbandwidth)
Direct transit latency (ns) between GPU pairs over NVLink/P2P utilizing SMSM: Streaming MultiprocessorThe basic processing unit of an NVIDIA GPU, which runs the threads of a kernel in groups, in parallel. Each SM has its own cores, registers, shared memory, and caches, and a GPU has from a few to more than a hundred SMs, so its compute throughput scales with their number.CUDA programming guideGlossary (Streaming Multiprocessor).
Benchmark details
- Benchmark ID
nvbandwidth:p2p:latency- Unit of measurement
- ns
- Server types evaluated
- 33
- Total measurements
- 33
- Range of measurements
- 394.7 – 1.6K ns
- Better
- ↓ Lower is better
Score distribution (33 measurements)
GPU-to-GPU P2P bandwidth (nvbandwidth)
Single NVLink/P2P lane write speed (GB/s) between any two GPUs utilizing CE (Copy Engine).
Benchmark details
- Benchmark ID
nvbandwidth:p2p:single- Unit of measurement
- GB/s
- Server types evaluated
- 33
- Total measurements
- 33
- Range of measurements
- 9.9 – 774.7 GB/s
- Better
- ↑ Higher is better
Score distribution (33 measurements)
Per-GPU GPU-to-host bandwidth (nvbandwidth)
Average physical PCIe slot egress speed (GB/s) per single GPU utilizing CE (Copy Engine).
Benchmark details
- Benchmark ID
nvbandwidth:slot:gpu_to_host- Unit of measurement
- GB/s
- Server types evaluated
- 180
- Total measurements
- 180
- Range of measurements
- 6.6 – 57.4 GB/s
- Better
- ↑ Higher is better
Score distribution (180 measurements)
Per-GPU host-to-GPU bandwidth (nvbandwidth)
Average physical PCIe slot ingest speed (GB/s) per single GPU utilizing CE (Copy Engine).
Benchmark details
- Benchmark ID
nvbandwidth:slot:host_to_gpu- Unit of measurement
- GB/s
- Server types evaluated
- 180
- Total measurements
- 180
- Range of measurements
- 6.1 – 57.9 GB/s
- Better
- ↑ Higher is better
Score distribution (180 measurements)
Per-GPU host-device latency (nvbandwidth)
Average round-trip transit time (ns) between CPU and individual GPUs utilizing SMSM: Streaming MultiprocessorThe basic processing unit of an NVIDIA GPU, which runs the threads of a kernel in groups, in parallel. Each SM has its own cores, registers, shared memory, and caches, and a GPU has from a few to more than a hundred SMs, so its compute throughput scales with their number.CUDA programming guideGlossary (Streaming Multiprocessor).
Benchmark details
- Benchmark ID
nvbandwidth:slot:latency- Unit of measurement
- ns
- Server types evaluated
- 180
- Total measurements
- 180
- Range of measurements
- 558.8 – 8.6K ns
- Better
- ↓ Lower is better
Score distribution (180 measurements)
Per-GPU device memory copy bandwidth (nvbandwidth)
Average on-GPU memory copy bandwidth (GB/s) per single GPU (device-to-device copy within the same GPU's memory) utilizing CE (Copy Engine).
Benchmark details
- Benchmark ID
nvbandwidth:slot:local_copy- Unit of measurement
- GB/s
- Server types evaluated
- 82
- Total measurements
- 82
- Range of measurements
- 13.1 – 3.2K GB/s
- Better
- ↑ Higher is better
A copy both reads and writes every byte, so the scores are roughly 35-45% of the memory bandwidth listed on GPU datasheets (e.g. ~1,490 vs 3,350 GB/s on H100 SXM), and should not be compared to those figures directly. Time-sliced virtual GPU profiles (e.g. T4-4Q) can report proportionally lower values than the full GPU. Only recorded with nvbandwidthnvbandwidthAn open-source NVIDIA tool that measures memory copy bandwidth and latency between the CPU and NVIDIA GPUs, between GPUs, and within one GPU. It copies with the GPU's copy engines or with kernels on its streaming multiprocessors, and reports one matrix per test, with a row and a column for each CPU or GPU taking part. It prints plain text or JSON, and an optional multinode build measures GPUs across several servers.GitHubGlossary v0.7 and later.
Score distribution (82 measurements)