NCCL · NVIDIA Collective Communications Library
NVIDIA's library implementing optimized multi-GPU collective operations like all-reduce over NVLink and the fabric.
Current numbers
ROCm 10.1.0 (release snapshot)Current AMD ROCm release; its RCCL implements NCCL's host collective API, trailing NCCL releases
~16 → ≤6 SMsGPU SMs consumed by a reduction after composing NVLink-SHARP + IB-SHARP in NCCL 2.27
≈370 of 400 GB/s (~92%)NCCL all_reduce busbw vs node aggregate line rate: Together AI's published rule of thumb; your golden-node baseline sets the gate