Collective
A coordinated communication pattern across many GPUs (all-reduce, all-gather, reduce-scatter) central to distributed AI.
Current numbers
ROCm 10.1.0 (release snapshot)Current AMD ROCm release; its RCCL implements NCCL's host collective API, trailing NCCL releases
1.8 → 3.0 TB/sNVLink Gen5 Blackwell / Gen6 Rubin per-GPU bidirectional ratings; NVL72 sums 72×1.8=129.6 and 72×3.0=216 TB/s (~130/216), not delivered collective throughput
400Gspeed of Meta's 2024 production RoCE back-end: PFC only, no DCQCN, receiver-driven admission in the collective library