GB300 NVL72 / GB300 NVL72 specs
GB300 NVL72 specs
What you get in one rack, what each number means for your workload, and where each figure comes from.
The spec table
| Item | Per rack | Source |
|---|---|---|
| GPUs | 72 Blackwell Ultra | nvidia.com |
| CPUs | 36 Grace (2,592 Arm Neoverse V2 cores) | nvidia.com |
| GPU memory (HBM3e) | 20 TB total, 288 GB per GPU | learn.microsoft.com |
| HBM bandwidth | Up to 576 TB/s | nvidia.com |
| CPU memory | 17 TB LPDDR5X | nvidia.com |
| Fast memory | 37 TB | nvidia.com |
| NVLink bandwidth in the rack | 130 TB/s | nvidia.com |
| FP4 Tensor Core | Up to 1.44 exaFLOPS per rack | learn.microsoft.com |
| Network per GPU | 800 Gb/s (ConnectX-8) | nvidia.com |
| Scale-out fabric | Quantum-X800 InfiniBand or Spectrum-X Ethernet | nvidia.com |
| Layout | 18 compute trays (4 GPUs + 2 CPUs each), 9 NVLink switch trays | learn.microsoft.com |
| Cooling | Fully liquid-cooled | nvidia.com |
Numbers are for a full rack. NVIDIA's own page is the reference for most of them; the per-GPU memory, FP4 figure and tray layout come from Microsoft's documentation for the ND GB300 v6 series, which runs on GB300 NVL72.
How the rack is laid out
A GB300 NVL72 is 18 compute trays and 9 NVLink switch trays in one rack. Each compute tray holds four Blackwell Ultra GPUs and two Grace CPUs. The switch trays sit in the middle and connect all 72 GPUs into one NVLink domain, which is why the rack behaves like one very large GPU for many workloads.
In the cloud you will often see the rack split up. Azure, for example, exposes it as 18 VMs of four GPUs each (Microsoft). If your job needs the full 72-GPU domain, ask for whole racks and confirm the scheduler gives you the domain intact.
Memory: the headline change
Each GPU has 288 GB of HBM3e, 20 TB across the rack. Add the Grace CPUs' 17 TB of LPDDR5X and you get 37 TB of "fast memory" that the GPUs can reach coherently. For inference on large models this matters more than raw FLOPS: more of the model and its KV cache stay close to the GPU.
Networking
Inside the rack, NVLink carries 130 TB/s. Between racks, each GPU gets 800 Gb/s through ConnectX-8, on either Quantum-X800 InfiniBand or Spectrum-X Ethernet (NVIDIA). When you compare quotes, ask which fabric the cluster uses and whether it is non-blocking across all your racks.
Power and cooling
The rack is fully liquid-cooled. Integrator Introl reports a draw of about 120 kW per rack (Introl, Jun 2025), far beyond what an air-cooled hall supports. That is why most buyers rent GB300 NVL72 from an operator rather than colocating their own. More on power and cooling.
Questions
How much memory does a GB300 NVL72 have?
Is GB300 the same as B300?
RackShore runs this site and may be paid a referral fee by the provider you choose. Buyers pay us nothing. Spot a mistake? Tell us.