When NVIDIA unveiled the GB200 NVL72, it was already a big deal: a full rack-scale AI supercomputer in a box. But the story didn’t stop there. In September 2025, NVIDIA introduced the GB300 NVL72, the next-generation evolution of this architecture. As someone who’s been closely following NVIDIA’s roadmap and the arms race in large AI compute, I want to take you through a detailed comparison of the GB300 NVL72 vs. GB200 NVL72 — highlighting the differences, similarities, and what this means for the future of AI infrastructure.
The Basics: What Are NVL72 Systems?
Both the GB200 NVL72 and GB300 NVL72 are rack-scale AI systems designed to train and run the largest AI models in existence. Instead of just offering GPUs as standalone accelerators, NVIDIA bundles together 72 Grace Blackwell Superchips, a liquid-cooled infrastructure, and a rack-ready form factor that can literally be dropped into a datacenter and powered up.
Each NVL72 rack contains:
● 36 dual-GPU Grace Blackwell Superchips (72 GPUs in total)
● Up to 1.4TB of GPU memory (for GB200 NVL72)
● NVLink Switch System for all-to-all GPU communication
● Integrated liquid cooling
● Grace CPUs tightly coupled to GPUs
The idea here is simple: AI workloads are no longer scaling linearly. To push into trillion-parameter model territory, we need enormous GPU clusters with ultra-fast interconnects, dense memory, and rack-level integration. That’s exactly what NVL72 systems provide.
GB200 NVL72 Recap
The GB200 NVL72 was NVIDIA’s first shot at rack-scale AI, announced in early 2024. It combines 72 Blackwell GPUs (B200) with Grace CPUs, connected via NVLink 5. Key specs:
● 72 B200 GPUs per rack
● Each GPU: ~8 TB/s memory bandwidth
● Up to 1.4TB total GPU memory
● Integrated Grace CPU with LPDDR5X
● Peak performance: 1.4 ExaFLOPs (FP4) per rack
It was aimed at training 100B+ parameter LLMs and scaling AI inference across billions of tokens. Companies like Microsoft and Google have been early adopters, integrating GB200 NVL72 racks into their hyperscale datacenters.
GB300 NVL72 — The Next Leap
Fast forward to 2025, and NVIDIA unveiled the GB300 NVL72, essentially the next-generation version. While the overall design philosophy remains the same, the GB300 generation pushes performance and efficiency further:
● 72 GB300 GPUs per rack
● Memory per GPU increased (early reports suggest up to 48GB HBM4 per GPU vs 24GB HBM3E in GB200)
● Peak performance: ~2.8 ExaFLOPs (FP4) per rack — nearly 2x the GB200 NVL72
● NVLink upgraded, providing lower latency and higher all-to-all bandwidth
● Energy efficiency improved through second-gen liquid cooling and optimized Grace CPUs
The result is a rack that not only supports 200B+ parameter models, but is designed to scale towards 500B–1T+ parameter models.
Key Comparison: GB300 NVL72 vs. GB200 NVL72
|
Feature |
GB200 NVL72 |
GB300 NVL72 |
|
GPU Count |
72 × B200 GPUs |
72 × GB300 GPUs |
|
GPU Memory |
Up to ~1.4TB (HBM3E) |
Up to ~2.8TB (HBM4, higher density) |
|
Peak AI Perf (FP4) |
~1.4 ExaFLOPs |
~2.8 ExaFLOPs |
|
NVLink Gen |
NVLink 5 |
NVLink 6 (lower latency, higher bandwidth) |
|
Grace CPU |
Grace 1st-gen |
Grace 2nd-gen (improved bandwidth & efficiency) |
|
Cooling |
1st-gen liquid cooling |
2nd-gen liquid cooling, better thermal density |
|
Model Scale Target |
100B–200B parameters |
200B–1T+ parameters |
Key Differences & Improvements in GB300 over GB200
Here are where GB300 claims to improve over GB200:
|
Feature |
GB200 NVL72 |
GB300 NVL72 |
Improvement / Notes |
|
GPU architecture |
Blackwell (base) |
Blackwell Ultra |
The “Ultra” variant adds performance and enhancements |
|
Inference / AI reasoning uplift |
— |
Up to 10× boost in user responsiveness; 5× throughput per watt over prior generation (Hopper) |
For reasoning/inference tasks, GB300 is optimized to push much higher throughput and lower latency. |
|
Memory & throughput |
~ (specs not always published) |
Increased HBM3e memory (more dense) |
GB300 supports larger batch sizes etc thanks to more memory per GPU / denser configurations |
|
Interconnect / NVLink |
5th-gen NVLink (1.8 TB/s per GPU) |
Maintains or enhances NVLink, NVSwitch and internal connectivity |
GB200 already uses 5th-generation NVLink with 1.8 TB/s GPU-to-GPU communication. GB300 continues this, possibly with improvements in switch design, topology, latency. |
|
Power management & smoothing |
Standard power supply |
New power supply with energy storage to smooth power spikes (reducing peak grid demand by ~30%) |
This is a differentiator: GB300 integrates capacitors for energy storage to buffer transient power surges. |
|
Performance gains over GB200 |
— |
~45% higher inference throughput on some benchmarks (DeepSeek R1) reported in media |
According to Tom’s Hardware, Nvidia claims GB300 achieves ~45% better DeepSeek R1 inference performance vs GB200. |
Arithmetic / capacity calculations (explicit, digit-by-digit)
GB200: per-GPU memory (from vendor total)
● Vendor states 13.4 TB total GPU memory.
● Treating TB as decimal (1 TB = 1000 GB): 13.4 TB = 13.4 × 1000 = 13,400 GB.
● Per GPU (72 GPUs): 13,400 GB ÷ 72 = 186.11111111111111 GB per GPU (≈ 186.11 GB).
GB300: per-GPU memory (from Supermicro / OEM datasheet)
● OEM datasheet / Supermicro calls out 288 GB HBM3e per GPU for B300.
● Total GPU memory = 288 GB × 72 = 20,736 GB = 20.736 TB.
NVLink bandwidth (representative numbers)
● Per-GPU NVLink (NVLink 5.0) ~ 1.8 TB/s per GPU; claimed system NVLink aggregate ~ 130 TB/s.
Practical implications (what these numbers mean)
● Memory per GPU: GB300’s ~288 GB/GPU vs GB200’s effective ~186 GB/GPU (based on datasheet total) means bigger model partitions per GPU and fewer cross-GPU page-moves for very large models — that improves inference latency and simplifies model placement for very large LLMs.
● Compute microarchitecture: Blackwell Ultra adds 2× attention acceleration and claims higher TFLOPS (1.5× more FLOPS in some modes), which directly benefits transformer attention-heavy workloads (reasoning / inference). That’s why NVIDIA advertises large responsiveness and throughput gains.
● Power smoothing: the integrated energy-storage PSU on GB300 lowers peak site demand (up to ~30% reduction in peaks), which can materially reduce data-center infrastructure cost (demand charges, generator sizing). NVIDIA is rolling that capability to GB200 as an option per their blog.
Why the Upgrade Matters
1. Scaling LLMs Beyond 200B — The GB200 was enough for GPT-4 and Claude 3 scale models, but next-gen AI (GPT-5, Gemini Ultra, etc.) is pushing toward 500B+ parameters. GB300 NVL72 racks are designed with that future in mind.
2. Double the Performance per Rack — For hyperscalers, doubling compute density without doubling datacenter footprint is game-changing. Power and cooling efficiency also make the economics far better.
3. Interconnect Evolution — NVLink upgrades reduce communication bottlenecks in massive multi-rack clusters, making distributed training more efficient.
Who Will Use GB300 NVL72?
Just like GB200 NVL72 found homes in Microsoft Azure, Google Cloud, AWS, and Oracle, the GB300 NVL72 will likely be the standard rack-scale system for next-gen hyperscale AI deployments. Expect early adoption in 2026 by:
● Cloud providers training multi-hundred-billion parameter models
● Enterprises building AI copilots, robotics brains, or foundation models
● Research labs pushing AI science beyond text into multimodal, 3D, and scientific workloads
Final Thoughts
The GB200 NVL72 was revolutionary, but the GB300 NVL72 feels like the system that truly unlocks trillion-scale AI. If the GB200 gave us GPT-4-class models, the GB300 is what will power GPT-5.5, Gemini Ultra, and beyond.
For datacenter architects, the decision point is simple: if you’re still planning GB200 deployments today, you need to evaluate whether to wait for GB300 NVL72. The jump in performance, efficiency, and scalability is significant enough that it could redefine the economics of AI infrastructure in 2026 and beyond.
John’s Take: As we move into the trillion-parameter era, the GB300 NVL72 isn’t just an upgrade — it’s NVIDIA laying the foundation for the next decade of AI compute. Hyperscalers who adopt early will have a decisive edge.
FAQs: NVIDIA GB300 NVL72 vs GB200 NVL72
1. Is the GB300 NVL72 backward compatible with GB200 deployments?
Yes. Both systems share similar rack-scale architecture and NVLink-based interconnects. However, mixed clusters may not achieve optimal efficiency compared to uniform GB300 racks.
2. How much power does a GB300 NVL72 consume compared to GB200?
Exact figures vary by configuration, but GB300 racks are expected to draw slightly more power. That said, they deliver nearly 2× the performance per watt, thanks to improved GPUs, memory, and cooling.
3. What workloads benefit most from GB300 NVL72?
The GB300 NVL72 is built for 200B–1T+ parameter AI models, multimodal training (text + image + video), 3D simulation, and scientific AI workloads that demand ultra-high memory bandwidth.
4. When will GB300 NVL72 systems be available?
Industry reports suggest availability in 2026, with hyperscalers likely being the first customers. Broader enterprise access may follow in 2027.
5. Is upgrading to GB300 NVL72 worth it if I already use GB200 racks?
For organizations running workloads under 200B parameters, GB200 NVL72 is still extremely capable. But if you’re planning for next-gen trillion-parameter AI, GB300 NVL72 offers a clear path forward.
Sources (the most important pages I used)
●NVIDIA GB300 NVL72 product page. (NVIDIA)
●NVIDIA GB200 NVL72 product page. (NVIDIA)
●Supermicro GB300 NVL72 datasheet (PDF — B300 / 288 GB HBM3e per GPU). (Supermicro)
●Supermicro / OEM GB200 NVL72 datasheet (PDF). (Supermicro)
●NVIDIA developer blog about GB300 power smoothing and energy storage. (NVIDIA Developer)
John-Fu
Tech Blogger & AI Infrastructure Enthusiast
John Fu is a Hong Kong-based tech blogger and deep enthusiast of AI infrastructure. His career is marked by the intersection of cross-cultural technology. His early years living and working in the San Francisco Bay Area have honed his keen understanding of cutting-edge technology. While working as a technical architect at a leading cloud computing company in Silicon Valley, he was deeply involved in the development of distributed computing frameworks and led the deployment of AI training clusters supporting millions of concurrent users.
Related reading:
NVIDIA B300 (Blackwell Ultra): The Next Leap in AI Superchips
NVIDIA GB200: Inside the Blackwell Superchip Powering AI at Scale
The NVIDIA A100 GPU: The AI Workhorse That Changed Everything