US xAI Deploys 200,000 H100 Clusters

en.Wedoany.com Reported - The performance competition of large language models (LLMs) is deeply intertwined with the "infrastructure strategy" of AI companies. Advanced Micro Devices (AMD) stated in its blog that simply expanding GPU cluster scale faces multiple constraints such as power, cost, and network bottlenecks. Therefore, "hardware-software co-design," which considers hardware limitations from the early stages of software design, has become a core strategy for AI companies.

xAI's "Grok 4.0" and DeepSeek's "DeepSeek-V2" demonstrate two distinctly different infrastructure paths. xAI chose a large-scale infrastructure strategy by deploying the "Colossus" cluster with 200,000 H100 units. However, at this scale, network bottlenecks in data transmission between GPUs reduce overall computational efficiency. To address this challenge, xAI bound nine 400G network interface cards (NICs) per server when developing Grok 4.0, achieving 3.6 Tb/s bandwidth. Meanwhile, to reduce costs, xAI adopted RoCE based on general-purpose Ethernet instead of NVIDIA InfiniBand. To tackle the "entropy collapse" issue caused by static computation graphs, xAI introduced "Adaptive Routing" technology that changes paths in real-time on a per-packet basis.

In contrast, DeepSeek chose to maximize hardware efficiency in relatively smaller H800 GPU cluster environments, ranging from 2,000 to 50,000 units. Unable to blindly increase hardware, DeepSeek started from the architecture level to fundamentally reduce network communication. While adopting a Mixture of Experts (MoE) structure, DeepSeek-V2 designed "Device-limited expert routing" technology, overlapping the inherent bottleneck of MoE models—expert routing communication—with GPU hardware computation simultaneously. Thanks to this software optimization, the time GPUs remain idle due to network congestion is minimized, allowing DeepSeek to achieve competitive performance at just one-tenth the hardware infrastructure cost of Grok 4.0.

The opposing strategies adopted by the two models indicate that blindly expanding infrastructure has clear limitations in terms of cost and efficiency. The ability to accurately understand hardware infrastructure constraints and finely tune the model accordingly through "co-design" will become a key factor in future AI market competition.