en.Wedoany.com Reported - The core task of AI infrastructure has shifted from supporting large model inference to supporting the large-scale operation of massive intelligent agents and the continuous production of high-quality tokens. According to IDC data, the enterprise market size for AI Agents in China in 2025 is approximately 19 billion yuan, with a projected compound annual growth rate exceeding 110% from 2025 to 2028. Gartner's assessment is even more direct, predicting that by 2026, 40% of enterprise applications will integrate task-oriented AI Agents. During the large model inference phase, AI infrastructure only needs to support one input and one output. Entering the Agent phase, the infrastructure must support task decomposition, tool invocation, multi-round collaboration, and continuous operation, which imposes entirely different requirements on computing power. Behind this lie two capability gaps: one is whether massive Agents can operate at scale and stably, and the other is whether multiple models can collaborate to make Agents smarter. At the 2026 Open Computing Conference, Inspur Information introduced new product solutions targeting these two gaps.
In the Agent era, people have new requirements for AI infrastructure. In the past, enterprises deploying AI mostly connected one or two models to handle relatively independent tasks—one call, one response, and the task was done. Agents are different. An Agent application first needs to decompose a task, then step by step invoke tools and collaborate back and forth, potentially running an entire group of Sub-Agents simultaneously behind the scenes.
After deployment in an enterprise, the number of Agents can easily reach tens of thousands. How to enable such a large-scale group of Agents to operate stably and collaboratively becomes an unavoidable new challenge. Besides the increasing number of Agents, the pressure on a single model is also growing. Some models excel at logical reasoning, while others are good at writing text. This imbalance in capabilities cannot be solved by simply increasing parameters. However, actual tasks are becoming increasingly complex, making it difficult to rely on a single model to handle everything. Therefore, enabling multiple models to divide labor, collaborate, and complement each other has become another problem that must be solved. For both of these to be truly realized, the underlying computing infrastructure must take the lead. With the arrival of new demands, the first change in infrastructure is that the CPU becomes more important and plays a larger role. In the previous question-answering mode, large model inference involved one input and one output, relying more on GPU operation. But Agents are different; they require task decomposition, tool invocation, multi-round collaboration, and result aggregation—these integer operations and logical reasoning run on the CPU. Moreover, Agents don't just run once and "clock out"; many Agents must be online year-round, significantly extending their runtime. Therefore, in the AI Infra industry, the ratio of computing power is also changing, shifting from a GPU-centric model to a collaborative multi-computing system. As the importance of the CPU rises, the issue of power density immediately follows. Zhao Shuai, Deputy General Manager of Inspur Information, stated that the power of domestic AI racks will surge to 300 kilowatts this year, and some global racks have already entered the megawatt level. If the CPU side remains at a density of a few kilowatts per rack, it simply cannot match the new power infrastructure of data centers. As rack power continues to climb, heat dissipation becomes a problem. The traditional air-cooling limit of 40 to 50 kilowatts per rack has long been exceeded, making liquid cooling solutions a necessity.
Running 40,000+ Agents in a single rack. To achieve the large-scale operation of Agents, Inspur Information released the industry's first CPU-native liquid cooling whole-rack server. This server supports up to 384 CPU processors based on the open OCM (Open Computing Module) architecture in a single rack, compatible with x86 and ARM, and can support the collaborative operation of 40,000+ Agents.
This scale is 40 times that of the "Qi Qian Xia" solution released by Inspur Information in April this year. The "Qi Qian Xia" solution deployed 1000 OpenClaw instances on a single 2U server. This time, Inspur Information has directly packed Agents into an entire rack. Moreover, this rack adopts the OCM architecture, which is compatible with processors of different generations and architectures, eliminating the need to redesign the entire system for each new generation of chips, thereby significantly shortening the development cycle. To pack 384 CPUs into a single rack, heat dissipation is an unavoidable hurdle. Inspur Information proposed a new cooling concept: native liquid cooling. This idea completely overturns the traditional air-liquid hybrid cooling logic. Previous cold plate liquid cooling server designs involved attaching cooling cold plates to computing components, while other components like memory, network cards, and hard drives still required figuring out how to dissipate the generated heat through fans.
This time, the approach is completely revolutionized. Computing and cooling are co-designed, breaking through the traditional limitation of liquid cooling covering only the CPU. All heat-generating components, including memory, network cards, optical modules, and SSDs, are integrated into the liquid cooling system, restructuring the entire computing system. Inspur Information's specific approach is to create a 2U ultra-thin form factor for the computing unit, packing 16 CPUs into one node. Meanwhile, components like memory, network cards, and optical modules, which originally relied on fans and cables for cooling and connection, are laid flat directly on the motherboard. A single large cold plate uniformly handles the cooling, even eliminating the need for server trays. This frees up space previously occupied by fans, coolant pipes, and cables, allocating it to computing and IO resources. The entire rack thus achieves a cable-free design, supports hot maintenance, ensures zero business interruption, and improves rack O&M efficiency by over 100%.
Multiple large models collaborate to complete a single task. To make Agents smarter, Inspur Information also launched a multi-model fusion API on its YuanBrain Enterprise AI (EPAI) platform, simultaneously releasing the YuanBrain SD200 Super Node AI Server Enterprise Edition. What multi-model fusion does is to simultaneously assign the same task to multiple candidate models, allowing them to generate answers independently. Then, a review fusion model steps in to compare the consensus, differences, omissions, and unique perspectives among these answers, ultimately piecing together a unified output. This process is not applied to every task. For short tasks like simple Q&A, tool invocation, and format conversion, the system directly routes them to a lighter single model for processing. Only complex, long-chain tasks trigger the system to dispatch multiple candidate models for collaborative processing, avoiding overkill. In the DRACO test, this mechanism achieved a score of 53.9%, higher than any single model in the same test candidate pool. Currently, this capability is available externally as a multi-model fusion API. It can be directly integrated into applications like a regular model service, or configured within agents and development frameworks, retaining the original dialogue, reasoning, and tool invocation workflows. However, a problem arises: having multiple large models participate in a single task simultaneously undoubtedly places higher demands on the underlying computing power. It must be able to load multiple trillion-parameter models at once without sacrificing output speed. This is precisely the part the YuanBrain SD200 Super Node is designed to handle. When the YuanBrain SD200 Super Node was released last year, it could already deploy four trillion-parameter large models simultaneously, achieving a token generation time of 8.9 milliseconds, making it the first domestic product to break the 10-millisecond barrier. This year, this figure has been further reduced to 4.77 milliseconds, making it the first domestic solution to break into the 5-millisecond range, with first-token latency also reduced by 35%.
These improvements are driven by software-hardware co-optimization techniques like Multi-Token Prediction, the W4A8 precision scheme, and JIT (Just-In-Time) compilation. Multi-Token Prediction generates multiple candidate tokens at once during the decoding phase before verification, reducing the number of iterative word-by-word generations. W4A8 reduces the computation precision of MoE modules in trillion-parameter models from BF16 to INT8, alleviating memory bandwidth pressure. JIT dynamically generates dedicated GPU kernels at runtime based on tensor shapes, making computing power more closely match hardware characteristics. Currently, the YuanBrain SD200 Super Node has been adapted for mainstream open-source models like Kimi K2.6, DeepSeek V4, GLM 5.2, and MiniMax M3. However, the threshold for this architecture remains high for many small and medium-sized enterprises. Therefore, Inspur Information also launched the YuanBrain SD200 Super Node Enterprise Edition, which can be understood as a smaller version of the YuanBrain SD200.
It reduces the Scale-Up computing domain from 64 GPUs to 16 GPUs, reducing the first-token latency for trillion-parameter models by over 40%, providing enterprises with a lower-cost option for migration and adaptation. This allows enterprises that previously could only deploy hundred-billion-parameter models for auxiliary tasks to now effectively use trillion-parameter models in production environments.
The competition in Agent infrastructure has already changed. Today, the division of labor among CPUs, GPUs, and software platforms is becoming tighter. The software platform is responsible for model access, task orchestration, resource scheduling, permission management, and result fusion. The CPU handles Agent instances, tool invocation, sandbox execution, and business system interaction. The GPU is responsible for model inference and token generation. Only through the collaboration of these three can massive Agents operate stably and complex tasks be executed efficiently. If any link in this chain falls behind, the entire Agent application will not run smoothly. This has also shifted the focus of competition in Agent-era infrastructure. In the past, the competition was about who had stronger support for a single model. Now, it's about who achieves better system-level collaboration. Strength in a single point is no longer sufficient; whether the entire chain runs smoothly and collaboratively is the key. This is precisely the answer Inspur Information aims to provide for Agent infrastructure this time.
