en.Wedoany.com Reported - US-based Neocloud company Parasail is deploying NVIDIA Hopper and Blackwell GPUs alongside d-Matrix Corsair inference accelerators to build a heterogeneous AI inference infrastructure. This solution targets large model inference services, boosting token generation rates by up to 10x and improving inference service cost efficiency.
The key to this infrastructure lies in splitting the inference task into two distinct stages. The prefill stage at the front end of large model inference handles input context, prompts, conversation history, and long text content. It is computationally intensive and better suited for NVIDIA Hopper and Blackwell GPUs. The decoding stage at the inference back end generates tokens one by one and is more sensitive to latency, memory access, and sustained throughput, making it more suitable for dedicated inference ASICs like the d-Matrix Corsair. Parasail uses automatic kernel optimization technology to dynamically distribute workloads across different hardware, allowing GPUs and dedicated ASICs to run tasks better suited to their respective architectures, rather than placing the entire inference chain on a single chip.
The d-Matrix Corsair employs a digital in-memory computing architecture, placing compute units and storage resources more closely together within the same chip structure. In traditional inference systems, data must be frequently moved between the processor and external memory, incurring time, power, and cost overheads. The Corsair reduces this data movement through its D-IMC architecture. The chip is built on TSMC's N6 process node, using an organic substrate and LPDDR5 memory, designed for scalable inference deployment in data centers.
Parasail's deployment represents a production-grade heterogeneous disaggregated inference case. The company's platform aggregates high-performance GPU resources from over 40 data centers across 15 countries, providing developers with model endpoints and inference services. By adding Corsair to the existing NVIDIA infrastructure, the inference chain can establish a division of labor between GPU prefill and ASIC decoding, improving the utilization efficiency of existing Hopper and Blackwell clusters.
Heterogeneous inference is not simply about adding a new type of chip; it changes the execution path of inference services. The prefill stage focuses on large-scale matrix computations and context processing, where the parallel computing power of GPUs is well-suited for handling such high-throughput tasks. The decoding stage continuously reads model states, caches, and newly generated tokens, making it more sensitive to memory access, response latency, and energy efficiency. By bringing computation and storage closer together, the Corsair reduces data movement pressure during the decoding process, making token generation more suitable for low-latency output.
d-Matrix's product portfolio also includes the Corsair inference accelerator, JetStream network accelerator, Aviator software, and SquadRack rack-level solutions, aiming to provide low-latency and energy-efficient infrastructure for data center-scale AI inference. By integrating Corsair into the NVIDIA GPU resource pool, Parasail has created not a GPU replacement, but a hybrid deployment model where GPUs and dedicated inference accelerators share different stages of the inference process.
Further performance details will be disclosed through test results and case studies following the initial deployment. The current architecture is already clear: Hopper and Blackwell GPUs handle compute-intensive prefill, Corsair ASICs handle latency-sensitive decoding, and Parasail uses dynamic routing and kernel optimization to allocate different model workloads to the appropriate hardware, ultimately delivering cloud-based token services for developers and enterprise customers.
