en.Wedoany.com Reported - Qualcomm recently stated in its technical blog that the key to AI performance improvement over the next decade will no longer be processor speed, but a structural change in computing architecture. The article points out that as large generative models become mainstream workloads, the performance bottleneck of artificial intelligence systems has shifted from computing speed to data transfer efficiency, a trend known as the "memory wall" problem.
The article argues that modern AI inference processes are highly dependent on memory. Generating each new token requires reading a large number of model parameters and contextual information from memory. The real challenge lies in providing this data at a sufficiently fast rate, rather than the computation itself. It is estimated that over the past decade, Transformer model sizes have grown 240 times every two years, but AI hardware memory capacity has only increased by 2 times, causing processors to spend a significant amount of time waiting for data.
Traditional high-bandwidth memory (HBM) plus XPU architectures improve bandwidth by widening the channels between memory and processors, but this approach is facing physical and economic limitations. Diminishing returns from interfaces, energy and time costs due to data transmission distances, and the high price of advanced packaging are increasingly constraining traditional paths. The article emphasizes that the energy consumed in transferring operands across chip boundaries can far exceed the arithmetic operations performed after the data arrives.
To address this issue, Qualcomm proposes the concept of "Near-Memory Computing" and calls its implementation High Bandwidth Computing (HBC) technology. This approach places computing units directly near memory, allowing data-intensive operations to be executed where the data resides. The main processor, such as the Qualcomm Dragonfly AI accelerator, still handles complex and flexible tasks, while HBC specifically manages operations constrained by data movement. The article believes that the maturity of 3D integration technology, particularly the ability to directly bond DRAM onto logic circuits, has provided a commercial foundation for this solution.
The Dragonfly series products based on this concept have already demonstrated significant performance improvements. Data shows that the Dragonfly AI250, using first-generation Qualcomm HBC technology, achieves 18 times the effective memory bandwidth of the Dragonfly AI200 using LPDDR5X; while the Dragonfly AI300, using second-generation Qualcomm HBC technology, achieves 54 times the effective memory bandwidth of the Dragonfly AI200.
The article points out that near-memory computing will have multifaceted impacts on the industry. Performance evaluation metrics will change, with buyers increasingly evaluating platforms based on performance under actual memory-intensive workloads and performance per watt and per dollar, rather than peak theoretical throughput. As deployment scales up, energy becomes a major operational cost and physical bottleneck, and architectures that minimize data transfer will have a structural advantage. Additionally, the role of memory will shift from a passive storage medium to an active participant in computation, a change that will affect everything from chip design to data center economics.
Qualcomm believes that in the era of artificial intelligence, the scarce resource is no longer computing power, but how to deliver data to where it is needed in a timely and cost-effective manner. The key to designing systems lies in the ability to intelligently limit data transfer, rather than simply pursuing processor speed.
