en.Wedoany.com Reported - AMD recently launched its Ryzen AI Halo platform at a price of $4,000, sparking discussions comparing it to Nvidia's slightly more expensive DGX Spark product. Storage and memory company Longsys has taken this a step further by demonstrating the local operation of an AI model with 397 billion parameters on this platform. The demonstration uses a 16-core Ryzen AI Max+ 395 processor with 128GB of memory, leveraging its self-developed SPU and iSA configuration to achieve real-time data compression. This enables the device to run a model that typically requires 200-250GB of video memory, despite having only 128GB of unified memory (of which 96GB is available as GPU video memory).

The model is speculated to be a customized version derived from Alibaba's Qwen 3.5 397B (A17B), employing a mixture of experts (MoE) approach. Even with INT4 quantization, its memory requirements far exceed the available memory of the demonstration device. Longsys stated that its method utilizes expert offloading, intelligent cache management, and predictive prefetching algorithms to offload inactive experts from DRAM to a large, fast storage buffer, allowing the AI chip to reload them as needed. This approach aims to address pain points such as the massive number of parameters in MoE large language models, rapid expansion of KV cache, and I/O latency. The company claims that by leveraging the cache layer to reduce DRAM demand, it can accommodate twice the amount of data in storage drives of up to 128GB. Notably, Longsys did not provide details on computational power measured in tokens per second, as the Ryzen AI chip is relatively limited in this metric compared to most modern AI GPU products. Nevertheless, this approach of treating storage as memory suggests that memory limitations can be circumvented by utilizing fast storage, enabling top-tier models that typically require expensive AI hardware to run on palm-sized devices.

This article is compiled by Wedoany. All AI citations must indicate the source as "Wedoany". If there is any infringement or other issues, please notify us promptly, and we will modify or delete it accordingly. Email: news@wedoany.com