en.Wedoany.com Reported - JEDEC (Joint Electron Device Engineering Council) has released a new specification aimed at lowering the cost of ultra-expensive HBM (High Bandwidth Memory) that powers the fastest AI processors. The new standard supports installing SPHBM4 memory stacks without requiring advanced packaging and uses cheaper organic substrates, potentially making high-bandwidth memory more affordable. However, it does not alleviate DRAM shortages as it utilizes large HBM4 DRAM devices.

The standards body has released the specification for SPHBM4 (Standard Package High Bandwidth Memory, JESD330-4), which combines HBM4 DRAM ICs with a standard package and a fast, narrow 512-bit interface. While the 1024-bit and 2048-bit interfaces used by HBM3 and HBM4 memory offer unparalleled performance, the wide interfaces consume significant silicon area within the processor and require expensive interposers and advanced packaging technologies with limited capacity (such as TSMC's CoWoS) for integration with the main processor. The upcoming SPHBM4 memory continues to use the same HBM4 DRAM stacks as JESD270-4 but replaces the traditional HBM4 base chip with a new SPHBM4 PHY/buffer chip featuring a narrower 512-bit interface, enabling installation on standard organic substrates without complex packaging methods. To compensate for the narrower interface, SPHBM4 supports higher data transfer rates ranging from 22.4 GT/s to 46.0 GT/s.

Unlike HBM4, which connects to the main processor using a 2048-bit memory interface, SPHBM4 uses 32 independent 16-bit DDR channels organized into eight quad-channels. Internally, an HBM4 stack contains 32 memory channels, each 64 bits wide, with a total external interface width of 2048 bits. SPHBM4 requires "converting" the 2048-bit internal I/O to a 512-bit external interface by grouping every four HBM4 channels into a quad-channel. As a result, externally, a quad-channel exposes 64 data pins (4×16 bits), replacing the 256 data pins (4×64 bits) typically required by these four HBM4 channels. To maintain bandwidth, these 64 pins operate at four times the data rate of the original HBM4 interface.

SPHBM4 significantly increases I/O bandwidth but does not make the DRAM array itself faster. The HBM4 memory core retains the same basic architecture and timing, including core frequency, row activation, precharge, and refresh operations, although the additional PHY is expected to introduce some latency. For example, the DRAM core operates at only one-quarter of the external interface frequency, which is 2 GHz in a 32 GT/s speed-grade SPHBM4. The main change lies in the new base chip, which implements a high-speed SerDes-like PHY, mapping each 16-bit external channel to four traditional 64-bit HBM4 channels. Consequently, SPHBM4 introduces equalization, channel training, BER requirements, and other high-speed signal characteristics unnecessary in HBM4's slower, wide parallel interface. To support transfer rates up to 46.0 GT/s per pin, each quad-channel uses a shared command/address interface protected by Forward Error Correction (FEC), while data transfer relies on dedicated differential write clocks (WCK) and read clocks (RCK), along with ECC and error reporting signals.

In terms of capacity, SPHBM4 can use stacks containing 4, 8, 12, or 16 DRAM chips with 24 Gb or 32 Gb densities. Therefore, the largest standardized SPHBM4 configuration is a 64 GB memory stack built from 16 32 Gb DRAM chips, matching the maximum capacity supported by HBM4E.

The standard supports bump pitches greater than 90 µm and channels up to 20 mm long, both features that allow eliminating expensive interposers and using cheaper organic substrate routing. However, eliminating the interposer and CoWoS (or similar) packaging does not automatically make SPHBM4 cheap. SPHBM4 still requires a large number of HBM4 DRAM ICs, 2.5D packaging, a complex base chip (potentially more expensive than those used in traditional HBM4), and advanced packaging assembly with through-silicon vias. Additionally, SPHBM4's narrow interface consumes significantly less chip perimeter and silicon area within the processor, making it more attractive for companies aiming to install more computing power and/or more memory stacks around the processor.

In terms of maximum performance, HBM4 transmits data at 8 GT/s (though most controllers and chips support higher data rates), so one HBM4 stack provides 2 TB/s of bandwidth. HBM4E increases the data transfer rate to 12–12.8 GT/s, boosting peak bandwidth per stack to 3–3.3 TB/s. In comparison, an SPHBM4 with a 46 GT/s interface can achieve 2.944 TB/s, but initial versions of SPHBM4 should not be expected to reach top speeds. Therefore, for the foreseeable future, HBM4, HBM4E, and C-HBM4E are likely to maintain a performance lead over SPHBM4 in bandwidth.

HBM4 latency may still be superior to SPHBM4. HBM4 essentially connects almost directly to the main processor through a very simple interface. In contrast, SPHBM4 inserts a more complex PHY that performs serialization/deserialization, channel training, FEC processing, and other operations that may add several nanoseconds of latency. This may not be a major issue for some applications, but inference tasks greatly benefit from low latency. In terms of power and voltage, HBM4 and SPHBM4 share the same DRAM core voltage because SPHBM4 reuses standard HBM4 DRAM stacks. However, the I/O differs: HBM4 leaves the interface voltage to the memory supplier's discretion and allows implementation at 0.7V, 0.75V, 0.8V, or 0.9V, depending on the desired balance between power, speed, and signal integrity. In contrast, the SPHBM4 standard standardizes the external I/O at 0.75V. Additionally, HBM4 transmits data over a very wide interface with many slow parallel links, which tend to be very power-efficient. In comparison, SPHBM4 transmits the same amount of data over one-quarter the number of wires, running approximately four times faster. High-speed data transmission tends to be less power-efficient than "slow" data transmission over a wide interface. Considering SPHBM4's rather complex PHY that converts a wide interface to a narrow one, this is likely a power-hungry process. Nevertheless, a fourfold reduction in the number of drivers and receivers may actually reduce SPHBM4's power consumption.

SPHBM4 essentially transforms the manufacturing challenges associated with using silicon interposers into the engineering challenge of developing an extremely complex base chip/PHY. Developing and manufacturing such a base chip should not be a problem for foundries. However, whether DRAM manufacturers can design and produce SPHBM4 with good energy efficiency remains to be seen. After all, Micron and SK hynix both collaborate with TSMC to manufacture C-HBM4E and HBM4E base chips, while Samsung's memory division uses base chips produced by Samsung Foundry.

An interesting aspect of SPHBM4 is whether Chinese AI accelerator developers can benefit from this technology. In theory, Chinese developers like Biren, Huawei, Moore Threads, and others that are blacklisted and unable to use TSMC's chip manufacturing or packaging services could become some of the biggest beneficiaries of SPHBM4, potentially even more than US companies. First, the shorter chip edge interface perimeter (shoreline) directly benefits chips manufactured using older process nodes, as it allows packaging more computing power without sacrificing memory bandwidth or capacity. Second, Chinese OSATs (Outsourced Semiconductor Assembly and Test providers) currently do not offer CoWoS-like technology, so eliminating the interposer and using advanced organic substrates is an advantage. However, SPHBM4 still requires HBM4 DRAM stacks, which currently only Samsung, SK hynix, and Micron can produce, while China's CXMT can barely produce HBM2E. Furthermore, building a 46 GT/s PHY is very difficult and may be challenging for Chinese IC developers. Nevertheless, assembling SPHBM4 packages on organic substrates is arguably more aligned with China's existing manufacturing base, and if local DRAM manufacturers eventually develop competitive HBM4-class memory, SPHBM4 could significantly narrow the country's remaining infrastructure gap.

JEDEC's SPHBM4 appears to be a promising standard that, due to lower integration costs, may be able to cover a broader range of applications than HBM4 itself. Nonetheless, HBM4, HBM4E, and C-HBM4E will maintain their performance leadership, making them the preferred choice for flagship AI accelerators in the coming years.