> **来源：[研报客](https://pc.yanbaoke.cn)** ```markdown # Total Recall: How AI Is Supercharging Memory Demand ## Core Content Summary This report explores the transformative impact of AI on the memory market, highlighting how the shift to AI-driven computing has created a surge in demand for high-bandwidth memory (HBM) and other advanced memory solutions. It outlines the historical evolution of memory technologies, the current state of the market, and future trends in memory innovation. ## Key Takeaways - **AI is driving memory demand**: While the AI platform shift has been centered on compute, the memory market is experiencing renewed growth due to increasing performance bottlenecks caused by memory bandwidth limitations. - **Memory bandwidth is the key AI bottleneck**: AI models, especially large language models (LLMs), require moving vast amounts of data with minimal computation, making memory bandwidth a critical performance factor. - **HBM is the solution for AI accelerators**: High-bandwidth memory (HBM) improves data access speed for AI accelerators by integrating memory stacks with the processor via silicon interposers and TSVs (through-silicon vias), offering much higher bandwidth than traditional DDR5. - **HBM is becoming more profitable for memory vendors**: HBM delivers higher ASPs and gross margins compared to traditional DDR DRAM, prompting vendors like Micron, SK Hynix, and Samsung to shift focus toward HBM production. - **DRAM remains essential but faces pressure**: Despite the rise of HBM, high-performance non-HBM DRAM is still in demand for AI clusters that include CPUs, NICs, DPUs, and storage controllers. Traditional DRAM is seeing significant ASP growth due to limited capacity and high demand. - **NAND storage benefits from AI inference growth**: As inference becomes the dominant AI workload, there is growing demand for NAND storage to support persistent and reusable data like model weights and KV cache. - **Memory controllers and interfaces are evolving**: Memory controllers are becoming more sophisticated, managing heterogeneous connections and integrating compute-adjacent logic. This trend allows vendors with strong IP and firmware to capture more value. - **Next-gen memory solutions are emerging**: Innovations like compute-in-memory, neuromorphic chips, high-bandwidth flash, and resistive RAM (ReRAM) are being developed to address the memory wall. Protocols like Compute Express Link (CXL) are also enabling faster memory interconnects. ## Main Points ### 1. Memory Bottlenecks in AI - AI performance is increasingly constrained by memory bandwidth, not compute power. - The "memory wall" refers to the growing performance gap between compute and memory, with compute FLOPs increasing faster than memory bandwidth. - HBM is a critical innovation that helps mitigate this issue by providing significantly higher bandwidth than DDR5. ### 2. HBM: A Game-Changer for AI - HBM is a type of DRAM that integrates memory stacks with the processor, using wide interfaces and TSVs for faster data access. - HBM4 offers double the memory bus capacity (2,048 bits) and higher pin speeds, significantly improving memory bandwidth for AI infrastructure. - HBM is becoming a key revenue driver for memory vendors, with higher margins and a shift in capacity allocation. ### 3. DRAM and NAND: Still Vital in AI Ecosystem - High-performance DRAM is still required for AI clusters that include CPUs, NICs, DPUs, and storage controllers. - NAND storage is gaining traction due to the need for persistent, large-scale model data, especially for inference workloads. - AI adoption is driving demand for enterprise SSDs (eSSDs), with a projected 35% CAGR from 2024 to 2030. ### 4. Memory Hierarchy and Technologies - The memory hierarchy includes registers, caches, main memory (DRAM/HBM), and storage, each with different trade-offs between speed, capacity, and cost. - SRAM is used for internal caching, while DRAM is the standard for main memory. HBM offers a balance between bandwidth and capacity. - pSRAM (pseudo-static RAM) is a niche variant that mimics SRAM behavior without the need for complex refresh mechanisms, suitable for edge devices. ### 5. Future of Memory: Next-Gen Solutions - Emerging technologies like compute-in-memory, neuromorphic computing, high-bandwidth flash, and ReRAM are being developed to overcome memory limitations. - CXL is an emerging protocol that allows for higher-bandwidth memory interconnects, enabling shared memory access between CPUs and accelerators. ## Key Companies Covered - **Micron**: Shifting focus to HBM and terminating its consumer flash business. - **Rambus**: A leader in memory interface technologies. - **Silicon Motion**: A key player in NAND and flash storage. - **Nvidia**: Using SRAM in its Groq acquisition for low-latency inference at the edge. - **SK Hynix**: Leading HBM3 market and expanding HBM production. - **Samsung**: Restructuring DRAM roadmap to prioritize HBM. - **Other innovators**: Anaflash, Astera, BrainChip, Cerebras, Credo, Crossbar, EnchargeAI, SynSense, Syntiant, Upmem, and Weebit Nano are developing next-gen memory solutions. ## Conclusion The AI revolution is reshaping the memory landscape, with HBM and other high-bandwidth solutions becoming central to AI infrastructure. Traditional DRAM and NAND storage are still in demand, but their margins and availability are under pressure. As AI adoption continues, the memory market is expected to evolve rapidly, with next-gen technologies and more sophisticated controllers playing a pivotal role in the industry's future. ```