Memory Scaling for Artificial Intelligence Systems

Date30 Jul 2026
Read3 min
Memory Scaling for Artificial Intelligence Systems
The meteoric rise of Large Language Models (LLMs) is redefining the benchmarks for data throughput and storage density. Legacy bottlenecks in data transmission have emerged as the primary hurdle to scaling AI infrastructure effectively. Kioxia’s new CM10 series aims to dismantle these barriers by implementing the PCIe 6.0 standard alongside advanced thermal management techniques. This solution signals a strategic shift toward the deep integration of storage systems within the unified memory hierarchy of modern high-performance servers.

The enterprise storage industry is hitting a critical inflection point where data transfer speeds are no longer merely a specification, but a fundamental survival factor for AI infrastructures. Kioxia is taking a decisive leap forward with the introduction of the CM10 series—the first solid-state drives based on the PCIe 6.0 standard. At the heart of these devices lie tenth-generation BiCS FLASH TLC NAND chips boasting an impressive density of 322 layers. This extreme layering enables massive storage capacities, ranging from a baseline of 1.6 TB to an expansive 61.44 TB.

Kioxia’s technical strategy employs a clear segmentation of models based on operational scenarios. For read-intensive workloads, specialized versions with a rating of one Drive Write Per Day (DWPD) are provided; for mixed-use workloads, more durable variants rated for three full write cycles are available. While the manufacturer is currently keeping exact IOPS and raw throughput figures under wraps, the claimed performance gains are promising: sequential read speeds have increased by 92% and random reads by 85% compared to the previous CM9 generation.

Significant attention has been paid to the physical implementation of these devices. The E3.S and E1.S (9.5mm) form factors are engineered to meet the rigorous demands of modern data centers. In ultra-dense hardware deployments, traditional air cooling is often insufficient; consequently, these models support direct liquid cooling. This prevents thermal throttling and ensures peak performance even within the most heavily loaded nodes of an AI cluster.

Regarding the hardware-software stack, the CM10 leverages the latest NVMe 2.1 specifications and the Open Compute Project (OCP) Datacenter NVMe SSD Specification 2.7. A key technological addition is support for NVMe Flexible Data Placement (FDP), which optimizes data placement to minimize latency. Security is integrated at a deep architectural level: beyond standard self-encryption and FIPS 140-3 compliance, the devices support SPDM 1.4 attestation and—crucially for long-term viability—post-quantum cryptography in accordance with CNSA 2.0 recommendations.

Perhaps the most compelling aspect of the series is its synergy with the NVIDIA ecosystem via CMX technology support. This concept allows flash memory to extend the memory hierarchy beyond the expensive VRAM of GPU accelerators. In the context of Large Language Models (LLMs), this unlocks the ability to store massive context caches, which is critical for models with vast parameter counts and extended context windows.

It is worth noting that the series is not monolithic. The model in the classic 2.5-inch form factor stands as an exception: it is limited to the PCIe 5.0 interface and utilizes eighth-generation BiCS FLASH TLC memory. This decision was likely driven by the need to ensure backward compatibility with existing infrastructure where the transition to PCIe 6.0 has not yet become ubiquitous.

Currently, these devices are in a closed testing phase for key customers, allowing Kioxia to refine stability before a full-scale market rollout.

Tala knows • The use of materials from this website is permitted solely on the condition that an active, direct, and search-engine-friendly hyperlink to the original source is included. The link must be clickable and placed directly within the body of the publication — either before or after the borrowed text. Any copying, reproduction, or citation of the content without complying with this condition will be considered a violation of copyright.
© 2007 – 2026 Tala Knows LLC