As AI transitions from simple chatbots to autonomous agents, the demand for massive context windows has pushed standard DRAM and on-chip memory to their physical limits. The OceanStor M900 addresses this by utilizing a UnifiedBus network to create a global, multi-tier cache. This architecture extends the KV cache—the data generated during model inference—from localized memory into SSDs, allowing a single cluster to reach a capacity of 64 PB.

The system integrates the CPU, network controller, and NAND controller to bypass traditional protocol conversion. This integration reduces access latency to 60 microseconds, a 90% improvement over existing architectures. With an aggregate bandwidth of 40 TB/s, the setup effectively doubles token throughput while halving the time required to generate the first token in complex inference tasks.

Beyond performance, the hardware focuses on the economic sustainability of large-scale AI. Its KV-aware adaptive storage technology intelligently manages data lifecycles, enabling up to 24 drive writes per day. This approach extends SSD endurance by 16 times, significantly curbing maintenance costs and hardware replacement cycles in high-intensity production environments.