Longsys is tackling the memory wall for local Large Language Models through a partnership with AMD. By integrating its proprietary AIDIMM high-bandwidth memory and iSA scheduling engine, the company demonstrated a system capable of running 122B-parameter models on just 64GB of memory. The AIDIMM module features a 256-bit interface and peak bandwidth of 307.2 GB/s, while the iSA engine manages resource allocation through predictive prefetching and MoE offloading to maintain efficiency during complex inference tasks.

For the PC sector, Longsys introduced a 5nm Storage Processing Unit that enables DRAM-less PCIe Gen5 SSDs to hit 14.8 GB/s read speeds while consuming under 6.3W. This controller design includes lossless compression to effectively double usable storage capacity. Meanwhile, in the mobile space, the company’s HLCache technology aims to optimize DRAM usage on budget devices. Tests on a Google Pixel 7a showed that offloading cold data to storage allowed the device to keep 28 apps in the background—a 64% increase—while facilitating the local operation of 20B-parameter models in a compact 22x22mm footprint.