Key Findings
Marvell is targeting the key-value (KV) cache bottleneck in AI inference to prevent redundant computations during long conversations in reasoning models and agentic applications. Simulation-based analysis indicates that a shared-memory configuration can support approximately six times as many concurrent conversations at a comparable time-to-first-token using the same compute resources. These results are based on workload-specific modeling and pre-silicon estimates.
Technical Details
The PFMA is a rack-mountable appliance featuring 128 standard DDR5 DIMM slots and approximately 1 TB of HBM3e, which functions as a cache for the larger DDR5 pool. The system utilizes sixteen Photonic Fabric Memory Modules connected through a passive fiber shuffle to create a unified shared address space. Marvell provides two integration options:
- PF-NIC card: Uses PCIe Gen6 x16/CXL 3.x for server connectivity.
- PF-Chiplet: Integrated directly into an XPU package.
The practical optical reach for memory sharing across racks is identified as 25–30 meters to balance physical separation against propagation delay. Marvell positions the chiplet-attached PFMA as a G1.5 tier below local GPU HBM, while the server-attached version serves as a G2.5 tier between system DRAM and storage.
Background & Context
The technology, acquired through Celestial AI in February 2026, expands Marvell’s memory strategy beyond Bravera storage controllers and Structera CXL expansion. The goal is to keep reusable KV cache closer to compute than SSD-based offloading, allowing inference software to flexibly assign workloads to different processors without repeating prefill work.
Significance & Outlook
This architecture shifts KV cache placement from a local component to a cluster architecture decision. However, realized performance remains dependent on cache hit rates, DDR bandwidth, fiber length, and the ability of serving software to manage compatible cache entries. The PFMA represents a move toward treating memory as a shared cluster resource rather than a standalone server component.
Source: https://convergedigest.com/pecc-summit-marvell-optical-memory-kv-cache/
Get our weekly technology intelligence — free
Receive an infographic that lets you judge at a glance whether each field’s analysis report is worth reading.
Subscribe Free — Weekly Tech Intelligence
By subscribing, you’ll receive Troy-Technical’s weekly technology intelligence newsletter.
- Your email and selected fields are used only to deliver the newsletter.
- We never share your information with third parties.
- You can unsubscribe anytime via the link in each email.
See our Privacy Policy for details.
Takes about a minute · Unsubscribe anytime
