Memory channel flash storage vs. PCIe flash storage

What is memory channel flash storage and what does it offer that PCIe connected flash doesn't?

Memory channel flash storage, also known as "in-memory storage," is flash storage that inserts or connects directly into the DIMM slots of a server motherboard. Currently, in-memory flash storage is only available from SanDisk as ULLtraDIMM. SanDisk has an exclusive agreement with Diablo Technologies that marries their DDR3 DIMM-based flash storage to SanDisk's Guardian software. PCIe flash connects to the PCIe channel. In-memory flash storage has several advantages over PCIe flash storage.

The number of server DIMM slots greatly exceeds the number of PCIe slots. Take the example of Intel's latest E7-8800 v2 family. A quad socket motherboard delivers 96 DIMM slots. At least 2 DIMM slots per socket are required for DRAM, leaving 88 slots for in-memory storage. That same motherboard has 4, 8 or 16 PCIe slots. Some of those slots must be utilized for network interface cards, storage adapters, clustering or other peripherals. That leaves only a small number for PCIe flash storage cards. There are almost always surplus DIMM slots in a server, whereas PCIe slots are precious and rare. Advantage: In-memory flash storage

Because in-memory flash storage looks like DDR3, writes and reads are performed in parallel. Parallel writes and reads means consistent high performance. Multiple PCIe flash storage cards with parallel writes and reads is still a work-in-progress. Advantage: In-memory flash storage

The general advantage of server flash over external flash storage systems is lower latency. Lower latency comes from the shorter distance from the application to the flash storage. PCIe flash storage has lower latency than embedded SATA or SAS flash SSDs because it has less distance to travel. The SAS or SATA SSDs must traverse an additional controller sitting on the PCIe channel and then the PCIe channel. In-memory flash storage has lower latency than PCIe flash storage because it does not have to traverse the PCIe controller or deal with contention from other cards for bandwidth on the PCIe channel. Advantage: In-memory flash storage

Flash latency consistency is an issue for many high performance applications such as high-frequency trading, life sciences, HPC applications in general, seismic data interpretation, reservoir modeling, fluid/flow simulation, aircraft simulation, 3-D imaging/modeling, and more. In-memory flash storage has tested out to have a much more consistent and smaller latency range (an order of magnitude smaller range) than the best PCIe flash cards. Advantage: In-memory flash storage

Cost per gigabyte tends to be a little higher for in-memory flash storage than PCIe flash storage. Advantage: PCIe flash storage

PCIe flash storage is a more mature technology today than in-memory flash storage, with far more vendor products and choices. There is also more software available written specifically for PCIe flash storage. Advantage: PCIe flash storage.