New enterprise PCIe 4.0 SSDs from Kioxia, Samsung double speed

NVMe 1.4 updates

NVMe 1.4 adds improvements in performance, quality of service, scalability, security, and congestion management. Matt Hallberg, senior product marketing manager at Kioxia, also highlighted namespace granularity to help customers use all the capacity of their drives, and persistent event log to expose deeper drive history for troubleshooting and debugging at scale.

The U.3 specification enables NVMe PCIe, SAS and SATA SSDs to use a single backplane/connector pinout, midplane/expander board and controller. By contrast, U.2 requires a separate backplane, midplane and controller for each of the three protocols.

"Today, when you go to configure servers on OEM sites, you've got to choose carefully what kind of storage you want to have inside of the drive. You can have all NVMe, or you can have all SAS/SATA," Hallberg said. "And, on certain servers, you're able to get one specific backplane for NVMe drives and one specific backplane for SAS/SATA drives."

Users eventually will be able to buy a single server that supports NVMe PCIe, SAS and SATA SSDs, since any U.3 bay could accept any of the three drives. Hallberg said customers could configure cold, warm and performance-sensitive tiers within the server, using a mix of drive types. The SSDs would be hot-swappable, if the U.3 adapters support the necessary firmware.

Kioxia's U.3 support so far is largely for enterprises that want to future-proof their storage purchases. Hallberg said Broadcom and Microsemi are shipping tri-mode-enabled host bus adapters (HBAs) for NVMe PCIe, SAS and SATA SSDs, but many server and storage vendors are still evaluating U.3 support. Customers that don't have U.3-compatible gear can use Kioxia's CM6 and CD6 SSDs with older U.2/SFF-8639 connectors, Hallberg noted.

Nascent PCIe 4.0 ecosystem

PCIe Gen 4 product support is also at an early stage. Intel has not shipped PCIe Gen 4-compliant server processors, so the main option is AMD's EPYC CPU, Hallberg noted. Server vendors that support PCIe Gen 4, with AMD Epyc processors, include Dell, Lenovo, Quanta Cloud Technology (QCT) and Supermicro.

"Although few CPUs ship today with a PCIe 4.0 interface, that is going to change relatively quickly. When it does, the migration of NVMe SSDs to PCIe 4.0 should be pretty swift," said Jim Handy, general director and semiconductor analyst at Objective Analysis.

The 2019 enterprise SSD breakdown was 51% PCIe, 27.5% SATA and 21.6% SAS based on a total shipment of 51 exabytes, according to Don Jeanette, vice president of Trendfocus. Five years out, Jeanette forecasts PCIe at 70.2%, SAS at 19.1% and SATA at 10.7%, with enterprise SSD shipments totaling 178 exabytes.

The vast majority of PCIe demand to date is hyperscale.

"The vast majority of PCIe demand to date is hyperscale," Jeanette said. "At financial institutions, it's a 'nice to have,' at a premium."

Kioxia's CM6 and CD6 SSDs will work in a PCIe Gen 3 system, but they would operate at PCIe 3 speeds. Users need a server with a PCIe 4-conformant processor and chipset to gain the performance benefits. They can use add-in cards to plug directly into a PCIe slot connected to the CPU, or they could use PCIe 4 SSDs that directly connect to the motherboard via a cable, Hallberg said. The nondirect connection option is a PCIe 4-compatible HBA with a PCIe 4-compliant backplane or switch, he said.

PCIe 4.0 performance benefits

The performance benefits with PCIe 4.0 can be substantial. A demonstration at Microsoft Ignite 2019 showed Kioxia's CM6 PCIe 4.0 SSDs could deliver more than double the storage I/O over its CM5 PCIe 3.0 SSDs. The tests used the same virtualized application running on an Azure Stack HCI two-node cluster with QCT server hardware, AMD's chipset and CPU, and Broadcom network interface cards.

Kioxia claims the CM6 enterprise SSDs can deliver up to 1.4 million IOPS for random reads, 6.9 GBps for sequential reads and lower read and latency over its CM5 predecessor by 40% and 67%, respectively. The CM6 has single-port and high-availability, dual-port options at capacities ranging from 800 GB to 30.72 TB, doubling the maximum capacity of the CM6 predecessor.

SSD manufacturers and storage system vendors are starting to ship drives that use dense quad-level cell (QLC) NAND flash to achieve capacities approaching 40 TB. But the new high-performance Kioxia enterprise PCIe 4.0 SSDs are built on 96-layer, higher-endurance triple-level cell (TLC) 3D NAND flash, the company's largest capacity 512 gigabit die, and three printed circuit boards, Hallberg said.

"We have evaluated QLC, and in my mind, it's not a question of if but when," said Alvaro Toledo, vice president of SSD marketing and product planning at Kioxia. "But, right now, our belief was that the use cases that we were addressing with our SSDs were better served with TLC."

Kioxia's new single-port CD6 PCIe 4.0 SSDs are available at 15.36 TB, double the capacity of the prior CD5. Just like the enterprise CM6, the data center CD6 has options for three drive writes per day (DWPD) and 1 DWPD, unlike the CD5's sub-1 DWPD. Hallberg said OEMs and users told him their main cause for hesitation with the NVMe CD5 over SATA SSDs in servers was endurance, since SATA SSDs generally offer 3 DWPD and 1 DWPD options.

The Kioxia CD6 supports the same die failure recovery, double-parity protection and security features as the enterprise CM6. But the CD6 reduces other features, such as overprovisioning and the number of namespaces, and targets read-intensive workloads such as read-centric databases and content delivery networks, Hallberg said.

Kioxia is shipping the new CM6 and CD6 SSDs to select OEM customers for qualification. The drives are due to become generally available in mid-March, but enterprises typically buy drives with their OEM-branded systems. OEMs determine SSD pricing, but Toledo said the CM6 enterprise drive aims to compete with SAS and the CD6 with SATA. He expects the new SSDs will be available as part of PCIe Gen 4-enabled server launches later this year.