Kioxia flash storage sews NVMe fabric onto Kubernetes

NVMe-optimized Kubernetes

Kioxia uses native TCP/IP to create an NVMe-oF layer that knits x86 servers as a storage array. KumoScale's API-driven federated architecture maps virtualized storage volumes to physical NVMe SSDs. Clients see the capacity as local storage.

Flash vendors typically run data-plane services on a dedicated array controller, but Kioxia KumoScale uses a different technique. To reduce latency, KumoScale Management delivers data services by making a roundtrip circuit of the network.

"KumoScale is meant for storing logical volumes as physical devices. We see a powerful analogy between the automation we do for storage and the automation Kubernetes does for containers," said Joel Dedrick, a VP and general manager of Kioxia's networked storage software.

Initial versions of the NVMe-oF industry specification were limited to Fibre Channel and Remote Direct Memory Access (RDMA) fabrics, but required specialized host bus adapters and application rewrites. The NVM Express consortium credentialed NVMe over TCP support in 2018. That allows vendors to develop NVMe-oF using standard TCP/IP drivers.

Eric Burgener, a research vice president of storage at analyst firm IDC, said TCP will "democratize" NVMe-oF, but thus far only a handful of storage arrays support it. In addition to Kioxia, startups Lightbits Labs and Vast Data sell storage based on native TCP/IP Ethernet drivers. Pavilion Data is preparing a similar version that works only with its Hyperparallel Flash Array.

"If you think about who needs to implement NVMe over TCP in production, it's a pretty small group," Burgener said. "Telcos and cloud providers [are] good place[s] for Kioxia to start, since those are all web-based infrastructure and they have sophisticated development organizations to implement it. Container-based development is the future, but it's going to take a while to get there."

For performance, the Kioxia flash software avoids remapping of logical and physical storage. It can virtualize local storage volumes for size or performance. Data services get storage from the local volumes. Dedrick said KumoScale delivers performance similar to that of bare-metal NVMe.

"We worked hard to keep the software path clean and short so the latency to your system is a few microseconds," Dedrick said.

Formerly known as Toshiba Memory Corp., Kioxia ranks second to South Korean rival Samsung in the production of NAND flash chips. Kioxia flash products are based on Toshiba's 96-layer BiCS flash memory and SLC NAND flash management. (Toshiba kept control of its branded hard disk drive business). The Kioxia KumoScale software can use TCP/IP or RoCE as an NVMe-oF transport layer.

Toshiba, based in Japan, spun out Kioxia in 2018 in a deal led by Bain Capital. Toshiba holds a 40% stake in Kioxia, which it planned to divest following Kioxia's IPO in October. That stock sale, however, has been postponed indefinitely.