Microsoft Azure storage team wants simpler flash devices

SAN DIEGO -- Microsoft Azure wants storage vendors to change the way they build flash devices.

Through its participation in the Open Compute Project, the cloud computing giant is partnering to customize "bespoke SSDs" that can readily deploy new generations of NAND flash for multiple tenants.

Michael Cornwell, head of infrastructure for Microsoft Azure storage, spoke of Microsoft's cloud flash strategy last week at the NVMe Developer Days conference.

"A lot of customers want a simple storage device, and yet the device manufacturers are trying to add more and more intelligence into that device. Key-value store, encryption, compression ... all these things drive value, [but add] cost to that device. There is a tug and pull between the customers and the suppliers of the technology," Cornwell said during his conference keynote.

"We see a huge breakdown in how quickly cloud providers need to move based off services, and it's the storage industry that doesn't want to come along with us [at the same pace]," he said.

The NVMe conference included user sessions and technical talks from storage and network OEMs.

NVMe flash is widely considered the successor to iSCSI, although it is not expected to supplant the older protocol. NVMe streamlines the register interface and command set to reduce overhead in the I/O stack.

Microsoft Azure cloud storage is provided in 50 global regions, encompassing more than 100 data centers. With its open source work, codenamed Project Denali, Microsoft Azure storage is imitating the "Facebook model" to spur innovation in storage, Cornwell said. That was a reference to Facebook's PCI Express-based Gen 3 Lightning SSD. 

New data generated by AI, IoT and machine learning causes storage capacity to double every 18 months, Cornwell said. One of Project Denali's tenets is to abstract and isolate NAND to allow cloud providers to integrate flash, enabling them to build hundreds of massively shared services and broader application sets.

"One of our core fundamentals is on how quickly we can innovate. Can we find a problem and deploy fixes and enhancements every week? Only in a nonvolatile environment can you do that. Sometimes, when I deal with a storage provider, it takes me months to get firmware [updates] because of their validation cycles," Cornwell said.

Project Denali is proposing specifications intended to make it easier for any cloud provider to use commodity flash and differentiate by adding their own software and services.

"When we [started] Denali, we had a lot of NVMe [device makers] say, 'You're destroying our value.' But we responded by saying that NVMe is a transport," Cornwell said. "You shouldn't care what we do on NVMe. You should only care about making sure there is standard protocol to enable what you do."

Microsoft Azure storage expanded this year with the launch of a series of managed disk options for cloud customers. Azure Standard SSD service is a cost-optimized performance option for Azure virtual machines. In October, Microsoft started previews of Azure Ultra SSD service, a fault-tolerant option that provides guaranteed low latency and IOPS consistency. Microsoft Ephemeral OS Disk is designed to offer the performance of local storage, with faster boot times.