Promises, potential pitfalls of software-enabled flash
An open source project is aiming to upend how storage architecture runs, but for developers to see potential gains, they'll have to invest in both hardware and software.
Software-enabled flash moves beyond old protocols of HDDs to a new open source, flash-native API that is focused on redefining the host and NAND relationship for better efficiency.
For flash drives to work in arrays, they need to work alongside or replace HDDs -- the most common storage media in the enterprise -- and use HDD protocols and interfaces, which aren't flash performant.
"There are a lot of capabilities that are possible with flash that we can't get to when it is trapped in the limitations of HDDs," said Scott Stetzer, vice president of technology at Kioxia America, speaking on behalf of the Linux Foundation.
Software-enabled flash (SEF), a new interface aimed at hyperscalers and designed by SSD and flash manufacturer Kioxia, is built to enable better communication between flash storage and host devices such as server CPUs. Kioxia began developing SEF in 2020; in 2022, Kioxia donated the project to the Linux Foundation.
The promise behind SEF is to provide users with more flexibility to define and control how to best use their flash storage, according to Thomas Coughlin, president of Coughlin Associates.
"[Users being able to] manipulate the drive behavior and functionality associated with the storage device could mean a lot," Coughlin said. But for SEF to work, he said, users will also have to adopt new SEF hardware, which could affect how quickly and widely the technology is adopted.
In August at Flash Memory Summit 2022, Kioxia showcased an E1.L EDSFF quad-level cell drive made specifically for SEF. The drive, which is not yet generally available, uses SEF in place of any flash management and DRAM requirements. Now, the technology is in search of users -- another reason adoption might be affected.
"I think they could be more definite in terms of why this matters," Coughlin said. "The devil is in the details of what people can actually do with [SEF]."
Switching mindsets, gaining benefits
With SEF, software developers could gain control of tenant isolation, latency control, data placement and flash migration, all of which is becoming increasingly important as vendors try to improve storage performance and efficiency, such as with computational storage, Coughlin said.
The idea is to completely redefine the relationship between solid-state storage and the host itself.
Scott StetzerVice president of technology, Kioxia America
Handing over control of flash's functionality and behavior to developers means upending the current model of allowing SSD controllers to act independently and could potentially cause delays, according to Stetzer.
"The idea is to completely redefine the relationship between solid-state storage and the host itself," he said.
With SEF, developers can run not only multiple applications, but different storage types on a single drive. They can then deploy a single SSD type and customize it to fit their needs.
The flexibility might enable developers to find ways to use stranded or siloed capacity. This is important for data centers, according to Jim Handy, a semiconductor and SSD analyst at Objective Analysis. Eliminating silos increases efficiency, which in turn lowers costs, he said.
"In the end, it comes down to a cost thing, whether [SEF] will catch on," Handy said.
Moving beyond NVMe
SEF is an attempt to fix a problem the storage industry has faced since the introduction of SSDs more than 40 years ago. Flash storage was used on SSDs that came out about 20 years after HDDs, but were shoehorned to fit into the architecture already designed for the HDD and its interfaces.
The NVMe protocol, the first flash hardware interface, helped overcome some of the limitations, particularly around performance. But the interface, while designed for flash, was based on an HDD template.
"The software still addressed the flash the same way it would address a hard drive," Handy said. SEF aims to change that and remove the hard drive legacy approaches altogether, he said.
Like software-defined storage, SEF uses software to find new ways to more efficiently use hardware. But that's where the similarities end, according to Stetzer.
SEF focuses on gaining full utilization of the flash at a developer or applications level. Software-defined storage improves entire array storage utilization, while SEF goes into the individual drive level, Stetzer said.
Kioxia demonstrated its first software-enabled flash SSD this year.
Roadblocks to adoption
Sebastien Jean, CTO of SSD controller company Phison, isn't sold on SEF yet. The bulk of the cost of an SSD is the NAND, he said. Removing management from the controller or SSD RAM doesn't save significant costs and shifts this management burden to the server.
"The cost of DRAM moved to the server, the SSD compute moved to the server," Jean said. "The question becomes, 'Could that server have done something more useful that generates potentially more revenue instead of handling infrastructure capabilities?'"
Handy doesn't see SEF shifting the burden so much as freeing the host. Currently, the host and SSDs don't speak the same language; SEF changes that so that they can work more synchronously, he said.
But Handy does see SEF facing significant challenges ahead. The technology currently requires specific hardware, shifts commands between hosts and drives, and lacks specific use cases. But those barriers to entry are easier to overcome than the challenges of getting companies to adopt new software, he said.
"Field-proven software may end up jumping through all kinds of hoops to work with a system. SEF could take those hoops away, but people will worry about some bug creeping in and causing headaches," Handy said.
Adam Armstrong is a TechTarget Editorial news writer covering file and block storage hardware and private clouds. He previously worked at StorageReview.com.
