DDN has created a new software-defined object storage product, Infinia, for AI and HPC secondary storage. The tech can also be applied to some enterprise storage use cases.
DataDirect Networks Storage has developed a new product to simplify storing and managing distributed data at a time when AI is generating large amounts.
DDN's Infinia is a software-defined storage product that the vendor has been developing for the past eight years. It is designed for AI model development and high-performance computing (HPC) storage. Infinia will be released later this year initially as an S3 object store, with a parallel file system to be added in 2024. DDN's Infinia aims to address issues such as hardware dependencies and complexity that arises from that, including performance limitations, compatibility, multi-tenancy and limited metadata capabilities.
DDN's main product is its parallel file system, ExaScaler, known for its use in HPC and AI workload use cases, according to Mitch Lewis, an analyst at Futurum Group.
The Infinia system is for things that are HPC-adjacent, such as AI, where performance is still important.
Mitch Lewis Analyst, Futurum Group
"The Infinia system is for things that are HPC-adjacent, such as AI, where performance is still important," he said.
Infinia is aimed at unstructured data that requires both large capacity and high performance, putting the product in competition with Vast Data and Pure Storage FlashBlade, Lewis said.
Built for AI and HPC, but not exclusively
DDN's customers are primarily users of HPC, although now 30% to 40% of its revenue comes from pure AI work, according to the vendor. DDN considers its AI400 based on Nvidia DGX SuperPod and ExaScaler as its "inner ring" of AI and HPC storage, housing data currently in use, and the Infinia to be the "outer ring," which stores data that is not currently being processed. This cooler tier of storage would also appeal to other, more traditional enterprise markets, including storage for medical data and backup targets for VMs, according to DDN.
By providing a distributed, cloud-like storage tier, Infinia resembles a scale-out secondary storage system -- but one unlike those currently on the market, according to Mike Matchett, an analyst at Small World Big Data.
"This looks like HPC-class distributed storage, creating a new category of storage that's designed to surround core HPC infrastructure and extend high-end data services out to support increasingly complex workflows that involve not only AI, but also distributed databases, research analytics, simulations and other demanding applications," Matchett said.
Although Infinia has some secondary storage attributes, it is more performant than simple cloud storage, and more cost effective for wider primary storage use cases than a core parallel file system such as ExaScaler, Matchett said.
Management, governance and multi-tenancy
Infinia comes with several features, including automation for configuring and installing and dynamic data provisioning, which allows a tenet to be created for a specific demand. DDN's multi-tenancy capability is unique, according to Matchett.
Other storage systems can support multi-tenancy but in a more static manner with configured partitioning, he said.
"That requires a storage admin to keep going in there and tuning, dialing knobs and resigning space," Matchett said.
Infinia built the concept of tenets and sub-tenets into the keyspace for full multi-tenancy without the need for logical partitions, DDN said. This allows for better sharing of resources as users start with a baseline instead of a cap on what can be used.
Software-defined, sold in a box
Infinia can run on a CPU, memory, a network and flash storage, but DDN does recommend using an appliance with a PCIe Gen 5 AMD CPU, and 12 QLC SSDs of up to 60 TB each for 720 TB raw capacity for best performance.
The double message is common with software-defined companies and a long-standing critique of such tech, Matchett said.
"A lot of software-defined companies come out and say, 'We're software-defined. You could run us on anything, but we really only ship this appliance, and this is the one we recommend,'" he said.
However, the hardware is optimized for best results and gives built-in future proofing, enabling customers to move the software to new, upgraded hardware.
Using this container-based approach reflects a trend in array development, according to Lewis.
"It makes it easy to deploy, portable and fits in with everyone's containerized workloads and applications," he said.
Adam Armstrong is a TechTarget Editorial news writer covering file and block storage hardware, and private clouds. He previously worked at StorageReview.com.
