OpenZFS interest grows alongside rise of unstructured data
With the explosion of unstructured data, lower-cost file systems such as OpenZFS are gaining more attention as they continue to gain support from vendors.
Unstructured data is exploding and traditional storage for it can be expensive, leading some to look into open source alternatives such as OpenZFS.
OpenZFS, an open source version of the zettabyte file system (ZFS) developed by Sun Microsystems, is a file system with volume management capabilities that can be cheaper than traditional storage offerings and added to commodity hardware. A drawback to OpenZFS is its inability to scale out, given the limitation for how much storage can fit behind each controller.
The latest arrow in the quiver for OpenZFS is targeted at overcoming that problem. In August, Arcitecta, an Australian data management company, released Scale-Out ZFS with Mediaflux that combines its data management software with OpenZFS.
The new product comes at a time when unstructured data generation and storage continues to accelerate due to the onset of generative AI and the proliferation of IoT devices, such as sensors and cameras. ZFS users are already seeing interest in the open source file system on the rise, and storage experts think it could be time for enterprise storage to look beyond proprietary NAS, often used to manage unstructured data today, to OpenZFS.
The demand for more ZFS-based offerings has escalated in recent years, according to Nikita Sherbina, co-founder and CEO of AIScreen, a digital signage software company that uses OpenZFS for its storage needs. In the last year alone, Sherbina observed a 30% increase in demand for ZFS-based systems from customers, due to its secure data handling.
"The appetite for scale-out ZFS in the enterprise is substantial," Sherbina said.
Unstructured explosion
Unstructured data will triple in the next five years on premises, at the edge and in the public cloud, according to a recent report from Gartner.
The world is going unstructured.
Ray LucchesiPresident, Silverton Consulting
"The world is going unstructured," said Ray Lucchesi, president of Silverton Consulting.
He pointed to interest in large language models that underpin generative AI applications such as ChatGPT as an example of why unstructured data is growing so rapidly.
Structured data stored on block storage such as databases will remain important to the enterprise, but unstructured data will become the dominant data type in the next couple of years, and companies will need new ways of storing it, Lucchesi said.
That ever-increasing size of unstructured data makes choosing a scalable system integral, according to Vinika Garg, COO of Webomaze, an Australian SEO agency. Having used ZFS as a product manager and UX designer, she sees the scalability of ZFS as a major point of interest.
"It offers an adaptable and affordable method for addressing the exponential increase of data in the current digital environment, making it an appealing option for companies looking to secure their storage infrastructure for the future," Garg said.
Arcitecta's new Scale-Out ZFS with Mediaflux layers its flagship database and file management system onto ZFS systems, which can help scale OpenZFS capabilities to multiple systems across a single environment and in different data centers, according to Ken Clipperton, an analyst at Data Center Intelligence Group.
"[Arcitecta] has identified a really good target for solving the problem of data migration for organizations," he said.
Arcitecta works with both existing and new ZFS environments and can facilitate a method of data migration, which is a challenge due to large volumes of data consuming resources and time, Clipperton said.
Scale-Out ZFS with Mediaflux can be a cost-effective alternative to traditional enterprise NAS technologies, Clipperton said. Arcitecta stated that its product can be used on commodity hardware with lower hardware and licensing costs than proprietary file systems.
Lucchesi agreed that OpenZFS can be an alternative to proprietary NAS, but cautioned large enterprises to consider what it means to use true open source systems.
"The biggest question from the enterprise perspective is who is going to support it [24/7]," Lucchesi said.
While there will be support from the open source community, it takes a company that is willing to take it on and offer 24/7 support, regardless of deployment, he said.
OpenZFS comes with challenges
While it provides certain benefits, OpenZFS introduces other challenges beyond support, Lucchesi said. OpenZFS doesn't have native object storage support, and performance can be dependent on the underlying hardware it is deployed on.
With eight years' experience in software development and programming, Ryan Faber, founder and CEO of Copymatic, an AI copywriter service, said ZFS can be resource-intensive. Users can expect to increase their memory when using ZFS.
"[ZFS] uses too much memory for primary use only, let alone making copies of files," Faber said.
As a business scales up its data footprint, this could become increasingly tricky, requiring memory system upgrades, he said.
Adam Armstrong is a TechTarget Editorial news writer covering file and block storage hardware and private clouds. He previously worked at StorageReview.com.
