Gorodenkoff - stock.adobe.com
MinIO extends Snowflake analysis to any data location
MinIO scored a partnership with Snowflake to bring the cloud data warehouse analytics nearly everywhere a user's data resides.
MinIO's recent decision to add external table support for Snowflake analytics lets customers who use the company's object storage run these analytics wherever their data resides, versus only in data stored in a Snowflake location.
Snowflake, a cloud data warehouse, currently runs in the three major public cloud services, AWS, GCP and Azure. Snowflake offers internal tables for data, its own AWS S3 object store, and external tables, object store for data of the customer's choice, through one S3 API. Now, with MinIO Snowflake can query any external table on MinIO, under any Kubernetes wrapper or cloud wrapper.
The typical Snowflake customer deals with huge amounts of data -- terabytes to petabytes. It can be painful, in terms of time and cost, for these users if they must move that much data into the cloud to take advantage of Snowflake's analytic capabilities, explained Ray Lucchesi, president of Silverton Consulting in Broomfield, Colo.
"New customers don't have to pay the price of moving all the data around if it is sitting in a S3 object storage," Lucchesi said.
Making Snowflake more effective and efficient
What makes all this possible is MinIO's support for external tables. Externals are flat files that let users store data outside of a database. This can be done instead of staging data in internal tables for Snowflake to query.
"You could continue to stage [data], or you could query directly on the external table," Lucchesi said.
Options abound
Recently, Snowflake disclosed a partnership with Dell, and quickly thereafter, with Pure Storage Inc. Both partnerships let Snowflake query data that resides in the data center, but they do it in different ways.
Pure's FlashBlade, for example, is an appliance. In this situation, users would run Snowflake on premises or in a colocation facility, not in the public cloud. MinIO does not use an appliance. Dell has a range of options.
Still, if a customer already uses Pure or Dell, it makes sense to use the storage for analytics from Snowflake, Lucchesi said. On-premises usage will most likely provide higher performance, although the cost of the appliance would be higher.
MinIO enterprise pricing remains at $20 per TB, per month. Snowflake offers usage-based pricing and capacity storage paid upfront.
