Google Cloud storage offerings bolstered for AI apps
Storage for Google Cloud now supports many features akin to AWS' catalog for AI app development, but could lead to cloud lock-in as best practices and strategies emerge.
New products coming to Google Cloud's storage offerings aim to support AI application speed and availability, but buyers leaning on a single cloud might find that this is another form of vendor lock-in.
Google's two latest storage offerings include Rapid Storage, for submillisecond reads and writes to object storage, and Google Cloud Managed Lustre, a managed parallel file system built on DataDirect Networks' technology.
Such offerings show that Google and other cloud hyperscalers aspire to become the dominant cloud platform for AI creation, said Brent Ellis, an analyst at Forrester Research. Many storage capabilities previously sold separately or developed in-house by enterprises are becoming part of cloud portfolios.
Google's latest storage products are similar to AWS' S3 Express One Zone and other managed storage services, he said. However, relying on a vendor's specific offerings could lead to customers getting locked in over time, even as better and more cost-effective practices emerge.
"The way enterprises consume storage is about to change significantly," Ellis said. "The underlying storage that's coming to market is going to be much more intelligent and data aware. Storage primitives are evolving to meet AI."
What's new
These new services are for optimizing costs and increasing AI workload performance, Google said through blog announcements during the Google Cloud Next conference April 9-11.
Rapid Storage will provide submillisecond random read/write latency by collocating the object storage bucket with the GPUs and TPUs in the same data center zone, the company said. Customers can mount Rapid Storage buckets using Cloud Storage Fuse, an open source API for using object storage within file storage systems.
I thought they were coming up to speed on S3 Express One Zone, [but] this is beyond that.
Ray LucchesiFounder and president, Silverton Consulting
Google's proclaimed submillisecond speed makes it faster in performance than AWS' S3 Express One Zone, which targets around 2 to 10 ms, said Ray Lucchesi, founder and president of Silverton Consulting.
"I thought they were coming up to speed on S3 Express One Zone, [but] this is beyond that," Lucchesi said. "They're making steps to be more competitive with Amazon."
The technology for that speed comes from using gRPC, Google's open source and stateful call framework, over the standard RESTful API, Lucchesi said.
REST is the more common connective API used by most of the modern web, but it is stateless, he said. REST is useful for the massive scale and demands of object storage, but lacks the analytical capabilities or durability of the stateful gRPC. Stateful applications keep track of data and past actions -- an important aspect of performant AI and machine learning training.
The Rapid Storage launch is a commercialized version of Google's own storage service, Colossus, the file system on which almost all the company's products operate, according to Google.
Since object storage services have evolved over the decades to handle the scale of modern apps and data traffic, they've become a common storage primitive for enterprises to build on, Lucchesi said. Object capabilities will ultimately need to evolve over time to handle new AI workloads and demands.
"Object is becoming a common [storage] back end for functionality because of the scale," he said. "It can scale to the petabyte and exabyte scale without blinking."
Consider your options when choosing file or object storage.
The Google Cloud Managed Lustre, available in preview only, is a fully managed parallel file system built on DataDirect Networks' ExaScaler technology and provided by DDN as well. The service integrates with other Google Cloud capabilities and targets high performance for HPC and machine learning environments.
AWS also offers a managed Lustre service, Amazon FSx for Lustre, alongside other file systems like OpenZFS.
What's next
Performant storage meant for AI apps could result in new development paradigms, but best practices, capabilities and needs are still very much in flux, said Roy Illsley, an analyst at Omdia, a division of Informa TechTarget.
Many announcements around Google Cloud Next this year were about increasing the adoption of Google's Vertex AI machine learning platform and Gemini large language model (LLM), Illsley said. Google remains in the fight for enterprise supremacy against AWS' Bedrock or Microsoft's significant investment partnership with OpenAI.
"OpenAI is the clear leader in the market, but others are following," Illsley said. "We're in this stage of AI evolution where it's like the [internet] browser wars of the '90s."
None of these AI platforms or LLMs has emerged as a clear victor for enterprise AI. Still, buyers should prioritize their ability to jump between clouds, systems and services to avoid a lock-in situation down the line with AI, Illsley said.
A current example of IT rallying around a single technology is VMware's virtualization, which has led to some regret following its acquisition by Broadcom, he said.
"The killer app for AI hasn't been developed yet," Illsley said. "You're not locked in and can move between a few [vendors] with some effort."
Most enterprise AI implementations might not involve any of the larger LLMs or frontier models, according to Ellis. Industry-specific model development through open source alternatives or specialized AI will be done locally by the enterprise and adapted to run elsewhere, which could include hyperscaler clouds, other private clouds and on-premises.
"The hyperscalers were put on their back foot [about] what they thought they would take by right," Ellis said. "This is another chapter in the story of how AI will not just be the providence of a few foundation model builders."
Tim McCarthy is a news writer for Informa TechTarget covering cloud and data storage.
