buchachon - Fotolia
Google's Anthos drives new storage and data management API
Google's hybrid and multi-cloud vision for running stateful container-based applications sparks work on a new API for data and storage management.
Storage and data management are critical components of Google's vision for its Anthos platform for building and managing stateful container-based applications that can span cloud and local data centers. But not all of the storage pieces are in place yet.
Anthos -- formerly known as Google Cloud Services Platform -- works by default with on-premises storage through VMware volume plugins, and it also can use Kubernetes volume plugins. However, Google's Anthos currently supports only a beta version of the Kubernetes Container Storage Interface (CSI) to connect to third-party storage systems; the 1.0 release is expected to be available later this year.
The goal of Google's Anthos is to facilitate the portability of stateful applications and their data across hybrid and multi-cloud environments. Google is working with partners and the open source community on a new API to support advanced data and storage management functionality, including snapshots of the containerized application's metadata, configuration and code.
"It's not sufficient just to allow you to access a storage system," said Saad Ali, a senior storage engineer at Google. "We also want to be able to manage the data within those storage systems."
Ali said an alpha version of the new data management API could be available in the second quarter, although no timetable is set for the release. Google expects many customers will use Anthos to modernize and containerize legacy applications on premises and then move them to the cloud. The new API could help with the transition.
Google's Anthos components
Anthos includes cloud and on-premises versions of the Google Kubernetes Engine (GKE) to ease the deployment, management and orchestration of containers across server clusters. GKE for Kubernetes clusters in the cloud and GKE On-Prem provide a software abstraction layer between the applications and the storage and facilitate access to block, file and object storage.
The default storage class for GKE is Google Persistent Disk, which supports block and file interfaces. Other options include Google Cloud Storage for object and Cloud Filestore, the company's managed file system.
By default, GKE On-Prem uses the VMware vSphere in-tree volume driver that ships with Kubernetes to connect to storage. Google built GKE On-Prem on VMware "for a good out-of-the-box experience," Ali said, and customers can provision volumes with any type of storage that VMware vSphere supports.
GKE On-Prem also supports Kubernetes in-tree volume plugins. Ali said when Google's Anthos GKE On-Prem updates to Kubernetes 1.13, users will also be able to provision any storage system that supports Kubernetes CSI 1.0, which became available in late 2018. GKE On-Prem currently supports Kubernetes 1.12 and CSI 0.3.
"You can basically jump into any environment that's running on Kubernetes and have your stateful application request block or file storage and have it dynamically provisioned," Ali said. "As the workload is rescheduled from node to node, that storage moves around with that workload."
The new data and storage management API will expand the CSI and add support for application-level APIs to facilitate snapshots, backups and disaster recovery across clouds and on-premises data centers, Ali said. For instance, the API could help enable a user to take a snapshot of a GKE application on an on-premises cluster, back it up to the cloud and restore it to a different GKE environment in the cloud or another on-premises cluster.
Portability wasn't an issue with stateless applications running in Kubernetes clusters, but it can be challenging now that developers are building stateful applications, potentially with large amounts of data sitting in persistent local storage, Ali said.
Google courts storage partners
Scott Sinclair, a senior analyst at Enterprise Strategy Group in Milford, Mass., said storage is one of the most complex elements that users will face if they run Anthos on premises. He said Google and its storage partners need to provide tools to ease the transition from VMware-based workloads to containerized applications that could run in hybrid or multi-cloud environments.
Stuart Miniman, a senior analyst at Wikibon in Marlborough, Mass., said there have been advances in making storage work with Kubernetes, but more work is needed.
"When I look at the maturity of storage in Kubernetes, we're getting there. But it is not something that is fully ready," Miniman said. "It feels like we've gotten past some of the major hurdles, but I would not say, 'Oh, we know how to deploy containers at scale. We know how to do all the networking pieces. The storage has been baked out in lots of company environments.'"
Miniman urged potential users to test Anthos before jumping in with production data.
"Part of the value of Anthos is I should be able to develop my application and not think about what environment it is," he said. "But if I'm sitting in my data center with an HPE [Hewlett Packard Enterprise] SimpliVity running VMware, and I also have Anthos running in Google Cloud and possibility also in AWS, those storage environments are going to be very different. That's where the devil's in the details on making sure that works consistently."
Startup Robin.io, which sells container-native storage for Kubernetes, worked with Google on initial demos of the Anthos data and storage management functionality. Ali said Google is also working with Cisco, Dell EMC, Elastifile, HPE, NetApp, Portworx and other storage vendors. He said storage vendors will eventually need to implement a new set of APIs at the CSI layer, and Google will build proprietary functionality to "tie it all together." Ali said Google hopes to build out an Anthos storage qualification program for partners.
Hyper-converged infrastructure vendors that have committed to deliver Google's Anthos on their HCI products include Cisco, Dell EMC, HPE, Intel and Lenovo. HPE also expects to ship a new Validated Design for Google Cloud's Anthos next month that uses its Nimble Storage and ProLiant servers, through a VMware volume plugin, to target database and analytics workloads.
