Storage
Tomasz Zajda - Fotolia
Multicloud storage mitigates risk of public cloud lock-in
Multicloud storage strategy allows for the shifting of data across public clouds so you can avoid lock-in to a single provider, reduce cost and improve workload efficiency.
Public cloud utilization continues to grow at a phenomenal rate. Public and private cloud infrastructure spending is increasing at double digit rates while spending on traditional, non-cloud, IT infrastructure is declining. Within a few years, non-cloud spending will be less than half the entire infrastructure market. Amazon Web Services alone, as the gorilla of the public cloud market, continues to grow at over 40% year over year and has an annualized run rate of around $15 billion. Microsoft boasts similar revenue when its Office 365 software-as-a-service offerings are included.
This trend has been widely predicted for years. The surprise is how strong the inertia toward public cloud adoption has become. So much so that it's an open question as to when we will hit the long-term equilibrium point between public clouds and on-premises infrastructure.
Amazon Web Services (AWS) pioneered public cloud storage when it introduced Simple Storage Service (S3) more than 10 years ago. The public cloud vendors' approach has been to offer storage services at cut-rate pricing in what we call the "Hotel California" strategy: Once they have your data, it can never leave.
And once they have your data, they then offer a wide variety of higher-priced services, such as global content distribution, data analytics and a variety of compute capabilities, to complement access to it. Recently, we've heard increased grumbling from public cloud storage customers concerned about losing the option to change vendors and the resulting reduction in competition.
Primary multicloud storage defined
Multicloud primary storage is storage technology providing primary data services that operate simultaneously across multiple heterogeneous cloud environments, and it's where compute and applications can be colocated with these data services. The primary data storage services must support at least one large public cloud vendor, such as Amazon Web Services (AWS), Google Cloud Platform and Microsoft Azure.
The key defining concept is heterogeneous clouds, meaning the storage service must be able to simultaneously operate in disparate cloud environments, such as native to AWS and native to Azure. This definition doesn't include storage services that only work natively with Azure Stack in one instance and also with Azure-based public clouds, or with VMware cloud on premises combined with VMware Cloud on AWS. Those services would be considered homogeneous cloud storage services because they use the same underlying virtualization technology offered as both a private and public consumption alternative. Microsoft Storage Spaces Direct and VMware vSAN fall into the homogeneous cloud storage category as well.
The second key concept is the colocation of storage services with the native compute and analytical resources that the public cloud also provides. This capability enables the creation of a full data center environment within the public cloud environment, removing the need for any on-premises infrastructure.
Another approach to multicloud storage is to use private storage as a service physically colocated near public cloud data centers via a provider such as Equinix. The storage provider can use physical connections enabled by dedicated, high-speed connections, such as AWS Direct Connect or Azure ExpressRoute, to provide low-latency storage services across a subset of heterogeneous public cloud locations. Hewlett Packard Enterprise's Nimble Cloud Volumes are an example of this approach. While they're serving an important need, we aren't including these approaches in our definition of multicloud primary storage.
To gather data and develop insights regarding plans for public cloud and multicloud storage use, Taneja Group earlier this year surveyed more than 400 IT decision-makers and practitioners from businesses of all sizes and industries. The results of that survey are helping us understand current and planned deployments of applications to the public cloud.
Specifically, we wanted to assess the need for an emerging set of storage products that we call multicloud primary storage. Essentially, multicloud storage products provide data services across more than one public cloud simultaneously.
Obstacles to public cloud adoption
Increasing costs and vendor lock-in are top concerns with cloud services, just behind the ever-present need for data security (see Figure 1). One could argue increased costs and vendor lock-in concerns are related, as a lack of vendor choice often leads to higher prices.
Multicloud storage can help address vendor lock-in and data security concerns. Many of the ones we evaluated offer full control of encryption from cradle to grave and across the multiple clouds. Look for end-to-end security to be a major differentiator in the multicloud storage market going forward.
Only 12% of respondents said they had no apprehensions about moving to cloud services, indicating that there is a sizeable opportunity for providers to offer products that address the multicloud environment concerns of the remainder of respondents.
Multicloud benefits
The key benefits of multicloud storage products and services include the following:
- Data portability among heterogeneous clouds. This involves the ability to shift data among public clouds as business needs change. It mitigates vendor lock-in and lets customers more easily meet regulatory compliance demands.
- Ease of lifting and shifting applications between multicloud environments. Certifying applications against a common set of data services makes shifting an application to a new environment easier to manage. Most multicloud storage enables application testing on premises and in the cloud.
- Control of end-to-end data security. Multicloud storage should provide better visibility of data location and enable seamless security with access control and encryption across disparate cloud environments.
- Enhanced data availability and disaster recovery options. Although public clouds boast geographically dispersed availability zones, they often come with increased costs and occasional outages. The diversity of cloud providers in a multicloud environment can provide both continuous availability and disaster recovery capabilities.
- Consistent set of enterprise data services collocated with apps and compute resources. Features such as replication, data locality, snapshot and capacity optimization differ among cloud providers. Multicloud storage should offer a more consistent quality of service across clouds through a common feature set.
- Bridging data services between private and public clouds. A multicloud strategy enables a more seamless transition for customers seeking hybrid cloud services.
How vendors meet multicloud storage needs
Customers look for a variety of storage services and capabilities when it comes to applications running in a public cloud. Figure 2 shows survey results when respondents were asked what data access method they prefer most for the applications they plan to move to the public cloud.
Given these responses, it's not surprising that multiple vendors are emerging to meet these diverse customer requirements. The following are vendors that embrace the multicloud primary storage concept as we've defined it:
SoftNAS was one of the first born-in-the-cloud primary data services vendors. It focuses on companies that want to lift and shift predominately file-based applications from on premises to the cloud. The company offers a consistent set of advanced file services based on the popular ZFS technology. The SoftNAS software-defined NAS operates across popular public, private and hybrid cloud computing platforms, including AWS, Microsoft Azure and VMware vSphere.
Hedvig has one of the most comprehensive enterprise multicloud strategies and is the only vendor offering block, file and object services simultaneously across AWS, Google and Azure. Hedvig's Distributed Storage Platform is based on a scale-out software-defined storage architecture that supports these multiple data access protocols. The goal is to link the three market-leading public clouds with on-premises cloud infrastructure. Hedvig's focus is on end-to-end security, data locality control, advanced capacity optimization and performance tiering.
Qumulo provides enterprise-grade, scale-out file services running between AWS and private clouds. It plans to add other public clouds over time, with the goal of enabling high-performance, scale-out file services for popular public cloud workloads, such as video rendering. Qumulo Core, based on a modern flash-first design, can store many times the number of files compared to legacy scale-out NAS products.
Scality takes advantage of AWS S3's popularity by making a compatible S3 API available in all public clouds while still enabling data control and movement of data location. Scality offers a policy-based data control plane. The data control plane centrally orchestrates data services, such as provisioning, replication and migration across the multiple heterogeneous clouds, largely based on predefined policies. Native cloud applications can store and access data using either the Scality S3 API or the cloud's own native object API.
SwiftStack offers multicloud object storage services and provides a data control plane across the clouds. It maintains the public cloud's native object storage API and provides OpenStack Swift API support.
In addition to evaluating whether these storage providers can meet the data services requirements needed to host your applications in the public cloud, consider the following complementary attributes:
- integration of public cloud billing and security environments;
- ease of deployment to public cloud environments;
- data control planes that navigate across multiple public clouds;
- ecosystem support and integration with other IT orchestration frameworks;
- efficient use of the underlying public cloud resources to deliver data services cost-effectively; and
- whether they offer the same flexible pricing models as the public cloud.
Multicloud storage drivers
As the multicloud primary storage market emerges, it will evolve quickly over the next few years. The key question dictating the long-term success of this market is whether customers will value the business benefits that a consistent multicloud data service delivers versus the incremental storage-related costs they might incur.
There are various reasons why businesses might want to use multicloud primary storage technology to shift data from one cloud to another. In one example, a major financial services company was fully utilizing AWS, but wanted to use both AWS and Azure instead. It opted for multicloud storage technology to keep web-tier applications on AWS and moved the SQL and analytics tier to Azure, where SQL runs more efficiently. In the future, the financial services provider wants to remain on multicloud storage and, as applications become containerized, looks forward to being able to seamlessly move those applications between heterogeneous clouds.
Containers and the need for persistent storage
Containers provide a simplified approach to virtualization promoted as open source technology by companies such as Docker. They offer a simpler and more standardized resource-isolation approach that enables greater independence and density across a wide variety of heterogeneous cloud infrastructures. Applications based on containers are more portable and easier to maintain, and this approach has quickly become popular among app developers.
Initially, containers only supported applications where the data required could be created and destroyed along with the launching and closing of the application itself. In order to make containers useful across a wider set of workloads, they needed to support persistent storage assigned to a particular container application. With persistent storage now a supported container feature, cross-cloud replication will become an increasingly important capability to enable application mobility across heterogeneous clouds.
Another driver for multicloud storage is what we see as shifting business alliances. An example would be a major retailer requiring vendors to shift off what is considered a competitor's public cloud infrastructure if they want to continue doing business with the retailer. Mergers and acquisitions could be another form of shifting business alliances. New government regulations regarding data privacy and locality can also create a reason to move data from one cloud to another. All of these scenarios would make shifting data from one public cloud provider to another a business necessity rather than just a nice-to-have.
We expect the multicloud primary storage market will grow rapidly over the next few years as public cloud providers get bigger and businesses increasingly feel like their data is being held hostage. This trend will coincide with the maturing of container-based applications using persistent storage, which should free applications to move easily among public clouds and act as an accelerant to multicloud market growth.
