Hybrid cloud technology gets the most out of primary storage workloads
Learn what to look for in a hybrid cloud platform so you can take advantage of the scalability, agility and cost benefits it has to offer primary storage.
By
Jeff Kato and Jeff Byrne
Published: 02 Aug 2017
The promise of smooth-functioning, cost-effective hybrid cloud storage has long been of interest to IT professionals. "Hybrid" has been in the cloud lexicon from the beginning, when the National Institute of Standards and Technology issued its original definitions of various cloud deployment models.
Hybrid cloud storage broadens the workload deployment choice to more than one cloud and enables compelling use cases, such as off-site backup, disaster recovery and cloud bursting. Done right, an enterprise hybrid cloud improves IT agility while reducing cost.
Up until recently, however, major challenges kept companies from embracing the promise of hybrid clouds, particularly for primary storage. These obstacles fall into four categories:
Lack of standard interfaces and tools. Workload portability, a prerequisite for hybrid clouds, has been hampered by a lack of standard APIs and a lack of runtime environments that let companies easily move their apps and data between on-premises and public clouds. While containers have facilitated the movement of cloud-architected apps, the movement of legacy apps and data remains a challenge. Similarly, a lack of tools has made cross-cloud provisioning and orchestration difficult at best. As a result, most hybrid clouds have required compatible platforms and infrastructure on each end to work effectively, limiting customer choice, flexibility and ease of use.
Limited awareness and knowledge. Vendor hype has confused buyers about what it takes to deploy enterprise hybrid cloud technology, while often oversetting expectations about what it can do for them.
Concerns about enterprise readiness. Storage managers are often reluctant to move active, production-level data outside data center walls due to concerns that public clouds won't meet service-level commitments for security, availability and performance.
Fear of lock-in. Some companies have shied away from public cloud due to lock-in concerns: Once I migrate my apps and data to the cloud, how difficult and costly will it be to move them back on-premises? Hybrid clouds require unfettered portability, but proprietary hooks and high data egress costs -- the cost of moving data out -- built into public provider services can get in the way.
Adopt a cloud services mindset, in which resources are provisioned and services delivered on demand and paid for as they're consumed.
Fortunately, as the cloud market and technologies mature, most of these adoption barriers are gradually being dismantled. Based on our recent research, IT manager confidence in public clouds has increased significantly in the past couple of years, leading to their adoption for an expanding set of workloads. In addition, rapidly maturing cloud storage, networking and orchestration technologies are bringing hybrid cloud primary storage closer to reality, while products that enable simple and streamlined data portability are beginning to alleviate lock-in concerns.
Even as these obstacles are lowered or removed, buyers need a way to sort and distinguish competitors. To accomplish that, let's look at a set of criteria that defines what we believe is the sweet spot: hybrid cloud services that allow you to better and more fully support primary apps and data.
Characteristics of ideal hybrid clouds
To overcome the limitations of existing approaches and ensure that a hybrid cloud primary storage product meets all your needs, start with an on-premises private cloud. This must include self-service provisioning and pay-as-you-go billing for infrastructure and app services. Among other benefits, this approach will help your organization adopt a cloud services mindset in which resources are provisioned and services delivered on demand and paid for as they're consumed. A private cloud also will lay the groundwork for a hybrid IT infrastructure.
Stripping away the hype from hybrid cloud reality
For nearly a decade, tech pundits and vendors have promoted the vision of a hybrid cloud. Hybrid has been hyped as a panacea for various IT problems and has often been positioned as a way to make the public cloud more palatable -- offering the flexibility of cloud without putting key IT assets at risk. But what, really, is hybrid cloud?
To paraphrase the National Institute of Standards and Technology, which has long monitored and participated in the development of cloud technologies, a hybrid cloud is composed of two or more distinct cloud infrastructures -- generally one private and one public -- bound together by technology that enables data and application portability. Meeting that definition requires data and app portability, which has been difficult to achieve between clouds because of the use of proprietary technology and a lack of agreed standards. That is slowly changing.
As cloud technologies have matured, enterprises have become more comfortable with the idea of deploying use cases in the public cloud. Based on Taneja Group research, most organizations are running at least some of their business-critical apps and data in one or more public clouds. As of 2016, 61% of companies surveyed had active or planned app deployments in the public cloud, up from 43% in 2014.
Going further, a growing number of organizations have a cloud-first policy for new workload development and deployment, attracted by benefits such as greater agility and lower Capex. Once companies adopt public cloud infrastructure, many look next to hybrid as a way of bringing cloud benefits to on-premises assets, while also avoiding lock-in to an external cloud provider.
Beyond that starting point, here's a list of criteria that will let you get the most out of your enterprise hybrid cloud investment:
Full and effortless data portability between on-premises and cloud. The agility of your hybrid cloud depends on this, whatever the use case.
Support for heterogeneous infrastructure, including hardware and virtualization platforms, on each end. Avoid hybrid clouds that force homogeneity, which will limit flexibility and likely increase cost.
Full orchestration of compute, networking and storage resources between on premises and public cloud. Along with data portability, this is the linchpin of a hybrid cloud. You need the ability to dynamically deploy and manage your primary storage in whatever private or public cloud makes the most sense while always having full accessibility to your data.
Ability to run production workloads in the private or public cloud, regardless of how they're packaged or the type of storage supported. General-purpose hybrid cloud primary storage should support all your workloads through support on each end for block, file and object protocols and for any type of application, whether bare-metal, virtual machine or container-based.
A comprehensive set of enterprise data services that can be colocated with your apps and compute resources. For example, bidirectional replication is an essential feature of hybrid cloud technology, and data reduction technologies, such as compression and deduplication, streamline and accelerate data movement. Snapshots also are critical, affording data protection and the ability to recover or return to a previous version or application state. A hybrid cloud storage product should be able to invoke these kinds of data services in whatever cloud your apps happen to be running.
Support for multiple public clouds to run apps and use cases and avoid lock-in to a single provider. Your hybrid cloud technology product should always let you choose where to run primary storage workloads, based on the cloud that best meets your service-level and cost requirements.
Intelligent, policy-based data placement, in the location that best serves the needs of on-premises or externally hosted workloads. Flexibility of data placement is essential for meeting application performance objectives, such as minimizing latency, while also satisfying compliance requirements, like data sovereignty or privacy.
Simple, cloud-enabled management of the distributed storage environment. Though primary storage may be distributed across multiple clouds, you'll want to easily monitor, control and manage hybrid cloud deployment from wherever you happen to be.
Product offerings that satisfy the majority of these criteria will more likely provide the choice, agility, control and cost you should expect from hybrid cloud technology storage. A full set of these characteristics is seldom found in a single product, however. Let's briefly consider the field of existing offerings to see how they deliver these capabilities.
How do products stack up?
Existing products that connect on-premises storage with a public cloud service come in several flavors. While several of these claim to offer hybrid cloud capabilities, some come closer than others to meeting our criteria:
Cloud gateways and tiering products, including those from traditional storage vendors where a tiering capability is added to an existing system and cloud gateway providers. Dell EMC, Nasuni and Panzura are among the vendors providing cloud gateways. These push less frequently accessed -- or cooler -- data from on premises to a public cloud, and often claim to enable not just archiving but also primary storage use cases such as file sharing or collaboration. Gateways let customers take advantage of cloud scalability and pay-as-you-go economics without sacrificing too much application performance or control. However, these approaches can have significant limitations, including the inability to move apps along with data to the cloud and orchestrate resources between the data center and public cloud. Gateway and other tiering approaches have proven beneficial, but they're only a start when it comes to offering hybrid cloud capabilities.
Object storage software designed for on-premises and public clouds, providing a hybrid solution for apps that rely on object storage. Among the object storage software vendors are IBM with its Cleversafe-based Cloud Object Storage, Scality and SwiftStack. Object storage is ideal if you're primarily running cloud-architected workloads based on containerized apps and microservices. You'll enjoy full data portability and your apps can move as needed with your data. Data services such as replication and erasure coding reside with your apps and compute resources. Several object storage software vendors also orchestrate global distribution of your data to both public cloud and on-premises locations.
Object storage products are on track to provide the majority of the hybrid cloud capabilities on our wish list, except most don't adequately support traditional file- and block-based applications. These may include the apps on which you may be running your business. If you're focused on moving legacy workloads to the cloud and running them in a hybrid fashion, then object storage software likely won't meet all of your needs.
New wave of software-defined products designed to provide hybrid cloud storage capabilities for primary workloads. Vendors in this category include Hedvig and SoftNAS. The emerging products take a variety of architectural approaches, but all emphasize the ability to flexibly and transparently distribute storage across on-premises and public cloud environments. If you're looking for an enterprise-caliber product, focus on those that let you set up multi-tenant, elastic storage pools across on-premises and the cloud. These products dynamically support the storage and data services needs of both traditional business and cloud-architected applications. Ideally, you'll want built-in high availability and data redundancy, with intelligent, policy-based data placement and automated load rebalancing to optimize accessibility and storage performance.
As this category develops, we believe some products will support particular workloads and deployment scenarios, such as lifting and shifting existing apps to the cloud. Others will be more general purpose. Look for hybrid products that provide the scalability and flexibility you'll need to grow, along with automated cross-cloud orchestration and management to minimize hands-on admin support.
Go hybrid, but choose wisely
Hybrid cloud technology products open new possibilities for deploying production applications. For example, if you're already running workloads in the public cloud and find your monthly bill growing too large, these products give you the flexibility to run selected workload tiers -- such as the presentation layer -- in the public cloud where they will benefit from the elasticity, while bringing more cost-sensitive portions of the workload back on premises. Hybrid cloud may also benefit on-premises apps, such as data analytics workloads, by providing an opportunity to burst out workloads that run primarily on premises but access public cloud resources as needed.
However, hybrid cloud isn't a panacea, so choose your vendor wisely. Go beyond simply tire kicking and carefully evaluate each product against your objectives and existing environment to determine which the best fit is.
We believe that hybrid cloud will become a reality for production apps and their associated primary storage in 2017 and 2018. Keep an eye out for new approaches and products, including enhancements to those described in this article, particularly to cloud-enabled, software-defined storage. These promise to change the way we as IT professionals think about the hybrid cloud and its role in running the business.
