Multi-cloud databases: How to deploy and manage them

What is a multi-cloud database environment?

A multi-cloud database environment puts database instances or distributed databases on multiple cloud platforms. Multi-cloud architectures can include separate databases running in different clouds and single databases deployed across clouds as database clusters. Typically, multi-cloud environments involve the use of more than one public cloud. But they can be built on any combination of public and private clouds, as well as edge clouds that combine cloud and edge computing technologies.

For example, an organization might spread databases that support different applications across AWS, Azure, Google Cloud or other public clouds. Some databases might also run in a private cloud set up in the organization's own data center, a third-party colocation site or virtually in a cloud provider's facility.

The multi-cloud approach differs from a hybrid cloud, which combines a private cloud, a public cloud and perhaps conventional on-premises systems into a unified IT environment. But the databases in a hybrid cloud could conceivably be incorporated into a broader multi-cloud database strategy.

Benefits of a multi-cloud database strategy

Although the benefits of deploying databases across multiple clouds will vary from one organization to another, the following three are the most commonly cited ones.

• Cost savings. Organizations may get better pricing options and more negotiating power by having choices between multiple cloud providers.
• Freedom from vendor lock-in. In a multi-cloud environment, organizations are able to pivot more easily to another cloud provider if circumstances warrant.
• Flexibility to use best-in-class services. Because cloud providers and services aren't created equally, a multi-cloud approach enables organizations to take advantage of the strengths of each cloud to deliver optimized capabilities.

Additional benefits include compliance with contractual obligations that specify or restrict cloud providers, increased resiliency for systems requiring the highest availability levels, better geographic coverage for global organizations, improved response times from data being stored closer to end users, more efficient data processing closer to the source, and an easier path to assimilate mergers and acquisitions.

Multi-cloud database management best practices

To ensure success, here are 10 best practices for designing, creating and managing a multi-cloud database environment, including information about the challenges that they'll help you overcome.

1. Start with a comprehensive plan and a governance framework

Without question, the biggest challenge to multi-cloud database management is increased complexity, so much so that database environments can quickly become unmanageable. Organizations that enter into this space without addressing the complexity of multi-cloud deployments will struggle to scale their architecture -- and they run the risk of creating a chaotic, Wild West environment.

The first step is to develop a deployment plan that covers all aspects of database management in a multi-cloud environment. Starting with one will provide direction and guidance for activities ranging from choosing the right types of databases for applications to implementing data access methods and tools. A key tenet of the plan should be simplification of the environment. Resist the urge to confront complexity by adding even more technologies, processes and people. Get smarter, not bigger.

Perhaps equally important is creating a governance framework for the environment based on internal policies and standards. It will define the rules that govern database deployment, administration and access across the organization. Proper governance ensures that the multi-cloud strategy is managed and implemented in a consistent and reliable way.

2. Run the right database in the right cloud

Multi-cloud offers the freedom to run database workloads on the most optimal cloud platform for delivering the needed business capabilities. Matching a database with the "best" cloud involves many factors that will vary from organization to organization. In general, though, deployment decisions should include matching both functional and non-functional requirements to each cloud's strengths.

To do so effectively, IT and data management teams must understand their organization's application needs and use cases for cloud databases. The advantages and disadvantages of various cloud platforms should be considered on a case-by-case basis and aligned with the overall multi-cloud database strategy.

In many cases, the decision between public or private clouds comes down to control. If more control of a database system is desired or required, a private cloud is the better option. Organizations that want to offload management of the underlying platform will find public clouds a preferred alternative. Some other factors to consider include cost, security, scalability, latency, resource utilization, geographic availability, customization, deployment speed, current infrastructure and flexibility.

3. Use data services that support multi-cloud environments

Database administration challenges will grow with the addition of each new cloud platform. With more platforms come more technologies to work with, including tools and interfaces that are native to a specific platform.

Operational simplicity can be achieved through abstraction. For example, data fabrics can help companies move from a single cloud to multi-cloud. Their promise is that you'll no longer need to directly use native cloud interfaces and tooling to manage databases. Instead, centralized data services provide a unified view of databases to standardize end-to-end database administration across cloud deployments. This is an emerging area, though, and organizations should expect that some native management will still be required until data fabric technology matures.

4. Exploit managed database services

One of the alluring aspects of the cloud is the managed services that are available to organizations. Using SaaS and PaaS technologies managed by cloud providers is an IT shift with a big payoff. The same applies to database as a service (DBaaS) offerings: Exploiting them in your multi-cloud strategy can significantly reduce database administration work and operational overhead.

DBaaS may not be a good fit for some of your database workloads, but it should be your first choice when it is. Managed database services enable your database administrators to focus more on delivering business value and less on routine administrative tasks.

5. Consider database portability across multiple clouds

Strike a balance between cloud-agnostic databases and ones that are proprietary to a single cloud platform. Open source databases and commercial products that are independent of one of the top cloud platform providers enable multi-cloud portability and help organizations avoid getting locked into a particular cloud.

On the other hand, proprietary cloud database services can offer optimized integration, automation, tooling, performance, functionality and security. As a result, there may be some need for them in your database architecture.

6. Reduce the number of different databases

The successful deployment of databases as a mainstream technology for more than two decades, and the growth of different kinds of databases, has created an IT dilemma. On one hand, IT teams are delivering required database capabilities at the pace of business. On the other hand, a lot of databases have been deployed, adding to operational complexity.

The proliferation of databases, sometimes referred to as database sprawl, is a real concern in a multi-cloud environment. The cloud has already made provisioning databases so easy that organizations often struggle to keep the number of them in check. Add new cloud platforms, with a choice of various purpose-built databases and the opportunity to deploy redundantly purposed ones to reduce cross-cloud traffic, and the reality of having even more databases seems inevitable.

Database rationalization reduces the number of different database technologies that you need to manage. To further simplify things, the most effective remedy may be moving to multimodel databases that support more than one type of data model -- relational, document, graph, etc. The convergence of multiple data models in a single database management system enables it to handle more applications. For special-purpose data needs, a single-model database likely will still deliver the best data management capabilities. For most needs, though, multimodel databases are more than sufficient.

7. Reduce the number of the same databases

Database consolidation is another way to contain sprawl. It reduces the total number of the same databases, effectively pooling and sharing resources for different applications in a smaller number of systems. Management complexity is reduced through fewer deployments and configurations, as well as increased standardization of database versions. The good news is that many leading database systems provide built-in features that enable multi-tenancy for different workloads.

8. Optimize data access for applications and end users

Spreading data across cloud platforms without negatively affecting applications and end-user productivity is a challenge that organizations will need to tackle immediately. If applications and users need to know technical details, such as where data resides and what the underlying database technology is, that will quickly result in inefficiencies and suboptimal user experiences.

One way to address the data access challenges in a multi-cloud database environment is through data virtualization. It's a data integration approach that builds an abstraction layer between different databases, giving applications and users seamless access without having to know where the data is physically located. Creating a single virtualized view of all the data in a multi-cloud environment also reduces isolation and fragmentation of data assets.

9. Keep data local in one cloud platform when possible

While the cloud provides increased flexibility and nearly unlimited scalability, every byte of data that leaves a cloud platform comes at a cost. Most leading cloud providers don't charge for data transfers into their cloud, known as data ingress, but they do charge fees when data is moved out of it, or even from one region to another in the same cloud. Those fees, referred to varyingly as data or network egress costs, can be large if a lot of data is being transferred.

Data traffic between different clouds will also increase latency, as database responses encounter performance degradation primarily resulting from increased distance and multiple network hops. This is especially problematic on high-volume database requests with stringent performance requirements. Worse yet, you'll likely need to deal with a lack of interoperability between cloud platforms.

As data volumes and velocities grow, the cost and latency of moving and copying data across a multi-cloud database environment can become unsustainable. To help avoid that, look to colocate highly integrated workloads and their databases in the same cloud platform. A federated data architecture keeps data in the local cloud ecosystem, minimizing inter-cloud data traffic and egress charges, while allowing interoperability when needed.

10. Connect cloud networks to reduce data latency

Despite the cost and latency issues, some data movement across cloud platforms is inevitable for many organizations. Some database requests may be able to tolerate the speed limitations and unpredictability of the public internet. For those that can't, there are a number of ways to connect cloud networks to address multi-cloud networking latency. For critical processes, private lines are a great option. Alternatives to dedicated circuits from your telco provider include VPN tunnels and private connectivity through a virtual router.