Strategies for Kubernetes multi-cluster management

Benefits of multi-cluster Kubernetes

Similarly, using multiple clusters has several advantages when it comes to Kubernetes management. Losing one cluster when your applications are appropriately spread across several other clusters is much better than losing a single cluster with everything on it.

Multi-cluster Kubernetes environments have many advantages, even more so when geographically redundant -- if they are configured correctly and managed appropriately. The benefits of running multiple Kubernetes clusters include the following:

• Ability to run different cluster software versions. Upgrading a cluster isn't risk free. Separating a properly architected application across multiple clusters and regions means that if an error occurs in one cluster, the other appropriately configured clusters can take the load. Sometimes a cluster must remain on a specific software version, but there's no reason not to have that version on one cluster and other versions on different clusters.
• Ability to separate environments. It is considered best practice to have separate development and production clusters to reduce the risk of being served beta or non-production code versions. In addition, some environments are subject to data sovereignty rules requiring they be geographically located in a certain data center or region.
• Better end user experience. No one likes a slow internet browsing experience. Splitting workloads using a load balancer such as Azure Traffic Manager can improve performance and reduce latency by providing users with a geographically close environment.

Challenges of multi-cluster Kubernetes

All this resilience doesn't come free. Most modern cloud providers, such as Azure, have a reference design for multi-cluster setups. Because of the highly ephemeral nature of containers, it makes sense to design for failure as much as possible at the outset.

When properly configured with appropriate traffic routing, providing resources from several different geographic areas enables requests to be served quicker because the cluster is geographically closer than it otherwise might have been. Obviously, this requires some upfront investment to configure the appropriate load balancers and routing.

Isolating data can also be necessary for legal reasons. The legal considerations are much simpler if all your data is located and served within a country or region that's required to follow well-defined data sovereignty and management rules, such as the GDPR. However, the details depend upon your customer base and their locations in the world. This is a key consideration, and mistakes can be financially ruinous. In short, there is more to consider than availability and technology.

From a technical standpoint, managing multiple Kubernetes clusters is more complex than managing a single cluster. Multiple clusters require multiple cluster configuration files. Centralized management avoids duplication of effort and reduces management overhead and scope for errors.

Tools for Kubernetes multi-cluster management

There are many options for automation and desired state tools, and the decision often comes down to the level of control needed and the environmental complexity. These factors need to be appropriately considered before adopting a tool.

Infrastructure as code

Kubernetes and infrastructure as code (IaC) go hand in hand. IaC helps ensure deployment consistency and reduce human error. Terraform is the go-to tool when creating an IaC environment because it can destroy and create infrastructure on the fly.

All these different deployment templates require version control to be useful. When deployment templates are properly managed, reverting to a previous configuration becomes as simple as using Git's git revert command and rerunning the process.

Managed Kubernetes services

While it is possible to create your own clusters on virtual or physical machines, there are many reasons to use the Kubernetes services that most major CSPs provide, such as Google Kubernetes Engine or Amazon Elastic Kubernetes Service.

With these managed Kubernetes services, the vendor does all the heavy lifting and maintenance. When implemented correctly, this can make multi-cluster management substantially cheaper.

Service meshes

Finally, service meshes provide tools to manage more complex infrastructures. Open Service Mesh is a popular example, but most service meshes have a broad range of capabilities:

• Service discovery.
• Traffic management.
• The ability to monitor the whole environment rather than individual service components.

Service mesh tools also help enforce security constraints, such as encryption between all services using Mutual Transport Layer Security. Service meshes also help ensure conformity of the running environment at scale.