Cluster API is considered production-ready by early adopters of multi-cluster Kubernetes, while a new multi-cluster API and control plane prototypes take shape.
The next phase of development for Kubernetes and its related open source projects has become squarely focused on multi-cluster management.
As GitOps spreads among cutting-edge IT teams, Kubernetes-based infrastructure as code strategies are being applied to multi-cluster Kubernetes management through Cloud Native Computing Foundation (CNCF) projects such as Cluster API, which in the last year grew from a nascent proof of concept to a tool some IT pros consider production-ready.
New efforts to support multi-cluster Kubernetes are also afoot in the Istio service mesh community, and Red Hat added two early-stage hybrid cloud control plane prototypes to the mix this year during the KubeCon + CloudNativeCon EU virtual event.
Multi-cloud and hybrid cloud management form the backdrop to this shift in direction, amid a larger movement to cloud computing spurred by the COVID-19 pandemic, where Kubernetes is the common layer that ties increasingly distributed environments together. The rise of edge computing will also make managing multiple clusters, sometimes thousands or even millions of them in Internet of Things environments, a key aspect of enterprise IT strategy.
"There is growing interest from enterprise IT organizations on understanding what role Kubernetes can really play across any cloud architecture as a management [and] orchestration engine," said Stephen Elliot, an analyst at IDC. "Vendors also have to consider their role as Kubernetes [becomes] just one piece of a broader multi-cloud management and orchestration strategy."
With early versions of the Red Hat OpenShift Kubernetes platform five years ago, customers would start with single clusters for small teams, and as those initial clusters grew, Red Hat focused its product development around multi-tenancy within them. This led to features such as Kubernetes namespace management, resource provisioning quotas and role-based user access controls, along with an OpenShift software-defined network that segregated application traffic and managed network policies within multi-tenant clusters.
"Despite these capabilities, customers always found requirements that would call for creating yet one more cluster," said Joe Fernandes, vice president and general manager of core cloud platforms at Red Hat, speaking at an OpenShift Commons virtual event presented as part of this week's conference. "All the tenancy in the world doesn't eliminate the need for multiple clusters, and then as the number of OpenShift clusters grew, so did the need for multi-cluster management … that's really what's driven our roadmap recently."
Red Hat hybrid cloud control plane presentation.
Cluster API matures amid GitOps buzz
Efforts to standardize a declarative API that could be used for multi-cluster Kubernetes management across multiple clouds first began in 2017 within the SIG Cluster Lifecycle group in CNCF. By last year, the resulting project, Cluster API, was about to enter its third alpha phase, but even bleeding-edge Kubernetes users were still hesitant to deploy it in production.
However, with that third alpha version, Cluster API added crucial features for users such as Marcel Müller, platform engineer at Giant Swarm. That update introduced features that were important for Müller's team, such as support for MachinePools, a way of defining common server configurations in multi-cluster environments that can be managed by cloud providers, rather than by the Cluster API controller.
"This [more closely] reflects how we manage clusters," Müller said. "In the upcoming [version 0.4], the added support of [the] ignition [command] was important to us [because] we use Flatcar as an operating system, which uses ignition during bootstrap." Flatcar Linux is a fork of CoreOS Container Linux, which reached end of life in May 2020.
Within the Kessel Run project that develops command and control software for the US Air Force, Cluster API has become essential to multi-cluster GitOps.
Cluster API is the technology that allows us to use GitOps patterns for our cluster lifecycle.
Michael MedellinDirector of engineering, Kessel Run, Department of Defense
"Cluster API is the technology that allows us to use GitOps patterns for our cluster lifecycle," said Michael Medellin, director of engineering at Kessel Run, in an online Q&A following a presentation at KubeCon EU. "Cluster API is the controller that lets us reconcile cluster configuration state from declared to actual. Tools like Flux let us define that state in version control and sync that to the Cluster API resources."
The Cluster API GitHub page still warns that the project is in alpha and not encouraged for production use. Alpha and beta projects within CNCF, unlike stable versions, may introduce breaking changes as APIs develop.
Medellin acknowledged this could become a risk, but said his team works with upstream maintainers to prepare for and influence such changes.
Multi-Cluster API targets cross-cluster communication
While Cluster API handles resource provisioning, another project has sprung up within the CNCF to address cross-cluster network communications. CNCF members said they expect the first implementations of this proposed Kubernetes Multi-Cluster API project to be incorporated into Istio service mesh by the end of the year.
"One of the challenges with Istio [in] multi-cluster [environments] is you can't easily declare, 'These are the services I want to export out of my cluster, and these are the services I want to import from other clusters,'" said Lin Sun, an Istio maintainer and director of open source at Solo.io. in a KubeCon EU panel discussion. "The Kubernetes Multi-Cluster API is an attempt to … allow service owners to declare what services they want to make available outside their cluster or consume from other clusters."
For now, financial software maker Intuit uses its own Admiral tool to manage Istio in multiple clusters but will evaluate the Kubernetes Multi-Cluster API.
"There's a lot of variability out there today that makes multi-cluster more complicated," said Jason Webb, distinguished engineer at Intuit, in the panel session. "The more the industry consolidates around some conventions, it'll make things easier for companies to adopt going forward."
Red Hat tinkers with multi-cloud control planes
The newest multi-cluster Kubernetes management projects, officially unveiled in a KubeCon EU keynote presentation and discussed during the OpenShift Commons event this week, consist of two prototypes created by Red Hat engineers for hybrid cloud control planes using the Kubernetes API.
Both new projects are focused on separating the Kubernetes control plane, governed by the API, from specific infrastructure such as containers, nodes and pods. One of them, called Hypershift, is based on the architecture IBM Cloud teams use to manage the Red Hat OpenShift on IBM Cloud service. Hypershift would allow users to "bring their own nodes" to an existing Kubernetes control plane, according to Red Hat's Fernandes.
The other project, which first published its GitHub repo this week under the name kcp, would delegate infrastructure resource requests to underlying infrastructure that is completely abstracted from the user. Hypershift and kcp are not related to each other so far but could overlap as they develop, Red Hat officials said this week.
The kcp project may eventually become a fresh way to address some of the same issues as Cluster API, said Clayton Coleman, architect for Kubernetes and OpenShift at Red Hat, in an interview.
"Everyone's ideal cluster API is different, [so] depending on use cases, every API is subtly wrong," Coleman said. "Part of kcp actually is trying to make … multiple sets of APIs more coherent."
Beth Pariseau, senior news writer at TechTarget, is an award-winning veteran of IT journalism. She can be reached at [email protected] or on Twitter @PariseauTT.
