DETROIT -- Red Hat revealed OpenShift roadmap details this week aligned around a common theme: managing tens of thousands of Kubernetes clusters in locations that range from data centers to embedded edge devices.
The IBM subsidiary rolled out Red Hat Device Edge this week, a supported version of its upstream MicroShift project bundled with a stripped-down Linux operating system for edge computing. MicroShift is a variation of Red Hat's OpenShift Kubernetes distribution (OKD) designed for highly resource-constrained devices used in IoT environments. MicroShift follows previous updates to OpenShift that support a growing number of edge computing clusters at locations such as restaurant and retail branches as well as remote offices.
Lockheed Martin was already using OpenShift Advanced Cluster Management (ACM) software to control Kubernetes multi-cluster deployments from data centers to remote locations. It has now added MicroShift to the mix for its smallest edge devices.
Engineers from Lockheed demonstrated an AI training application running on MicroShift within an Nvidia Jetson AGX Orin device during a presentation at the OpenShift Commons Gathering this week. That device, onstage with the presenters, was connected to an instance of OpenShift running on an HPE Edgeline EL8000 Converged Edge System server in Denver.
"You can do a lot of your AI model training on large clusters in your data center where you have all these GPUs, but [eventually] ... you need it to point elsewhere," Ian Miller, MLOps engineering manager at Lockheed Martin, said during a presentation.
In Lockheed's case, "elsewhere" might mean a battleship, fighter jet or another environment that has limited hardware resources and intermittent network access. Training AI in those environments then requires monitoring data from previous device runs to be repeatedly fed into the model.
Having a consistent software automation platform that can update embedded devices means that AI model training loops can be performed quickly -- or even swapped out for another model entirely -- without downtime between runs, according to Miller.
That's where Kubernetes and MicroShift come in, he said.
"It's certainly possible to do [edge computing] without container orchestration. But we find that container orchestration has lot more flexibility in some of these dynamic environments," he said.
Ian Miller, left, and Matt Wittstock, right, of Lockheed Martin demonstrate Red Hat MicroShift running on an edge device, center, at OpenShift Commons Gathering.
OpenShift support matures as edge goes mainstream
OpenShift isn't the first to expand into Kubernetes multi-cluster management for edge computing at scale. The U.S. Air Force, for example, put Kubernetes in fighter jets using Rancher Kubernetes Engine (RKE) and its distribution for edge computing, K3s, three years ago. RKE also already supported up to 1 million edge clusters prior to its acquisition by SUSE in 2020.
"I spent dozens of meetings with Red Hat on their edge stuff, but their timelines were always way off from what we needed -- at least two years," said Nicolas M. Chaillan, former Air Force and Space Force chief software officer, in an interview. Chaillan oversaw the DoD's Rancher K3s deployment for edge computing in 2019.
As a result, Red Hat OpenShift's usage declined within the DoD in favor of Rancher, according to Chaillan. He left the department in 2021 and now serves as an independent consultant and member of several advisory boards for IT security startups.
Rancher also began to specialize in multi-cluster Kubernetes management that was agnostic to the underlying Kubernetes distribution in 2019 -- another topic that has since emerged as a priority for OpenShift ACM. ACM has limited support so far for Amazon Elastic Kubernetes Service, Azure Kubernetes Service, and Google Kubernetes Engine workload clusters. But product development teams have already started experimenting with importing observability data from such clusters into ACM, according to a presentation here by Karena Angell, OpenShift principal product manager. Support for full cluster lifecycle management beyond OKD is also on the long-term roadmap.
While these additions to OpenShift were too late for bleeding-edge applications such as the Air Force project, they may be well-timed for mainstream enterprises given Red Hat's partnerships with cloud providers, one analyst said.
"OpenShift has become so embedded with Google, Microsoft and AWS that it gives them a chance to be a preferred vendor," said Larry Carvalho, an independent analyst at Robust Cloud. "Rancher wasn't even on the radar for a lot of companies [before the SUSE acquisition]. But Red Hat OpenShift was first to jump on the Kubernetes bandwagon [in 2014]."
OpenShift ACM fleshes out new control planes
The proliferation of Kubernetes clusters, especially for edge computing, forced a new perspective within Red Hat about cluster fleet management, Angell said in her presentation.
"We've embraced a hub-and-spoke pattern as our approach to operating fleets of Kubernetes clusters," she said. "The hub is the management cluster and then the spoke clusters can be [owned by] different teams, which provide a modular approach to repeatable deployments of similar workloads."
OpenShift roadmap presentations here also included updates on product plans for two upstream projects Red Hat engineers first began talking about publicly last year: HyperShift and Kubernetes control plane (KCP). Both projects use a "hub of hubs" architecture for Kubernetes multi-cluster management at massive scale but have started taking distinct paths that reflect the rising popularity of enterprise platform engineering.
It's certainly possible to do [edge computing] without container orchestration. But we find that container orchestration has lot more flexibility in some of these dynamic environments.
Ian MillerMLOps engineering manager, Lockheed Martin
HyperShift, the more mature project, creates a management control plane and a pooled data plane for massively federated fleets of OpenShift clusters in multiple data center and cloud computing locations. A platform engineering team might use HyperShift to instantiate many physical clusters at scale, said Stefan Schimanski, senior principal software engineer and architectural lead for KCP at Red Hat, during a presentation.
"Now I need a way to deploy and continue applications and services on those clusters easily and scalably," Schimanski said.
That's what KCP would do, through API platform consumers could use without having to install and manage Kubernetes operators on a cluster, Schimanski said.
"A contract between two teams should be just a CRD-backed API, with everything else hidden behind that API," he said. "[With KCP, platform consumers] don't have to install an operator and maintain it and fix it when it doesn't work."
HyperShift is in technical preview as hosted control planes in ACM and is currently focused on AWS and bare metal deployments. Support for vSphere and Azure are in the works. Further updates are planned in the version 2.7 release slated for the first quarter of 2023.
KCP remains an early-stage project, and the community is preparing to apply for sandbox status within the Cloud Native Computing Foundation, but "ACM already has a very close relationship with KCP and we expect that to continue as we move forward," according to a Red Hat spokesperson.
Lockheed's Miller added during his presentation that he wants Red Hat to develop a control plane specifically to manage fleets of edge device clusters.
"Even though we can use [existing] control planes on the edge devices, obviously there's some overhead since they're built to run in cloud environments," he said. "One area that's ripe for improvement is a control plane that could be embedded on optionally disconnected devices."
Beth Pariseau, senior news writer at TechTarget, is an award-winning veteran of IT journalism. She can be reached at [email protected] or on Twitter @PariseauTT.
