Kubernetes edge computing architectures have emerged over the last 18 months as the next significant frontier in enterprise container management.
Edge computing, which refers to a distributed IT architecture that locates processing power close to the network periphery, has grown separately among enterprises because of the availability of 5G wireless bandwidth, the popularity of smartphones and mobile apps, and the proliferation of "smart" devices on IoT networks. As this trend has emerged, Kubernetes and edge computing have also begun to intersect, because the container orchestration platform offers a way to automate software deployment and management on a variety of devices, with centralized administration.
"There's a huge match between IoT and edge computing and Kubernetes and containers, as users need to act quickly on data from edge environments and IoT sensors," said Jay Lyman, an analyst at 451 Research. "Containers offer a lighter footprint, which is what you need on edge devices [as well as] easy scalability."
Early high-profile cases of Kubernetes edge computing architectures surfaced in 2018, at stores run by mega-retailer Target and fast food restaurants owned by Chick-fil-A. The Kubernetes community took notice, forming the Kubernetes IoT Edge Working Group in 2019, as did Kubernetes management vendors such as Rancher Labs, which made a distribution of Kubernetes tailored for edge computing architectures generally available in November 2019 with k3s. Other open source Kubernetes edge distros that emerged in 2019 included KubeEdge, developed by Huawei engineers, and MicroK8s, by Ubuntu Linux vendor Canonical.
IT pros get a central point of management for Kubernetes edge deployments,with tools such as Rancher.
Rancher herds Kubernetes edge clusters with k3s
There's some debate in the industry about whether Kubernetes variants such as k3s represent a fork of the upstream project, but their proponents argue that Kubernetes edge computing architectures require a fundamentally different approach.
"A lot of people didn't understand [Kubernetes edge computing] at first -- they were doing a single Docker worker node at the edge connected to a master in the cloud or data center, but that's not what Kubernetes was designed to do," said Sheng Liang, CEO of Rancher. "Master and worker were designed to be closely coupled, and keeping nodes at the edge connected to a master in the data center required that the SRE team babysit that deployment as a high-maintenance 'pet.'"
K3s supports nodes that may become disconnected from the control plane, and strips out some Kubernetes features so that the software fits in a very small footprint -- as small as 512 MB in the latest version of k3s, Liang said.
"Kubernetes becomes like an application server for the edge" in such environments, Liang said. "It maintains high availability and distributes software automatically and nondisruptively, without requiring anyone to reimage the Linux OS."
The ability to manage thousands of branch locations remotely without hands-on effort has intrigued enterprise Rancher users such as the Amalgamated Banks of South Africa (ABSA), which is considering rolling out k3s to more than 8,000 bank branch locations in more than 12 countries this year.
Kubernetes becomes like an application server for the edge. It maintains high availability and distributes software automatically and nondisruptively, without requiring anyone to reimage the Linux OS.
Sheng LiangCEO, Rancher Labs
"Right now, we don't have a standardized branch office setup -- we need different components for different locations," said Zak Anderson, head of container platform management at ABSA, based in Johannesburg. "We use physical servers now, and it requires an engineer to install hardware, configure the OS and install the apps, which can take up to five days at a cost of about $2,000 U.S."
K3s, by contrast, could be preloaded onto a server, booted remotely through PXE, and auto-joined to the Rancher Kubernetes environment for remote management, Anderson said.
Rancher Kubernetes Engine version 2.4, slated for release mid-March, will boost the number of clusters the software can centrally manage, to accommodate ever-expanding edge and IoT environments, Rancher's Liang said.
"The 2.4 release will manage over 1,000 clusters, and we're working on the next version that will scale up to a million," he said. "Ultimately, this will benefit enterprises whether they plan to use edge computing or not."
Red Hat scopes further forays into branch offices, disconnected Kubernetes
By Red Hat Summit 2020 in April, Red Hat will roll out updates to OpenShift to better accommodate branch office edge environments, especially environments that may be disconnected from the main data center network.
"Many customers may have tens of thousands of branch office devices, and they're not super excited about having to touch all of those every few months [to do updates]," said Brian Gracely, senior director of product strategy at Red Hat OpenShift. "They'd rather have a 12- or 18-month window."
While Red Hat is less interested in Kubernetes on cellphones or Raspberry Pi devices than other Kubernetes edge computing vendors, branch office environments can still come in exotic forms, such as remote military bases and even fighter jets and weapons systems, as in the case of the U.S. Air Force.
The Air Force uses Rancher k3s for remote bases in Iraq, and already runs Kubernetes aboard F-16s. But its IT staff is also working with Red Hat to refine OpenShift support for the type of real-time operating systems used in weapons and some fighter jet functions.
"Real-time operating system support will be very interesting for telcos, planes and cars," said Nicolas Chaillan, chief software officer for the Air Force. "In a fighter jet or a car, you need a real-time CPU and operating system -- if the system takes five minutes to respond, you're in trouble."
Red Hat officials did not specify when real-time OS support will be available for Kubernetes, but Chaillan said he expects an early access version to be available with OpenShift 4.4 in the first half of the year.
The Air Force is also waiting for some advanced Red Hat OpenShift Service Mesh features to be supported in disconnected environments, which Red Hat officials said is also forthcoming.
"Red Hat Service Mesh is more than just Istio -- it's a Kubernetes Operator that performs some complex automations [that] include ... integrated observability with easy-to-select application patterns [such as] circuit breakers," said Mike Barrett, senior director of product management at OpenShift, in an email. "Because of those ease-of-use features, we ran into a small issue in specialized disconnected public cloud regions such as AWS GovCloud."
Red Hat has an early version of a workaround in progress for such environments that will become available to all users in OpenShift 4.4, a Red Hat spokesperson said.
How far will Kubernetes edge computing go?
Rancher and other Kubernetes edge distro purveyors believe Kubernetes will eventually find its way into tiny IoT devices, from surveillance cameras to cable TV set top boxes. Red Hat plans to focus its Kubernetes edge computing efforts on more substantial branch office environments for now, however.
"Edge can mean a lot of different things," Red Hat's Gracely said. "Kubernetes was built to be a large-scale system, and we want to build small form-factor Kubernetes, while avoiding creating any sort of fork or fork-like [version]."
The overall edge computing trend is here to stay, but some analysts said they have doubts about whether Kubernetes will reach the smallest extremes of edge devices.
"You don't run Kubernetes on cellphones and embedded devices -- you run it in local offices that have significant [infrastructure]," said Charles Betz, an analyst at Forrester Research. "I can also see it showing up in the huts that are built at the bases of cell towers now -- Kubernetes plays a role as a highly flexible architecture closer to the edge, but not all the way."
