Getty Images/iStockphoto
Why and how to use eBPF for Kubernetes scaling
Scaling Kubernetes clusters has always been a challenge -- but eBPF makes the process smoother with service-specific monitors and a lack of special instrumentation.
Rightsizing Kubernetes clusters is complicated. With eBPF, however, a relatively new Linux kernel framework, scaling Kubernetes can be much simpler.
The extended Berkeley Packet filter, or eBPF, is a framework that makes it easy to deploy kernel mode programs in Linux. With these programs, admins can collect low-level data about resource consumption on a process-by-process basis. And when it comes to Kubernetes, eBPF eases the traditional challenges admins face when scaling cluster resources.
Kubernetes scaling challenges
Traditionally, it has been difficult to monitor and scale Kubernetes clusters for several reasons, including the following:
- Distributed architecture. A Kubernetes cluster usually consists of multiple nodes, each running different types of containers. Collecting metrics and log data from across the environment was challenging.
- Multiple services and components. Kubernetes consists of many different parts: an API server, a key-value store and a scheduler, for example. Each service generates different types of metrics and exposes them differently. This makes it difficult to track the performance of all Kubernetes services.
- Abstracted metrics. Kubernetes reports some metrics and events data. However, the data is fairly high-level. It doesn't record which specific processes within a Kubernetes cluster are doing what.
- Limited log data. Kubernetes applications can, and usually do, generate logs -- and these logs help IT admins monitor application health. However, the log data is typically application centric. In other words, logs record information about the state of applications, but not the servers that host them. The application logs therefore reveal little about Kubernetes host infrastructure. And although admins could collect logs like syslog from individual nodes, the information recorded in syslog is not very granular. For example, admins can't use syslog to figure out which network packets are associated with which processes on a node.
Due to these challenges, admins struggle to determine how many resources their Kubernetes clusters require and how to scale them appropriately. The log and metrics data provided by Kubernetes itself, applications running on Kubernetes and the OSes that manage Kubernetes nodes don't provide the kernel-level visibility into application resource consumption that teams need to fine-tune resource allocations on a dynamic basis.
And that, of course, is problematic because over-provisioned clusters waste money -- especially in the cloud, where businesses usually pay based on the resources they allocate. Clusters that are under-provisioned risk performance problems due to lack of available resources.
How eBPF simplifies Kubernetes scaling
By using eBPF programs to collect granular, kernel-level data from across all the nodes in a Kubernetes cluster, teams can simplify several of the issues associated with Kubernetes scaling.
- Service-specific monitoring. With eBPF, it's easy to monitor specific processes. Teams can use eBPF to track the resource consumption trends of each process running within their cluster. And because every Kubernetes service is ultimately just a process running on a node somewhere in the cluster, eBPF offers a consistent means to monitor all services across the cluster via a single tool set.
- More detail than logs. Because eBPF programs can collect highly granular, process-specific data, they offer a greater degree of detail and visibility than most logs and metrics can provide.
- No special instrumentation. Unlike logs, which typically collect in Kubernetes with log agents that run in sidecar containers, eBPF works directly at the OS level and is installed by default. That makes it easy to deploy. It also means that -- unlike sidecar containers -- eBPF programs add very little resource overhead to the cluster because they consume minimal resources when they run.
With eBPF, Kubernetes admins can simplify collecting the monitoring data they need to scale clusters, while also getting access to more detailed and granular information.
Using eBPF with Kubernetes
Of course, eBPF doesn't actually scale the Kubernetes cluster. It provides insights that can be helpful for admins to manage scaling operations.
Additionally, because eBPF is an OS-level framework, it must deploy on each individual node within a Kubernetes cluster. Currently, there are no tools that make it easy to centralize eBPF monitoring across a cluster of servers. This makes orchestration a challenge if admins want to deploy eBPF programs directly.
However, teams can address these limitations by taking advantage of third-party Kubernetes management tools that use eBPF internally. For example, Cilium is an eBPF-powered open source network monitoring tool that can manage and scale Kubernetes network resources. Some proprietary monitoring platforms, like Splunk and Kentik, are also now beginning to use eBPF.
Tools like these deploy eBPF-based monitoring agents automatically within Kubernetes clusters, while also helping teams make sense of the data eBPF provides. In these ways, they make it easy to use eBPF in Kubernetes without having to write eBPF programs from scratch or orchestrate their deployment across all the nodes in a cluster.
