kalafoto - Fotolia
Reality strikes when enterprises deploy Kubernetes in production
It's easy to run containers with Kubernetes, but Kubernetes in production -- with enterprise security, monitoring and compliance demands -- is a different story.
Developers spin up a Kubernetes cluster in a sandbox and unlock application design and delivery capabilities that they want. Then they try to implement it into an enterprise IT environment and everything goes awry.
The hang-ups with enterprise Kubernetes implementation arise in all the stuff beyond the app code -- security, auditable logs and performance tracking. To accommodate the diverse IT setups and needs around containers, the open source community that steers the Kubernetes roadmap prioritized dynamic extensibility, rather than conquer every possible feature within the core project.
These real-life concerns of Kubernetes in production push most enterprise adopters to an ecosystem of supporting tools that work at scale. Authentication, log aggregation and other backing ecosystem requirements also open the door for a range of managed Kubernetes choices that incorporate these tools. The right approach for enterprise Kubernetes adopters depends not only on their vendor strategy, but also why they brought in containers to begin with.
From developer laptop to production deployment
Kubernetes has widespread appeal among developers, because it enables them to deploy applications with distributed components in an elegant configuration.
"Developers are asking their companies of all sizes to adopt Kubernetes as a solution so they can bring what they're familiar with to the office," said Michael Jacobi, director of global field services at Altoros, a professional services and consulting firm headquartered in Sunnyvale, Calif.
When developers create or refactor code in containers, they can pull any public image from Docker Hub with no red tape -- but when it comes time to deploy, they hit a roadblock of protective policies. Operations and security teams, for example, require an internal registry of vetted images, which must pass vulnerability analysis, Jacobi explained. The head of security wants assurances that no rogue code can infiltrate the app during the development and deployment process.
That single code release prompts IT teams to check Sarbanes-Oxley Act compliance, multifactor authentication, reliable connections between the app code and a backing service also deployed on Kubernetes, performance tracking, auditability via aggregated logs, and other ops concerns. Stateful databases and networked storage volumes must deal with multiple stateless container instances' requests at the same time, in a predictable way.
While developers led the charge to get Kubernetes into the enterprise, DevOps and IT operations teams have largely caught up, according to upcoming research from Enterprise Management Associates (EMA), "The State of Kubernetes in 2019." However, Kubernetes still is a new technology with a small number of production deployments compared to virtualization platforms such as VMware vSphere and Microsoft Hyper-V, said Torsten Volk, analyst at EMA.
Container orchestration and dynamic extension
Kubernetes is developed as an open source project under the direction of the Cloud Native Computing Foundation (CNCF). While its Kubernetes roadmap includes a focus on enterprise concerns such as authentication and authorization, it cannot address all operations needs for all deployment scenarios.
All the features required for real-life deployments became a bottleneck, said Mike Danese, a software engineer on the Kubernetes team for Google Cloud, who first contributed to the project in November 2014, when it had about 2,000 commits. "It became a real focus of the project to make sure that people could build extensions outside of core Kubernetes," Danese said. He co-chairs the Auth special interest group for Kubernetes within the CNCF.
Kubernetes has forged ahead with dynamic extension rather than all-encompassing core features. Dynamic extension relies on the Kubernetes API to enable policy expression against its control plane. Examples include dynamic admission control via webhooks and the Container Storage Interface.
"It's components that can exist -- and make sense to exist -- beyond just Kubernetes," said Maya Kaczorowski, product manager of container security at Google. These types of controls and services for containers and microservices code apply across the application lifecycle and are not specific to the container orchestration layer.
"[Kubernetes setup and management] is a fairly narrow view of container security," Kaczorowski said. "There are a lot of other components that users need to get right beyond just Kubernetes." Call together security, operations and developers to nail down what's running in containers and how you'll protect deployed instances.
Altoros, which is a member of the CNCF, identifies dozens of integration points for enterprise Kubernetes in production: messaging queues, logging, metric systems, access management and more. "There are opportunities to plug those holes, and so we see a lot of vendors in this ecosystem coming to the table with products to augment and complement the Kubernetes container orchestration piece to build out a platform that enterprises are interested in," said Andrei Yurkevich, CTO of Altoros.
DIY or managed enterprise Kubernetes
Organizations can catalog and address integration points for open source Kubernetes in-house or with a partner, or they can pick a vendor distribution. Industry experts agree that the first question is why the company needs Kubernetes, then whether its value lies in a bespoke build or a supported option.
Some companies want the extreme flexibility of rolling their own Kubernetes, or they're still experimenting and not ready to commit to a managed option. Others run open source Kubernetes because they were early adopters, when managed options were more limited. Large, dedicated operations teams can maintain secure, efficient and compliant Kubernetes in production.
Hosted options eliminate the difficult task of initial configuration, and the burden of ongoing cluster operations and management, Kaczorowski said. In addition, vendors cover security patches, direct integrations with cloud features, and control plane management.
There's still a degree of control and customization with a managed product, Danese said, although vendors don't expose all the knobs.
Red Hat OpenShift and Pivotal Container Service are popular for on-premises enterprise deployment, Alteros' Jacobi said. Cloud Foundry offers an opinionated platform powered by Kubernetes. Even Rancher, Docker Inc. and Mesosphere promote Kubernetes products. Microsoft Azure Kubernetes Service (AKS), Amazon Elastic Container Service for Kubernetes (EKS) and Google Kubernetes Engine (GKE) are core choices for off-premises deployments; EMA's research found these public cloud container services were among the fastest growing areas of interest for DevOps professionals.
Many adopters want a hybrid approach, but consistency between managed Kubernetes instances is easier said than done, Volk said, because performance, availability, upgradability, compliance and even cost are affected by each Kubernetes vendor's secret sauce. "For example, a placement policy that works very well for GKE creates availability problems in EKS or AKS," he said.
Not sure which open source or vendor-driven Kubernetes model is best for your production deployment? Take a step back and think about the bigger system in play. Altoros' Yurkevich and Jacobi insist that enterprises must know what they're bringing in Kubernetes to fix or to enable for the business. Assess what's driving the move, not just what the tool can do. Understand the business decisions -- needs and challenges -- that lead to a container deployment, and the workloads in those containers. Kubernetes is flexible, running workloads from databases to Java-based microservices, but it won't accomplish anything unless the user has a clear plan.
