Kubernetes policy project takes enterprise IT by storm
A Kubernetes-friendly compliance as code project hosted by the CNCF has caught on among large enterprises in the first half of 2019, largely through word of mouth.
An open source compliance as code project has gained a groundswell of popularity over the last six months among enterprise IT pros, who say it simplifies and standardizes Kubernetes policy management.
The Open Policy Agent (OPA), an open source compliance as code project founded by former VMware employees, was used at Netflix as early as 2017 and accepted into the Cloud Native Computing Foundation (CNCF) as a sandbox project in March 2018. Netflix gave an OPA demonstration at KubeCon in December 2017, and Intuit and Capital One followed at KubeCon in December 2018. After the project advanced to the CNCF's incubating stage in April 2019, and was demonstrated a third time at KubeCon EU in May 2019, it began to generate mainstream buzz.
"It's been showing up everywhere," said Mike Ryan, a DevOps consultant who advises ABN AMRO Bank NV in the Netherlands on its cloud-native strategy implementation. ABN AMRO has not deployed OPA in production yet but has tested it. "It's found a really sweet market spot of doing generic policy enforcement."
OPA strikes a nerve among enterprise IT pros who want to put cloud-native infrastructures and applications into production, but must not run afoul of regulatory compliance rules. OPA creates a code-based subsystem that automates policy decisions in real time as workloads consume infrastructure.
"All of us who are trying to build workloads that run in the cloud are trying to solve the same problems," said Kevin Hoffman, a software engineer at Capital One. "Once people knew that OPA was there, and that we don't have to solve that particular problem on our own anymore, it just became a no-brainer to use it."
OPA rides wave of Kubernetes policy development
While OPA, which describes policies as code in a language called Rego, isn't limited to Kubernetes policy enforcement, it's been most widely used so far in Kubernetes environments.
The project's founders haven't disclosed how many companies use OPA, but it boasts a healthy public reference list of early adopters, most of them also Kubernetes users, and OPA is the basis for the Gatekeeper Kubernetes policy controller project developed by Google, Microsoft and others. OPA has also been publicly endorsed by AWS for use with its Amazon Elastic Container Service for Kubernetes and Amazon Elastic Container Registry.
"Let's say that you want to define that Person A can deploy resources but Person B cannot," said Fernando Montenegro, analyst at 451 Research. "This is a way to define that policy in a much simpler way than the native policy language [in Kubernetes]."
Styra's OPA dashboard coordinates policies among multiple IT environments
OPA offloads policy decisions from the Kubernetes infrastructure itself; Kubernetes admission controllers query OPA's Kubernetes policy engine before they allow access to resources, instead of responding to policy logic encoded within the application. This means policies can be updated independently from the application, while they remain deployed alongside application functions in a sidecar, library or host-level daemon, so Kubernetes policy decisions happen in real-time. OPA can also be used as a quality check on application policies in the CI/CD pipeline, through integrations with unit testing tools and a plugin for Visual Studio Code.
Treating compliance as code means adopting best practices from the software development process. One of these is Don't Repeat Yourself. Decoupling policy from applications, and reusing policy definitions in multiple locations, is a good implementation of this rule.
Mike RyanDevOps consultant, ABN AMRO Bank NV
Consistent Kubernetes policy enforcement within the DevOps pipeline as well as within Kubernetes production infrastructure is an important part of OPA's appeal, said ABN AMRO consultant Ryan in a blog post.
"Treating compliance as code means adopting best practices from the software development process," Ryan wrote. "One of these is Don't Repeat Yourself. Decoupling policy from applications, and reusing policy definitions in multiple locations, is a good implementation of this rule."
As Kubernetes environments grow to encompass Istio service mesh and Knative event-based orchestration in what Google calls the open cloud stack, the fact that OPA lends itself to Kubernetes policy enforcement but can expand to include those adjacent utilities boosts its appeal.
"There's a lot of promise in how we might scale and better leverage OPA as the ecosystem grows," said Andy Domeier, senior director of technology operations at SPS Commerce, a Minneapolis-based communications network for supply chain and logistics businesses, which uses OPA in production. "We're thinking through governing entitlements and access within the service [mesh] architecture."
Early adopters say OPA is simpler to use in the long run than native Kubernetes policy features, and like its strict focus on policy enforcement, without attempts to replace other infrastructure as code tools. Its open source governance within the CNCF also appeals to Kubernetes users over vendor-driven tools such as Chef InSpec and HashiCorp Sentinel.
However, almost every OPA early adopter who has spoken publicly about the tool acknowledged a steep learning curve for Rego.
"When I look at [Rego] at a distance, it looks like regular programming language code," said Capital One's Hoffman. "But it doesn't work that way – it's more logical and higher-level than that, so it takes a little while to context-switch until you get used to it."
Once past the initial confusion, Rego turned out to be a powerful way to express detailed policy concepts, Hoffman said.
The challenge of the Rego learning curve isn't lost on OPA's founders, who also run Styra Inc., a commercial implementation and support package for OPA with a centralized user interface for policies across multiple clouds. Styra came out of stealth at the RSA conference in March of 2019, and won't disclose how many paying customers it's garnered so far.
"A decent GUI would be helpful for displaying rules to other teams and giving people more visibility, and I've seen the demo of what the commercial company is building," said ABN AMRO consultant Ryan. "As more people use OPA with tools like Terraform and Jenkins, it will be interesting to see what frameworks emerge around that to make these things more reliable and repeatable."
Dig Deeper on Systems automation and orchestration
