BOSTON -- Blue-chip IT shops have established production container orchestration deployments. Now, the question is how to make them fully secure within large, multi-tenant infrastructures.
For starters, users must make changes to default settings in both Docker and Kubernetes to close potential security loopholes. For example, a default component of a container image, called docker.sock, that's mounted without proper security tools to control its usage is vulnerable to an attack that can use it to access the host operating system and then back-end databases to exfiltrate data. Similarly, the Kubernetes API's default setting could potentially let containers access host operating systems through a malicious pod.
"Containers also have the same problem as any other VM: They can talk to each other via an internal network," said Jason Patterson, application security architect at NCR Corp., an Atlanta-based maker of financial transaction systems for online banking and retailers, in a presentation at the DevSecCon security conference held here this week. "That means that one misconfiguration can compromise pretty much all the containers in the environment."
Container security configuration settings are critical
NCR uses Red Hat's OpenShift, which restricts the Kubernetes API settings out of the box, but OpenShift users must set up security context constraints, Patterson said.
Heroku's Etienne Stalmans presents on container security at DevSecCon.
In general, it's best to constrain a user's permissions and each container's capabilities as tightly as possible and, ideally, configure container images to whitelist only the calls and actions they're authorized to perform -- but this is still uncommon, he added.
It's possible to limit what a container root user can do outside the container or the host on which the container runs, said Etienne Stalmans, senior security engineer at Heroku, based in San Francisco, in a separate DevSecCon presentation. To do this, container administrators can adjust settings in seccomp, an application sandboxing mechanism in the Linux kernel, and configure application permissions or capabilities.
"That still makes them a privileged user, but not outside the container," Stalmans said. "Overall, it's best to drop all capabilities for all container users, and then add them back in as required."
Some highly sensitive applications require isolation provided by a hypervisor to remove any possibility that an attacker can gain host access. Vendors such as Intel, Google and Microsoft offer modified hypervisors specifically tuned for container isolation.
DevSecCon presenters also touched on tools that can be used to minimize the attack surface of container and host operating systems.
Beeline, which sells workforce management and vendor management software, uses an Oracle tool called Smith that strips out unneeded OS functions. "That shrank our Docker image sizes from as much as 65 MB to 800 KB to 2 MB," said Jason Looney, enterprise architect at Beeline, based in Jacksonville, Fla.
Container security experts weigh host vs. API vulnerabilities
Overall, it's best to drop all capabilities for all container users, and then add them back in as required.
Etienne Stalmanssenior security engineer, Heroku
Most of the best-known techniques in container security restrict attackers' access to hosts and other back-end systems from a compromised container instance. But prevention of unauthorized access to APIs is critical, too, as attackers in recent high-profile attacks on AWS-based systems targeted vulnerable APIs, rather than hosts, said Sam Bisbee, chief security officer of Boston-based IT security software vendor Threat Stack, in a DevSecCon presentation.
Attackers don't necessarily look for large amounts of data, Bisbee added. "Your security policy must cover the whole infrastructure, not just important data," he said.
Kubernetes version 1.8 improved API security with a switch from attribute-based access control to role-based access control (RBAC). And most installers and providers of Kubernetes, including cloud container services, now have RBAC Kubernetes API access by default. But users should go further with configuration settings that prevent untrusted pods from talking to the Kubernetes API, Stalmans said.
"There is some discussion [in the Kubernetes community] to make that the default setting," he said. It's also possible to do this programmatically from container networking utilities, such as Calico, Istio and Weave. But "that means we're back to firewall rules" until a new default is decided, he said.
