Container security improves overall enterprise IT posture
Enterprise IT pros worried about security when containers were first introduced, but as containers are put into production, they can actually present ways to tighten security practices.
Container security has evolved from a potential security risk to a security boon as enterprises put Docker and Kubernetes into production.
Containers aren't without their security downsides. Without careful configuration and management, container security vulnerabilities can allow an attacker to break out of a container and access the underlying host, with serious consequences. However, in the right hands, the way containers lend themselves to granular isolation of individual application processes, along with immutable infrastructure concepts, can boost IT security overall.
"In the past, when you didn't have containers, you'd have ops teams go in and make changes to [individual servers], and sometimes those changes weren't uniform, they weren't pushed out to everything, and then you'd have issues or errors," said Ross Hosman, head of information security at Recurly, a subscription and billing management service provider in San Francisco. "We're looking for uniformity and scalability and allowing our developers to go a lot faster."
Uniformity of compute resources also helped from a container security standpoint, as did container security tools that automate security policy enforcement. Recurly, which has over 2,000 customers, including Sling TV and CBS, specializes in high-volume and high-velocity transactions. To keep up with the pace of its business, the best container security approach is to stay out of the way of developers, Hosman said.
"I can take a container and run it and profile it and say, 'It makes these network calls, it runs these binaries and it has these packages,' and restrict that container to only do that," Hosman said. "That's something we didn't have in virtualization or on stand-alone servers, unless you went and did the SELinux policies and got really deep down into it. It wasn't automated like it is today."
Recurly uses Terraform templates to install Aqua Security agents automatically in GKE clusters at deployment. The company also embedded Aqua's container image scanning tool in its CI/CD pipeline, so that container security flaws break developer builds before they're deployed to production. Aqua also prevents developers from deploying container images from outside the company's container registry.
Ross Hosman
Once apps reach the production Kubernetes environment, security policies enforced through Aqua allow all developers and IT ops pros read-only access to their activities. This improves and speeds up application development, and lets IT pros troubleshoot faster than they could with VMs -- in the past, Recurly's security staff more carefully restricted such access without automated whitelisting tools available for containers. Also, since containers separate application processes from the underlying host, admins can more strictly lock down the host itself with tools such as Google's Container-Optimized OS.
"We are heavily running immutable hosts today, so even if you break out of a container and get on a host, good luck," Hosman said. "You can't run anything, install anything, or pivot to anything, and if we restart the host, everything just resets."
Recurly's goal is to move away from human responses to alerts, whether they refer to IT monitoring or container security issues, and toward a remediation response to issues through code.
"So many companies are stuck in that alert-response model where you have to have an engineer respond to a problem, but that's just not quick enough in the cloud," Hosman said. "It takes four minutes for somebody to find a key in GitHub and start spinning up instances -- if we see [something like that] we're going to alert, as we should, but our goal is to remediate it, via code, within 60 seconds."
Looker centralizes multi-cloud container security
Containers appealed to developers because they abstract application processes from the underlying infrastructure, which makes application code more portable between developers' laptops and dev, test and production environments. As containers evolved, IT ops pros used this portability to centralize multi-cloud management. Now, the portable nature of containers has caught on among IT security pros as well.
We are heavily running immutable hosts today, so even if you break out of a container and get on a host, good luck. You can't run anything, install anything, or pivot to anything, and if we restart the host, everything just resets.
Ross HosmanHead of information security, Recurly
"When I started here almost two years ago, we [used] classic EC2 instances in AWS, but last year we got word of a change in direction to be in every cloud, using Kubernetes and multi-tenancy," said Richard Reinders, manager of security operations at Looker, a business intelligence firm in Santa Cruz, Calif. "And I had to come along for that ride in security."
Reinders initially saw Kubernetes management in AWS, GCP and Azure as a challenge, but he began to see its advantages as Looker put container security tools from StackRox and Sumo Logic into production in the quarter of 2019. These tools constantly scan running containers, as well as container images predeployment, which means Reinders' security team gets faster alerts to vulnerabilities and where they're located in the IT infrastructure.
"I could go and tell a [host] scanning solution to scan every day, but then I still don't know where those vulnerable images are," Reinders said. "It's that combination that's valuable."
Looker locks down commands, containers
Like Recurly, Looker has used container security tools to automate container security configurations and troubleshooting. It also views containers as immutable resources, which eliminates error-prone patches and updates at the container level, and whitelists the commands that can be run against the container infrastructure using custom processes in Sumo Logic.
"We don't want to detect if a particular command is run -- we want to know if any command is run at all on the command line, and we will whitelist specific commands we approve of," Reinders said. "We can use that for network connections, processes and a number of other things because of investments we've made over time [in customizing Sumo Logic]."
Richard Reinders
StackRox also offers visibility for compliance record-keeping and documentation in multiple cloud environments, regardless of whether the public cloud provider or Reinders' team directly manages them. The vendor beefed up these features as a result of Looker's early requests.
"If it's out of my control, maybe it's not something I'd ask the ops team to do, but it's something I'd still want to be aware of," Reinders said. "If it is under our control I want to know about it and what to do about it."
It's unknown how Looker's acquisition by Google, expected to close later this year, will affect its container management -- Looker wouldn't comment on the still-pending merger. But overall, Reinders said he'd next like to unify IT monitoring across clouds the way StackRox and Sumo Logic have centralized container security management.
"We use the services every cloud offers, but we want to centralize and simplify everything down to get a consistent outcome without multipoint solutions," Reinders said.
Ross Hosman
Richard Reinders
