Capital One hack highlights SSRF concerns for AWS

The theory: An SSRF vulnerability was exploited

However, some experts have theorized that a server-side request vulnerability was used by the threat actor to obtain the credentials.

Evan Johnson, a security professional at Cloudflare Inc., wrote on his personal blog about the Capital One hack: "Every indication is that the attacker exploited a type of vulnerability known as server side request forgery (SSRF) in order to perform the attack." An SSRF vulnerability, Johnson explained, allows a threat actor to trick a target server into connecting to a system to which it did not intend to connect.

Johnson theorized the threat actor exploited an SSRF vulnerability to connect to a Capital One EC2 instance and then access the AWS metadata service, which can be used to retrieve temporary credentials. Once the metadata service is accessed, Johnson wrote, it's "extremely easy" for someone to access IAM roles within AWS.

"There is no authentication and no authorization to access the service," he wrote. "Using curl, it's pretty easy to explore the static files that are available within your metadata service."

Following Johnson's post, infosec journalist Brian Krebs reported that an SSRF vulnerability was used to access a server on which the web application firewall was running, according to an anonymous source familiar with the Capital One breach investigation.

An infosec professional who previously worked at Capital One and who wished to remain anonymous told SearchSecurity that Johnson's theory holds water. An SSRF vulnerability, the source said, would allow an unauthorized party to connect to the metadata service on the WAF instance. The source, who stressed they had no knowledge of current IAM policies or configurations within Capital One's AWS environment, said that once access to the metadata service was established, the attacker could obtain the WAF role credentials, as well as potentially others.

Erik Peterson, founder and CEO of CloudZero Inc., also agreed with Johnson's theory and said SSRF has been a known threat for cloud providers, including AWS, for some time. Peterson, then with Veracode, gave a presentation at RSA Conference (RSAC) 2015 that in part described how an SSRF vulnerability could be used to obtain metadata such as API keys and credentials for a cloud environment.

Several experts, including Johnson and Peterson, said requiring a special HTTP header to communicate with the AWS metadata service would mitigate some SSRF threats (Google currently requires such headers for requests that attempt to access its cloud metadata).

"There are simple things that you can do with HTTP headers that can prevent these attacks, but Amazon doesn't do them," Peterson said. "I recommended this to members of the AWS team after my talk, but nothing ever happened."

It's unclear if AWS made attempts to address its exposure to SSRF since Peterson's 2015 RSAC session. But at least one high-profile AWS customer indicated the problem still exists. In a now-deleted tweet responding to Johnson's blog post, Netflix senior security software engineer Michael Wardrop said "Netflix asked AWS to follow GCP [Google Cloud Platform] and require a special header for metadata service requests. Unfortunately we didn't get a satisfactory response."

When asked about Johnson's blog post and Wardrop's tweet, an AWS spokesperson maintained that its cloud infrastructure was not to blame.

"It's inaccurate to argue that the Capital One breach was caused by IAM in any way. The intrusion was caused by a misconfiguration of a web application firewall and not the underlying cloud-based infrastructure," the spokesperson's statement read.

"Customers can always better protect themselves through additional layers of security depth, including how they set permissions, what access they allow for each authenticated role, what additional protections they deploy from our platform, and what intrusion detection features and alarms they employ and monitor."

The spokesperson added that AWS offers "more security capabilities and layers than customers can find anywhere else including within their own data centers, and when broadly used, properly configured and monitored, offer unmatched security -- and the track record for customers over 13+ years in securely using AWS provides unambiguous proof that these layers work."

Wardrop's tweet was removed some time after AWS sent its statement to SearchSecurity.

Addressing SSRF threats

Several security professionals not only support Johnson's theory regarding the Capital One hack but also warned that future SSRF attacks could jeopardize enterprise customers.

The first concern we had when we saw the [Capital One] breach was finding out if anyone else was at risk.

"The first concern we had when we saw the [Capital One] breach was finding out if anyone else was at risk," said Ray Watson, vice president of innovation at managed SD-WAN vendor Masergy. "We looked at the possibility that a zero-day was involved, but we ruled that out."

Based on discussions with infosec professionals and AWS experts, Watson said an SSRF vulnerability provided the most plausible explanation for how Thompson allegedly accessed the WAF role credentials via Capital One's metadata service. And while he said it's not a zero-day, SSRF vulnerabilities are a significant concern for public cloud clients.

Scott Piper, an AWS security expert at consulting firm Summit Route in Salt Lake City, said he "strongly agrees" with Johnson's post and said SSRF threats on AWS have been a concern for some time.

"This is a known weakness in AWS that has been discussed at infosec conferences since 2014 and has been a repeated request for AWS to improve, and is something that should be trivial for AWS to fix," Piper said.

But Peterson offered another view regarding the HTTP header requirements. "It would be a big change because all the APIs that connect to the metadata service -- dozens, maybe hundreds of them -- would be broken," he said.

Another way to address the issue, Peterson said, is through IAM.

"Even if an attacker gets the WAF credentials, it wouldn't matter if the credentials were limited to very specific permissions," he said. "WAFs are never perfect, and application security is never perfect, but if you follow the principle of least privilege you can at least reduce the blast radius of an attack and limit what the attacker can achieve."

The source who previously worked at Capital One agreed, though they said navigating AWS IAM tools can be challenging. While AWS has given enterprises the ability to create extremely granular IAM policies and permissions, they said it has also made IAM on the cloud provider "esoteric."

"When I worked at Capital One, they had an entire team devoted to [AWS] IAM because that's how complex it can be," the source said. "Setting permissions can be tricky, too. You may be in a rush to push out code, and if you set permissions that are too narrow, you'll be flooded with troubleshooting requests. So what ends up happening is, you set wider permissions and greater access than is necessary."

The source also said Capital One's homegrown AWS security tool, Cloud Custodian, was designed to detect excessive permissions in, for example, a WAF role but for whatever reason wasn't able to avert the breach. "Cloud Custodian does a lot of things like [AWS] security group validation and IAM policy validation," the source said. "But there's so much going on in the cloud that there's bound to be one small slip-up, and unfortunately it can lead to a breach like this." Cloud Custodian was released to open source last fall.

While it may not satisfy all customers, there's a possible reason for AWS' public response to the matter, Summit Route's Piper said.

"AWS' philosophy is to draw a very strict line with regard to the 'shared' responsibility model, which I tend to refer to as the split responsibility model because there is no sharing involved," he said. "Access to the metadata service is the customer's problem to secure, so there is no sense to them in securing it, because the only way it can be abused is if someone exploits something the customer installed on an EC2. That is my understanding of their reasoning, at least."

Other potential breaches

Additional organizations may have been compromised, according to the Slack messages from the alleged attacker. Those organizations include Vodafone, the Ohio Department of Transportation, Michigan State University and Infoblox.

Several of those organizations told SearchSecurity they're conducting investigations but have not found any evidence that their networks had been compromised.

However, the former Capital One infosec worker said such evidence could be hard to come by because, on the surface, this kind of activity will resemble normal traffic.

"The signal-to-noise ratio would be very low," the source said. "It's like looking for a needle in a haystack."

Peterson agreed and said that while security tools like AWS CloudTrail collect a lot of data, it would be a challenge to find where an SSRF attack may have been used and which instance and corresponding metadata service were accessed. It would also be difficult to find anomalous activity related to specific temporary credentials like the WAF role.

"All activity is logged in CloudTrail, but it's very verbose and a lot of people don't pay attention to all that data until after the fact," Peterson said. "Every API call is logged, but if I had credentials and was hopping through other EC2 instances, then it wouldn't look surprising at all."

If an SSRF vulnerability was used in the Capital One hack, it's not a good look for AWS, Piper said.

"Capital One has one of the most respected teams for AWS security. They have one of the most popular open source tools for cloud security. Their CISO was one of only two non-AWS people who shared the stage with the CISO of AWS at the re:Inforce conference," he said. "I think the situation could be a wake-up call that AWS has some improvements they need to focus on still."