Corero: Telecom carriers have fallen behind on DDoS defense

Telecom service providers have struggled with distributed denial-of-service attacks for years. But according to Corero Network Security's Dave Larson, they've failed to upgrade their DDoS defense strategies in the wake of new threats, like the Mirai botnet.

Larson, CTO and COO of Corero, based in Marlborough, Mass., recently spoke with SearchSecurity about how telecom carriers have failed to change their approach to DDoS protection and what should be done in light of powerful new IoT-enabled botnets.

Editor's note: Larson recently left Corero to join Hewlett Packard Enterprise.

In part one of this interview, Larson discussed the growing threat of powerful, 1 TB DDoS attacks and the potential for government regulation to address them. In part two, he discusses how the DDoS defense game has changed in recent years, and why telecom carriers have fallen behind.

Here are excerpts of the conversation with Larson. For the audio version of the interview, listen to this episode of the Risk & Repeat podcast.

You've talked about how telecom carriers aren't doing enough on DDoS defense. Why is this eluding them? I know you said it has to do with cost.

Dave Larson: No, it is cost. It's entirely business. It's cost, 100%. But it has to do with how they got here. DDoS attacks have been around for two decades. And for the first decade, no one could do anything about it, and it also wasn't very big. There wasn't enough bandwidth capacity to actually focus on events of size.

But then, a decade ago, that started to change. And Arbor Networks was in the market with a very, very powerful visualization tool, which they still have in the form of Peakflow. And they recognized that, yes, you can black hole the victim.

Actually, this is all done with routing protocols. They said, 'Why don't you just shift it off to the side, and we'll do something about it? And we'll see if we can do some deep packet inspection and mitigate some of it.' And they built a tremendous business doing that. The problem is that every carrier adopted it and has instrumented it.

And they don't want to change?

Larson: They don't want to change. So, they want to be in these scrubbing-center, localized environments, and they don't want to change. And, literally, it costs them, in terms of operational costs alone, more than it cost them to acquire it. We're starting to get some interest and gains on traction with our [DDoS defense] model, but we're making a statement that you can reliably put our [automatic DDoS mitigation] product in line with your traffic.

Now, from a carrier perspective, it doesn't mean deploy it on every Peering edge link, because just the logistics of that alone are difficult. But it means building your capacity in such a way that you can divert people constantly through it. So, in effect, it's always on. And you can start to think about using some new MPLS [virtual routing and forwarding] structures the way they were intended, and not be using route reinjection and [Generic Routing Encapsulation]backhaul and all this other stuff that's fragile, which are some of the reasons that it's operationally intensive. And so, we see these hosting operators that basically have clean edges and really high-capacity clean edges say, 'OK, I'll just do that.'

This hosting operator customer that has us at 720 GB -- we're not in line with all of their peering. They happen to be massive in their peering because they have a very large client base, but they scaled us to what they perceive is the size of the attack. And now, they also notice that the size of the attack is bigger, so we're engaged with them about how we can expand their footprint next year.

If the big operators like the Verizons and the AT&Ts and Deutsche Telekoms don't take that same approach, they're going to lose this revenue to the top services like Prolexic and Cloudflare, or they're going to lose it to the hosting providers, and they're just going to be relegated to selling bandwidth. And they say publicly that they want more revenue-bearing services, but they also are giant, hard-to-turn aircraft carriers.

You've talked about how the DDoS threat landscape has changed. What are you seeing in terms of new attacks?

Larson: In the last month or so, we announced the discovery of a new zero-day. And that zero-day is connectionless [Lightweight Directory Access Protocol (LDAP)], which is a [DDoS] reflection and amplification vector. Now, you should say LDAP is an open service on the internet.

We saw that and couldn't believe it.

Larson: I couldn't believe it, either. I don't spend my life deep in Shodan, but I did a Shodan search and I see 100,000 North American entries that will respond to LDAP openly on the internet. That's just staggering. And so, that's another area where the service providers need to get involved, in two dimensions. They could just be performing some consultative services to their customers using tools like Shodan and saying, 'Do you realize this is not a good idea? And it's not allowable, according to our [service-level agreements (SLAs)], for you to do this. Get that off. Get it behind a firewall.'

That's No. 1. But two, any zero-day reflection and amplification technique is dangerous when it's a zero-day, because it will be used by the one person who knows it. Reflection and amplification techniques, once they're public, they become diffused because everybody reflects. And, eventually, it gets remediated, and people fix the problems because they don't want to deal with the inbound request traffic that's going on. So, more and more of these are going to occur because tools like Shodan exist.

The thing about this reflection attack is, the very first time we ever saw it, it was instantaneously 70 GB against one website. It was in a hosted environment that we protect. We monitor and we manage for them. So, our security operations team saw it. The cool thing about our solution is that we 100% mitigated it without false positives. No one ever knew. Our customer, the hosting data center, was unaware that it was going on, except that they could see it in our reports after the fact. We did send them alerts. They respond to the alerts. And then the customer that was the victim, the intended victim, never knew and has not been told. No human did anything. It just works.

The scary thing about it is that there are going be other ones like it. And in environments that aren't protected by our product, people need to know that port 389 UDP is a problem so that they can prestage a countermeasure. We made the press release in that way so that Arbor can put up a countermeasure, and Prolexic can do it, and CloudFlare can do it, and everyone can do it. It's actually in the best interest of the community to share that information and get it out there. It's also self-serving in that it allows us to claim victory in something and get some marketing benefit.

Shouldn't the end-user customer know?

Larson: We don't know [if they do]. Maybe they do know. The victim was not our customer -- it's the customer of our customer. And as far as we knew, they had not communicated it for whatever reason. But this is a case where this particular service provider is not yet offering it as a paid service. It is just default in their environment. They claim, in generic terms, that they give DDoS protection to their customers. But they don't have SLAs associated with it. They're asking people to charge for it. They're looking to do it.

And I think the reason that they don't communicate downstream to their customers as much as you might expect is that they're still not sure what they want to charge for it. And they're in a business assessment model, so they'd rather just keep it quiet.

Is it a normal practice for DDoS defense?

Larson: Yes, actually, about 50-50. Block Communications, which is a regional service provider, does not charge. This is part of their service. They look at it as just good business. And because they don't charge for it, they also view themselves as being less likely to be taken to task if an event occurs that gets through us. So, they don't have an SLA that they have to meet. We give you premium service, and it includes a measure of DDoS protection capability. The reality of it is they're very happy about it because very little, if anything, actually gets through the system.

But one of the other talking points that we constantly ask folks in your position to help us carry water for the service providers is ingress filtering. The biggest problem with a volumetric DDoS attack is reflection attacks. All of them rely on the fact that services on the internet respond to spoofing. So, if you're Verizon and your clients are sending Chinese source IP addresses because someone's trying to attack something over in China, you should stop that. There's no reason for that. Every provider edge router or access router in the world is capable of implementing ingress filtering.

Why do you think that they don't do ingress filtering?

Larson: Because, again, it's costly. There's an underlying business driver for why they don't. All of the big operators rely on peering agreements that they audit for balance. It's good sometimes to send traffic -- even if it's junk. And I don't know if I fully understand it myself. It's very easy for them to drop the traffic in the last mile heading to the customer. They prefer not to from a balance of trade perspective in the other direction. It's very, very strange.