After Kaseya attack, MSPs should rethink RMM practices

A history of MSP software breaches

October 3rd, 2018: CISA warned about a previous two-year history of threat activity against managed services providers. CISA warned MSPs generally have direct and unfettered access to their customer's networks and may store customer data on their own internal infrastructure. By servicing a large number of customers, MSPs can achieve significant economies of scale. However, a compromise of one part of an MSP's network can spread globally, affecting other customers and introducing risk.

Also of note in the advisory: Managing the supply chain risk was highlighted.

Of course, in the timeline, [the July 2021 Kaseya attack was] not even the first breach. Kaseya was involved in a breach in 2018. [Kaseya] and ConnectWise had a breach in 2019. ConnectWise, too, was used as an entry point in September 2019 -- oh, and another in November 2019. Webroot -- yep, they had one, too, in June 2019. NinjaRMM had one in July 2019, but, to be clear, it was not a supply chain issue. It was just using RMM as an entry point. I didn't even mention SolarWinds and the supply chain attack there.

From my vantage point, this is a repeat of history, not something new.

Let's hear from Kaseya's own CEO on his take about the incident.

[Clip of video published by Kaseya on July 6, 2021, about the cyber attack]

Fred Voccola, CEO of Kaseya: This breach has gotten incredible scrutiny from the press. All of a sudden, cybercrime and ransomware has become the topic the day, and we're caught in the middle of it. And people make the story, make the impact of this, larger than what it is.

[Voccola], much like many in the community, presents the incident as something the world has just woken up to -- when this attack vector was highlighted three years ago. This is clearly not the first time this has happened. It's not even the first time for his company.

But for the record, [Voccola] is also wrong. ProPublica did an entire piece on MSPs as targets in September of 2019. I think it's quite disingenuous to claim the world is waking up to this. What happened instead is that it's escalating.

He's resigned to the status quo, too.

[Clip from the July 6, 2021, Kaseya video]

Voccola: We all have to take a step back and realize this is the world we live in.

Lastly, his message to you is that this is the plan:

[Clip from the Kaseya video cited above]

Voccola: … and recognizing that our security plans, the architecture of how we run IT Complete in our businesses, prevented what could have been something much greater. Due to the things I mentioned, the impact of this incredibly sophisticated attack is very minimal.

Now, I highlight all of this not to dump on Kaseya. Really. In fact, as is clearly the trend in the history, if any other vendor was hit, the IT services community would be hearing the same message from whoever was in the hot seat. This is the world we live in: 'Our security did what we promised, and the media is just waking up to this.' The vast majority of leaders in the software vendor space will sound just like that.

If you didn't see my video about financial incentives, I think you should. For companies of [Kaseya's] size, writing a check makes the security problem go away and lets them get back to business.

The problem with RMM toolkits

I want to offer another perspective.

Small customers are letting their IT providers install a piece of software with total administrative control into their network that links them to a vast network of other companies, where IT becomes a single point of access for criminal gangs to enter and exploit.

Seriously, think about it. You've installed a single system which has total and absolute administrative control over your entire customer base.

Twenty years ago, these technologies made a lot of sense. It was far more important to know that very small hard drives weren't running out of space or internet connections were functioning correctly. On top of that, a virus at best knocked a machine or two offline for a period of time. Worms were more denial-of-service attacks than anything else.

You just need to accept that every tool is an attack vector into your business, and you need to be examining those tools.

But today, that single pane of glass is also a single source of pain. If you tie your security into it, too, you've actually created a single administrative window for you, your customers and your fellow providers to be impacted with.

The argument for this toolkit has been that you gain enough efficiency for your staff that it's worth it. That calculus was done 20 years ago, because by using these tools, you assume this risk.

Security is all about risk management, and this is a risk you, the provider, picked. You decided that it's worth it to deliver an infrastructure management service offering, and now it's a matter of when the system will be compromised. Thus, if you continue to use these tools, you should be planning that they will be breached again and again and again and again. That's the situation.

Thus, what to do? My first statement is this: If you're running a system like this on premises, you are insane. I'll say it, because everyone else seems too nice to do it. Do you have the SOC [security operations center] and threat ops organizations of Microsoft or Amazon? How about the threat ops groups of even Kaseya? To Kaseya's credit, they shut the cloud down fast. The big exposure? All those on-premises servers. In fact, in the history of those breaches I listed, on-premises [infrastructure] is a big theme there. I don't care how safe you think you can be. That's a fiction you tell yourself. The big cloud providers have security operations that rival nation states. The Pentagon has decided that Department of Defense Systems are better served in the cloud, but you're a special flower who can manage it better. Give me a break. If you say, 'I can do it better than the vendor,' well, perhaps you selected the wrong vendor, not that you're more capable.

As we dig into solutions, kudos to Huntress Labs and the details they provide. They have a guide for what to do when your RMM has been breached, and they also have tons of resources on how to address security. Gavin Stone with MSPGeek also has a great list of technical mitigation ideas that he's collected. I'm going to share it as a resource.

Reconsidering RMM

Now, I'm going to offer you the business strategy that hasn't been said. Uninstall the damn thing. You have a weapon of mass destruction installed in your business. You could just disarm the thing.

Oh, I hear the cries now: 'Oh, my God. I can't run my business without it. It's so necessary.' But is it?

You have customers who have machines on a guest wireless network, right, ones you don't manage? And you still have to help them, don't you? You now have customers working from everywhere, and you have to support them, too. Many aren't in your management console. What if you reconsider the actual core premise because the actual business solution that you need to address is in a changing market? The core premise I focus on is the idea of delivering proactive IT services to a group of customers to help them implement and use their technology. Nothing in that statement says you need to use a specific RMM technology to do that. A super-simplified answer to that. You could just call and check on them.

I know, crazy, right? Talk to a person when you could use a system. But I'm serious. Start thinking about the systems you use and why you use them.

I'm going to use a couple of examples here.

First, everyone talks about patch management. Let me update you with the latest thinking on patch management. Let's quote Dr. Ian Levy, technical director of the National Cybersecurity Centre in the UK, speaking at the Cybersecurity Agency Cyber UK 2021 virtual event: 'Patching is now so much easier and so much less risky than it was when we first started doing this stuff. If there's one thing that anyone out there wants to take away, turn on automatic updates, please. Even if you're an enterprise, turn on automatic updates.' Think about that. Are you holding onto an idea from previous decades? What if you just turned on a policy that required patching? Change the game. The official recommendation is to not hold back.

Second example: the single pane of glass. Many of you have embraced this concept of a single pane of glass and that you truly get efficiencies here. But already, that's been broken. Azure and Office 365 already break that. So, if you're already broken on the concept and doing exceptions, why do you cling to this idea?

Third, question unattended access. What if you had to ask first? Think it through. What are you really asking to do? Removing it entirely removes an attack vector. Instead, you could leverage policy enforcement.

Fourth, inventory. Did you know that all the inventory information is right in Azure Active Directory? Think it out. You don't need that from your RMM when you can get it from other tools.

I want you to start questioning your offering. Are you actually keeping up with the technology itself or clinging to concepts that date back years or even decades? The best core value remains this: Help customers with their technology, make IT work for them, help ensure it's there for them. None of that is dependent on an RMM or a single system if you think about the problem you're solving and then reapproach the solution.

Look, you may think this is all too much work. You may love your RMM. You just need to accept that every tool is an attack vector into your business, and you need to be examining those tools. Start with your core business offering, then decide how you'll implement it. You don't start with your toolkit and work backward. Because doing IT infrastructure the same way as you have for the past decades is now operating a business with a ticking time bomb inside.

About the Author
Dave Sobel is host of the podcast The Business of Tech, co-host of the podcast Killing IT and authored the book Virtualization: Defined. Sobel is regarded as a leading expert in the delivery of technology services, with broad experience in both technology and business. He owned and operated an IT solution provider and MSP for more than a decade, and has worked for vendors such as Level Platforms, GFI, LOGICnow and SolarWinds, leading community, event, marketing and product strategies, as well as M&A activities. Sobel has received multiple industry recognitions, including CRN Channel Chief, CRN UK A-List, Channel Futures Circle of Excellence winner, Channel Pro's 20/20 Visionaries and MSPmentor 250.