Uncover and overcome cloud threat hunting obstacles

Threat hunting -- the process of proactively searching for signs of malware or an unauthorized intruder -- is a critical part of modern cybersecurity programs. Traditional antivirus programs and intrusion detection systems often miss cutting-edge malware, such as Emotet, or the subtle signs of an advanced persistent threat. An informed, manual threat hunting program can help to find these threats in time to prevent the next stage of attacks, such as ransomware installation.

But what happens when threats invade your cloud environment? Effective cloud threat hunting depends on strong threat intelligence: you need good information in order to successfully hunt down invaders. Many organizations have advanced threat intelligence capabilities in their on-premises environment, but when it comes to the cloud, they are nearly blind.

Now is the time to build your cloud threat hunting program. The problem is that, unlike in on-premises environments, defenders do not have ready access to the same wealth of threat intelligence in the cloud. Here are some of the challenges to threat hunting in the cloud, and tips for surmounting them.

Availability. The cloud is just "someone else's computer," goes the joke. When it comes to logging and monitoring, this is often painfully clear. Many cloud providers offer only very limited event logs, such as records of user authentication, and some do not even provide that. Under pressure from customers, some providers are expanding logging and monitoring capabilities, but security professionals are often foiled by decision-makers who see these features as nice to have rather than as required. 

Advanced environments, such as AWS and Azure, offer you an enormous amount of control over "your" systems -- but due to the nature of their shared environments, the ability for users to monitor network traffic is limited. In on-premise environments, defenders can collect network flow records and sniff traffic to detect malicious activity. In the cloud, tools for monitoring virtual networks are not as readily accessible. Amazon and Microsoft both introduced virtual network terminal access point (TAP) capabilities in recent years, but few security professionals have experience using these tools, and the Azure virtual network TAP appears to be under development (the feature has not been consistently available).

Aggregation. To hunt for threats efficiently, practitioners need to be able to easily access intelligence from various sources, ideally using one central console. In on-premise environments, it's easy enough to set up a central server or SIEM to collect logs from various applications and pieces of network equipment. When it comes to the cloud, however, aggregating logs is not so simple. Cloud providers may or may not support log export. When they do, the format of data can vary widely -- and it may change without notice, unexpectedly foiling SIEM ingestion.

Expense. Detailed logging in the cloud is rarely on by default. In AWS, for example, CloudWatch monitoring is disabled unless explicitly turned on -- and then a pop-up warns, "additional charges apply." In Microsoft's Office 365, exchange mailbox auditing is now on by default for all new commercial instances -- a change that took place in 2019 after a huge number of customers suffered business email compromise breaches and found that they did not have mailbox logs that they needed to investigate. However, the default retention time is limited to 90 days for many tenants, and customers have to pay for longer retention times.

When it comes to aggregating threat intelligence in the cloud, customers may be charged at every step of the way: for turning logging on, for storing log data in the cloud, for the bandwidth or processing power needed to transfer data to another system, and more. For example, let's say you want to collect log data from AWS and send it to a central Splunk server on Azure. Enabling CloudWatch on AWS requires opening a new Simple Storage Service bucket for local log storage, which costs money. You can use the firehose to push data to another source, which means you are charged for processing power. On Azure, you have to pay for the underlying VM that you use to set up Splunk, as well as for the Splunk license itself. All of this adds up.

Analysis Tools. Tools for cloud threat hunting are nascent. More advanced cloud providers, such as Microsoft and Amazon, have built-in analysis tools, but they often have surprising -- and poorly understood -- limitations. For example, security professionals frequently use Microsoft's graphical Security & Compliance Center to pull Unified Audit Logs (UAL) from Office 365 -- not realizing that the results are limited to 5,000 sorted records or 50,000 unsorted records. Incomplete threat intelligence, of course, leads to shoddy results! Instead, hunters need to use third-party products or custom Powershell scripts to recursively extract large volumes of UAL records. For analysis, products such as Splunk, Extrahop or the open-source Kibana are invaluable.

The cloud is the emerging battleground for bleeding-edge cybersecurity threats. Unfortunately, the constant evolution of threat intelligence, difficulty and expense of aggregation, and nascent cloud-based analysis tools are all challenges for today's defenders. The good news is that cloud monitoring and logging is slowly maturing, and security professionals who push for cloud threat hunting capabilities will reap the rewards.