Threat intelligence offers promise, but limitations remain

Threat data: Too much of a good thing?

The challenge underscores why threat intelligence has largely failed to live up to its promise. More than 40 companies sell some sort of threat intelligence service, according to analyst firm Forrester Research. The variety of the information produced by those organizations, along with the varying quality of information, makes using threat intelligence a challenge for most companies, said Andrew Morrison, cyberstrategy, defense and response leader at consultancy Deloitte.

"A few years ago, we did not have that much data, so we urged companies to collect everything," he said. "Now too much data is available, and the volume is having negative effects because companies have trouble processing it all."

The variety of threat intelligence is both a boon and a problem. Companies can subscribe to just about any type of threat information, and firms on average subscribe to five different feeds, according to Forrester Research. Blacklists of domains and IP addresses that have attacked companies? Check. Information gleaned from forums on the darknet? Check. Associations between common attack techniques and adversary groups? Mitre's ATT&CK framework has you covered there. Lists of indicators of compromise connected to specific attacks? Security firms regularly publish threat-analysis reports that offer IOCs.

Yet threat intelligence feeds, and the firms that provide them, frequently just publish everything they can, often without rigorous verification. The result is that the quality of threat intelligence is spotty, said Mike Smola, director of risk intelligence strategy at Flashpoint and a former senior analyst for a large retailer he declined to name.

Mike Smola

"More isn't always better," he said. "It is more about the quality of the feed and fidelity of the information. Can you make decisions based on that information? The less noise the better, but you have to be able to trust the information."

Moreover, companies often find threat intelligence more of a distraction than a source of information that helps them reduce their risk. In 2018, 85% of companies considered threat intelligence an important part of the security effort, but only 41% of firms effectively used the information to reduce risk -- a 44% gap in effectiveness, according to a 2019 survey by the Ponemon Institute.

Companies struggle to get some utility out of threat intelligence. The result is that the use of the information has fallen into three categories: prevention, response and detection.

Threat intelligence at work: Prevention and response

At the most strategic level, threat profiles of common actors targeting a particular industry can inform companies in that industry what their priorities need to be. If criminals are using phishing attacks to steal credentials, the business should emphasize training and antiphishing defenses. If nation-state actors are using the EternalBlue exploit against unpatched Windows systems, confirming that no vulnerable systems are in your network should be a priority, according to Josh Zelonis, principal analyst serving security and risk professionals for Forrester Research.

"By figuring out what the commonly attacked vulnerabilities and vectors are, you can prioritize the work that you are doing so that you can reduce risks," Zelonis said.

Two-thirds of organizations belong to an ISAO, according to the Ponemon study. The majority of firms see sharing threat intelligence on groups affecting their peers as the primary benefit of belonging to an ISAO.

"The most valuable intelligence you are going to get is information on internal threats, because it is the most relevant to you," Zelonis said. "The next most important is from your industry ISAC or ISAO, because it tells you who is targeting companies like yours."

A second way of using threat intelligence is to direct investigations and respond to attacks. If your security team finds a compromised machine, and you can identify the adversary, the team can then follow up by looking for signs of other tactics that the adversary is known to use. If the attacker, for example, is thought to be Gothic Panda, also known as APT3, the security team should search for machines that have the remote desktop protocol turned on, as that is a persistence tactic of that nation-state group.

"These are particular indicators of compromise that are linked to these particular actors who are attacking your industry," Zelonis said. "So you can use the information coming on that feed and do a threat-hunting exercise against your infrastructure to look for whether there is a compromise that you did not have eyes on."

Where threat intelligence falls short: Detection

Finally, threat intelligence can be used at the most tactical level to decide whether some network traffic is a sign of an attack. As the industry deals with alert fatigue and a shortage of knowledgeable people, it's natural to try to incorporate threat intelligence into the threat detection process.

Yet this is where threat intelligence most often comes up short. With the proliferation of data, the only way to distill that information into actionable intelligence is with automation and machine-learning techniques, but those approaches have failed to reduce the number of false positives in a detection scenario, said Deloitte's Morrison.

"As you can imagine, based on the quality of the feed, you are going to see more false positives," he said. "Using intelligence feeds in this manner is doing more harm than good, because what you are getting from it is more alerts, and that is taking away from anything else you would be doing."

For the most part, threat intelligence services focus on hard indicators. The software and services keep watch for specific payload hashes, blacklisted IP addresses and unauthorized registry-key changes. These pieces of threat intelligence are incorporated into products and communicated to machines.

Yet the most effective intelligence is soft intelligence, said Richard Rushing, CISO at Motorola Mobility. Knowing activity from one good IP address can be linked to malicious activity from another IP address is one example. Or while an executable may not have been classified as malicious, the behavior of that executable has been linked to a malicious actor.

The question for Rushing is how companies can get those soft indicators out of their feeds and into their products, where they can do some good. Most threat intelligence is not delivered in a machine-readable format. Most companies -- 62% -- deal with threat intelligence in unstructured documents, as a PDF or CSV file. A minority have used more machine-friendly formats, including Structured Threat Information Expression (STIX) and Trusted Automated Exchange of Indicator Information, to automate their use of threat intelligence.

"The power of the threat intel is 'How do I mine my soft indicators?' and 'How easy is it to take those soft indicators and enrich my other tools?'" Rushing said. "The problem right now is, you may have a great soft indicators, but you cannot get that information into your other tools."

Threat intelligence continues to be human-heavy

Because the more amorphous side of threat intelligence -- the soft indictors -- are not easily fed into other security products, the ability to use many threat intelligence feeds boils down to whether a company has a knowledgeable employee, said Motorola's Rushing.

Looking at reports of ransomware targeting local government and deciding whether that applies to your company and how you will pursue the threat requires an analyst to work through the process, he said. "Without a trained professional focused on threat intel, figuring out which soft indicators need to be followed -- to find what is bad -- is difficult."

Some companies or organizations have the resources to solve the problem. Texas A&M University System, for example, staffs its security operations center with a few full-time employees and a couple dozen students. The result is that it has a lot of research capability, TAMUS's Basile said.

"I can never get enough information sources to validate everything coming in," he said. "But I have 20 students right now who can do that work. I have a full-time threat intelligence analyst. Would I recommend that for a retail store with just a handful of locations? Probably not."

Even the threat intelligence vendors argue that companies should find the right fit for their needs.

Yet most companies do not have the luxury of throwing student interns at a problem, which means many security managers are stuck between failing to catch an attack and automating a process that might mistakenly block important traffic.

"This idea that you are going to block any malicious website is scary from an operations point of view," Flashpoint's Smola said. "There is a lot of trepidation to trusting machine learning and artificial intelligence by themselves when the company is dependent on the delivery of goods and services."

The end result is that companies that want to use threat intelligence for detection would do well to let the vendor integrate specific feeds into their own product. Rather than attempting to use third-party threat intelligence for detection, companies should focus most of their efforts on using threat intelligence for strategic security and to enrich investigations.

Even the threat intelligence vendors argue that companies should find the right fit for their needs.

"Find a vendor for the attack surface you are dealing with; they will have specialized in this aspect and have high-quality data for you to defend yourself with," said Yonathan Klijnsma, head researcher with RiskIQ, a threat intelligence provider. "You can't expect the same high quality on all aspects of an intelligence feed if a vendor is very broadly collecting and indexing. You can't be good at every different thing."