New cybersecurity report gets the hacker perspective

Attack methodology

Not all of the information in the report clashed with conventional wisdom, of course. The hacker survey found that 72% of respondents used social engineering to gather information before launching an attack, and 86% said they used vulnerability scanning to find the weaknesses of a target. Additionally, the two most popular attack methods were phishing (40%) and direct server attack (43%).

Pogue told SearchSecurity "it's important to understand how the adversary thinks and acts. The more defenders know, the better they can prepare themselves."

The cybersecurity report also found that the tools being used in an attack may not be what cybersecurity professionals expect, given the focus on exploit kits. The report found 60% of respondents used open-source tools and 21% used custom made tools.

Pogue noted that only 10% of responders used commercial tools, which was something of a surprise because "being employed by an organization usually means a budget to purchase tools. But, according to the research, even when the budget is available, the respondents opted for the free or self-created solutions."

Jake Williams, founder and CEO of Rendition Infosec LLC based in Augusta, Ga., said these findings could imply a sampling bias due to respondents all being attendees of a major hacking conference.

"This seems to imply that most attackers don't use Exploit kits and commercial tools. Very few people will admit to using illegal exploit kits, the vast majority of which are rented panels that would be unsuitable for commercial client engagements," Williams told SearchSecurity by email. "At DEFCON (somewhat less commercial) I would be surprised to find many people admitting that they use Core Impact or Cobalt Strike."

However, Monsegur said the low use of commercial tool and exploit kits (just 3% of respondents) could be because "exploit kits that are sold on the dark web are usually buggy, highly specific and more often than not, backdoored," so more experienced hackers wouldn't trust them.

"I think the percentage leaning toward open source tools is mainly because some of the best security tools are in fact open-sourced. It is because these tools are open and freely available, and accessible, is why most security researchers use them predominantly," Monsegur said. "These same security researchers could make their own exploit kits, write custom tools, or setup environments similar to Kali for their engagements."

Blankenship said the perception that assessors use more commercial tools may be "thanks to vendor marketing efforts and a misunderstanding of the pen testing process."

"Proposals for security testing often include a list of tools to be used by the assessors. Most of these include a list of commercially available software tools, open source tools, and tools developed by the assessor," Blankenship said. "Skilled assessors will use a variety of tools that mimic those used by adversaries which will likely be a combination of open source and self-built tools."

Defensive countermeasures

The cybersecurity report also found that half of respondents said they change tactics with every engagement with 38% changing tactics within every six months, and 56% said they changed just to keep learning new tricks.

Williams attributed the changing tactics to "security products evolving to cover the old techniques." 

"This is an unfortunate reality, but security countermeasures will always be playing catch up. It's in the name, that's why they are called countermeasures," Williams said. "But for the 56% who say they do it just to learn new techniques, they are doing it for more than the desire of knowledge. If they don't continue to learn new techniques, they will be stuck with only techniques that don't work."

Monsegur said tactics have to change out of necessity.

"Each engagement is different. Different scopes, targets and environments," Monsegur said. "Most researchers I know employ the use of a methodology, and it usually involves a streamlined process for any pen testing engagement, but eventually they begin to shift or change strategy along the way as they begin to find new vulnerabilities, configurations and software."

Pogue wrote in the Nuix cybersecurity report that these findings "indicate that if your defensive countermeasures are less flexible than the people trying to get around them, they have little to no chance of being effective."

"You will be protecting against an attack pattern that is no longer relevant. This underscores the importance of incorporating realistic, goal-oriented penetration testing into your security program," Pogue wrote. "Only by continuously evaluating and enhancing your security countermeasures can you follow constantly shifting attack strategies."

Unfortunately, the hacker survey results in terms of effective countermeasures left experts underwhelmed. Respondents said endpoint security (36%) presented the most challenge during a pen test, followed by intrusion detection or intrusion prevention systems (29%) and firewalls (10%); although 22% said no countermeasure could stop a successful attack given enough time.

But, experts noted that these findings contradicted the respondent suggestions for the most effective place to spend a security budget, which was led by IDS/IPS (37%), pen testing (25%) and data hygiene or information governance (21%).

"Endpoint security is not mentioned among the effective countermeasures, although 36% of respondents said that it represents the biggest challenge for them," Blankenship noted. "Overall, however, the results demonstrate that having controls at the endpoint in addition to robust security monitoring provide more effective protection, which is not at all surprising."

Williams said the survey results about penetration testing could be more proof of sampling bias. 

"IDS/IPS are becoming less and less effective with the ubiquitous use of encryption on the network," Williams told SearchSecurity. "I'm really surprised that more respondents didn't recommend spending more on endpoint detection since on the same page respondents note that it is their biggest challenge while penetration testing."

Monsegur said he wasn't surprised about the lack of consensus in terms of security countermeasures in the hacker survey "because, quite honestly, security is hard."

"When you have one environment with various operating systems, devices, users, configurations, settings, remote employees, open source and proprietary software then the attack surface expands to a degree that there is not one fix," Monsegur said. "There is no single solution to the security problem that most companies and people are prone to. There are various steps and areas to focus your budget on, before you reach a state of security enlightenment, if that is such a thing."

The hackers surveyed seem to agree that such "security enlightenment" may be far off, because 64% said their greatest frustration as an attacker was that people and enterprise don't fix things they know are broken and 75% said the most common response by an organization after an engagement was to perform "some remediation, usually focused on high and critical vulnerabilities."

Pogue wrote in the report that while this "appears to be a logical approach to remediation, it misrepresents the true nature of vulnerabilities and provides a false sense of security for decision-makers."

"If you only address specific vulnerabilities that you have chosen arbitrarily and devoid of context, it's the cybersecurity equivalent of taking an aspirin for a brain tumor; you are addressing a symptom as opposed to the root cause," Pogue wrote. "Simple remediation of specific vulnerabilities fails to take into account why that deficiency exists in the first place; it discounts strategic shortcomings such as poor or missing patch management policies, lack of a vulnerability management program, or an untrained security staff. This approach also fails to recognize the complexities of multistaged attack vectors."

Blankenship said this was "a fair assessment of the reality in many enterprises."

"Too many companies see pen testing as a compliance requirement that doesn't produce actionable results," Blankenship said. "Others may get the results of an assessment, see the need for changes, but don't have the resources (time, people or budget) to implement the changes."

Williams said the divide between the data found in the cybersecurity report with traditional cybersecurity wisdom could be due to perspective and because "the penetration testers are almost always lacking some data."

"What a penetration tester (or their tools) labels critical may not actually be critical. Organizations may mitigate some vulnerabilities with defense in depth, opting to create layered defenses around a vulnerability rather than completely mitigating it," Williams said. "In other cases, organizations don't remediate vulnerabilities because the penetration testers fail to communicate the risks effectively. Effective communication and report writing are key penetration tester skills that are commonly overlooked by security firms."

Monsegur agreed that "communication and education [are] key to ensuring companies fix their problems."

"Otherwise by sending off reports and disclosures to companies [that] are swamped out of time, budgets or manpower, you end up doing the opposite of what it was you set out to do," Monsegur said. "It is important to discuss the security problem and maintain a flow of communication with the right people and hope [that] by making them aware of their issues, risks and potential for compromise, they move on your remediation suggestions."