Do phishing simulations work? Sometimes

Do phishing simulations work?

Cybersecurity awareness trainings have remained relatively stagnant over the past decade, according to Jinan Budge, analyst at Forrester Research. The widespread adoption of phishing simulations has been the only major development in recent years, she added.

Phishing tests are used to collect data on employees' click and response rates to malicious emails, links and attachments. The simulations are meant to help employees recognize malicious emails.

But do they have any lasting effect?

A 2021 study from ETH Zurich, a public research university in Switzerland, concluded phishing simulations -- at least embedded phishing tests, which tell employees when they clicked a phishing link or send an employee who clicked a malicious email to voluntary training -- don't necessarily help reduce click and dangerous action rates among employees.

ETH Zurich researchers concluded that "embedded phishing training is … not effective and can in fact have negative side effects," suggesting tests make users more susceptible to attacks because employees either gain false confidence from the trainings or start to feel less responsible for stopping such attacks.

The researchers did find crowdsourced phishing simulations, on the other hand, to be effective. Participants in the experiment were given a button to alert the security team of a suspicious email. They correctly reported 68% of phishing emails, aiding the security team in phishing prevention.

The study did say, however, that even crowdsourced phishing simulations should complement other security awareness measures.

Gartner analyst William Candrick agreed. Phishing simulations alone aren't effective, he said, unless an organization has a program to engage repeat offenders.

Enterprises should use data from phishing tests to identify high-risk employees and then enforce mandatory and interactive cybersecurity awareness trainings.

Are phishing simulations ethical?

Despite their ability to identify risky employees, phishing simulations remain controversial, even among security professionals.

In September 2020, Tribune Publishing Company sent employees an email offering bonuses of up to $10,000. After clicking the link to see how much their bonus would be, however, employees learned they failed a phishing test -- and would receive no bonus. Employee backlash, viral social media posts and bad publicity ensued, especially as the email was sent out after several recent years of layoffs and furloughs -- not to mention amid a global pandemic.

But that doesn't mean the tests are wrong, per se.

"Everyone gets mad [about phishing simulations] … but the IT department is perfectly in the right, and the cybersecurity team is doing exactly what it should be doing," said Johna Till Johnson, CEO of Nemertes Research. "Hackers are not sensitive to the feelings of employees."

While it may be legal, it isn't always moral. "You have to consider many factors when conducting a simulation," Budge said. She suggested asking the following phishing simulation questions before deploying a test:

• How will it impact employees' mental health?
• Is the simulation necessary?
• How will the message be perceived?
• Does this benefit employees?
• Are we being smug, or do we genuinely want to change behaviors?
• Is there a better way to communicate this message?

Asking these questions also helps prevent security teams from ostracizing themselves from the rest of the organization, she added.

It's also important for security teams to teach and regularly reiterate that cybersecurity is a team sport. Security is often viewed as IT's problem; 70% of employees believe it is IT's responsibility to ensure company accounts are not breached or hacked. Without the proper messaging, phishing simulations only reinforce this mindset.

If a company does use phishing simulations, Candrick suggested security teams conduct extended phishing campaigns with increased difficulty over time. This helps employees who are doing well continue to improve and feel rewarded for their efforts.

Phishing is here to stay -- and so are phishing simulations, at least for the time being. While evidence shows phishing simulations alone are ineffective, they can be useful in conjunction with risk management practices and a security awareness program that uses the data collected from these tests to identity and train high-risk employees.

"Phishing simulations are a necessary evil at the moment," Budge said.