Researchers fool Cylance AI antimalware with 'simple' bypass

Security researchers found a bypass method to make Cylance AI antimalware classify malware as benign, but it is unclear how effective the method would be against other antimalware solutions.

Adi Ashkenazy and Shahar Zini, CEO and CFO, respectively, of Skylight Cyber, based in Sydney, analyzed the engine and model for Cylance PROTECT, the company's AI antimalware product, to find a way "to fool it consistently, creating a universal bypass."

"Combining an analysis of the feature extraction process, its heavy reliance on strings, and its strong bias for this specific game, we are capable of crafting a simple and rather amusing bypass," the researchers wrote in a blog post. "Namely, by appending a selected list of strings to a malicious file, we are capable of changing its score significantly, avoiding detection. This method proved successful for 100% of the top 10 Malware for May 2019, and close to 90% for a larger sample of 384 malware."

Ashkenazy told SearchSecurity that the analysis, development of the bypass and writing the blog post took about one week. He and Zini decided not to contact BlackBerry Cylance before posting their research on July 18 because they "did not consider this to be a software vulnerability, rather a passive bypass." Ashkenazy added that despite titling the blog post Cylance, I Kill You, "Cylance really wasn't the story."

"We were looking to specifically research pure AI [antimalware] vendors, to see if the approach holds versus a determined attacker. Symantec and TrendMicro are not pure play and CrowdStrike was technically harder to get (we did try). So Cylance was easy to download, pure AI vendor and known/respected enough to make the point," Ashkenazy wrote via email. "The way we view it, this research was not on Cylance, rather it was on the concept of pure AI defense, and Cylance just happened to be the test subject. It's the 'silver bullet' perception of AI which we wanted to test and expose."

Ashkenazy said that since releasing their research, he and Zini "provided Cylance with all the required technical information for a fix," but he thought it would take a while to fix the Cylance AI model.

"It really depends on the way they solve it. They can probably create a hotfix with a 'black list' type of mechanism quickly (which would be defeated easily). However, to solve the bias we found at its core, a lot of heavy lifting is involved," Ashkenazy said. "They would probably have to modify their feature selection process, and reduce their dependency on strings significantly, followed by a retraining of the model itself and rigorous validation. Also, they would probably want to make reversing harder and remove the black/white listing mechanisms or decrease their weight. This is certainly doable, but it is not comparable to sending out a new virus signature like in the legacy AV world."

Cylance posted a response to Skylight's research on July 21 and claimed to have a fix ready to roll out to customers over the next few days. Cylance contended that the method developed by Skylight was not a universal bypass, but a vulnerability in the Cylance AI model allowing for "the manipulation of a specific type of feature analyzed by the algorithm that in limited circumstances will cause the model to reach an incorrect conclusion."

"Our response to this vulnerability is three-fold: First, we have added anti-tampering controls to the parser in order to detect feature manipulation and prevent them from impacting the model score. Second, we have strengthened the model itself to detect when certain features become proportionally overweight. Lastly, we have removed the features in the model that were most susceptible to tampering," the Cylance team wrote in their announcement. "By leveraging the power of our cloud architecture, we are able to automatically deploy these enhancements, minimally impacting our customers."

Cylance also defended its use of machine learning and AI in its antimalware product, noting that AI antimalware is "designed to evolve."

"As we raise the bar against threats, those seeking to bypass these models will continue to search for new vulnerabilities. Nonetheless, machine learning remains the most effective tool in combatting malware, which is why the technique has been nearly universally adopted by security vendors," Cylance wrote. "We are on our 6th generation of machine learning models and the advancements we have made allow us to quickly adapt as the industry evolves."

While Skylight called its method a "universal bypass" of Cylance AI, Ashkenazy noted the broader effectiveness of this bypass method depends on "the weight of the AI model in the product." So antimalware products from Symantec and TrendMicro don't rely on AI enough to be at risk, but he suspected AI antimalware from DeepInstinct or Endgame could be susceptible to the same process, "but probably not the exact same method."

Ashkenazy did not respond to questions about Cylance's fix at the time of this post. In Skylight's original blog post, Ashkenazy and Zini argued for a hybrid approach rather than a "pure AI" antimalware product.

"We believe that the solution lies in a hybrid approach. Using AI/ML primarily for the unknown, but verifying with tried and tested techniques used in the legacy world," Ashkenazy and Zini wrote. "This means that the promise of a pure AI product may not be realized for [endpoint protection programs], and vendors will have to maintain and update multiple systems of detection. The promise of low resource consumption, with rare update cycles does not hold true for such a hybrid product, but it does provide a superior protective capability."