CrowdStrike responds to NSS Labs lawsuit over product testing

The NSS Labs lawsuit against several major antimalware vendors marks the latest chapter in an ongoing feud between the testing firm and CrowdStrike, one of the most successful and visible cybersecurity vendors in the world.

NSS Labs, the independent cybersecurity testing company based in Austin, Texas, announced Wednesday that it had filed an antitrust suit against CrowdStrike, as well as Symantec, ESET and the Anti-Malware Testing Standards Organization (AMTSO) accusing the antimalware vendors and the AMTSO of conspiring to "restrict competition in the testing of cybersecurity products" by boycotting NSS Labs. The lawsuit further claims that vendors like CrowdStrike have used end-user license agreements that restrict comparative or competitive testing of their products by independent organizations like NSS Labs.

CrowdStrike initially declined to comment on the NSS Labs lawsuit because it was currently in litigation with the company and did not want to comment about an ongoing case. However, the cybersecurity vendor issued an updated statement Thursday that took NSS Labs to task.

"NSS is a for-profit, pay-to-play testing organization that obtains products through fraudulent means and is desperate to defend its business model from open and transparent testing," CrowdStrike's statement read. "We believe their lawsuit is baseless."

Symantec, ESET and the AMTSO have not responded to requests for comment.

NSS Labs issued an additional statement to SearchSecurity Thursday indicating the antitrust suit could include additional parties in the future. "Those vendors (CrowdStrike, Symantec, ESET) and the vendor-driven AMTSO are named because they are the ones who are leading the conspiracy. There are other unnamed co-conspirators and their involvement will be determined during the course of this antitrust case."

CrowdStrike and NSS Labs have been engaged in a high-profile legal dispute since 2017 when the vendor filed a lawsuit against the testing company regarding NSS Labs' unflattering test results for CrowdStrike's Falcon endpoint protection platform. The vendor sought a temporary restraining order and preliminary injunction against NSS Labs to prevent it from releasing a report for advanced endpoint protection product tests, which included a "caution" rating for CrowdStrike Falcon.

CrowdStrike accused NSS labs of faulty testing and argued that publishing the test results constituted a misappropriation of trade secrets and posed "irreparable harm" to the vendor. According to the lawsuit, the two parties struck a private agreement in April of 2016 to have NSS Labs test CrowdStrike Falcon and then provide a detailed report of the results to the vendor. But what happened after that is a disputed matter.

CrowdStrike claimed that NSS Labs "failed to perform the tests in a way that CrowdStrike deemed accurate and acceptable." The lawsuit describes a second round of tests to address the issues that CrowdStrike identified, but during discussions of a third round of tests, NSS Labs informed the vendor that it planned a public test of the platform in early 2017. CrowdStrike objected to that test, which led the company to take legal action against NSS Labs to prevent the testing company from making the results public at RSA Conference 2017.

The day before RSA Conference began, a U.S. District Court judge denied CrowdStrike's request for a temporary restraining order or preliminary injunction, and NSS Labs' advanced endpoint protection report was published on Feb. 14.

In an accompanying blog post, CrowdStrike noted that NSS Labs admitted in its advanced endpoint protection report that the testing of CrowdStrike Falcon was "incomplete" and claimed that all "Falcon prevention capabilities" were incorrectly turned off during the tests.

"Including Falcon in the report based on an incomplete analysis is contrary to basic industry standards for testing," CrowdStrike wrote.

NSS Labs, however, disputes that version of events. "We were unable to complete testing of the CrowdStrike Falcon Host product via a few attack vectors because CrowdStrike remotely disabled the product partway through our tests," NSS Labs CEO Vikram Phatak wrote in a blog post. 

Phatak added that if NSS Labs had in fact disabled all Falcon prevention capabilities, then the platform would not have been able to stop any of the attacks the testers used, which was not the case.

The dispute over the advanced endpoint protection report marked a turning point, according to the NSS Labs lawsuit filed this week. Just as NSS Labs' report was made public at RSA Conference 2017, the complaint alleges CrowdStrike organized a meeting of antimalware and endpoint protection vendors at the conference "with the express intent, purpose and effect of obtaining agreement among the competitors to refuse to do business with companies, including specifically NSS Labs" that used testing methodologies other than the AMTSO Testing Standard.

While NSS Labs is currently a member of AMTSO, the testing firm pushed back against the AMTSO Testing Standard, which NSS Labs argued gives vendors an unfair advantage. "Knowing how one's product will be tested in advance defeats the entire purpose of independent third-party testing, no less than a student knowing the questions and answers before a test defeats the entire purpose of a school test," the complaint reads. "Indeed, obtaining such knowledge is usually called 'cheating.'"

AMTSO, meanwhile, issued a statement Friday responding to the NSS Labs lawsuit that categorically denies all claims made against the organization. In addition, AMTSO president Dennis Batchelder wrote a blog post contesting NSS Labs' characterization of the organization and the testing standard.

"We were disappointed that one of our members chose to file an antitrust lawsuit against us and other AMTSO members. We were even more surprised at their allegation that our organization is only vendor-driven, which just isn't true," Batchelder wrote. "Our standard, as well as our guidelines and fundamental testing principles, were developed by both vendors and testers working together to ensure that our industry can provide customers with the best information possible."

UPDATE: Symantec issued a statement on Sept. 26 regarding the NSS Labs lawsuit. Here is an excerpt from the statement:

"We are aware of the lawsuit filed by NSS Labs and we believe that their claims against us are entirely baseless. While it's understandable that NSS Labs' desire for profits may be inherently at odds with a non-profit, standards-based organization such as AMTSO, the integrity of the testing process should be of utmost importance, starting with transparency and equity for all participants. We welcome the opportunity to bring the discussion of fair and open testing further into the public conversation, while also shining a light on certain business practices within the testing industry."