Experts debate Vulnerabilities Equities Process disclosure
Experts debated how the government should weigh disclosure in the Vulnerabilities Equities Process and whether to err on the side of offense or defense.
LAS VEGAS -- Experts discussed various metrics that the government might use to determine whether or not to disclose bugs under the Vulnerabilities Equities Process, or whether there is ever a reason to hold on to flaws.
The Vulnerabilities Equities Process (VEP) was introduced to the government as a way to standardize the practice of deciding whether a software bug obtained by a federal agency would be kept for offensive intelligence purposes or disclosed to the vendor. More recently, the PATCH Act, introduced in Congress earlier this year, has sought to codify the VEP into law rather than leave it as an optional policy, though that bill is awaiting action from the House Committee on Oversight and Government Reform.
A panel of experts at the Black Hat conference discussed the topic, including Jason Healey, senior research scholar at Columbia University, who said the Vulnerabilities Equities Process has two major aims.
"The main purpose of this process is to try and buy ourselves some security; to try to figure out if the United States and the internet are better if we know this or should we use it for some time to figure out if the Russians or ISIS are going to attack us, use it for espionage or potential offensive purposes and try and do a smart trade off," Healey said. "It's also there just for that intra-agency process, so that the defensive-minded agencies feel like they've got some voice on what's going to happen to these bugs."
Trey Herr, fellow at the Harvard Kennedy School, said a significant factor should be bug collisions -- or how often two independent researchers discover the same bug -- because "the government's decision to disclose a given vulnerability hinges in part on that vulnerability's likelihood of being discovered and used maliciously by another party," as was noted in a white paper describing a study conducted by Herr; Bruce Schneier, security expert and lecturer at the Harvard Kennedy School; and Christopher Morris, research assistant at the Harvard School of Engineering and Applied Sciences.
Part of understanding the rate of bug collision includes gauging whether an adversary might be holding or exploiting the same vulnerability. Lillian Ablon, information scientist for the RAND Corporation, studied a data set of information regarding 16 years of zero-day vulnerabilities and exploits in order to create a virtual database mimicking a cyberweapons stockpile that a nation state might have.
"The debate of whether to retain or disclose these vulnerabilities is often fueled by how much overlap there might be between the zero-day vulnerabilities or exploits the U.S. government keeps and those its adversaries are stockpiling," Ablon wrote in a report with co-author Timothy Bogart, statistics expert at RAND. "If both sides have the same stockpiles, then some argue that there is little point to keeping them private -- whereas a smaller overlap might justify retention. But without information on the over-lap, or concrete metrics based on actual data, it is challenging to make a well-informed decision about stockpiling."
Katie Moussouris at Black Hat USA 2017
Criticisms of the data
However, Katie Moussouris, CEO of Luta Security, asserted that the Vulnerabilities Equities Process was "completely unrelated" to bug collision because "you can't actually extrapolate anything from one fixed piece of software to even the next version's bug collision rate."
"You add new features, things change, you add new researchers to the scene and then there's also the problem of omnipotence in this space. There's no real way to use any of the published data -- or the private data that some folks are correlating -- to know what is in the mind of hacker number one, two or three," Moussouris said. "For the purposes of informing policy decisions around the Vulnerabilities Equities Process, we certainly need to look at the balance between our duty to preserve national security, protect our critical infrastructure and our offensive capabilities."
Herr noted that the value of research into zero-days and bug collisions isn't only in the statistics because "both of these data sets describe not only the characteristics of software but the behavior of the community of researchers doing the discovery."
You can't actually extrapolate anything from one fixed piece of software to even the next version's bug collision rate.
Katie MoussourisCEO, Luta Security
Healey said one trouble with the debate is that people tend to "make too much" about the Vulnerabilities Equities Process because it's one of the few parts of government where it's possible to decide to prioritize defense over offense and have real control.
"There's a really strong argument that says that's stupid because out of the VEP you get so little value from each new bug that the amount of time you're spending on this is just a waste and it's a dumb idea anyway," Healey said. "So we're really desperate for these things like the collision rate ... because that helps us figure out if we're actually getting in on this with all that we can do."
Healey also relayed a thought from Joe Nye, describing the unique aspect of vulnerability disclosure -- unlike nuclear disarmament where removing a weapon doesn't affect an adversary's stockpiles, patching a bug does remove that threat.
Ablon said disclosing a flaw and making a patch available doesn't necessarily mean users will apply that patch, which is the biggest issue in discussing how to choose what to do with a bug. And, she added, the issue becomes even more confused because bug density metrics don't necessarily speak to how easy or hard a bug is to find.
Moussouris said zero-days are not necessary for intelligence agencies to achieve their offensive goals and discussions shouldn't focus too much on data, because although the research is well-intentioned, it "doesn't help us make policy decisions" in the Vulnerabilities Equities Process.
"It's much more beneficial to err on the side of disclosure. The reason for that is if you think about it, if we're playing a game of capture the flag and we've all got the same flags, what do you think is the smart thing to do to protect your critical infrastructure and in terms of preserving your offensive capabilities?" Mousourris asked. "Do you actually need zero-days to achieve most of your goals? The answer is a resounding no. We see that mirrored in general exploitation of software in the world."
Next Steps
Learn why the CIA was criticized for its lack of vulnerability disclosures.
Find out how the NSA disclosure policy aims to balance offense and defense.
