Every once in a while, a vulnerability disclosure incident occurs that significantly changes the game. Recently, Digital Bond released vulnerability information in conjunction with exploit code packaged in Metasploit for 6 different SCADA system devices. This time around, the stakes have been raised with much bigger consequences.
 
With consequences this high, it is worth re-evaluating the impact of vulnerability disclosure on risk in the IT environment.
 
First, a brief reminder about how risk works. Even though we can’t measure it with precision, we can do a fairly good job in understanding when it increases or decreases. We do this intuitively every day regardless.
 
Risk is a function of threats, vulnerabilities, and consequences. Threats and vulnerabilities combine to contribute to the likelihood that a breach will occur. Consequences, or impact provide the expected losses if a breach happens. We will not deal with measuring consequences since discovery/disclosure rarely impacts the level itself, so it remains constant for any scenario in this model.
 
Threat levels increases whenever the attacker costs are reduced or the benefits are increased. Attacker costs can be reduced by providing more information about specific vulnerabilities, and go lower still the more information is provided, with weaponization providing the lowest cost to entry (with the exception of actually pointing out targets). Benefits increase with larger promise of fame, fortune, etc.
 
Vulnerability levels increase when the attack surface increases – typically when new software is introduced into an environment or a configuration change is made on existing assets. It is important to note that the act of discovering or disclosing vulnerability information does nothing to increase this level unless the act of addressing the known vulnerability results in the introduction of more new ones, for example deploying a patch that has more vulnerabilities than the ones it intends to fix (possible, but unlikely).
 
The promise of managing vulnerability levels, which is the basket bugfinders have their cards in, is in attempting to reduce the level through the identification of individual vulnerabilities and the application of controls – a configuration change, modified ruleset on intrusion prevention, and most often the application of a patch.
 
In their seminal work on the patch ecosystem, Timing the Application of Security Patches for Optimal Uptime, Beattie et.al. model the patch process. They properly demonstrate that the decision to patch is more complex than the simplistic notion that all patches should be applied immediately. The real decision involves a cost-benefit analysis between the costs to patch (including effort and also potential patch failures) and the costs not to patch (risk of compromise). Given that the large majority of vulnerabilities are never acted upon by attackers in the wild, this is a challenging analysis.
 
With the preliminaries out of the way, we can evaluate the situation at hand. With Digital Bond, et.al. in conjunction with Rapid7 and their Metasploit team, releasing weaponized exploits for several vulnerabilities on several systems, they have artificially increased the threat.
 
Now, the question is whether this increase in threat can be offset by a decrease in vulnerability. Unlike with general-purpose computing, SCADA systems are often configured and left alone for significant periods of time because the operations are highly sensitive. This typically means that organizations are very reluctant to touch the systems. Accordingly, a vulnerability disclosure is highly unlikely to reduce vulnerability by enough to justify the increase in threat.
 
Perhaps more importantly, the consequences are worth discussing. SCADA systems increase the significance of vulnerability disclosure activities simply by introducing critical physical infrastructure into the consequence equation.
 
Ultimately, the release of these weaponized exploits and the corresponding vulnerability information has increased the threat and is likely to have minimal impact on vulnerability level for some time, meaning the risk is increased significantly.
 
Bugfinders do what they do for various reasons. Some are motivated by ego and pride, and even spite. It is worth identifying these people because often these emotions are given higher priority in their decisionmaking than the impact on risk. Therefore, they are willing to increase risk to society simply due to an emotional response because “the vendor was a jerk” or “the deserved it” because they were somehow slighted or ignored. All of us experience situations like these and need to work harder to overcome the spiteful response that demonstrates a lack of professional care.
 
Many researchers, truly the best and brightest technical minds in our profession, have positive intentions and believe their actions are beneficial to society. This is primarily because they are focused on vulnerabilities rather than compromises. They mistakenly believe that reducing the vulnerable state is beneficial because they fail to acknowledge that threat levels more than increase enough to overcome the slight benefit to vulnerability levels. 
 
There are a number of reasons that researchers use to justify their actions. We will ignore the petty ones for now and address the risk-oriented ones.
 
“It is better to know than not to know…”
 
The most common justification for vuln disclosure is the assumption that one needs to know about the vulnerability so it can be mitigated. While it is an attractive thought, the real problem with this idea is that nobody can know about every vulnerability. So, for example, though vulnerabilities that will be disclosed in months and possibly years in the future currently exist in an environment, they are not addressed in any way. While patching vulns often makes people feel like they are being productive, there are just too many to focus only on specific ones. This phenomena, by the way, is well known in psychology as “the certainty effect.”
“We must find the vulnerabilities before the attackers do…”
 
This justification is attractive as well, and can work in some highly constrained environments where the population of vulns is relatively small. Unfortunately the evidence for success for all software is not promising. The problem occurs because there are too many vulnerabilities – whether that be a single IT environment or the entire software universe. A conservative approach to understanding this would be to do the math on likelihood of collisions – which I can’t do, but it is similar to that done in determining the likelihood of a hash collision or to the birthday problem where 365 days is replaced with an estimate of the number of vulnerabilities in your population – that is, number of vulnerabilities that exist whether or not they have been found. The collision rate is clearly not zero, but it isn’t very high, either.
 
“It is the only way to defend ourselves…”
 
Many security pros still believe that identifying specific vulnerabilities and patching them is our best hope in protection, even if it is a hopeless cause. This simply isn’t true. We see that in the evidence – since we know that we can’t find all the vulns before the bad guys, it is inevitable that we find out in other ways – either through behavorial analysis, heuristics, or some other means. It happens a lot less than compromises of known vulnerabilities (for obvious reasons) but it still happens. This is the area where we really need to get better, since we know there are unidentified vulns in our environment currently.
 
“It reduces risk in the long run by making vendors and developers more secure programmers…”
 
This justification is often used but completely unjustified. A simple look at the total number of vulnerabilities found every year shows an increase more often than not, and when it doesn’t people are quick to point out that there are still many more vulns out there that remain undiscovered. Just comparing the amount of new code written every day to the number of vulnerabilities found in the same period is a sobering thought exercise.
 
On the other hand, people make qualitative claims of success in certain contexts and perhaps this is true. In these situations, it is worth considering alternative approaches that might provide the same benefit at lower cost. Presumably, everyone agrees that compromises would occur with or without public vuln disclosure. If this is the case, then breach evidence could provide just as effective, perhaps even more effective, justification for training developers to be more secure. Consider, for example, that perhaps the most publicized success in this area – the Trustworthy Computing initiative at Microsoft – was only initiated after Code Red and Nimda ran wild across the Internet, many thousands of disclosed vulnerabilities after the fact.
 
As a profession, and really a society, we have tolerated and even lauded bugfinding activities without any evidence that it helped. It can be exciting because many of the characters involved have colorful personalities and are clearly among the best and brightest, as mentioned earlier. Risk management is challenging to begin with, so being able to cling to specific data and resolve it makes us feel like we are demonstrating progress. SCADA consequences increase the stakes significantly – into the physical realm. It is time to consider the implications of our actions and respond accordingly.
by Pete Lindstrom
Through the years, the value proposition of vulnerability discovery and disclosure has been batted about quite a bit in the Information Security field. Every once in a while, a vulnerability disclosure incident occurs that significantly changes the game. Recently, Digital Bond released vulnerability information in conjunction with exploit code packaged in Metasploit for 6 different SCADA system devices. This time around, the stakes have been raised with much higher consequences and the profession needs to step back and evaluate whether conditions supporting discovery and disclosure in the past still exist today. It is worth re-evaluating the impact of vulnerability disclosure on risk in the IT environment.
First, a brief reminder about how risk works. Even though we cannot measure it with precision, we can do a fairly good job in understanding when it increases or decreases. We do this intuitively every day and the basic guidelines are straightforward.
Risk is a function of threats, vulnerabilities, and consequences. Threats and vulnerabilities combine to contribute to the likelihood that a breach will occur. Consequences, or impact describe the possible losses if a breach happens. We will not deal with measuring consequences since discovery/disclosure rarely impacts the level itself, so it remains constant for any scenario in this model. (more…)
