by Pete Lindstrom

Through the years, the value proposition of vulnerability  discovery and disclosure has been batted about quite a bit in the Information Security field. Every once in a while, a vulnerability disclosure incident occurs that significantly changes the game. Recently, Digital Bond released vulnerability information in conjunction with exploit code packaged in Metasploit for 6 different SCADA system devices. This time around, the stakes have been raised with much higher consequences and the profession needs to step back and evaluate whether conditions supporting discovery and disclosure in the past still exist today. It is worth re-evaluating the impact of vulnerability disclosure on risk in the IT environment.

First, a brief reminder about how risk works. Even though we cannot measure it with precision, we can do a fairly good job in understanding when it increases or decreases. We do this intuitively every day and the basic guidelines are straightforward.

Risk is a function of threats, vulnerabilities, and consequences. Threats and vulnerabilities combine to contribute to the likelihood that a breach will occur. Consequences, or impact describe the possible losses if a breach happens. We will not deal with measuring consequences since discovery/disclosure rarely impacts the level itself, so it remains constant for any scenario in this model.

Threat levels increases whenever the attacker costs are reduced or the benefits are increased. Attacker costs can be reduced by providing more information about specific vulnerabilities, and get lower still the more information is divulged, with weaponization providing the lowest cost (with the exception of actually pointing out targets). Benefits increase with larger promise of fame, fortune, etc.

Vulnerability levels increase when the attack surface increases‚ typically when new software is introduced into an environment or a configuration change is made on existing assets. It is important to note that the act of discovering or disclosing vulnerability information does nothing to increase the vulnerability level unless the act of addressing the known vulnerability results in the introduction of more new ones, for example deploying a patch that has more vulnerabilities than the ones it intends to fix (possible, but unlikely).

The promise of managing vulnerability levels, which is the basket bugfinders have their eggs in, is in attempting to reduce the level through the identification of individual vulnerabilities and the application of controls – a configuration change, modified ruleset on intrusion prevention, and/or most often the application of a patch.

In their seminal work on the patch ecosystem, ‘Timing the Application of Security Patches for Optimal Uptime‘, Beattie et.al. model the patch process. They properly demonstrate that the decision to patch is more complex than the simplistic notion that all patches should be applied immediately. The real decision involves a cost-benefit analysis between the costs to patch (including effort and also potential patch failures) and the costs not to patch (risk of compromise). Given that the large majority of vulnerabilities are never acted upon by attackers in the wild, this can be a challenging analysis.

With the risk construct working as a model, we can evaluate the situation at hand. The immediate impact of Digital Bond and Rapid7’s release of weaponized exploits is clear – it significantly reduces costs to the attacker and likely introduces new attackers into the arena that had previously not had the skill set or interest. So, the initial impact is to artificially increase the threat level over and above whatever it was to begin with. Intuitively, security professionals acknowledge this change by raising threat and risk barometers around the world.

Now, the question is whether this increase in threat can be offset by a decrease in vulnerability. After all, that is the goal. For that assessment, we should keep in mind that these are SCADA systems. Unlike with general-purpose computing, SCADA systems are often configured and left alone for significant periods of time because the operations are highly sensitive. This typically means that organizations are very reluctant to touch the systems and will need to plan carefully in order to proceed – the cost to patch is high as they anticipate a manual patch process and the potential for patch failure. So we can expect patches to be deployed at a rate much slower than we have become accustomed to in the typical windows environment, for example. Add to the challenge of a single patch the notion that digital bond actually released a number of exploit modules that will need to be addressed. Throughout this process, system administrators will need to keep in mind that there are possibly other identified vulnerabilities already on these systems. Accordingly, a vulnerability disclosure is highly unlikely to reduce vulnerability by enough to justify the increase in threat. This notion is generally true and is easily supported by the numerous incidents that occur against known vulnerabilities compared to the relatively few exploits of unknown or ‘undercover’ ones.

Ultimately, the release of these weaponized exploits and the corresponding vulnerability information has increased the threat and is likely to have minimal impact on vulnerability level for some time, meaning the risk is increased significantly.

Perhaps more importantly than acknowledge the increased likelihood, we should evaluate the consequences and decide whether the benefits are worth the increased risk. SCADA systems increase the significance of vulnerability disclosure activities simply by introducing critical physical infrastructure into the consequence equation.

Bugfinders do what they do for various reasons. Some are motivated by ego and pride, and even spite. It is worth identifying these people because often these emotions are given higher priority in their decisionmaking than the impact on risk. Therefore, they are willing to increase risk to society simply due to an emotional response because “the vendor was a jerk” or “they deserved it” because they were somehow slighted or ignored. All of us experience situations like these and need to work harder to overcome the spiteful response that demonstrates a lack of professional care.

Many researchers, truly the best and brightest technical minds in our profession, have positive intentions and believe their actions are beneficial to society. This is primarily because they are focused on vulnerabilities rather than compromises. They mistakenly believe that reducing the vulnerability level is beneficial because they fail to acknowledge the impact on threat levels, which exceed the benefit level provided by a vulnerability disclosure.

There are a number of reasons that researchers use to justify their actions. We will ignore the petty ones for now and address the risk-oriented ones.

“It is better to know than not to know”

The most common justification for vuln disclosure is the assumption that one needs to know about the vulnerability so it can be mitigated. While it is an attractive thought, the real problem with this idea is that nobody can know about every vulnerability. So, for example, though vulnerabilities that will be disclosed in months and possibly years in the future currently exist in an environment, they are not addressed in any way. While patching vulns often makes people feel like they are being productive, there are just too many to focus only on specific ones. This phenomena, by the way, is well known in psychology as “the certainty effect.”

“We must find the vulnerabilities before the attackers do”

This justification is attractive as well, and can work in some highly constrained environments where the population of vulns is relatively small. Unfortunately, in most cases, there are simply too many vulnerabilities – whether that be a single IT environment or the entire software universe.  A conservative approach to understanding this would be to do the math on likelihood of collisions – which is similar to that done in determining the likelihood of a hash collision or to the birthday problem where 365 days is replaced with an estimate of the number of vulnerabilities in your population – that is, number of vulnerabilities that exist whether or not they have been found. The collision rate is clearly not zero, but it isn’t very high, either.

“It is the only way to defend ourselves”

Many security pros still believe that identifying specific vulnerabilities and patching them is our best hope in protection, even if it is a hopeless cause. This simply isn’t true. We see that in the evidence- since we know that we can’t find all the vulns before the bad guys, it is inevitable that we find out in other ways – either through behavorial analysis, heuristics, or some other means. It happens a lot less than compromises of known vulnerabilities (for obvious reasons) but it still happens. This is the area where we really need to get better, since we know there are unidentified vulns in our environment currently.

“It reduces risk in the long run by making vendors and developers more secure programmers”

This justification is often used but completely unsupported by the evidence. A simple look at the total number of vulnerabilities found every year shows an increase more often than not, and when it doesn’t people are quick to point out that there are still many more vulns out there that remain undiscovered. Just comparing the amount of new code written every day to the number of vulnerabilities found in the same period is a sobering thought exercise.

On the other hand, people make qualitative claims of success in certain contexts and perhaps this is true. In these situations, it is worth considering alternative approaches that might provide the same benefit at lower cost. Presumably, everyone agrees that compromises would occur with or without public vuln disclosure. If this is the case, then breach evidence could provide just as effective, perhaps even more effective, justification for training developers to be more secure. Consider, for example, that perhaps the most publicized success in this area, the Trustworthy Computing initiative at Microsoft, was only initiated after Code Red and Nimda ran wild across the Internet, even though many hundreds of vulnerabilities had been disclosed up until that point.

As a profession, and really a society, we have tolerated and even lauded bug-outing activities without any evidence that it helped. It can be exciting because many of the characters involved have colorful personalities and are clearly among the best and brightest, as mentioned earlier. Risk management is challenging to begin with, so being able to cling to specific information, like vulnerabilities, and resolve them makes us feel like we are making progress.  SCADA consequences increase the stakes significantly into the physical realm. It is time to consider the implications of our actions and respond accordingly.

Tags: Disclosure, risk, Vulnerability, weaponization

This entry was posted on Tuesday, January 24th, 2012 at 9:19 pm and is filed under Analysis. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.