How much better is it to have a world-class patching process compared to an average one? Could it ever be detrimental to patch too fast? And what does patching have to do with cholera? Two earlier Verizon Business Risk Team Studies shed more light on this subject.

The recently published “Verizon Business 2008 Data Breach Investigations Report” describes characteristics of more than 500 computer crime investigations performed over the past four years. Our data shows that in only 18% of cases in the hacking category (see Figure 11) did the attack have anything to do with a “patchable” vulnerability. Further analysis in the study (Figure 12) showed that 90% of those attacks would have been prevented had patches been applied that were six months in age or older! Significantly, patching more frequently than monthly would have mitigated no additional cases.

Given average current patching strategies, it would appear that strategies to patch faster are perhaps less important than strategies to apply patches more comprehensively. This is particularly true if you are trying to avoid a breach that would lead to a computer crime and forensic investigation.

For many, this is an unexpected finding, but two recent studies (Control Effectiveness and Sasser) performed by The Verizon Business RISK Team yielded similar but perhaps even more “unusual” conclusions. Like almost all things scientific, they also provide additional insight into truly understanding how patching adds (or diminishes) control effectiveness in enterprises.

Control Effectiveness Study

In a recent “Control Effectiveness Study” (partially published by Baker, W., in Infosecurity Today, Elsevier, May/June 2006, and IEEE Security and Privacy, Vol. 5, 2007) we interviewed the key person responsible for internal security (CSO) in just over 300 companies for which we had already established a multi-year data breach and malcode history. We asked the CSO to rate how well each of dozens of countermeasures were actually deployed in his or her enterprise on a 0 to 5 scale. A score of “zero” meant that the countermeasure was not in use. A score of “5″ meant that the countermeasure was deployed and managed “the best that the CSO could imagine it being deployed in any similar company in the world.” A score of “3″ represented what the CSO considered an average deployment of that particular countermeasure.

We then compared the actual loss experience to see if companies that deployed a given countermeasure exceptionally well (a self-score of 4 or 5) correlated with a reduction of actual hacking or malicious code events compared with companies who had average deployments (scores of 2 and 3) of the same countermeasure. Correlation coefficients were calculated for dozens of countermeasures with respect to the actual hacking and malcode experience at each company.

You will remember from your Statistics 101 course that the lower the correlation coefficient (p), the more “significant” the correlation between two parameters, and that a correlation coefficient of less than p < 0.05 is generally considered to indicate “significant” correlation between the two parameters, while those where p is greater than 0.05 are generally considered “not significant”. The further the coefficient is from 0.05, the more significant it is or is not, respectively. In this study, doing a “world class” job of either vulnerability patching (p=.096), or AV software updates (p=0.249) showed no significant correlation with those who did an average job. Incidentally, in the same study, both applying default deny ingress and egress router ACL’s (p=0.006) and doing light-weight hardening to a “minimum configuration” (p=0.007) were very highly correlated with lower malcode or hacking events.

To summarize the findings in our “Control Effectiveness Study”, companies who did a great job of patching (or AV updates) did not have statistically significant less hacking or malicious code experience than companies who said they did an average job of patching or AV updates. And companies who did other simpler countermeasures, like lightweight standard configurations, had very strong correlations with reduced risk. The Verizon Business 2008 Data Breach Investigations Report supports very similar conclusions.

Sasser Study

From 2000 through 2005 we performed a total of eight different “Large Security Event Studies.” For each study we surveyed between 1,350 and 3,500 companies beginning about 10 days after the peak of a significant malcode attack (Blaster, Code Red, MyDoom, Nimda, Sasser, Slammer, SoBig and Zotob). Sasser was a worm that exploited a Windows LSASS vulnerability (MS04-011). The worm propagated widely, and exploited a Microsoft management interface available via TCP port 445, along with (depending on the variant) FTP and Remote shell on TCP ports 5554 and 9996. Among other things, our study showed that the worm infected and caused what we termed moderate or major impact at an average of 18% of surveyed companies. The Microsoft patch was published on April 13, and Sasser.B came out on May 1 with other, prevalent variants following over the next two weeks.

We studied the power of numerous countermeasures in this study. The two with the most effectiveness were default deny ACLs on internet-facing routers or firewalls, and instructions for users to reboot laptops before using them on the corporate LAN. Just over half of the medium and larger companies in our study had “completed” their deployment of the relevant Microsoft patch before Sasser hit. Oddly, these companies were statistically worse off than average companies in the survey (28% suffered a significant infection, compared with 18% of average companies in the survey). No companies who had used both the Laptop and default deny controls were infected, and only 1% who had actually “achieved” 100 % of machines patched were infected (this was almost exclusively companies with less than 100 PCs).

In summary, the Sasser worm study analysis found that companies who had succeeded at “patching fast” were significantly worse off than “average” companies in the same study. This seemed to be because, as a group, these companies tended toward less use of broad, generic countermeasures. They also thought they had patched everyone, when in reality they hadn’t. You might say they spent more of their energy and money on patching and less on routing, ACLs, standard configurations, user response training, and similar “broad and fundamental” controls.

Conclusions

Collectively, our “Verizon Business 2008 Data Breach Investigations Report”, along with our earlier studies, suggests that getting the right mix of countermeasures in an enterprise is far from simple. Rather than “do more,” all three studies seem to suggest that we should “work smarter.” The Sasser study shows that in some cases working harder seems to not only consume significant resources, but is also sometimes counterproductive. Unfortunately, precious few of us have the data or risk models available to show us exactly how to focus our limited time and resources.

A control like patching, which has very simple and predictable behavior when used on individual computers, (i.e., home computers) seems to have more complex control effectiveness behavior when used in a community of computers (as in our enterprises).

Communities behave differently than individuals.

This reminds me of the differences between individual medicine and community health. After all, you can effectively treat an individual with cholera with a mixture of salt and sugar water, but putting salt and sugar in the drinking water does nothing to reduce cholera in the community.

If you don’t have a stomach ache by now, let us know your experiences.

Tags: Data Breach Report, forensics, Information Security, Patching, Personally Identifiable Information, Vulnerabilities

This entry was posted on Friday, June 13th, 2008 at 1:17 pm and is filed under Studies & Whitepapers. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.