1) A breach investigation focuses on the collection of evidence related to who did it, how, when, what was compromised, etc. Analyzing and quantifying financial losses to the victim organization is simply not what we’re paid to do. Although we do occasionally gather information relevant to the impact of a breach, we do not gather near enough pieces to complete the puzzle. Nor are we on the ground long enough after the breach to truly study the long-term consequences.

2) While we could include the bits and pieces that we collect on losses, we made a decision at the very beginning not to do so. One of the aspects of the DBIR that we (and we hope many others) like is that, from cover to cover, it is filled with objective, credible, factual information. Since we do not collect data of that caliber on losses during an investigation, we do not feel it fits with the rest of the report.

That said, we’re not blind. We realize that breach details along with a credible account of financial losses is the “Holy Grail” of our field. I won’t give away too much now, but let’s just say that we’re actively working on something that may please the masses.

Creating the single-year sequel to a four-year report on over 500 breach investigations is a daunting prospect. While it would be impossible to trump the sheer scope of the original 2008 DBIR, we’ve sought to preserve its strengths and introduce some key enhancements for 2009. Here is some of what you can expect in this release:

First, you’ll notice the report is quite a bit larger than last year. Hopefully it’s worth the extra disk space (which isn’t saying much given current prices) and/or toner (which *is* saying a lot given current prices). Rather than platitudes and pitches, we’ve worked to fill those extra pages with real substance. Everyone loves data.

More data was possible, in part, due to an important methodological change in 2008. Whereas the original DBIR reached back across four years in one massive data collection effort, this data set was assembled periodically throughout the year. This shift from historic to ongoing collection allows for more detail on existing data points and opens the door to new areas of study.

We’ve also listened to your feedback and requests since the last report was released. We couldn’t possibly address everything but we’ve tried to be responsive and accommodating. Your feedback was very much appreciated last year and we covet your input again on this report. Over the next few days, we will roll out posts for each major section in the report. If a section intrigues you, you’ll find pointers in the document to the accompanying post. It’s your opportunity to tell us what you think and what you’d like to see next year.

Finally, 2008 was a crazy year in the world of data breaches. One might argue it was a crazy year in general, but that’s a different discussion. In terms of cybercrime, the bad guys were really busy and, unfortunately, really successful. We saw much of the same in 2008 but new twists and trends undoubtedly emerged. The percentage of breaches in our caseload involving financial service organizations, targeted attacks, and customized malware all doubled in 2008. It’s sure to win me the “Captain Obvious Award” from the Securitymetrics list, but organized crime activity increased and was responsible for over 90% of the 285 million records compromised. The scales continue to tilt more and more toward servers and applications as the point of compromise. I don’t want to spoil the fun so I’ll close this out and let you get to the report.

As with last year, our goal is that the data and analysis presented in the report prove helpful to the planning and security efforts of organizations around the world. Beyond that, we also hope you simply find it an enjoyable read. Cheers.

Yes, you read that correctly. I’m as flabbergasted as you are. We knew the number was big when we recently started combing through last year’s statistics in preparation for the upcoming 2009 Data Breach Investigations Report (DBIR), but I don’t think we quite knew it was THAT big. To put this number in perspective, that means 9 records were compromised for every second that ticked by in 2008 – and that’s just among the cases Verizon Business investigated! To put that in further perspective, you may remember that in the 2008 DBIR we reported a figure of 230 million records from cases we worked between 2004 and 2007.

What happened? We’re currently up to our data-loving eyeballs trying to put together an answer to that question. We will have it to you on April 15 in the form of the 2009 Data Breach Investigations Report…so stay tuned.

Listen to Part I

Listen to Part II

Today, we released a supplement to our 2008 Data Breach Investigations Report (DBIR) that focuses on four major industry groups. As many of you know, the original document compiled four years of data from over 500 cases worked by our Investigative Response team and was intended to be a kind of “state of the union” look at recent security breach and data compromise trends.

The DBIR presented statistics in aggregate across all organizations in our caseload and did not delve into the state of affairs within each of the industries represented. Drawing from the same data set as the original, the Supplemental Report provides this analysis for the financial, technology services, retail, and food and beverage industries.

You probably can’t read it on the miniature cover shot above (you can if you download the full report here) but part of the subtitle reads “Industry Focus. More Analysis. Greater Insight.” It might sound like unadulterated marketing mumbo-jumbo but it’s actually a fitting description of what we feel the report accomplishes. Sure, it looks more closely at a few industries and provides more analysis, but the real value is greater clarity and insight into the original data. Why? Well, you can read the report for a complete answer to that question but one of the major reasons involves that age-old enemy of statistics, The Flaw of Averages. Any time you average a bunch of data points, the result is a middle-of-the-road expression of the data. Variations, fluctuations, and groupings (which often provide great insight) are lost. That’s not to say we shouldn’t average whole sets of data – this certainly has value – but other methods of slicing up and analyzing data are important to the understanding of what’s really going on.

We hope the new Supplemental Report sheds more light on the findings presented in the DBIR in ways that are helpful to your organization. Even if your industry is not included among the four discussed throughout the report, perhaps you can identify with certain characteristics of them or experience similar challenges in your business environment. At the very least, it should reinforce the notion that an efficient and effective information security program cannot be achieved through a standardized template applied without regard to the unique risks faced by each organization.

By Wade Baker

Since releasing the 2008 Data Breach Investigations Report (DBIR) in June, we’ve frequently been asked some form of the following question: “Do the findings presented in the report differ among industries?” It’s a good question, and one we’re working on answering at length in a supplemental report contrasting the four most frequently breached industries (Financial Services, Tech Services, Retail, and Food & Beverage) using the original dataset. We plan to release the report sometime next month, but would like to give you a sneak peak in this post.

You may remember that the 2008 DBIR considered three main sources, or origins, of data breaches: external, internal and partner. The upcoming supplemental report naturally adopts this same trio of sources. Based on Verizon Business caseload from 2004 through 2007, the figure below depicts the percentage of breaches attributed to internal, external and partner sources for each industry group.

The predominant pattern to note here is that each industry exhibits the same pattern or order (External sources being highest followed by Partner then Internal) except Tech Services, in which insider breaches were more common than those involving partners. Our explanation of this finding is straightforward: Tech Services are often in the role of “the partner” to the other industries, providing management, hosting and other services. It stands to reason that organizations in this industry likely employ a high percentage of tech-savvy staff and grant them high levels of access to numerous systems. Unfortunately, some find that access to sensitive and valuable resources is a temptation too difficult to resist. Facing similar temptations, insiders in the Financial Services industry were behind a large proportion of breaches as well.

The Food and Beverage industry shows a very different but yet striking series of statistics. Insider breaches fall well below other industries while the percentage for partners is extremely high – almost equaling that of external sources. At first this may seem counter-intuitive as staff within this industry constantly handle money, checks and credit cards. When incidents happen, however, they are more likely to be handled by law enforcement personnel than our Investigative Response team since such thievery doesn’t typically require hacking into systems.

The large percentage of partner breaches in the Food and Beverage industry is mostly due to the scenario mentioned earlier in which an external attacker compromises a partner and then uses that asset as a privileged platform to attack the victim. In the Food and Beverage industry, this is often a vendor supporting the Point of Sale system using default or shared credentials among many clients. Though not a willing accomplice, the partner’s lax security practices – often outside the victim’s control – undeniably allow such attacks to take place. This is obviously a much needed area of focus for security efforts within the Food and Beverage industry.

So, do the findings of the 2008 DBIR differ among industries? You betcha. We hope this gives you a taste of what those differences entail. The remaining statistics from the 2008 DBIR will be aligned with these key verticals in the upcoming 2008 Data Breach Investigations Supplemental Report.

Our 2008 Data Breach Investigations Report presents statistics on the percentage of breaches involving outsiders, insiders and partners (73%, 18%, and 39% respectively). Public reaction to these statistics runs the gamut from revulsion to revelry. This is especially true with respect to the relatively low percentage of breaches tied to insiders. Some seem to think we’ve blasphemed the sacred doctrines of our trade handed down from on high long ago. Others are glad to see their oft-ridiculed beliefs finally vindicated by objective data. Many in the middle are cautious about drawing conclusions, and are unsure what to make of the statistics.

Which reaction is appropriate? We won’t weigh in on that question; we’ll stick to providing data rather than dictating the reactions of others. We would, however, like to address the underlying questions fueling such reactions – whether these statistics are bogus, biased or believable.


Before we begin, let’s make sure we’re all on the same page, and reacting to statistics rather than semantics. First of all, our findings relate specifically to the likelihood of security breaches leading to data compromise – not attacks, not general security incidents and not risk (although we do touch on insider risk elsewhere in the report). Though these terms are often used interchangeably in media coverage of the report, they are not the same. Secondly, our inclusion of partners as a third source seems to have caused some confusion, as most other reports contrast only insiders and outsiders. We can’t be sure in which of those two buckets these other sources traditionally place trusted business partners. Admittedly, this confounds efforts to compare our results to other reports, but we find isolating partners as a distinct source to be helpful for many reasons (Perhaps others could assist by adopting a three-source system when reporting such results?). With that stumbling block out of the way, let’s move on.

Are these statistics BOGUS? We don’t think so – they are what they are. Keep in mind the report is based on first-hand observations from professional investigators rather than on surveys, or other more subjective means of data collection. Although in some cases we were unable to reliably confirm the source of the breach (denoted by the shaded bar segments in the figure above), the determination is typically straightforward. We believe the statistics to be reliable within the scope of our caseload.

Are these statistics BIASED? Without question – we never claimed otherwise. Anytime statistics within a sample (our caseload) differ from those of the overall population, bias is present. So, exactly how biased are our statistics on insider breaches? Unfortunately, it is impossible to know. To answer that, we’d need to analyze all undiscovered, all discovered but unreported and all reported data breaches everywhere. The first two are (and will always be) unavailable for comparison. We can speculate (i.e., perhaps insider breaches are less likely to be discovered or reported) but we could never measure bias. Comparing our results with the final group (all reported breaches) is no cakewalk either. While public sources of data breach disclosures exist, they are often incomplete, vague or flat-out wrong. Though we were engaged to investigate roughly ¼ of disclosed breaches (a very large sample) according to one source (http://www.idtheftcenter.org/), we noted significant differences between the two samples. If you demand unbiased statistics on data breaches, you’d best look elsewhere than the “2008 Data Breach Investigations Report”…just please let the rest of us know when you find that pot o’ gold.

Are these statistics BELIEVABLE? We think so, but it really depends on the expectations one has for these results. If perfect statistical accuracy is the goal, then they clearly won’t make a believer out of you. We suggest, however, that statistical accuracy isn’t the goal; it is a means to an end. That end is facilitating better, more justified actions to reduce the risk of data compromise. Bogus or overly biased statistics work against this goal, but it can still be achieved with less than perfect data. We believe our findings accomplish this or we would not have published the report. Supporting this conclusion, other comparable sources of data breach statistics show results very similar to our own. A tally based on Attrition.org’s Data Loss Database (DLDOS) between 2000 and 2007 finds roughly 668 (76%) breaches were caused by outsiders and 208 (23%) by insiders. The Identity Theft Resource Center reports an even lower percentage for insider breaches. This corroboration among independent sources, along with a lack of conflicting evidence, enhances the believability of our insider breach statistics. Is it enough to make you a believer?

We appreciate the dialogue taking place on this topic, and invite you to take up the discussion on our blog.

The recently published “Verizon Business 2008 Data Breach Investigations Report” describes characteristics of more than 500 computer crime investigations performed over the past four years. Our data shows that in only 18% of cases in the hacking category (see Figure 11) did the attack have anything to do with a “patchable” vulnerability. Further analysis in the study (Figure 12) showed that 90% of those attacks would have been prevented had patches been applied that were six months in age or older! Significantly, patching more frequently than monthly would have mitigated no additional cases.

Given average current patching strategies, it would appear that strategies to patch faster are perhaps less important than strategies to apply patches more comprehensively. This is particularly true if you are trying to avoid a breach that would lead to a computer crime and forensic investigation.

For many, this is an unexpected finding, but two recent studies (Control Effectiveness and Sasser) performed by The Verizon Business RISK Team yielded similar but perhaps even more “unusual” conclusions. Like almost all things scientific, they also provide additional insight into truly understanding how patching adds (or diminishes) control effectiveness in enterprises.

Control Effectiveness Study

In a recent “Control Effectiveness Study” (partially published by Baker, W., in Infosecurity Today, Elsevier, May/June 2006, and IEEE Security and Privacy, Vol. 5, 2007) we interviewed the key person responsible for internal security (CSO) in just over 300 companies for which we had already established a multi-year data breach and malcode history. We asked the CSO to rate how well each of dozens of countermeasures were actually deployed in his or her enterprise on a 0 to 5 scale. A score of “zero” meant that the countermeasure was not in use. A score of “5″ meant that the countermeasure was deployed and managed “the best that the CSO could imagine it being deployed in any similar company in the world.” A score of “3″ represented what the CSO considered an average deployment of that particular countermeasure.

We then compared the actual loss experience to see if companies that deployed a given countermeasure exceptionally well (a self-score of 4 or 5) correlated with a reduction of actual hacking or malicious code events compared with companies who had average deployments (scores of 2 and 3) of the same countermeasure. Correlation coefficients were calculated for dozens of countermeasures with respect to the actual hacking and malcode experience at each company.

You will remember from your Statistics 101 course that the lower the correlation coefficient (p), the more “significant” the correlation between two parameters, and that a correlation coefficient of less than p < 0.05 is generally considered to indicate “significant” correlation between the two parameters, while those where p is greater than 0.05 are generally considered “not significant”. The further the coefficient is from 0.05, the more significant it is or is not, respectively. In this study, doing a “world class” job of either vulnerability patching (p=.096), or AV software updates (p=0.249) showed no significant correlation with those who did an average job. Incidentally, in the same study, both applying default deny ingress and egress router ACL’s (p=0.006) and doing light-weight hardening to a “minimum configuration” (p=0.007) were very highly correlated with lower malcode or hacking events.

To summarize the findings in our “Control Effectiveness Study”, companies who did a great job of patching (or AV updates) did not have statistically significant less hacking or malicious code experience than companies who said they did an average job of patching or AV updates. And companies who did other simpler countermeasures, like lightweight standard configurations, had very strong correlations with reduced risk. The Verizon Business 2008 Data Breach Investigations Report supports very similar conclusions.

Sasser Study

From 2000 through 2005 we performed a total of eight different “Large Security Event Studies.” For each study we surveyed between 1,350 and 3,500 companies beginning about 10 days after the peak of a significant malcode attack (Blaster, Code Red, MyDoom, Nimda, Sasser, Slammer, SoBig and Zotob). Sasser was a worm that exploited a Windows LSASS vulnerability (MS04-011). The worm propagated widely, and exploited a Microsoft management interface available via TCP port 445, along with (depending on the variant) FTP and Remote shell on TCP ports 5554 and 9996. Among other things, our study showed that the worm infected and caused what we termed moderate or major impact at an average of 18% of surveyed companies. The Microsoft patch was published on April 13, and Sasser.B came out on May 1 with other, prevalent variants following over the next two weeks.

We studied the power of numerous countermeasures in this study. The two with the most effectiveness were default deny ACLs on internet-facing routers or firewalls, and instructions for users to reboot laptops before using them on the corporate LAN. Just over half of the medium and larger companies in our study had “completed” their deployment of the relevant Microsoft patch before Sasser hit. Oddly, these companies were statistically worse off than average companies in the survey (28% suffered a significant infection, compared with 18% of average companies in the survey). No companies who had used both the Laptop and default deny controls were infected, and only 1% who had actually “achieved” 100 % of machines patched were infected (this was almost exclusively companies with less than 100 PCs).

In summary, the Sasser worm study analysis found that companies who had succeeded at “patching fast” were significantly worse off than “average” companies in the same study. This seemed to be because, as a group, these companies tended toward less use of broad, generic countermeasures. They also thought they had patched everyone, when in reality they hadn’t. You might say they spent more of their energy and money on patching and less on routing, ACLs, standard configurations, user response training, and similar “broad and fundamental” controls.

Conclusions

Collectively, our “Verizon Business 2008 Data Breach Investigations Report”, along with our earlier studies, suggests that getting the right mix of countermeasures in an enterprise is far from simple. Rather than “do more,” all three studies seem to suggest that we should “work smarter.” The Sasser study shows that in some cases working harder seems to not only consume significant resources, but is also sometimes counterproductive. Unfortunately, precious few of us have the data or risk models available to show us exactly how to focus our limited time and resources.

A control like patching, which has very simple and predictable behavior when used on individual computers, (i.e., home computers) seems to have more complex control effectiveness behavior when used in a community of computers (as in our enterprises).

Communities behave differently than individuals.

This reminds me of the differences between individual medicine and community health. After all, you can effectively treat an individual with cholera with a mixture of salt and sugar water, but putting salt and sugar in the drinking water does nothing to reduce cholera in the community.

If you don’t have a stomach ache by now, let us know your experiences.

At considerable investment in time and resources, Verizon Business began an initiative in 2007 to identify a comprehensive set of metrics to record during each data compromise investigation. As a result of this effort, we pursued a post-mortem examination of over 500 security breach and data compromise engagements between 2004 and 2007 which provided us with the vast amount of factual evidence used to compile this study. This data covers 230 million compromised records. Amongst these are roughly one-quarter of all publicly disclosed data breaches in both 2006 and 2007, including three of the five largest data breaches ever reported.

The Verizon Business 2008 Data Breach Investigations Report contains first-hand information on actual security breaches rather than on network activity, attack signatures, vulnerabilities, public disclosures and media interpretation, which typically means information that IT Adminstrators tell them, but in reality, aren’t based on any factual evidence but instead, gut instinct.

It is a given that our data is not intended to state facts regarding all Internet crime or all computer criminal activity, but instead the findings from our own caseload. For instance, 55% of our cases involved the Retail and Food and Beverage industries, while only 14% of our cases involved the Financial Services industry. A study involving a different mix of business may well make other findings. We do believe, however, that given the volume of cases and the time period covered, our findings are significant to any industry.

Further, the reader may note findings that are at odds with commonly held beliefs. For example, only 22% of our cases involved exploitation of a vulnerability, of which, more than 80% were known, and of those all had a patch available at the time of the attack. This is not to say that patching is not effective, or necessary, but we do suggest that the emphasis on it is misplaced and inappropriately exaggerated by most organizations. For the sake of clarity, 78% of the breaches we handled would have still occurred if systems had been 100% patched the instance a patch was available. Clearly patching isn’t the solution to the majority of breaches we investigated.

With this understood, there are a number of important findings worth noting here.

While criminals more often came from external sources, and insider attacks result in the greatest losses, criminals at, or via partner connections actually represent the greatest risk. This is due to our risk equation: Threat X Impact = Risk

• External criminals pose the greatest threat (73%), but achieve the least impact (30,000 compromised records), resulting in a Psuedo Risk Score of 21,900
• Insiders pose the least threat (18%), and achieve the greatest impact (375,000 compromised records), resulting in a Pseudo Risk Score of 67,500
• Partners are middle in both (73 39% and 187,500), resulting in a Pseudo Risk Score of 73,125

While these are rudimentary numbers, the relative risk scores are reasonable and discernable. It is also worth noting that the Partner numbers rose 5-fold over the duration of the study, making partner crime the leading factor in breaches. This is likely due to the ever increasing number of partner connections businesses are establishing, while doing little to nothing to increase their ability to monitor or control their partner’s security posture. Perhaps as expected, insider breaches are the result of your IT Administrators 50% of the time.

While the study does indicate that Asian countries represent the largest number of geo-sources (35%), it strongly refutes the common suggestion that all attacks are coming from China. While we acknowledge that determining the source of an attack strictly from IP addresses is extremely difficult, our investigations don’t stop there and, therefore, are far more reliable than any other study on the topic.

We are continually dismayed at the number of breaches that involve what we term “omission”. An example of omission would be policies being established and thought to be in place, but in fact were not. 49% of all cases involved some form of omission. 66% of all cases involved data the victim did not know existed, or, did not know was being stored where it was.

Other significant findings:

• Three quarters of all breaches are not discovered by the victim
• Attacks are typically not terribly difficult or do not require advanced skills
• 85% of attacks are opportunistic rather than targeted
• 87% could have been prevented by reasonable measures any company should have been capable of implementing or performing

From our findings, Verizon Business Solutions Powered By Cybertrust offer several basic suggestions to every company hoping to avoid using our Forensic services:

• Ensure that basic and essential security controls are met across your entire organization consistently, and that these controls are actually implemented as you have stated within your policies. This simple step along removes you from among the low-hanging fruit that represents 85% of the attack targets
• Know what data you have, what its value is, and where it is stored and being used. You can’t secure something you don’t even know is there, and you won’t secure something unless you recognize its value. Criminals already know where data is being stored by applications that do not make that information obvious to the user/owner of it, so ask your vendors.
• Monitoring often creates information overload. Simply logging everything through your firewall could create sufficient work for several analysts daily. Be more strategic in your logging solutions and configurations. Heavily log those systems that are critical, or that are storing the critical data criminals want. By being more selective in your logging you may well make it more obvious for your burdened analysts when a criminal is active.

We believe you’ll find the information within this study to be a potential goldmine of data and insight.

But the Verizon Business 2008 Data Breach Investigations Report tells a very different story.

The successful attacks were almost universally multi-faceted and the various timeframes are truly astounding. The series of pie charts in Figure 21 are the most interesting data.

The first chart shows that more than half of attacks take days, weeks, or months from the point of entry of the attack (the first successful attack step) to the point of data compromise (not simply system compromise, but the point at which the criminal has actually done material harm). 90% take more than hours and over 50% take days or longer. Clearly if an appropriate log was instrumented and being regularly reviewed or an IDS alarm occurred, you would notice and could stop the attack in the vast majority of our cases.

The second pie chart in the series reveals that 63% of companies do not discover the compromise for months and that almost 80% of cases do not learn of attacks for weeks after they occur. In 95% of cases it took the organization longer than days after the compromise to learn of the attack. There are hundreds of cases in which the inside team either didn’t look at the logs (in 82% of the breaches in the study, the evidence was manifested in their logs), or for some other reason (were frustrated, tired, overwhelmed by the logs, found them to be not-interesting, felt they were too noisy after a few days or weeks) simply quit looking.

One of the Verizon Business (nee Cybertrust, nee Ubizen) MSS zealots is Bart Vansevenant. The study surprised him for entirely different reasons: “We in the industry focus on things like correlation engines, security intelligence, and expertise in various platforms, etc … but the simple fact that we as an MSSP will monitor activity consistently around the clock probably is the most compelling reason for MSS. As a CISO, try to get an internal SLA in place that guarantees you that suspicious activity will be seen and followed-up on within 30 minutes; clearly impossible.”

Another significant motivator for MSS is the fact that the majority of attacks required low to moderate skill (not rocket science), and victims were found opportunistically by the criminals. In other words, simply stop yourself being the “low hanging fruit” and you’re likely to significantly reduce the likelihood you’ll have us, or more importantly, the criminals visiting you. MSS definitely accomplishes this, partly by ensuring the best practices have been adhered to, but also by continually ensuring you’re aware of your exposure. Just because a system is connected and lit up shouldn’t mean it is quickly compromised, even if it’s vulnerable, if you’ve defined what should be accessible to your MSSP. Without MSS, systems are frequently exposed inadvertantly, and left exposed for considerable amounts of time without the business being aware.

So what do you think? Torpedo warnings or real value?