Insider Breach Stats: Bogus, Biased, or Believable?
By Wade Baker
Our 2008 Data Breach Investigations Report presents statistics on the percentage of breaches involving outsiders, insiders and partners (73%, 18%, and 39% respectively). Public reaction to these statistics runs the gamut from revulsion to revelry. This is especially true with respect to the relatively low percentage of breaches tied to insiders. Some seem to think we’ve blasphemed the sacred doctrines of our trade handed down from on high long ago. Others are glad to see their oft-ridiculed beliefs finally vindicated by objective data. Many in the middle are cautious about drawing conclusions, and are unsure what to make of the statistics.
Which reaction is appropriate? We won’t weigh in on that question; we’ll stick to providing data rather than dictating the reactions of others. We would, however, like to address the underlying questions fueling such reactions - whether these statistics are bogus, biased or believable.
Before we begin, let’s make sure we’re all on the same page, and reacting to statistics rather than semantics. First of all, our findings relate specifically to the likelihood of security breaches leading to data compromise - not attacks, not general security incidents and not risk (although we do touch on insider risk elsewhere in the report). Though these terms are often used interchangeably in media coverage of the report, they are not the same. Secondly, our inclusion of partners as a third source seems to have caused some confusion, as most other reports contrast only insiders and outsiders. We can’t be sure in which of those two buckets these other sources traditionally place trusted business partners. Admittedly, this confounds efforts to compare our results to other reports, but we find isolating partners as a distinct source to be helpful for many reasons (Perhaps others could assist by adopting a three-source system when reporting such results?). With that stumbling block out of the way, let’s move on.
Are these statistics BOGUS? We don’t think so – they are what they are. Keep in mind the report is based on first-hand observations from professional investigators rather than on surveys, or other more subjective means of data collection. Although in some cases we were unable to reliably confirm the source of the breach (denoted by the shaded bar segments in the figure above), the determination is typically straightforward. We believe the statistics to be reliable within the scope of our caseload.
Are these statistics BIASED? Without question - we never claimed otherwise. Anytime statistics within a sample (our caseload) differ from those of the overall population, bias is present. So, exactly how biased are our statistics on insider breaches? Unfortunately, it is impossible to know. To answer that, we’d need to analyze all undiscovered, all discovered but unreported and all reported data breaches everywhere. The first two are (and will always be) unavailable for comparison. We can speculate (i.e., perhaps insider breaches are less likely to be discovered or reported) but we could never measure bias. Comparing our results with the final group (all reported breaches) is no cakewalk either. While public sources of data breach disclosures exist, they are often incomplete, vague or flat-out wrong. Though we were engaged to investigate roughly ¼ of disclosed breaches (a very large sample) according to one source (http://www.idtheftcenter.org/), we noted significant differences between the two samples. If you demand unbiased statistics on data breaches, you’d best look elsewhere than the “2008 Data Breach Investigations Report”…just please let the rest of us know when you find that pot o’ gold.
Are these statistics BELIEVABLE? We think so, but it really depends on the expectations one has for these results. If perfect statistical accuracy is the goal, then they clearly won’t make a believer out of you. We suggest, however, that statistical accuracy isn’t the goal; it is a means to an end. That end is facilitating better, more justified actions to reduce the risk of data compromise. Bogus or overly biased statistics work against this goal, but it can still be achieved with less than perfect data. We believe our findings accomplish this or we would not have published the report. Supporting this conclusion, other comparable sources of data breach statistics show results very similar to our own. A tally based on Attrition.org’s Data Loss Database (DLDOS) between 2000 and 2007 finds roughly 668 (76%) breaches were caused by outsiders and 208 (23%) by insiders. The Identity Theft Resource Center reports an even lower percentage for insider breaches. This corroboration among independent sources, along with a lack of conflicting evidence, enhances the believability of our insider breach statistics. Is it enough to make you a believer?
We appreciate the dialogue taking place on this topic, and invite you to take up the discussion on our blog.
Tags: Data Breach Report, forensics, Information Security, insiders, statistics
Is number of breaches indider vs. outsider relevant, or are we looking at the wrong statistic.
If I’m managing risk instead of metrics, I want to know the amount of damage (exposure) done by one or the other. The data clearly shows that the insiders _do the most damage_ to an organization in terms of data exposure.
My experience is that it is also more costly to clean up after an insider.
Posted by: Hawke on July 11th, 2008 at 3:45 pm@Hawke: You bring up an interesting point and, to a certain extent, we agree - managing risk is clearly the right focus. We do not, however, think the number of breaches is the “wrong” statistic. Since risk is the product of likelihood (number of incidents) and impact, statistics on both parameters are necessary. Concentrating solely on impact or damage quickly leads to a form of decision making we like to call “WIBiHI” (Wouldn’t It Be Horrible If…”) and can often result in gross overspending. As you suggest, managing security based on likelihood, alone, isn’t a winning strategy either.
Keep in mind the original blog post is in response to public reaction to our findings regarding the percentage of breaches involving insiders. We’re not suggesting likelihood is more important than impact (or damage); it’s just that nobody seemed to argue against or misconstrue the latter parameter.
Posted by: Russ Cooper on July 21st, 2008 at 2:49 pm