What is an “effective” control?
Wade BakerJanuary 12th, 2009
An interesting question went out to one of my favorite mailings lists a few days ago (SecurityMetrics.org) regarding a definition for “effectiveness”. It’s one of those words that we in the security profession use constantly but there seem to be differing opinions on what qualities a control (or group of controls) must have in order to be ‘effective’. For instance, does it need to be foolproof? Prevent at least 90% of attacks? Provide more value than it costs? Satisfy its purchasers? Make auditors happy? Something else?
After thinking over it a bit, I offered up the following definition to the group:
“If it does what it’s supposed to, to the degree it’s supposed to, it’s effective (no matter how much risk, or what % of attacks, etc it reduces). If it does that for a cost that is low relative to its effectiveness, it’s efficient. At the point where the cost of increasing effectiveness exceeds the incremental benefit of doing so, it’s optimal.”
I know this isn’t a new question nor do I feel I’ve offered up some novel, ultra-insightful definition. I’d simply like to know what other folks out there think. Agree / Disagree? Have something better?
Tags: controls, Information Security
If you define price as more than just money (i.e. what % of attacks it reduces, time required from personnel as well as money) then it makes sense. In which case, possibly using the term ‘benefit’ and ‘cost’ may help to split that up. If you leave price as referring to just money, then the additional reasons for using a control may be lost in that definition. i.e. a company could be using a control for a very good reason that would not fit with the definition of efficient or effective.
Posted by: Dominic White on January 13th, 2009 at 10:04 amI think we should consider the organizations capability to manage/use the control.
Posted by: Alex on January 13th, 2009 at 2:12 pmA Couple of Links on Risk & Decision Making…
First, I wanted to point you over to Chris’ Risktical blog. He’ll be doing a FAIR analysis over there that looks interesting. It’s nice that Chris is dedicating his time on this, given the amount of PCI work he’s got on his …..
Posted by: RiskAnalys.is on January 13th, 2009 at 2:58 pmI like your definition, David. I agree that the cost/benefit should be separate. Something may be highly effective, but the cost of the control or the negative impact to the mission make it unworkable. Or it may be marginally effective and practically not worth the effort (for example, if another, more effective, control already mitigates the related risk.
Posted by: Fred on January 13th, 2009 at 3:26 pmI referenced one of your quotes from the securitymetrics.org list at http://chuvakin.blogspot.com/2009/01/making-pci-easy.html#respond.
Posted by: Chris Hayes on January 13th, 2009 at 3:49 pm@Dominic:
Good point about ‘cost’ being a better word than ‘price’. It was not my intention to limit ‘price’ to only refer to $, but I see (and agree) that it reads that way. I’ve modified accordingly. Thanks.
Posted by: Wade Baker on January 13th, 2009 at 4:15 pm@Alex:
Would ‘effectiveness’ and ‘cost’ cover (even if indirectly) the organization’s ability to manage/use the control? In other words, if they are not equipped to manage it well, it may not do what it’s supposed to do (effectiveness) or it may be too expensive or difficult (cost) to maintain/use it at effective levels. I completely agree an organization’s ability to manage and use a control is an important ingredient…I’m just thinking it’s encompassed somewhere within cost and effectiveness. Am I off or missing your intent?
Posted by: Wade Baker on January 13th, 2009 at 4:51 pmGreat thread.
In some past work we opined that control effectiveness was a function of two factors: (1) the potential strength of the control , and (2) the coverage of the attack surface by the control. Perhaps I should add a competence factor to my next version of the algorithm.
Posted by: David Shaw on January 13th, 2009 at 9:10 pm@David
Hello David. Nice to hear from you. I still like those 2 factors and see them as very similar (”does what it’s supposed to” ~ “coverage of attack surface”, “to the degree it’s supposed to” ~ “potential strength”), except your 1&2 get closer to measurement/calculation of effectiveness.
As for “adding competence”…from my experience, anything you do has that built-in already.
Posted by: Wade Baker on January 14th, 2009 at 8:30 pm