Evidence Based Risk Management & Applied Behavioral Analysis
Alex HuttonSeptember 29th, 2010
To begin with, let me freely admit that I am not the fastest runner off the line. I have had the pleasure over the past few months to collaborate with Allison Miller on topics such as threat and risk modeling and fraud prevention, conversations that have provided me with much food for thought. These discussions reminded me of how over the past few years I’ve had the pleasure of listening and engaging colleagues as they patiently explain to me their frustrations with risk expression, risk management, and trying to turn their GRC programs into something useful. I have actually been told things such as, “Alex, we’re just doing GRC so that we can say, ‘yeah, we have GRC’”. While mulling all this over, I have been trying to get my head around why current approaches to risk expression seem to be so disconnected from operational security.
These experiences, combined with our work here around VERIS has lead me to believe that the whole idea of Evidence Based Risk Management is going to prove significant, but exactly why that was the case wasn’t apparent to me until recently. I had this epiphany while reading a tweet from Richard Bejtlich. He said,
“I’m a fan of appsec, honestly. However, we can’t code our way out of a problem that, at its core, involves crime and national security.”
If you have ever used Twitter, you will realize what a great achievement it is to fit your thoughts (particularly if they are in any degree profound or possess noteworthy knowledge or insight) into 140 characters. In my quest to express myself as efficiently as possible it was my desire to tweet some form of pithy statement of wholehearted agreement. Richard’s tweet had something to do with the fact that you can’t engineer ‘secure’, and that security was really in large part about understanding the behaviors of your adversary. It was at this moment when all the discussions Allison and I had shared regarding behavioral economics and how that field might play into infosec came to mind, and it dawned on me all at once. I had an epiphany, or at least a moment of clarity for discordant thoughts that have been jumbling around in my mind for a while.
SOME THOUGHTS ON INFORMATION RISK
Risk meets much of the definition of a hypothetical construct. While we can identify factors that, in combination (impact, threat, vulnerability, etc.) we believe contribute to a state, risk is not a directly observable object or action. It is an inherently subjective description of a state, one that we tend to attempt to “calculate” (be that qualitatively or quantitatively) to create meaning. This has caused InfoSec a lot of problems. It has also cost us a lot of money.
And it’s confounded me for some time. At B-Sides in San Francisco this spring, and again at Secure360 in Minneapolis, I talked about how I was becoming a “deconstructionist” with regard to risk and risk management. I discussed how I have been thinking that analysis of factors that contribute to risk was much more important than bubbling up some ALE-like statement for “risk” itself. My presentations at these events showed how, in my research, I came to think that we’re dealing with complex adaptive systems. If that’s true, then how point probabilities such as our ALE statements are of limited validity – for complex adaptive systems the only thing we can really do is identify and respond to patterns in information (for InfoSec, the information we worry about are those risk subcomponents).
Evidence Based Risk Management and Applied Behavioral Analysis
This line of thinking allowed me to better grasp and define this concept of Evidence Based Risk Management. Observable behaviors and traits (for failures see the DBIR, for successes we might look to Visible Ops For Security) are much more addressable or “pragmatic” than hyper-probabilistic calculations for a hypothetical construct.
And then, in my research around Applied Behavioral Analysis as a discipline, I came across this:
behavior analysts reject the use of hypothetical constructs and focus on the observable relationship of behavior to the environment.
If you’re one of those self-confessed “risk/security pragmatists”, this ought to light your toes on fire. For anyone who has thought that maybe game theory might be applicable to InfoSec, this ought to get your heart racing. For everyone who has ever thought that understanding business processes was more important than signatures and packet contents – here’s your fix.
And think about it from an information theory standpoint. The purpose of all the information we receive via network devices, logs, SIEMs, it’s really all done for us to understand behaviors. Now, we may be more focused on system behavior, or traffic behaviors or looking in a wrong or ineffective manner, but when you distill the information to try to actively defend against an intelligent attacker you’re talking about understanding their behaviors in the context of how your systems will (probably) behave and what you might be able to do about that (detect/respond/prevent). The measurement concepts of ABA (repeatability, extent, locus, and derivative measures) can be easily applied to SIEM and log management programs.
In addition we must consider the threat landscape. At SourceBarcelona, I had the opportunity to sit down with Chris Nickerson for a while and discuss modeling threat intent; a conversation that has lead me to develop a quick and interesting intent modeling construct based on VERIS. This sort of intent modeling is very much an exercise in behavioral analytics.
VERIS, IT ISN’T JUST FOR INCIDENT RESPONSE TEAMS ANYMORE
In this manner VERIS, as an object model, now becomes HUGELY important for organizations. VERIS is not primarily significant because it gives great definitions (though it provides that benefit), it is beneficial because its A^4 model, in the context of a data breach, is a very real description of this above behavioral collision between threat and assets/business processes.
So now I finally get it. A little slower than most others, maybe, but I’m there now. The benefit of metrics, risk analysis, threat modeling, it’s all just ABA, and EBRM is simply the study of those behaviors.
Alex -
It seems to me that you have identified yet another piece of the puzzle – behaviors.
I would agree that understanding behaviors that occurred in the past, including 10ms ago, can be very useful – at least to the extent that those behaviors don’t evolve.
But surely, at some point, the past will fail to inform the future, and a malicious behavior will go undetected in the present moment and have an adverse impact, small or large, before we have time to react.
Were that not the case, then antivirus and other types of signature based defenses would be all that we would need, no?
As we discussed in Barcelona, frequency of attack becomes almost infinite for some types of attack, and, consequently, frequency becomes meaningless as a term of a risk quantification equation.
But impact/loss exposure/unrealized loss exposure still exists and has to be estimated, however poorly – otherwise what’s the justification for spending money on all of the behavior detection technologies and methodologies. And how do you know whether you are spending enough, too little, or too much, unless you have an estimate of your loss exposure/risk?
For me, it comes down to the fact that risk means many things simultaneously, and we do ourselves a disservice by locking into a single concept.
Posted by: Patrick Florer on September 29th, 2010 at 6:19 pmAlex,
one of the best security posts of the past months, for sure.
It really seems that ABA has its place in the infosec field. I’m just curious about why you are talking about systems and traffic behaviour, when ABA theory has a better place for that, the “enviroment”. Even when we start thinking about actions to change behaviour (from the attackers? “users”?), that’s usually done through manipulating the environment. And if we end up finding that those subjects usually have similar behaviours, we’ll probably find out that the differences are mostly in the different envionments.
the interesting part of going through ABA is that it drives us to experimental control for attempts to change behaviour. The implications would certainly force us into finding ways to verify if our controls cn really induce behaviour change. That’s one of the key issues we have Today in our field. If the attackers are behaving “accordingly” (i.e. not performing successful attacks), is that due to our attempts to change their behaviour or because of other external stimulus? Since you mentioned Bejtlich, isn’t one of his favorite ideas, the continuous testing by a “red team”, a good away to assess if the stimuli we are generating are really successful in causing behaviour change?
Posted by: Augusto Paes de Barros on October 1st, 2010 at 3:44 pm