FAIR and betaPERT Distributions

Since posting my previous blog entry a couple of weeks ago I have been doing some research on how I could use the openPERT tool to improve the rigor of my selection of the metrics used in the FAIR methodology.  This comes after listening to the discussion in the RiskHose podcast 9 where it was discussed how to attribute an “expert” opinion into your metrics.

While I am no statistician, and I am only just started to read about PERT, Beta distributions and Monte Carlo simulations, I think I have obtained a base understanding of how they are used, especially in the normal context of project management. However ,I am grappling with how I can make use of these tools in the context of the FAIR risk methodology.  So, as a way to try draw out my thinking, I thought I would jot down my current thoughts and hope for some feedback/direction to determine if I am heading in the correct direction.

To ensure this post doesn’t turn into War and Peace, I will limit my thoughts to the initial FAIR metric selection of Threat Community capability. The FAIR Threat Community capability suggests five categories:-

Rating Description
Very High (VH) Top 2% when compared against the overall threat population
High (H) Top 16% when compared against the overall threat population
Moderate (M) Average skill and resources (between bottom 16% and top 16%)
Low (L) Bottom 16% when compared against the overall threat population
Very Low (VL)  Bottom 2% when compared against the overall threat population

The problem is, a particular generic Threat Community will be made up many smaller Threat Communities, each of which have their own capabilities. So, it would be hard to define a Threat Community, such as “Organised Crime”, as just having a High (H) threat capability.

I believe that you need to be able to justify any metric you use and it has to pass the “smell” test to the lay reader. That is, if a metric doesn’t “smell” right, then it will create doubt in the reader for the rest of the assessment.  Given the diversity of a Threat Community such as “Organised Crime”, I believed that their capabilities would range across Moderate (M) to Very High (VH).

Before going further it is probably best to further define the example Threat Community of “Organised Crime”. This Threat Community would be those criminals working as a group to misuse computer systems for the purpose of monetary or material gain. To better understand what potential Threat Vectors this Threat Community may use we can define the motivation even further to include:

  • Illegal Information Disclosure
  • Unauthorised Data Alteration
  • Fraud
  • Blackmail
  • Supporting Traditional Crime

When I wrote my original risk assessment I tried to visualise a Threat Community’s capability for the reader. To do this I displayed the metric using a chart with a coloured normal distribution curved such as:-

Medium_to_High

Other communities with more defined capabilities would be shown as the below.

High

Low

It was when I visualised a broad capability such as “Organised Crime” that I became uncomfortable with my capability choice. Basically, it didn’t “smell” right to me. While it is probably true that “Organised Crime” will have such a broad capability as a Threat Community, from an Medium to Large Enterprise point of view, there will be a large part of that Threat Community that would not be relevant. Both because Enterprises don’t meet specific sub-threat communities target or motivation, and also because the techniques used can be expected to be ineffective in an Enterprise environment (assuming firewalls, proxies with authentication, patching, anti-malware software etc ). This sub-threat community would be targeting “consumer” end users, and would need a much lower “force” to be applied due to the low control strengths of a “normal” home personal computer.

This is the point at which I had reached before I wrote my previous entry. While I was really satisfied with the results of using the FAIR methodology for doing a generic risk assessment, there were some areas that I wanted to introduce more rigor. This was especially around metrics that had a strong dependence on my “expert” opinion.

At this point I will attempt to use the openPert excel add-in to better display how a particular generic Threat Community may be applied in a context such as an Enterprise environment (NB: I am using the tool for the first time as I write this post). First some assumptions.

  • While we may see the less capable “consumerware” threats, because of the Enterprise environment we can assume they will be mitigated (and therefore will have little impact on the risk assessment).
  • As we move towards the High category, the force applied by the Threat Community will become more successful in the initial stages, but are more likely not to cause a significant impact to the business. This may be because the community ignores Enterprise environments as it is not their target, or because additional stages are ineffective to complete the community’s motivations. So we can set the minimum capability for our betaPERT calculation at 67% which is the middle between 50% and 84% (boundary for High).
  • The upper boundary will still be 100%, because it is likely that only the most capable of the “Organised Crime” community would target larger Enterprise environments.
  • Given the context of the Enterprise for a Medium to Large enterprise, the most likely capability to be successful would be 88%. This is just an initial estimate so that we can run a simulation and produce a distribution histogram from the openPERT tool

To make use of openPERT, we run Excel and from the Add-In menu select the betaPERT Simulation option

OpenPERT Excel Add-on

The betaPERT Simulation will ask you for a minimum, maximum and most likely estimate for your values. By entering the values from the assumptions above we get the following output.

BetaPERT Simulation Table

betaPERT Distribution

Having a look at these results we can see from the produced distribution that approximately 35% of the “Organised Crime” Threat Community would be considered to be of Moderate (M) capability. At the other end of the scale only 1% of the Threat Community would be estimated to have Very High (VH) capability.

Using the openPERT Add-in, the produced distribution starts to “smell” right to me. If we take it one step further I think we start to see more rigor added to the metrics used to feed into FAIR. That is, we use the betaPERT simulations to produce distributions for Threat Communities based on a particular “victim” context and then adjust again to take into account what we normally only see the Threat Communities of a certain capability make use of for a Threat Vector. For example, would we expect the most capable attacks (e.g. use of zero day exploits)  from “Organised Crime” to be using Spam as a Threat Vector.

So after writing all this I am left with the following questions in my mind.

  1. Does making use of the openPERT tool like this really add value to improve the rigor of the metrics used in a FAIR assessment?
  2. How do I present/visualise the difference between the general Threat Community (which was my first normal distribution graphic) with an adjust betaPERT distribution so that lay readers of any assessment can easily recognise the differences due to the risk assessments context.
  3. What other metrics should also make use of betaPERT simulations to add rigor, and then how do you calculate the intersection of two distributions in FAIR to produce distributions for Vulnerability and Loss Event Frequency. I have found the following article that gives advice on how this could be done: Why You Cannot Add Two PERT Estimates
Advertisements
1 comment
  1. Jack said:

    Another great demonstration of the kind of thinking FAIR is intended to stimulate. Once again, you’re spot-on. Thanks for a great post.

    Jack

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: