Key Performance Indicators (KPI)
Justin SomainiOne of the conversations that I've had with many people is around metrics and KPI's. I'm a strong believer that we have not gotten to actual KPI's in industry. To take it a step further, most of us feel that our metrics are pretty bad (See S3 results). Last week I sat down to put the final touches on our new risk methodology document and came up with some ideas that I'm not batting around. Nothing earth shattering but I can finally see the last pieces of the puzzle fall into place.
If you remember, I posted a while ago about the equilibrium theory as well as flow management. Part of the equilibrium is to gather data from leaders in the company to get to an approximation on how much loss they are willing to accept. Not to be confused with ALE. To do this, I decided to gather 20'ish questions that focus on the risk categories (Value, Brand and Operations) to put risk into context. What came out was very interesting.
Not only was I able to gather an understanding on what is acceptable to govern the level of controls we implement, but I also got a metric. I now have the ability to establish the acceptable level of security performance for the company from the business. i.e. $2M is allowed in remediation costs, 2 public breaches, etc. If done right I should have 3-5 metrics for all of Security.
More to come on this later.
Reader Comments (4)
Interesting Justin. I've done something similar here using the OCTAVE Allegro impact criteria. Would love to see the questionnaire you developed.
Niels
Thanks. I should have something good in a month or so after I work it through a bit in practice
I use to think Metrics were useful until I realized they did very little but cause a false sense of security for the C-levels when they are dealing with the board. Half the time the metrics are wrong, skewed, or a straight up lie. Im not sure that security as a whole can be summed up with Metrics, at least not like the rest of IT? A companies overall risk rating/posture changes daily due to the consistent changes in the industry, One day you think your safe, the next day you find out that youve been compromised for the last few months. It always comes back to the want of that one size fits all magic silver bullet that doesnt exist to give the appearance of due-diligence. I find that unless the sky is falling, no one wants to hear how bad it is, ie "Cross that bridge when we get to it" mindset that seems to do wonders for many ;) I wish more took the time to understand their business model and how to wrap IT around it through true EA. Too few understand their business and almost no-one understands how IT delivers their business driven services to their clients. Without knowing your business and how IT is architected and configured, how in the hell can any security professional provide the proper level of service to the company? I wish more cared, but too few do unless it’s a metric ;(
I've always had mixed thoughts on metrics when it comes to security.
On one hand, it's great to be able to show how much "good" security professionals do in terms of saving companies money from data breaches, etc.
On the other hand, it seems like sometimes we have to justify our existence. This creates an "them vs. us" situation, where everyone wants the "annoying security people to go away", because they believe they'll never get a virus infection or be attacked by a malicious user.
Many times I will see security as an afterthought... "what software can we throw on here at the last minute to make this system secure?" Nevermind the fact that your original software application is full of security holes such as buffer overflow or SQL injection attacks. "But Antivirus will fix that, right???"
Sigh.
Ken
CaffeineSecurity
http://caffeinesecurity.blogspot.com