Information Security Metrics

Imagine for a moment a scientific experiment where no one measured anything. How reliable would the results be? Firstly, no one would be able to test a hypothesis with any real certainty. Even if observations were made, the experimenters would have to recall these from memory – with all of its inherent shortcomings. The experimenters would likely even disagree among one another over what they had observed. Then of course they would have to present their findings. They could describe the experiment, but all of the key questions like how much, how fast, etc. would all have to be left unaddressed or described in a qualitative manner: “The material combusted quite quickly.” or “The bacteria multiplied quite fast.” This would be enough for any respectable scientist to howl in laughter or pull their hair out in frustration. There would be no confidence in any of the findings and the study itself would be impossible to replicate. An unscientific scientific experiment indeed.

In a similar vein, no information security programme can be effective if it fails to gather relevant data to create metrics and track progress. While mistakes and biases can still skew results, the beauty of the scientific method is that it places our faith in facts and evidence. Like an astute detective scanning a crime scene with a keen eye, we must derive meaning from chaos. We can only do so by gathering information pertinent to the investigation.

What would be ill advised is to gather information without first knowing what questions we are trying to answer. Key questions to ask oneself when developing information security metrics include:

• What are the goals of my information security programme?
• How will metrics demonstrate the progress of my information security programme?
• What data do I have access to and what data will I need?
• What tools will I use to gather that data?
• How much time and money will it take to implement these metrics?
• Which metrics will be key indicators?
• How can I present these metrics in a way that can be understood by senior management and translates to the broader goals of the organisation?

We also need to be aware of the potential limitations and pitfalls of metrics. If it becomes too great of an obsession, organisations and individuals can often lose sight of the main strategic purpose of the metrics. The numbers become an obsession, to such an extent that it becomes a game of sorts; people and organisations desperately look for any way to show an improving metric by any means necessary, even if it happens to run counter to the spirit of that metric.

The old adage of “you can’t manage what you don’t measure” remains true in almost every field and organisational department. Ultimately the question we are aiming to answer with information security metrics is: Am I (or are we) spending time and money on what matters most? In that sense, an effectively implemented and sustained metrics initiative will prove to be invaluable in harnessing the full power of an information security programme.