Jarquith’s Book, also published in 2007, is still a valid resource once you are concerned with IT-Security Metrics. Within the first chapter the author argues why metrics are important and what makes a good metric. Thereby he extends far beyond the famous quote: “you can only manage what you can measure”. One can approximately summarize his arguments as follows: A good metric can be collected simply. It tells a story about what is done and what should be done. Also it urges the responsibility in management to act upon it. Also Andrew Jarquith strongly disagrees with any ROI calculation claiming these are based on too many estimates to have any valid point he still does not provide an alternative. Funny enough he tells a view stories about a Start Up he founded (@Stake) and how they developed various metrics which are based on the same amount of estimates as ROI calculations. In the second part of the book the author describes various metrics. He clearly explains what data to collect and how to interpret them. His discussion also includes a description of the intent and the limits of each metrics. Let’s take the ratio of incoming malware to outgoing malware. This example gives you an estimate of how “clean” your environment really is. But this estimate is based on software that needs to be regularly updated with new antivirus signatures. In addition antivirus signatures are mostly developed once malware was recognized as such and a sample of it had been analyzed. This implies that also you think to see how clean your environment is you are only able to see a part of that - like looking through dirty glasses. Further examples are the average time until a patch is rolled out or the mean time to recover from an incident. Jarquith discusses metrics for all management disciplines. This implies that a certain amount of metrics presented can questioned as being a real it-security metric. So not ever proposal he makes can be within your responsibility as security professional. Within the book Jarquith heavily mentions his mailing list “security-metrics.org” which I can only recommend. The discussions within this list go far beyond metrics and present interesting and sometimes challenging views of the it-security business as a whole. The third part of the book discusses how to present the collected metrics. I was very pleased to find such big part of the book spend on this topic because other books rarely talk about the visual aspects of metrics. The author starts with a view statistical principles and definitions like what is a median, what is the difference between a median and an average with or without standard deviation. Also he provides advice in which situation you should use which mathematical technique to gain an appropriate and meaningful value. Following this rather dry and theoretical discussion Andrew Jarquith starts to elaborate on how to create a diagram that catches management attention. Quickly one can see that he spend some time a Forrester as well that he is a big fan of stock market reports which he cites both as examples for clean, lean diagrams with a message. In the end this is a very crucial point since you will be selling your efforts through a presentation. However, I missed a view references on literature that discusses how not to present data - like Daren Hall’s “How to lie with statistics”. Thereby I mean that simple things like setting the scales of the diagram in a certain way will strongly influence the first impression and the message that is associated with the presented data. Marcus Ranum has a small series of blog posts onto this topic - just as a starting point. Summarizing my opinion on Andrew Jarquith book: “Security Metrics - Beyond Fear, Uncertainty And Doubt” I do strongly recommend reading this book if you have never worked with or collected metrics. Also if you are interested in evaluating your current efforts as a security professional and the security of your company this book is a good read for you and provides with a lot impulses and starting points for further improvements.

If you want to continue on this topic - read the book by Andrew Jarquith “Security Metrics”.