Security Metrics Special Interest Group

1. Security MetricsSpecial Interest Group Key Points Presentation

2. WARNING This presentation is confidential and purely for the attention of and use by organisations that are Members of the Information Security Forum (ISF). If you are not a Member of the ISF or have received this document in error, please destroy it or contact the ISF on isfinfo@securityforum.org or on +44 (0)20 7213 1745. Any storage or use of this document by organisations which are not Members of the ISF is not permitted and strictly prohibited. This document has been produced with care and to the best of our ability. However, both the Information Security Forum and Information Security Forum Ltd accept no responsibility for any problems or incidents arising from its use.

3. Key findings (1 of 2)

4. Key findings (2 of 2)

5. About this presentation • The presentation summarises the research and conclusions from the ISF Special Interest Group (SIG) on Security Metrics. • The presentation can be used by Members to: • understand the topic, without reading the associated report • gain an overview of the key issues and findings of the project • provide material for their own presentations on this topic.

6. The SIG project approach • The approach taken included: • Holding nine Member Work Group meetings • Over 120 attendees • Average attendee evaluation 4.3 out of 5 • Analysing 56 Member-completed questionnaires • Interviewing 12 Members • Covered most sectors and geographical locations • Researching published material on security metrics

7. Project Deliverables Report SIG meeting minutes Key point presentation These deliverables are also available on MX2

8. Outline of presentation • Defining security metrics • Member usage of security metrics • Main issues • Key actions

9. A. Defining security metrics

10. What are security metrics? • Objective, quantifiable measures against specific targets that enable an organisation to judge the effectiveness of information security in that organisation. “ “

11. Security metrics should be: Quantifiable Consistently measured Repeatable The information provided should: Allow effective analysis Enable reporting Enhance understanding Assist in managing information security Demonstrate the value of information security to the business What are security metrics? “ Metrics should be: timely; reliable; trustable; accurate; simple (at a certain level); provable; meaningful and easily understandable; repeatable; verifiable; and scaleable. “

12. Characteristics of security metrics

13. Examples of security metrics by category

14. B: Member usage of security metrics

15. A model for understanding security metrics

16. Managing information security Providing information for management reporting Indicating compliance to legislation, regulation and standards Showing efficiency, effectiveness and performance against objectives Demonstrating the value of information security Supporting risk-based approach to information security Supplying information for risk management Providing information about information security risks Highlighting information security strengths and weaknesses Benchmarking information security arrangements Common reasons for using security metrics “ We need to continuously improve and justify what we do to management. “

17. Incidents Number of incidents Number of business-critical incidents Cost of individual incidents Virus protection frequency of virus incidents in a specific period frequency of virus incidents compared to previous periods number of viruses blocked at gateway/perimeter defences Risk management number of information risk analyses performed number of high/critical information security risks identified number of high/critical information security risks mitigated Patch management number of vulnerabilities recorded/patches issued (per period) time to patch (eg estate or critical systems/applications) percentage of systems patched, against Service Level Agreement/policy What security metrics are currently used? “ We only use the data we can get our hands on easily. That may not be the right thing to do. “

18. Compliance number of staff attending awareness training number of inappropriate internet sites accessed Virus protection Audit findings number of internal audit findings number of external audit findings (eg failure to comply with regulation) percentage of major information security-related findings left unresolved over a stated period of time Cost total financial losses (eg lost sales, orders or production) caused by information security incidents total financial value of regulatory or other fines imposed after information security incidents total financial losses due to fraud (including legal and recovery costs) total cost of security (cost of controls + cost of incidents) What security metrics are currently used?

19. Audiences for security metrics • Most common audiences: • CISO • IT function • Senior Management “ Metrics are a way of communicating with the board to gain backing for your projects. “

20. Examples of presentation methods

21. C: Main issues

22. Main issues with security metrics

23. Addressing the issues • Members agreed that the concepts of measuring security and security metrics have considerable merit. • The management saying “you can’t manage what you can’t measure” still holds true and many attendees agreed with this statement. • The issues identified here are not about security metrics in themselves but about using the right security metrics for an organisation • Using the right security metrics delivers benefit and improves communication with non-information security professionals (eg business people, accountants, executives and managers).

24. D: Key actions

25. Key actions • A. Define requirements • B. Identify relevant security metrics • C. Collect data required • D. Produce security metrics • E. Prepare presentations • F. Use dashboards and scorecards • G. Review the use of security metrics

26. A. Define requirements Define and understand audience requirements Seek input from managers and staff Obtain funding B. Identify relevant security metrics Decide which security metrics to use Review against objectives Review the chosen security metrics for ‘balance’ Key Actions “ “ Metrics round off the picture – but don’t forget the intangibles! You have to understand the requirements and have objectives before you start to collect metrics. You don’t want to spend man-hours collecting useless information. “ “

27. A. Collect data required Define data required for use in security metrics Collect data for use in security metrics Collect context data Normalise and store the data D. Produce security metric Perform analysis and/or aggregation of data Analyse metrics Test for correlation in dataset Key Actions “ “ Metrics must have a context – otherwise they may not be understandable. Business isn’t always interested in numbers; trends matter too. “ “

28. E.Prepare presentations Match the presentation to the audience Select presentation formats F. Use dashboards and/or scorecards Dashboards Balanced scorecards Key Actions “ “ Fewer reports are required if you have a security dashboard – you can field many enquiries with a general response. The idea of using a balanced scorecard elegantly links information security and business. “ “

29. G. Review the use of security metrics Review security metrics used Review presentation format Key Actions

30. Mapping the key actions with the model

31. Possible future development Balanced scorecard based on the Meta Standard Dashboard based on ISF products (Survey, Healthcheck, Meta Standard)

32. Project contacts Adrian Davis Project Programme Manager: Tel: +44 (0)207 213 3372 Email:adrian.davis@securityforum.org Christopher Petch Project Associate Tel: +44 (0)207 212 3012 Email: christopher.m.petch@securityforum.org