Security Measures & Metrics

1. Security Measures & Metrics Pete Lindstrom, CISSP Research Director Spire Security, LLC www.spiresecurity.com petelind@spiresecurity.com

2. Security Metrics I Security Metrics (Part 1): Building the Framework There are obvious benefits to charting and quantifying the success of your security program. But where do you begin? This session -- part 1 of a 2-part mini-workshop -- outlines a practical approach to security metrics that links standard business practices with security functions. Find out from Information Security magazine contributing editor, Pete Lindstrom, Research Director for Spire Security, how to build a rock-solid foundation based on a model known as the "Four Disciplines of Security Management." Then learn about the elements of a cohesive security metrics program from a functional and resource-usage perspective. Plus, you leave with a solid understanding of the relative utility metrics for productivity, process efficiency, cost effectiveness and risk management.

3. What is the Four Disciplines Model? • A way to think about security • High-level without losing clarity • Detailed enough for technical folks • Identifies relationships • A taxonomy of objectives, functions, activities, and products. • A framework for security measurement.

4. Introducing the Four Disciplines 2 Trust Mgt: Designing security policy and process Identity Mgt: Managing Users and other sources 3 1 Vuln. Mgt: Hardening the systems Threat Mgt: Monitoring activities and events 4

5. 1. Harden Systems

6. Vulnerability Mgt Functions • Evaluate and harden configurations • By platform • Identify and remediate vulnerabilities • Software bugs • Configure firewalls / other access control • Reduce/filter anomalous traffic

7. 2. Identify/Manage Users

8. Identity Management Functions • Validate user information • Create/modify user accounts and privileges • Disable/delete user accounts • Change/reset passwords • Validate sessions • Authorize access

9. 3. Design/Strengthen Processes

10. Trust Management Functions • Create/modify user policies • Create/modify system policies - technical baselines • Design security architecture • Design/implement controls to prevent sniffing or copying data. • Design/implement controls to prevent modifying data.

11. 4. Monitor Environment

12. Threat Management Functions • Identify anomalous activities • Monitor network and components • Aggregate alerts and logs • Collect physical information • Manage/resolve incidents • Incident response - take corrective action • Conduct forensic analysis of systems/data

13. Putting It All Together

14. Q1: Most Important? Which Discipline is most important to a strong security program? • Vulnerability Management (firewalls, vuln assess, patch) • Identity Management (provision, acct mgt, authent.) • Trust Management (policies, tech guides, crypto) • Threat Management (monitor, incident, forensics)

15. Q2: Most Time? Which Discipline does your organization spend the most time on? • Vulnerability Management (firewalls, vuln assess, patch) • Identity Management (provision, acct mgt, authent.) • Trust Management (policies, tech guides, crypto) • Threat Management (monitor, incident, forensics)

16. Fundamental Security Elements Activities: Four Disciplines People: Departments Admins Time: Hr/Day Month/Yr Costs: Salaries, Consulting HW, SW, Maint. Resources: User accts, systems, apps

17. Types of Metrics • Process Effectiveness – doing things right. (measure quality) • Staff Productivity – people doing more things. (measure volume) • Cycle Time – transaction time. (measure process efficiency) • Staff Efficiency – people doing things faster. (people / transaction / time) • Cost Effectiveness – transaction costs. (cost / activity)

18. Process Effectiveness Metrics “doing things right” • Key Elements: • Activities • errors • Examples: • Acct request errors • Remediation errors • False alarm rate • Policy exceptions error rates

19. Process Effectiveness • Measure quality by identifying error rates of activities • Identity Management • User account request errors • Vulnerability Management • Vulnerabilities not remediated • Threat Management • Improper incident management • Trust Management • Policy violations

20. Staff Productivity Metrics “people doing more things” • Elements: • People • Activities • Examples: • Accts per person • Vulns per person • Patches per person

21. Staff Productivity • Productivity and workload for all manual activities (activities/people) • Identity Management • Requests per administrator • Account disablements per admin • Password resets per admin • Vulnerability Management • Vulnerabilities resolved per administrator • Threat Management • Incidents per person • Trust Management • Policy changes per person

22. Cycle Time Metrics avg “time to perform activity x” • Elements: • Time • Activities • Examples: • Accts per month • Vulns fixed per month • Patches per month

23. Cycle Time • Process efficiency • Identity Management • User account request time to complete • Vulnerability Management • Remediation time to complete • Threat Management • Incident response time to complete • Trust Management • Policy creation time to complete

24. Staff Efficiency Metrics “people doing things” quicker • Elements: • People • Activities • Time AdminsbyDepartment • Examples: • Accts per person/hr • Vulns per person/hr • Patches per person/hr 2000 Hours per FTE

25. Staff Efficiency • Combines staff productivity and cycle time metrics. • Identity Management • User account requests completed per person per day/week/month • Vulnerability Management • Vulnerabilities remediated per person per day/week/month • Threat Management • Incidents closed per person per day/week/month • Trust Management • Policies reviewed per person per day/week/month

26. Cost Effectiveness Metrics Cheaper transactions • Elements: • Activities • Costs • Examples: • Cost per acct • Cost per vuln fixed • Cost per patch

27. Cost Effectiveness • Dollars/activities; dollars/resources; dollars/demographics • Identity Management • Cost per request • Cost per password reset • Vulnerability Management • Cost per vulnerability • Cost per system setting • Threat Management • Cost per incident • Trust Management • Cost per policy • Cost per project

28. When to Use Metrics • Process Effectiveness • Six Sigma • Staff Productivity • ROI / promotions • Cycle Time • Balanced Scorecard • Staff Efficiency • ROI • Cost Effectiveness • Activity-based costing • ROI/TCO

29. Q3: Most Useful? Which metric type is most useful to your security program? • Process Effectiveness • Staff Productivity • Cycle Time • Staff Efficiency • Cost Effectiveness

30. Conclusions • Security functions are spread throughout organizations. • You can’t improve security until you measure it. • Ultimately, security is a business operation that should be run like a business operation.

31. Agree? Disagree? Pete Lindstrom petelind@spiresecurity.com www.spiresecurity.com