1 / 48

Towards Total Security Quality Management (TSQM): Definition and Measurement

Towards Total Security Quality Management (TSQM): Definition and Measurement. WORK IN PROGRESS 8 September 2005 (Major changes to slides 43-47) MIT TEAM Yang Lee Stuart Madnick Michael Siegel Diane Strong Venkataramana Thummisi (Venkat) Richard Wang. Overview of Project. Key Findings.

mavis
Download Presentation

Towards Total Security Quality Management (TSQM): Definition and Measurement

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Towards Total Security Quality Management (TSQM): Definition and Measurement • WORK IN PROGRESS • 8 September 2005 • (Major changes to slides 43-47) • MIT TEAM • Yang Lee • Stuart Madnick • Michael Siegel • Diane Strong • Venkataramana Thummisi (Venkat) • Richard Wang

  2. Overview of Project Key Findings Comprehensive List of Aspects of Security Survey 1 and 2 Gap Analysis Key Dimensions And Aspects Academic Literature Gap Analysis Instrument Stakeholders And Roles Industry Literature Gap related hypotheses • - Extensive Gap Analysis • - Case Studies • - Best Practices • Benchmarking • Metrics • Security Methodology • Security Maturity Model Extended Enterprise Hypotheses Survey 3 Interim Results

  3. Brief Description of Surveys Survey 1 Open-ended: What does holistic Security mean to you? Survey 2 Semi-structured: What does holistic Security mean to you? Similar to Survey 1, but starts with 20 security aspects. Survey 3 13 semi-structured questions regarding Extended Enterprise security covering issues such as Security Return on Investment, Benefits of Security, and Extended Enterprise Security.

  4. Overview of Project Key Findings Comprehensive List of Aspects of Security Survey 1 and 2 Gap Analysis Key Dimensions And Aspects Academic Literature Gap Analysis Instrument Stakeholders And Roles Industry Literature Gap related hypotheses • - Extensive Gap Analysis • - Case Studies • - Best Practices • Benchmarking • Metrics • Security Methodology • Security Maturity Model Extended Enterprise Hypotheses Survey 3 Interim Results

  5. Comprehensive List of Aspects of Security

  6. Comprehensive List of Aspects of Security (continued)

  7. Overview of Project Key Findings Comprehensive List of Aspects of Security Survey 1 and 2 Gap Analysis Key Dimensions And Aspects Academic Literature Gap Analysis Instrument Stakeholders And Roles Industry Literature Gap related hypotheses • - Extensive Gap Analysis • - Case Studies • - Best Practices • Benchmarking • Metrics • Security Methodology • Security Maturity Model Extended Enterprise Hypotheses Survey 3 Interim Results

  8. Vulnerability Accessibility Confidentiality Technology Resources For Security Financial Resources For Security Business Strategy For Security Security Policy & Procedures Security Culture Dimensions of Security

  9. Good Security Good Security provides Accessibility to data and networks to appropriate users while simultaneously protecting Confidentiality of data and minimizing Vulnerabilities to attacks and threats. Good Security Practice goes beyond technical IT solutions. It is driven by a Business Strategy with associated Security Policies and Procedures implemented in a Culture of Security. These are practices are supported by IT Resourcesand Financial Resources dedicated to Security.

  10. Overview of Project Key Findings Comprehensive List of Aspects of Security Survey 1 and 2 Gap Analysis Key Dimensions And Aspects Academic Literature Gap Analysis Instrument Stakeholders And Roles Industry Literature Gap related hypotheses • - Extensive Gap Analysis • - Case Studies • - Best Practices • Benchmarking • Metrics • Security Methodology • Security Maturity Model Extended Enterprise Hypotheses Survey 3 Interim Results

  11. Stakeholders of Extended Enterprise Security General Public Extended Enterprise Enterprise Ring 1: Enterprise Ring 2: Extended Enterprise Ring 3: General Public

  12. Stakeholders & Roles

  13. Overview of Project Key Findings Comprehensive List of Aspects of Security Survey 1 and 2 Gap Analysis Key Dimensions And Aspects Academic Literature Gap Analysis Instrument Stakeholders And Roles Industry Literature Gap related hypotheses • - Extensive Gap Analysis • - Case Studies • - Best Practices • Benchmarking • Metrics • Security Methodology • Security Maturity Model Extended Enterprise Hypotheses Survey 3 Interim Results

  14. Top Level Hypotheses H1: Addressing security can have a positive Return on Investment (ROI) H2. Security Practice, if not careful, can lead to counter-productive impacts H3. Security is not just about technology but requires correct corporate policies and incentives H4. Public Policies and Regulations regarding Security are important H5. Security must be understood holistically in the context of the Extended Enterprise

  15. H1. Hypotheses on ROI H1: Addressing security can have a positive Return on Investment (ROI) H1.1: Security is not necessarily an added cost H1.2: Benefits of security can outweigh the cost H1.3: Resilient enterprises are able to seize opportunities from security problems H1.4: Security can be a competitive advantage H1.5: Security needs to be designed into organizational strategy and translated into organizational performance

  16. H2. Hypotheses on Productive & Counter-productive Impacts H2. Security Practice, if not careful, can lead to counter-productive impacts H2.1 Security must be designed into all products and processes. H2.2 Security should be a part of organizational process design. H2.3. Security is not just an IT function. H2.4. Security must be based on statistical and economic models for investment and risk. H2.5. Software and hardware products are often not secure. Security updates are not well managed. H2.6. Security can be perceived to be a burden to users.

  17. H2. Hypotheses on Corporate Policies & Incentives H2. Security is not just about technology but requires correct corporate policies and incentives H2.1. Managing security of distributed heterogeneous systems and technologies need to catch up with rapid use and advancement of this technology. H2.2. Public not well-trained on Security. H2.3. Lessons learned from security are often not shared. Confusion between security and secrecy.

  18. H2. Hypotheses on Public Policies & Regulations H2. Public Policies and Regulations regarding Security are important H2.1 Rules of law and regulations needs to catch up with global digital security. H2.2 Security crimes need clear and appropriate punishment.

  19. H5. Hypotheses on the Holistic Nature of Security in the Extended Enterprise H5. Security must be understood holistically in the context of the Extended Enterprise H5.1 Solving local security problems can lead to larger global security problems [“Stopping small fires may lead to big fires.”] H5.2 Security and Privacy are closely related with complex interdependencies H5.3 Security problems at disparate parts of the enterprise (or members of the extended enterprise) can ripple throughout the extended enterprise H5.4 Different stakeholders within the organization (and within partner organizations) can have different perceptions and requirements for security

  20. Overview of Project Key Findings Comprehensive List of Aspects of Security Survey 1 and 2 Gap Analysis Key Dimensions And Aspects Academic Literature Gap Analysis Instrument Stakeholders And Roles Industry Literature Gap related hypotheses • - Extensive Gap Analysis • - Case Studies • - Best Practices • Benchmarking • Metrics • Security Methodology • Security Maturity Model Extended Enterprise Hypotheses Survey 3 Interim Results

  21. Purpose of Gap Analysis • Purpose of Gap Analysis is to understand perceptions of Differences between factors such: • (A) Security Status Assessment and Security Importance • (B) views of diverse Security Stakeholders • within Enterprise and across Extended Enterprise

  22. Purpose of Gap Analysis (cont.) • Gaps represent Opportunities for Improvement within the Enterprise and across the Extended Enterprise • (A) When Status is below the Needs, these represent Areas for Improvement • (B) When Status among Stakeholders show differences, these represent areas for Investigating sources of the differences • Gaps may represent misunderstandings • Gaps may represent differences in local knowledge and needs

  23. Three Types of Gaps • Performance Gaps • Role Gaps • Inter-Enterprise Gaps • For today, we will focus on Performance Gaps, insufficient data for analyzing Role and Enterprise Gaps at this time • Issue: Gathering of enough data from same organization and partner data

  24. Performance Gap 1. Gaps between Security Assessment and Security Importance Example: High importance for Confidentiality vs. Low assessment status of Confidentiality

  25. Role Gap 2. Gaps among Enterprise Roles Example: Business Managers vs. IT managers Example: Business Executives vs. Technology Executives Example: Executives vs. Line Managers

  26. Enterprise Gap 3. Gaps between Enterprise andExtended Enterprisepartners Example: Internal (IT or Line managers) vs. Suppliers Example: Internal (IT or Line managers) vs. Customers

  27. Gap Analysis Questionnaire • Questionnaire respondents are comprised of the diverse roles (IT, IT security, Users, Business managers, Executives, etc.) within the enterprise and across (suppliers, customers, collaborators, etc.) the extended enterprise. • 2. Each respondent reports his/her view of actual assessment and importance of each aspect for both his/her organization and a partner organization.

  28. Gap Analysis Questionnaire (cont.) • Questions on the questionnaire cover the 8 constructs of security: • Accessibility • Vulnerability • Confidentiality • Financial resources for security • Technology resources for security • Business strategy for security • Security policy and procedures • Security culture • 4. To ensure construct validity, 5 questions are included for each construct.

  29. Extended Enterprise Security Survey Form # 01-20-____________ Towards Total Security Quality Management (TSQM) MIT’s Extended Enterprise Security Survey Introduction The following survey is part of a research project at MIT to develop a holistic framework to study enterprise security within and between organizations. Your responses to the following survey will provide us valuable insight about extended enterprise security. The extended enterprise includes an organization and its suppliers, customers, partners, and competitors. Extended enterprise security is concerned with security both within and between these organizations. The survey should take you about 20 minutes to fill out. Note about confidentiality: Your responses to questionnaire items will not be revealed to your organization or to any other organization. Only aggregate results will be used in our analyses. If you would like to receive a copy of our research results, please provide your email address at the bottom of the survey. General Instructions 1. What does it mean by “assessment” and “importance”? The survey asks you to give your impression of the “assessment” and “importance” of various security issues. “Assessment,” means your view of how well your organization is doing on these issues. “Importance” means your view of how important this issue is to you. 2. There is no right or wrong answer to any question. We are asking for your view. You may not know exact details about your company’s security. We are not asking for these details, but asking for your views. Please give your best estimate. 3. What is “Partner Organization”? The survey also asks you to give your impressions of “assessment” and “importance” for ONE partner organization. This partner organization should be one of your suppliers, if feasible. Alternatively, please select a customer or a collaborator organization. 4. There is no right or wrong answer about a partner’s security. We are asking your views of the partner organization’s security, you do not need to know exact details. Please give your best estimate. If you have no knowledge at all of an aspect of your partner security, you may leave that question blank. Thank you MIT TSQM team

  30. Your Organization & Partner Extended Enterprise Security Survey Section 1: Your Organization Your Organization/Company Organization Name__________________________________________________________ Industry____________________________________________________________________ Approximate total number of employees in your entire organization: ________________ Your Job Title and Work Role ________________________________________________ ___________________________________________________________________________ Department/Division/Group___________________________________________________ In my organization, I am a: _____(1) Executive (CEO,CFO, VP etc.) _____(2) Functional or Line Manager _____(3) Professional (Consultant, Engineer, In-house Expert, etc.) _____(4) Other Organizational Member In my organization, I work in the area of: _____(1) Business Security Policy and Management _____(2) IT Security _____(2) IT but not in Security, _____(3) General/Physical Security, _____(4) Not in Security or in IT. Section 2: Your Partner Organization Pick one partner organization for answering these questions. The survey administrator may give you additional instructions about picking a partner origination. All answers about your partner organization should be about ONE specific organization. Your Partner Organization/Company Partner Organization’s Name (optional)__________________________________________ Partner’s Industry_____________________________________________________________ Approximate total number of employees in your partner organization: ________________ Your Partner Organization is your organization’s: _____(1) Supplier ____(2) Customer ____(3) Collaborator ____(4) Competitor Major Group/Division/Department you usually work with: _______________________________________________________________________

  31. Security Questions (40)

  32. Lots of Survey Data (more to come)

  33. Gap Analysis Procedures • Assess Construct Validity • Compute Cronbach Alphas • Check inter-item correlations • Delete and revise questions as needed • 2. Form Constructs • Aggregate questionnaire items into constructs • Check inter-construct correlations • 3. Compute Gaps: • Performance Gap, Role Gap, Enterprise Gap • 4. Test Gaps for significance • 5. Interpret the results • Further analysis of interesting results

  34. Gap Analysis Preliminary Findings Performance Gaps Explore: at item level (yet not construct level) - Data just now being received - Only very limited analysis so far - All Findings that follow are preliminary

  35. Gap Analysis FindingsAccessibility Question 40 Gap = 1.40 Example data: 5.40 (Assessment) vs. 6.80 (Importance) Availability of data and network when needed

  36. Gap Analysis FindingsVulnerability Question 1 Gap = 1.20 Example data: 4.60 (Assessment) vs. 5.80 (Importance) Tampering with data and networks is rare.

  37. Gap Analysis FindingsConfidentiality Question 24 Gap = 1.20 Example data: 5.40 (Assessment) vs. 6.60 (Importance) Protects privacy of personal data.

  38. Gap Analysis FindingsFinancial Resource for Security Question 2 Gap = 1.17 Example data: 5.50 (Assessment) vs. 6.67 (Importance) Security is adequately funded.

  39. Gap Analysis FindingsIT Resource for Security Question 5 Gap = 2.00 Question 17 Gap = 1.33 Example data: 4.33 (Assessment) vs. 6.33 (Importance) 5.00 (Assessment) vs. 6.33 (Importance) Business managers are involved with IT security policies. Adequate technology for supporting security.

  40. Gap Analysis FindingsBusiness Strategy for Security Question 4 Gap = 1.50 Question 19 Gap = 2.00 Example data: 5.00 (Assessment) vs. 6.50 (Importance) 4.00 (Assessment) vs. 6.00 (Importance) Security strategy sets directions for security practices. Well-defined and communicated security strategy.

  41. Gap Analysis FindingsPolicy and Procedures for Security Question 25 Gap = 0.60 Question 30 Gap = 1.00 Example data: 5.20 (Assessment) vs. 5.80 (Importance) 5.20 (Assessment) vs. 6.20 (Importance) Adequate procedures for physical security. Procedures for detecting and punishing security violations.

  42. Gap Analysis FindingsSecurity Culture Question 11 Gap = 1.83 (3.67 vs. 5.50) Question 18 Gap = 1.83 (3.83 vs. 5.67) Question 26 Gap = 1.60 (4.40 vs. 6.00) Question 39 Gap = 2.40 (4.20 vs. 6.60) People are knowledgeable about IT security tools and practices. People carefully follow good security practices. People can be trusted not to tamper with data and networks. People are aware of good security practices.

  43. Recent Activities (since last meeting) • Developed web-based survey instrument • Developed secure (https) web-based survey instrument • Collected more data • About triple • Considerable “partner” company data • Both “miscellaneous” and two companies • Valuable for intra-company stakeholder gap analyses • Preliminary analysis of increased pilot data • Some sample analysis follows …

  44. Gap Analysis Findings - UpdatedSecurity Culture Question 39: People are aware of good security practices. Gap between Assessment and Importance – for your company Complete = 1.83 (4.71 vs. 6.54) Miscellaneous* = 2.40 (4.20 vs. 6.60) Company C = 1.83 (5.00 vs. 6.83) Company W = 1.89 (4.61 vs. 6.50) * Original sample: • diverse array of companies • many middle-managers

  45. Gap Analysis Findings - UpdatedSecurity Culture Question 39: People are aware of good security practices. Gap between Assessment and Importance – for your company Complete = 1.83 (4.71 vs. 6.54) Company C = 1.83 (5.00 vs. 6.83) Gap between Assessment and Importance – for partner company Complete = 0.99 (4.82 vs. 5.81) Company C = 1.40 (4.80 vs. 6.20)

  46. Next steps: Phase 2 (near-term) • Collect much more data – especially for intra-company stakeholder analysis • Complete Round 1 Survey (within Sponsor) • Conduct Round 2 Survey (4-6 Sponsor partners) • Perform construct analysis • Analysis of pilot data • Refine stakeholder and dimensions • Refine questionnaire items • Revise gap analysis instrument

  47. Next steps: Phase 3 (longer-term) • Large-scale Gap Analysis Study • Extensive Gap Analysis Results • Pursue other hypotheses, through • Other Survey Instruments • Case Studies • Best Practices • Benchmarking • Security Methodology • Security Maturity Model

More Related