CMMI vs. ISO

1. CMMI vs. ISO David S. Craft CIRM, PMP

2. Agenda • Process • ISO • CMMI

3. The Process Management Premise . The quality of a system is influenced by the quality of the process used to acquire, develop, and maintain it, the analysis and forethought that goes into an architecture that supports business goals and requirements, and the training provided to teams involved in the project. Using proven methods for process and product quality, software success is predictable and achievable, and failure is avoidable. Once coding starts, teams trained in mature software engineering processes can remove defects early, when defect removal is 10 to 100 times less costly than it is during test. This dramatically reduces test costs and only marginally increases costs upstream

4. Process • To Develop Software and Systems You Need A Process • So what is a process: • A systematic series of actions directed to some end • A continuous action, operation or series of changes taking place in a definite manner • A series of actions, changes or functions bringing about a result • A series of operations performed in the making or treatment of a product • Process or processing typically describes the action of taking something through an established and usually routine set of procedures or steps to convert it from one form to another (such as processing paperwork to grant a loan, processing milk into cheese, converting computer data from one form to another, etc.

5. Common Misconceptions • I don’t need defined processes I have: • Really good people • Advanced Technology • An experienced manager • Defined Processes: • Interfere with creativity • Equals bureaucracy + regimentation • Isn’t needed when building prototypes • Is only useful on large projects • Hinders agility in fast moving projects • Costs too much

7. Why We Need Structured Processes • Estimating (History) • Scope • Cost • Time • Tools • Deliver the Product to Estimate (Visibility) • Time • Cost • Quality • Handling/Controlling Changes • Planned • Unplanned • Scope Creep

8. Why We Need Standard Processes • Organizations and governments worldwide will spend about $1 trillion this year on IT projects. Recent data suggested only about 35 percent of those projects are likely to be completed on time and on budget, with all their originally specified features and functions. Many projects, perhaps 20 percent, will be abandoned, often after multimillion-dollar investments—and the biggest projects will fail most often. • One well-documented $170 million software failure was blamed on a lack of defined requirements in the original contract; a lack of software engineering, program, and contract management skills; and underestimates of the complexity of interfacing the new system with legacy systems, addressing security needs, and establishing an enterprise architecture. • Other software-development failures have brought down entire companies, such as the $5 billion drug-distribution firm in Texas that declared bankruptcy as a result of a poorly implemented resource planning system.

9. How to Achieve Quality Processes • ISO • CMMI

10. Meet The International Organization forStandardization (ISO) • A global federation of national standards bodies from some 164 countries • Representing approximately 95% of worldwide production. • The world's largest developer and publisher of International Standards. • A non-governmental organization established in 1947 • Promotes the development of standardization and related activities with a view to facilitating international exchange of goods and services and development of cooperation in the spheres of intellectual, scientific, technological and economic activity • Many of its member institutes are part of the governmental structure of their countries, or are mandated by their government. On the other hand, other members have their roots uniquely in the private sector, having been set up by national partnerships of industry associations. Therefore, ISO enables a consensus to be reached on solutions that meet both the requirements of business and the broader needs of society.

11. Standards are documented agreements containing technical specifications or other precise criteria to be used consistently as rules, guidelines, or definitions of characteristics, to ensure that materials, products, processes and services are fit for their purpose. For example, the format of the credit cards, phone cards, and "smart" cards that have become commonplace is derived from an ISO International Standard. Adhering to the standard, which defines such features as an optimal thickness (0,76 mm), means that the cards can be used worldwide. International Standards thus contribute to making life simpler, and to increasing the reliability and effectiveness of the goods and services we use. What are standards?

12. What ISO Standards Do • Make the development, manufacturing and supply of products and services more efficient, safer and cleaner • Facilitate trade between countries and make it fairer • Provide governments with a technical base for health, safety and environmental legislation, and conformity assessment • Share technological advances and good management practice • Disseminate innovation • Safeguard consumers, and users in general, of products and services • Make life simpler by providing solutions to common problems

13. Where are theStandards 19,500+ Standards 832,000+ Pages

14. ISO 9000 - Quality management • The ISO 9000 family addresses various aspects of quality management and contains some of ISO’s best known standards. The standards provide guidance and tools for companies and organizations who want to ensure that their products and services consistently meet customer’s requirements, and that quality is consistently improved. • There are many standards in the ISO 9000 family, including: • ISO 9001:2008 - sets out the requirements of a quality management system • ISO 9000:2005 - covers the basic concepts and language • ISO 9004:2009 - focuses on how to make a quality management system more efficient and effective • ISO 19011:2011 - sets out guidance on internal and external audits of quality management systems • The ISO 9000:2008 standard has been implemented by over 1,000,000 organizations in 176 countries

15. ISO 9000:2008 Key Principles • Customer Focus • Leadership • Involvement of People • Process Approach • System Approach to Management • Continual Improvement • Factual Approach to Decision Making • Mutually Beneficial Supplier Relationships

16. Level 1 Defines Approach and Responsibility Quality Manual Level 2 Defines Who, What, When Procedures Work/Job Instructions Level 3 Answers How Level 4 Results: shows that the system is operating Records/Documentation Quality System Documentation

17. ISO 9001:2000 Structure • Quality Management System • 4.1 General requirements • 4.2 Document requirements • Management Responsibility • 5.1 Management commitment • 5.2 Customer focus • 5.3 Quality policy • 5.4 Planning • 5.5 Responsibility, authority, communication • 5.6 Management review • Product realization • 7.1 Planning of product realization • 7.2 Customer-related processes • 7.3 Design and development • 7.4 Purchasing • 7.5 Production and service provision • 7.6 Control of monitoring and measuring devices • Measurement, Analysis & Improvement • 8.1 General • 8.2 Monitoring and measurement • 8.3 Control of nonconforming product • 8.4 Analysis of data • 8.5 Improvement • Resource Management • 6.1 Provision of resources • 6.2 Human resources • 6.3 Infrastructure • 6.4 Work environment

18. Standard Examples • 5.2 Customer Focus “Top management shall ensure that customer requirements are determined and are met with the aim of enhancing customer satisfaction.” 6.1.b Resource Management “The organization shall determine and provide the resources needed (…) to enhance customer satisfaction by meeting customer requirements. 8.4 Analysis of Data The organization shall determine, collect and analyze appropriate data to demonstrate the suitability and effectiveness of the quality management system and to evaluate where continual improvement of the effectiveness of the quality management system can be made. This shall include data generated as a result of monitoring and measurement from other related sources. The analysis of data shall provide information relating to (…) customer satisfaction (..).

19. Steps to Implement ISO (and CMMI) • Plan for ISO and gain commitment of people, particularly upper management. • Assign the responsibility of the implementation process to someone (internal or external). • Train all personal in ISO requirements • Perform assessment of current processes and find the gaps • Fill the gap by revising, adding or improving the current processes and documentation to meet ISO requirements. • Perform internal audit • External audit

20. ISO’s Impact In The Global Economy • ISO 9001:2000 is now firmly established as the globally accepted standard for providing assurance about the quality of goods and services in supplier-customer relations. • The positive roles played in globalization by ISO’s standards for quality and environmental management systems include the following: • a unifying base for global businesses and supply chains – such as the automotive and oil and gas sectors • a technical support for regulation – as, for example, in the medical devices sector • a tool for major new economic players to increase their participation in global supply chains, in export trade and in business process outsourcing; • a tool for regional integration –  as shown by their adoption by new or potential members of the European Union • In the rise of services in the global economy – nearly 33 % of ISO 9001:2000 certificates in 2005 went to organizations in the service sectors.

21. CMMI

22. CMMI History • Active development of the model by the US Department of Defense Software Engineering Institute (SEI) began in 1986 when Watts Humphrey joined the Software Engineering Institute located at Carnegie Mellon University after retiring from IBM. • At the request of the U.S. Air Force he began formalizing his Process Maturity Framework to aid the U.S. Department of Defense in evaluating the capability of software contractors as part of awarding contracts. • The result was a model for the military to use as an objective evaluation of software subcontractors' process capability maturitybased on Philip Crosby’s book, “Quality Is Free”. • Humphrey's approach differed because of his unique insight that organizations mature their processes in stages based on solving process problems in a specific order. He based his approach on the staged evolution of a system of software development practices within an organization, rather than measuring the maturity of each separate development process independently. • The CMMI has thus been used by different organizations as a general and powerful tool for understanding and then improving general business process performance.

23. SEI • The Software Engineering Institute (SEI) is a federally funded research and development center sponsored by the U.S. Department of Defense and operated by Carnegie Mellon University in Pittsburgh, Pa. The SEI helps advance software engineering principles and practices and serves as a national resource in software engineering, computer security, and process improvement. The SEI works closely with defense and government organizations, industry, and academia to continually improve software-intensive systems. Its core purpose is to help organizations improve their software engineering capabilities and develop or acquire the right software, defect free, within budget and on time, every time. The SEI transitions its technologies to the global software engineering community through its public courses, conferences, technical reports, and Partner Network.

24. Meet CMMI • CMMI® (Capability Maturity Model® Integration) models are collections of best practices that help organizations to improve their processes. These models provides a comprehensive integrated set of guidelines for developing products and services. The SEI’s body of work in technical and management practices is focused on developing software right the first time, which results not only in higher quality, but also predicable and improved schedule and cost • The CMMI-DEV model provides guidance for applying CMMI best practices in a development organization. Best practices in the model focus on activities for developing quality products and services to meet the needs of customers and end users. • CMMI helps you to meet your organizations business objectives and improve performance.

25. What is a Maturity Model • A structured collection of elements describing characteristics of effective processes • A maturity model provides: • A place to start • The benefits of companies prior experience • A common language and shared vision • A framework for prioritizing actions • A way to define what improvement means for your organization • The model can be used to assess where your organization is against other organizations

26. CMMI Organization • CMMI is organized as a process framework clustering related practices into process areas that, when performed collectively, satisfy a set of goals. It requires that you define specific practices to meet specific goals but does not define how they are to be implemented. • The CMMI provides two representations – staged and continuous, each containing 22 Process Areas (PA). The staged view provides five maturity levels: Initial, Managed, Defined, Quantitatively Managed, and Optimizing. The PAs at each maturity level build on the previous level. Alternatively, continuous representation is used to focus on a process capability in a desired functional area (project management, process management, engineering and support) rather that maturity levels

27. Process Areas

28. CMMI Standard Example • SP 1.2 Specify Measures • Specify measures to address measurement objectives. Measurement objectives are refined into precise, quantifiable measures. Measurement of project and organizational work can typically be traced to one or more measurement information categories. These categories include the following: • schedule and progress • effort and cost • size and stability • quality. • Measures can be either base or derived. Data for base measures are obtained by direct measurement. Data for derived measures come from other data, typically by combining two or more base measures. Examples of commonly used base measures include the following: • Estimates and actual measures of work product size (e.g., number of pages) • Estimates and actual measures of effort and cost (e.g., number of person hours) • Quality measures (e.g., number of defects by severity) • Information security measures (e.g., number of system vulnerabilities identified) • Customer satisfaction survey scores

29. CMMI Standard Example con’t • Examples of commonly used derived measures include the following: • Earned value • Schedule performance index • Defect density • Peer review coverage • Test or verification coverage • Reliability measures (e.g., mean time to failure) • Quality measures (e.g., number of defects by severity/total number of defects) • Information security measures (e.g., percentage of system vulnerabilities mitigated) • Customer satisfaction trends • Derived measures typically are expressed as ratios, composite indices, or other aggregate summary measures. They are often more quantitatively CMMI for Development, Version 1.3 Measurement and Analysis (MA) 180 reliable and meaningfully interpretable than the base measures used to generate them. • There are direct relationships among information needs, measurement objectives, measurement categories, base measures, and derived measures. This direct relationship is depicted using some common examples in Table MA.1.

30. Capability and Maturity Models

33. Capability and Maturity Levels

35. Evaluation • This is not a certification model, but ratings may be announced and published. • The SEI publishes ratings provided the company gives it permission. • Formal appraisals are typically 5 – 10 days and led by SEI-authorized internal or external lead appraisers, using trained teams and a formal methods. The method is named SCAMPI (Standard CMMI Appraisal Method for Process Improvement).

36. Examples of CMMI Impact: ROI • 5:1 ROI for quality activities (Accenture) • 13:1 ROI calculated as defects avoided per hour spent in training and defect prevention (Northrop Grumman Defense Enterprise Systems) • Avoided $3.72 M in costs due to better cost performance (Raytheon North Texas Software Engineering) as the organization improved from SW-CMMI level 4 to CMMII level 5 • 2:1 ROI over 3 years (Siemens Information Systems Ltd, India) • 2.5:1 ROI over 12st year, with benefits amortized over less than 6 months (reported under non disclosure) • (reported by the American Society for Quality)

37. ISO – CMMI Differences Netta Dotan, Quality Assurance & project management, Ronkal Office Technologies

38. ISO – CMMI Differences Netta Dotan, Quality Assurance & project management, Ronkal Office Technologies

39. ISO – CMMI Similarities • Both require the organization be explicit about what their processes and quality systems are • Say what you do; do what you say • The organization records and tracks data for objective analysis • Require strong management support to succeed • Provide a structured and measured approach to quality improvement • Require an outside audit for “certification” • Both are refined/improved over time

40. So What • Why Should You Care