Chap.10 The CMM Implementation and Assessment

1. Chap.10 The CMM Implementation and Assessment Cheng, Jing (2/04/04) Software School,Hunan University

2. Objectives • To Introduce a Smooth Way to • implement the CMM • To Introduce the CMM Assessment • Process • To Introduce Some Issues of CMM

3. When we go in for implementation, we set up what we call “cooperation channels” first and move on from there. Considering the flexibility at hand, implementing CMM really starts best with that free exchange of informed option. It’s a good way to get the team on board. And it’s even better for learning the best way to go forward from there. -- Jenna Doelux, SSAC international Implementing the CMM in a Smooth Manner

4. Know the mission Establish the mission Create knowledge base Orient your organization Develop a plan Pre-assess Refine your processes Train your people Use the program Support your people Assess your program Key Success Factors –Steps of Implementing CMM

5. Step 1: Know the mission Who: CMM program manager Resource: Executive Management; CMM materials Time: As needed up front Adopting the CMM means you are choosing to become a different kind of IT shop, one focused on itself now as well as its products. Usually the idea of being CMM compliant comes into the organization from the executive level. Knowing the mission may require that you take time to become trained in CMM. Know the Mission

6. Step 2: Establish the mission Who: CMM program manager Resource: Executive Management Time: As needed up front You should look to define four things out of this effort: Time commitment – 12 ~ 20 months General budget commitment (1) Time (2) Expenses Commitment of resources (1) How many people/Over time Commitment to rewards – milestone benefits Establish the Mission

7. Step 3: Create an in-house CMM team Who: CMM program manager; team members Resource: Group Management; line workers Time: 2-4 weeks Step 1 and 2 give you the go-ahead for CMM activity. You will defined in broad terms where the organizations needs to be, why it needs to be there, and what approach you ‘ll take to move it in the direction. The goal is establishing a knowledge base to gather together a team of people who can initiate CMM, getting it working in the environment, and keep it working smoothly. Create an In-House Knowledge Base

8. Step 4: Create an implementation plan Who: CMM program manager; team members Resource: Group management; line workers Time: 3 ~ 5 weeks Use CMM V.1.1 KP to create an implementation plan which will guide that the whole organization can follow for the course of the implementation effort. The plan should also address the following broad activities: Mission definition Program commitments CMM team formation and identification Workforce orientation Reassessment planning Assessment activity Process refinement and publication Organization training Program use Program support Reassessment milestones Create an Implementation Plan

9. Step 5: Orient the workforce to CMM Who: Managers, leads, line workers Resource: Training services, facilities Time: 3 weeks; includes preparation and planning Most problems that crop up during an implementation come from lack of knowledge, from not knowing what to do or not knowing why certain things ought to be done. Both of these can be overcome with training and a series of orientations. (1) CMM training (2) Training in your shop’s own processes and procedures To initiate the implementation process, you begin with executive buy-in to the procedures and benefits of CMM. You followed by establishing a CMM knowledge base in your shop and then assembling a team of people to help you roll the program out. Orient the Organization’s Managers and Line Workers to CMM

10. Step 6: Pre-assess Who: CMM team Resource: Access to organization Time: 4 ~ 8 weeks; A pre-assessment is simply an analysis of how your current work habits (your shop’s software development methodology) match up against what’s in the CMM. You assign areas of investigation to your process team members. The assignment is based on the scope of CMM KPAs that you will be adopting for your shop. From this pre-assessment, your team will develop an assessment report. The report will identify your areaa of CMM strength as well as areas of CMM weakness. Pre-assess Your Current Work Habits

11. Step 7: Refine Your Processes Who: CMM team; Managers; line workers Resource: Access to input and review workers Time: 6 ~ 12 weeks; This is a major step in your CMM implementation. You can align the shop with CMM recommendations. To accomplish this, CMM team should work with the organization’s managers and line workers to modify existing processes or create new ones. This will be a big job. Process refinement may touch many areas within your organization. If you are now to Level 2, you may have to create processes within all six KPAs; if you are working toward Level 3, you may consider an additional seven KPAs. Refine Your Processes

12. Step 8: Training organization in new processes Who: Managers; leads; line workers Resource: Training services; facilities Time: 3 weeks; includes preparation & planning The CMM specification mentions training throughout its structure. A cornerstone of CMM effectiveness is shared knowledge, the dissemination of what I call commonalities. For your program to work at its peak, your people need to understand and use it in a consistent manner. They should have a common understanding of what it’s all about. The best way to do this is with training and on-the-job support. Training Your Staff

13. Step 9: Use the program Who: Whole organization Resource: Business as usual Time: 18 weeks and on You can define and document the best process improvement program in the world, but it’s nothing if it’s not used. The whole key to CMM – to any process improvement effort – is measured use. Repetition is the key to effective use, and repetition takes its own time. Most importantly, support your people with ample attention, guidance, and patience. Use the Program in the Workplace

14. Step 10: Support the program Who: CMM team Resource: Business as usual Time: 18 weeks and on In this role your job is to keep the shop on track with CMM. You will want to make sure that your people are using the processes as described and that they are acting in ways harmonious to your shop’s CMM goals. By shaping a carefully thought-out CMM program you are making an effort to usher your shop into a new realm of software development. Your people will need the time, resources, and support to follow you. If you provide these to them, the process should go smoothly. Support Your People

15. Step 11: Assess progress Who: CMM team Resource: Managers and line workers; CMM users Time: As planned The reasons for staring with an unofficial assessment are pretty clear: > You might as well know where you stand unofficially before you make it public. > If this your first real look at your progress, you might find it managed more easily with familiar people, not with some outside consultants. Official or unofficial, partial or full, the concept of periodic assessments is crucial to the CMM and to SPI in general. Your assessment of where your shop stands should be conducted as a careful review, free of judgment, exclusively focused on process and practice. ( It’s important to remember that assessments have nothing to do with the skills, talents, or performance of your people.). The result should be shared with the whole shop and used to form the basis for improvements. Assess Your Progress

16. You may run a process improvement program for years, and it may bolster your operations in a multitude of areas. But you’ll still be in the dark about it all until you make a formal assessment of where you are, measure it from where you started, and use it as a benchmark to sight where you want to go. -- Kendell Reese, Quality Improvement Consultant Understanding the Assessment Process

17. Decide to conduct an assessment Contract a leader assessor Select the assessment team Choose the projects Choose the participates Create the assessment plan Determine Plan review and approval Train the team Prepare the team Hold kick-off meeting Administer the maturity questionnaire Examine questionnaire results Examine process and practice documents Conduct on-side interviews Consolidate information Prepare draft findings Present draft findings Publish assessment findings Hold executive session Deliver report Steps for Conducting an Assessment

18. Assessments deliver the best results when the following conditions exits: Your processes have been in place long enough for your project teams to have been actively using them. Assessments deliver the best results when you look at projects well underway in their life cycle. Assessments that seek to address the organization as a whole should look at projects that represent the organization as a whole. Assessments require executive sponsorship Deciding to Conduct an Assessment

19. The assessment only can be executed by the assessors authorized by SEI. The SEI publishes a semi-annual list of its authorized assessors. You can find them from the SEI’s web-site: http://www.sei.cmu.edu Contracting a lead assessor according to your choice. Contracting a Lead Assessor

20. All members of the team must have received CMM training in two areas: the SEI’s Introduction to the CMM course (3 days) and CBA IPI Team Training course (also 3 days). Your team must have at least 25 years of combined software engineering experience. The overall (combined) average for each member must have at least 6 years, no one individual on the team can have les than 3 years’ experience. Your team must also have at least 10 years of combined management experience, and one of the members individually needs to have at least 6 years’ experience. 75% of the team needs to be experienced in at least one-third of the shop’s development life cycles. At least one member of the team has to have deep familiarity with the organization structure, workings, and culture of the entity being assessed. Select the Assessment Team

21. Those five items constitute the hard composition qualities of the assessment team. The soft composition qualities are less tangible but just as important. To begin with, your team members must have knowledge of the CMM KPA areas being assessed. It also helps if they are familiar with software process improvement philosophies and approaches. Finally, the team members must have the motivation, abilities, and objectivity required to carry out an assessment. Select the Assessment Team

22. The central purpose of a CMM assessment is to measure your shop’s compliance with specific CMM Key Process Arera. The first thing is define KPA scope. Once the KPA scope is defined, you can then select the projects that will come to the forefront in the assessment effort. When it comes to choosing the projects for an assessment consider the following: (1) The projects should be representative of the entity being assessed. Choose projects that give the team a valid feel for the size, scope, and type of projects the organization typically undertakes. (2) When possible, choose projects that have been underway at least 6 months and try to include full life cycle projects. Choose the Projects

23. There are three broad categories of people who you need to identify as assessment participants: Project leaders Questionnaire responders Participants in functional area interview Choose the Participants

24. Because a CMM assessment is a serious undertaking, involves people, across your organization, and requires an investment of time and effort, an assessment plan becomes especially important. The plan will serve three chief purposes: (1) It’s the document that tells executive management just what is entailed in the assessment. (2) The plan will serve as a management guide for the assessment team, helping them work within a schedule of time and activity. (3) The plan will communicate to the participants (and the rest of organization) what will be expected in terms of preparation and cooperation on their parts. Create the Assessment Plan

25. The plan usually has to address seven areas: Goals CMM scope Schedule Team members Participants Reports Risks/contingencies A template of Assessment Plan

26. A recommended light five-step process to run the final draft of your assessment plan is: Conduct action review Make executive presentation Polish Publish Distribute Assessment Plan of Approval

27. An assessment team will use a standardized maturity questionnaire to foster commonality. Likewise, the team will need minimal common background in CMM. The assessment training requirements help ensure this. Everyone on the team must have completed two SEI CMM courses: (1) “Introduction to the CMM” (2) “CBA IPI Team Training” class Train the Team in CMM

28. The first status meeting: review assignment, go over materials required, and make sure you have everything linked up that needs to be ready for the various parts of the assessment. Go over the tool of data collection. The following data should be collected: (1) instrument data; (2) interview data; (3) document data; (4) presentation data Four deportment techniques for helping set the individual up within a role that supports both information facilitation and refinement: (1) interview techniques; (2) Confidentiality; (3) plan your interviews and meeting sessions so that no two people present are linked in the chain of command in the organization; (4) orient the team to the rules of corroboration. Prepare the Team

29. An assessment kick-off meeting can include the assessment team members and as many of the participants who are free to attend. The real point of the meeting is to mark the beginning of the assessment. Introduce the members of the assessment team Brief the participants again on their roles and responsibilities Take the opportunity to go over the schedule and the major milestones in the process. Another good use of the kick-off meeting is to request from the participants the first round of documents that you will need as part of assessment analysis. Keep the meeting as brief as possible, but you might also consider this final opportunity. If the managers you have chosen to response the maturity questionnaire are at the meeting this is a good time to get the questionnaire in their hands. Hold Kick-Off Meeting

30. A logical starting point in your investigation is to get a high-level look at where the projects stand in relation to the CMM. A tool to get this view is the CMM maturity questionnaire. This is the basic starting point for any assessment. As you give the questionnaire out, it’s a good idea to go over them briefly with the respondents, as a group or one at a time. How much time should you give people to get the questionnaire back to you? (1) A knowledgeable project manager could probably tick off all Level 2 and Level 3 questionnaire in half an hour. (2) Let them have as long as they like, and as long as you can allow. (3) It depends on how you schedule it in your plan. Administer the Maturity Questionnaire

31. Response of the Questionnaire

32. You first feel for the degree of CMM compliance comes at this step. Once your team has collected the questionnaire, they can go about the process of analyzing the responses. For each single questionnaire response you should first tackle the issue of clarity. The next issue is compliance of CMM areas. Based on the group analysis of the questionnaire, your assessment team can produce an overall map of CMM compliance. Examine Questionnaire Results

33. This will involve probing into two areas: Are the development rules governing how the project ought to run CMM compliant? The answer can be found from documented processes and practices. Is the project actually run according to those rules? The answer can be revealed from a look at project artifacts and discussion with team members. Examine Process and Practice Documents

34. The step of on-site interviews is an on-site assessment. It is where the true and full CMM picture takes shape. You can do this in any order or combination you like, but you’ll probably end up speaking with three groups of participants: (1) Project leadership (2) midline managers (3) functional area representatives. During these interviews, seek not only to define the areas of weakness (where CMM compliance is not well-supported) but also to get a full picture of where CMM compliance has been strong. Conduct On-Site Interviews

35. Get the team together and review what you have uncovered for each KPA against which you were assessing. There should be a team consensus. It is easy to identify significantly lacking areas from interview questions. The flexible spirit of the CMM An organization can have weaknesses in a KPA, but that weakness needn’t prevent it from satisfying the goals of that KPA. You can satisfy goal and still have room for improvement. Carefully review the goals for each KPA assessed. Ultimately you have to rate each as Satisfied, Unsatisfied, Not rated (if you skipped it.), or Not Applicable (if it doesn’t apply to your shop.). You can’t count a KPA as a whole as being satisfied if any of the goals remain unsatisfied. For each KPA you team should be able to identify both areas of process and maturity strength, as well as areas of process and maturity weakness. Rating a level for the organization. Consolidate the Information

36. Prepare a draft of the assessment findings The purpose here is to get the assessment findings in what may end up being close to their final shape. The report produced will be authored by all members. The team must be ready to present its conclusions to the organization. Take the final quality control step of running the findings past the participants for their take on the conclusions. Prepare Draft Findings

37. The final data gathering session in the assessment is the presentation of the draft findings to the assessment participants. This is a final chance for the participants to review the collected data, to ask questions about it in relation to CMM, and to provide further data should they feel some areas have not be fully represented. The assessment might consider going over the course of the assessment: from the collection methodology, to the points looked at in CMM, to the summation and general conclusions. Areas of disagreement should be noted and addressed, and ample time should be provided for the reconciliation of conflicting views. Present Draft Findings

38. You have reached the endline here at step 18. The assessment is now finished. Your final assessment report can take any shape you think best ( Your lead assessor should have some good practical experience here.). But it should probably mimic your assessment plan in some way. At any rate, what you should probably always provide in the report include: (1) a restatement of the purpose of the assessment (2) recaps of the projects and the KPA assessed (3) identified areas of strength (4) identified areas of weakness (5) a KPA-satisfaction summary (6) a final Level rating (if desired) (7) recommendation for action: short-term and long-term activities to address the findings in the report. (options) Publish the Final Assessment Report

39. Example: CMM L2 & L3 Profiles

42. With the final report finished, you can deliver it to the sponsor. The sponsor owns the results of the assessment, and all materials should at this time be relinquished to the sponsor. Even if this is a formal CMM assessment, the sponsor need not release the findings of the report. This is the sponsor’s choice exclusively. The sponsor can also elect not send the results to the SEI. Your final step will be to present the findings to the sponsors. Deliver Assessment Report

43. The assessment process closes by opening a door on the future. Usually report delivery is followed with a presentation to the sponsor’s executive management in order to present the findings and discuss any implications the findings hold. This can usually be accomplished in an overview meeting. The general objective of the meeting should be to clearly establish where the organization stands with regard to CMM and then to establish where the organization should move from here. Hold Executive Session

44. Typical Assessment Activity Time Requirements

45. Key Points • Implementing the CMM in a Smooth Manner • Steps of Implementing CMM • Understanding the Assessment Process • Steps for Conducting an Assessment