CHAPTER 8 : CMM-BASED SOFTWARE PROCESS ASSESSMENT [zahran]

1. CHAPTER 8 : CMM-BASED SOFTWARE PROCESS ASSESSMENT[zahran]

2. OBJECTIVE : ·to understand the use of CMM as a process model to assess processes · to know the usage of SEI questionnaire as an assessment instrument

3. CMM-Based SPA • Introduction of CBA • Context of CMM-Based Assessments • SEI-Assisted Assessment Process • SEI Software Capability Evaluation(SCE) • SEI Maturity Questionnaire

4. CMM-BASED APPRAISALS • The CMM is used as the basis for different types of software process assessment – known as CMM-based appraisals • CMM-based appraisals are any assessments based on the Capability Maturity Model’s key process area and the SEI questionnaire

5. Generally, an assessment has a number of essential components • Aprocess model – against which to assess the current state of your process • An assessment method – specifies the phases and steps of the assessment • One or more assessments tools to assist in the activities of fact gathering, fact analysis and reporting

6. Contexts of CMM-bBased Assessments SEI-assisted assessments -Conducted by a trained team of software professionals from both the SEI and the organization being assessed -The scope – usually a section or division of an organization or possible an entire site

7. Software Capability Evaluations • Subset of an assessments • SEI does not perform evaluations, but it does offer training in conducting software capability evaluation • Conducted as part of DoD software acquisition process • May also be used by software organizations to evaluate subcontractors

8. Self- assessments • Conduct by a team of software professionals from the same organization undergoing the assessments • SEI trains in-house assessment teams to conduct self-assessment • Scope - division organization, an organization site or entire organization • Provide a way for an organization to examine its own software process and monitor process improvement

9. SEI-ASSISTED ASSESSMENT PROCESS • Have the following features: • Uses the CMM as a process maturity framework • Uses a questionnaire as a tool for the assessment team

10. Objectives of SEI-assisted assessments • To determine the current state of the software process that is practiced on a day –to-day basis in an organization • To identify the high-priority key areas for software process improvement on which the organization should focus its improvement efforts

11. To enroll key software practitioners in the software process improvement and change process • To help an organization focus on monitoring and improving its software process • To identify areas of strength and spread them across the organization • To send a copy of the assessments report back to the SEI to be stored in the assessments database

12. PHASES OF SEI-ASSISTED ASSESSMENT • PRE-PLANNING • PLANNING • ASSESSMENT • POST-ASSESSMENT

13. Selection phase • Involves identifying an organization as a candidate for assessment • Setting up an executive-level briefing

14. Commitment phase • Agreement is signed by the senior representatives of the organization and the SEI • This commitment includes the personal participation of the senior site manager, site representation on the assessment team, and agreement to take action on the assessment recommendations

15. Activities • Conducting an executive-level briefing • Signing the assessment agreement • Selecting the assessment team Main

16. Preparation for PLANNINGphases • Preparing for the on-site assessment • Assessment team receives training • Plan on-site assessment • Participants are selected and briefed about the assessments process, times, duration, and purpose • The questionnaire can also be filled at this time

17. Activities • Pre-assessment team training • Project selection criteria • Organizational briefing • Assessment team training

18. Assessment team training • Process management overview • Organizational briefing • Assessment process overview • Maturity questionnaire • Review of project responses • Assessment discussions • Team-building exercises • Assessment findings • Final report preparation • Planning for the assessment

19. Preparing for the on-site period • Selecting the assessment participant • Briefing the assessment participant • Assessment principles • Assessment overview • Detailed schedule • Filling up the questionnaire by project representatives of the selected projects Main

20. ASSESSMENT • Fact Gathering and analysis –assessment phase • On-site assessment is conducted • Takes a minimum of four days; most assessments take 5-8 days

21. Assessment Structure 1st day • brief senior management and assessment participantsabout the objectives and activities • complete the assessment questionnaire • data/information are received/analyzed • hold discussion

22. 2nd day • conduct discussion with the key software practitioners – provide further insight into the software proces 3rd day • formulate findings • get feedback from project representatives

23. 4th day • review finding to ensure assessment team understand the issue • findings are revised and presented - ends with formulation of the recommendation that address the findings

24. Reporting- reporting phase • assessment team prepares a formal written report and gives formal presentation of the recommendations • usually include: - scope of the assessment - how the assessment was conducted - composite organization status - examples of any strength found - summary of the findings - next steps (recommendations)

25. guidelines for formulation recommendations address each key finding - limit the number of recommendations - make recommendations specific and concise - prioritize the recommendations - focus on ‘what’ the recommendations are, not ‘how’ they will be implemented - make the recommendations realistic • the final report is the only permanent record of the assessment – will be used as reference for the action plan formulation and execution Main

26. Assessment follow-up phase • After receiving the final report, the organization spends three to six months developing an action plan • An action plan should be sufficiently detailed to provide a clear guide for execution by including the followings: • What recommendations will be implemented (action tasks) • What recommendations will not be implemented, and why?

27. How each action task will be implemented • What kind of resources required to implement each action task • Which person is responsible for each task • How is the action task scheduled

28. The action plan should be reviewed with senior management and should specify regular checkpoints for periodic management reviews • After 18 months, a reassessment or self-assessment is done by the organization to determine progress

29. The follow-up assessment has the following purposes: • To assess the progress that has been made • To establish a target for completion of the most important actions • To establish new priorities for continued improvement

30. SEI SOFTWARE CAPABILITY EVALUATION (SCE) • SCE sometimes referred as capability determination • A method for evaluating the software process of an organization to gain insight into its software capability • Focus on the management aspects • SCE team evaluates the organization’s software project development practices against the key process area (KPAs) of the CMM – to determine whether the organization follows a stable, predictable software process

31. PHASES OF SCE Consists of five phases: 1.Evaluation start • Performed by the sponsoring organization • Determining the attributes of the desired software product and the project required to produce it • Determining the process capability that is most applicable for the planned development • Selecting the SCE team

32. 2.General Preparation • Performed by the SCE team • Identifying areas where the development organization lacks experience • Defining the scope of the SCE to be investigated

33. 3. Specific Preparation • Performed by the SCE team • Selecting projects for evaluation • Preparing specific topics corresponding to the sub-process areas for evaluation • Coordinating preparation for the site data collection activities 

34. 4.Site Data Collection • Performed by the SCE team • Visiting the site and investigating each critical sub-process are in enough depth to determine their strengths, weaknesses and observable improvement activities • Determining strengths, weaknesses, and improvement activities through interviewing and document reviews

35. 5. Findings • Performed by the SCE team • Consolidating the decisions made in phase 4 relating the findings to key process areas and sub-process area • Expressing the findings as KPA strength and weaknesses and improvement activities • Conclusion and completion

36. THE SEI MATURITY QUESTIONNAIRE • Assessments instrument in the Fact Gathering phase • Features in questionnaire CMM version 1.1: - Focuses solely on process issues - Organized by CMM KPAs and cover the 18 KPAs of CMM version 1.1 - Addresses the goals of each KPA, but not all of the key practices - Includes glossary of terms and KPA descriptions - Ample space is provided beneath each question - There is room for customizing the questionnaire

37. USE OF THE QUESTIONNAIRE • Serves primarily to identify issues to be explored further though further interviews and discussion • Analysis of the responses provides focus areas for further discussion and fact finding activities, to request supporting information • Should be filled by in by the assessment participants • They should be briefed on the structure, contexts and the guidelines for answering the questions

38. The answer could be of the following: YES • The practice is well established and consistently performed NO • The practice is not well established or is inconsistently performed DOES NOT APPLY • You have the required knowledge about the project or organization and the question asked, but you feel the question does not apply to your project

39. DON’T KNOW • You are uncertain about how to answer the question

40. BENEFIT OF USING THE QUESTIONNAIRE • Provide a detailed framework for identifying key issues for discussion later in the assessment • The questionnaire narrows the focus of the assessment team to selected key process area, and helps the team to focus on areas most important to the organization • The questionnaire identifies areas about which to request supporting information

41. Establish an initial rating of an organization’s software process maturity level • Uses as a ‘springboard’ to start the software process improvement programme and set it in the right direction • The assessment provides a baseline for the current state of the software process in the organization. This baseline will provide the foundation upon which to base the improvement actions

42. Using consistent assessment tools and models across assessments is likely to produce ratings and result in a reliable, consistent and repeatable manner • Provide an entry point through which the assessment team can get the truth about what really goes on in the software organization

43. CUSTOMIZING THE SEI QUESTIONNAIRE • For self-assessment purpose • Why customization? Need to align to: • Objective of assessment • Overall direction of SPI Programme • Business and software environment • Package acquisition strategy

44. Actions during customization: • Add question to cover additional key practices in KPA • Add questions to cover non-process aspect. • Customizing the number of KPA covered by questionnaire • Also the CMM focuses solely on process-related issues. Sometimes it may be desirable to include other domains such as the organization or the technology.

45. Summary • Assessments can be performed by using the CMM as a process model to assess the processes • Use SEI Questionnaire as an instrument