1 / 36

Convener: Houman Younessi

Software Engineering Management. Course # CISH-6050. Lecture 7:. Software Process Assessment. Convener: Houman Younessi. 07/09/2012. AGENDA …. Process Assessment SEI SW-CMM Improvement Approach Do organizational assessments pay off? Whistleblowers

adeola
Download Presentation

Convener: Houman Younessi

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Software Engineering Management Course # CISH-6050 Lecture 7: Software Process Assessment Convener: Houman Younessi 07/09/2012 1

  2. AGENDA … • Process Assessment • SEI SW-CMM Improvement Approach • Do organizational assessments pay off? • Whistleblowers • Reference: SPICE Assessment Documents 2

  3. SW-CMM Process Assessment • SEI SW-CMM Improvement Approach: IDEAL • Initiating • Diagnosing • Establishing • Acting • Leveraging • Diagnose process, establish priorities, & act on them 3

  4. SW-CMM: IDEAL • Initiating: • Stimulus for improvement • Set context & sponsorship • Establish improvement infrastructure • Diagnosing: • Appraise & characterize current practice • Develop recommendations • Document phase results 4

  5. SW-CMM: IDEAL … • Establishing: • Set strategy & priorities • Establish process action items • Plan actions • Acting: • Define processes and measures • Plan & execute pilots • Plan, execute, & track installation 5

  6. SW-CMM: IDEAL … • Leveraging: • Document & analyze lessons • Revise organizational approach 6

  7. SW-CMM Process Assessment • General Classes of SW-CMM Appraisal • Software Process Assessment • Determine state of organization’s software process • Software Capability Evaluations • Identify contractors qualified to perform software work 7

  8. SW-CMM Process Assessment … • Software Process Assessment: • Identify improvement priorities within organization • Assessment team uses CMM to guide identifying & prioritizing findings • Findings & KPA guidance used to plan improvement strategy for organization 8

  9. SW-CMM Process Assessment … • Software Capability Evaluations: • Identify risks associated with a project or contract to build high quality on schedule & budget • During acquisition process, capability evaluation may be performed on bidders • Findings of an evaluation may be used to identify risk with using a contractor • Performed on existing contracts to monitor process performance 9

  10. Software Process Assessment & Capability Evaluation Steps: 10

  11. SW-CMM Process Assessment … • Common Steps: • Team Selection • Select team trained in CMM • Knowledgeable in SE & mgmt • Maturity Questionnaire • Site reps complete questionnaire • Response Analysis • Analyze results of questionnaire • Investigation areas = KPAs 11

  12. SW-CMM Process Assessment … • Common Steps … • On-site Visit • Using results analysis, conduct on-site visit to view process areas • Using KPAs as guide, question, listen, review & synthesize info • Apply professional judgment • Document rationale for situations where KPAs not met 12

  13. SW-CMM Process Assessment … • Common Steps … • Findings • At end of on-site period, team produces list of findings • Identifies strengths & weaknesses of org’s software processes • Software Process Assessment -> Basis for PI recommendations • Software Capability Evaluation -> Findings part of risk analysis 13

  14. SW-CMM Process Assessment … • Common Steps … • KPA Profile • Team prepares KPA profile, showing where KPAs satisfied / not satisfied by organization • KPA can be satisfied and still have associated findings, as long as findings don’t identify major problems achieving goals of KPA 14

  15. SW-CMM Process Assessment … • Differences: Process Assessments & Capability Evaluation • Results of process assessment or capability evaluation may differ • Assessment / evaluation scope may vary: • Different definitions of ‘Organization’ • Org may be based on senior management, geo location, common app, profit/loss center, etc. • Sample of selected projects 15

  16. SW-CMM Process Assessment … • Differences … • Motivation, objective, outcome & results ownership differ • These factors lead to differences in dynamics of interviews, scope of inquiry, info collected, & results • Assessment & evaluation methods are different • Assessment training doesn’t prepare team to do evaluation, vice versa 16

  17. SW-CMM Process Assessment … • Differences … • Process Assessment – performed in open, collaborative environment • Commitment from mgmt & staff to do process improvement • Objective: surface problems & help improve organization • Emphasis on interviews as tool for understanding organization’s software process 17

  18. SW-CMM Process Assessment … • Differences … • Capability Evaluation – performed in audit-oriented environment • Objective tied to monetary considerations • Emphasis on documented audit trail that reveals software process actually implemented by organization 18

  19. AGENDA … • Process Assessment • SEI SW-CMM Improvement Approach • Do organizational assessments pay off? • Whistleblowers • Reference: SPICE Assessment Documents 19

  20. Software Process Assessments • Why do process assessment? • “Why Do Organizations Have Assessments? Do They Pay Off?”, CMU SEI Technical Report, July, 1999 • Panel discussion from SEPG ’99 in Atlanta, GA on March 10, 1999 • Each panelist is one of most active lead assessor for SEI for CMM-Based Appraisals for Internal Process Improvement (CBA IPI) 20

  21. Software Process Assessments … • Electronic Data Systems • Why do assessments? • Driven by customer: higher maturity to keep customers; extra bidding points for higher maturity; DoD requirements • Do assessments pay off? • Yes! • Need exec sponsorship, funding • Everyone learns, helps org culture 21

  22. Software Process Assessments … • Reuters • Why do assessments? • Framework for continuous process improvement; maturity levels build on previous level • Raises awareness in group • Do assessments pay off? • Clear view of cost, effort, resource • Measure key performance items • Method for introducing global change 22

  23. Software Process Assessments … • U. S. Navy • Why do assessments? • Early SEI affiliate • Beta test group for Maturity Questionnaire • Continued with SEI assessments • Do assessments pay off? • Invaluable to organization • In 1997 determined 4.1 to 1 ROI • Education for new team members 23

  24. Software Process Assessments … • Panel Q&A • Why pay cost of doing CBA IPI? • How often do you do full CBA IPI? • How small of organization to do CBA IPI? • How to reduce impact/stress on org when doing CBA IPI? 24

  25. AGENDA … • Process Assessment • SEI SW-CMM Improvement Approach • Do organizational assessments pay off? • Whistleblowers • Reference: SPICE Assessment Documents 25

  26. Software Process Assessments … • Blowing the Whistle on Troubled Software Projects • Article by Mark Keil & Daniel Robey • Controversy behind going forward to management when auditors find problems with projects • Case study of IS Auditors • 1999 Study • 75 Auditors 26

  27. Software Process Assessments … • Key Points • Auditor obligation vs. risk • Reluctance to blow the whistle • Reluctance to hear the whistle • Conditions that encourage or influence whistle blowing 27

  28. AGENDA … • Process Assessment • SEI SW-CMM Improvement Approach • Do organizational assessments pay off? • Whistleblowers • Reference: SPICE Assessment Documents 28

  29. SPICE Assessment: Reference • SPICE Software Process Assessment Information • SPICE documentation available free of charge from Software Quality Institute web site: http://www.sqi.gu.edu.au/spice/suite/download.html 29

  30. SPICE Assessment: Reference … • SPICE Part 3 – Rating Process • Section 4.2: Defining Assessment Input • Section 4.4: Assessing and Rating Process • Section 4.5: Recording the Assessment Output 30

  31. SPICE Assessment: Reference … • SPICE Part 4 – Guide to Conducting Assessment • Section 4.2: Process Rating Scheme • Section 4.3.1.1: Team-based Assessment • Section 4.4: Assessment Stages (8) • Section 4.5: Success Factors for Process Assessment • Section 5.1: Reviewing Assessment Inputs 31

  32. SPICE Assessment: Reference … • SPICE Part 4 … • Section 5.2: Selecting Process Instances • Section 5.3: Preparing for Team-based Assessment • Section 5.4: Collecting and Verifying Information • Section 5.5: Determine Actual Ratings for Process Instances • Section 5.6: Determining Derived Ratings • Section 5.7: Validating the Ratings • Section 5.8: Presenting Assessment Output 32

  33. SPICE Assessment: Reference … • SPICE Part 5 – Construction, Selection, & Use of Assessment Tools • Section 4.1: Form and Purpose of an Assessment Instrument • Section 4.2 (and Annexes A, B, C, D): Implementation of Standard Indicators • Section 4.3: Tailoring of Indicators Contained in Assessment Instrument • Section 4.4: Modular Assessment Instruments 33

  34. SPICE Assessment: Reference … • SPICE Part 5 … • Section 4.5: Capturing and Processing Assessment Data • Section 4.6: Using an Assessment Instrument • SPICE Part 6 – Qualifications & Training of Assessors • Section 4.1: Role of the Assessor • Section 5: Assessor Competence 34

  35. M. Paulk, B. Curtis, M. B. Chrissis, C. V. Weber, "Capability Maturity Model for Software", Version 1.1, Software Engineering Institute, Carnegie Mellon University, February, 1993. Available at http://www.sei.cmu.edu/publications/documents/93.reports/93.tr.024.html W. S. Humphrey, Managing the Software Process, Addison-Wesley, Reading, MA, 1989 M. Paulk, C. V. Weber, B. Curtis, M. B. Chrissis, The Capability Maturity Model: Guidelines for Improving the Software Process, Addison-Wesley, Boston, MA, 1995 “Process Maturity Profile of the Software Community 2002 Mid-Year Update”, Software Engineering Institute, Carnegie Mellon University, August, 2002. Available at http://www.sei.cmu.edu/sema/pdf/SW-CMM/2002aug.pdf References 35

  36. D. Dunaway, R. Berggren, G. des Rochettes, P. Iredale, I. Lavi, G. Taylor, “Why Do Organizations Have Assessments? Do They Pay Off?”, Software Engineering Institute, Carnegie Mellon University, July, 1999. Available at http://www.sei.cmu.edu/publications/documents/99.reports/99tr012/99tr012abstract.html M. Keil, D. Robey, “Blowing the Whistle on Troubled Software Projects”, Communications of the ACM, Vol. 44, No. 4, April 2001, pp. 87-93. PDF file (ACMp87-Keil.pdf) available online via ACM Digital Library when accessed from the RPI at Hartford Library ISO/IEC TR 15504:1998 - Software Process Assessment (SPICE) documentation suite. Available at http://www.sqi.gu.edu.au/spice/suite/download.html References … 36

More Related