1 / 40

Standards & Assessments CMMI, ISO 9000, TL9000

Standards & Assessments CMMI, ISO 9000, TL9000. Sources: ASQ CSQE Primer Introduction to CMMI CMMI Distilled. August 4 Class . CMMI Introduction & Configuration Management Appraisal ISO 9000/TL-9000 Due today (31-July): Cycle 2 Design & Code, hand off to System Tester

tanner
Download Presentation

Standards & Assessments CMMI, ISO 9000, TL9000

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Standards & AssessmentsCMMI, ISO 9000, TL9000 Sources: ASQ CSQE Primer Introduction to CMMI CMMI Distilled SE 652 - 2007_7_31_CMMI_Software_Quality.ppt

  2. August 4 Class CMMI Introduction & Configuration Management Appraisal ISO 9000/TL-9000 Due today (31-July): Cycle 2 Design & Code, hand off to System Tester System Test Plan Inspected & Baselined Project notebook updates including inspection records, meeting minutes, etc. SE 652 - 2007_7_31_CMMI_Software_Quality.ppt

  3. Topics Audits & Assessments CMM / CMMI & SCAMPI ISO 9000: ISO 9001:2000, ISO 9000-3:1997, TickIT Q9000, TL9000 SE 652 - 2007_7_31_CMMI_Software_Quality.ppt

  4. Capability Maturity Model (CMM) Created in 1987 by Software Engineering Institute (SEI) 5 level model based on proficiency in Key Process Areas (KPAs) Migrating to Capability Maturity Model Integration (CMMI) Three source models: • CMM for Software • Systems Engineering Capability model • Integrated Product Development CMM CMMI v1.1 SE 652 - 2007_7_31_CMMI_Software_Quality.ppt

  5. What is it? SE 652 - 2007_7_31_CMMI_Software_Quality.ppt

  6. Why Would I want one? Required • Contractual • Senior Management Decree (e.g. ROI of 7 to 1) Sales Tool Want to improve SE 652 - 2007_7_31_CMMI_Software_Quality.ppt

  7. Schedule Example SE 652 - 2007_7_31_CMMI_Software_Quality.ppt

  8. Process Capability Ability of a process to produce planned results • Predictable • Measureable SE 652 - 2007_7_31_CMMI_Software_Quality.ppt

  9. Process Models CMMI is model based Model = structured collection of elements that describes characteristics of effective processes Process Area = cluster of related practices that when performed collectively, satisfy a set of goals considered important for making significant improvement in that area Processes selected are those proven by experience to be effective (i.e. best practices, practical knowledge from previous endeavors) Notes: A process area is not a process A model is not a processmodels show what to do, not how to do it! Philosophy “All models are wrong, some are useful” – George Box SE 652 - 2007_7_31_CMMI_Software_Quality.ppt

  10. CMMI Models Model Options: Software Engineering (SW) Systems Engineering + Software Engineering (SE/SW) Systems Engineering + Software Engineering + Integrated Process & Product Development (SE/SW/IPPD) … + Supplier Sourcing (SE/SW/IPPD/SS) Representation Options: Staged (Maturity Levels) Migration from CMM to CMMI Continuous (Capability Levels) Migration from EIA/IS-731 to CMMI Recommended order for process improvements, but not prescribed … SE 652 - 2007_7_31_CMMI_Software_Quality.ppt

  11. Levels Zero – Ad Hoc One – Doing it (in Continuous, Ad Hoc in Staged) Two – Process performed for individual projects Three – Process focus at organizational level Four – Projects and processes are quantitatively managed Five – Projects and processes being optimized based on performance data & results SE 652 - 2007_7_31_CMMI_Software_Quality.ppt

  12. Representations Revisited Continuous Model • 25 Process Areas each assessed at level 0-5 Configuration Mgmt = capability level 3 Risk Mgmt = not done (capability level 0) Requirements Mgmt = capability level 2 • Result can be presented as a Kiviat chart Staged Model • 25 Process Areas assigned to each of 4 Maturity Levels (see next slide) • Result is a grade (1-5) SE 652 - 2007_7_31_CMMI_Software_Quality.ppt

  13. Staged RepresentationMaturity Levels (MLx) 5 Optimizing Focus on process improvement 4 Quantitatively Managed Process measured & controlled Organization Organization 3 Defined Process characterized by organization is proactive 2 Managed Process characterized for project & often reactive Project Project 1 Initial Process unpredictable, poorly controlled & reactive SE 652 - 2007_7_31_CMMI_Software_Quality.ppt

  14. Staged RepresentationProcess Area Mapping to Maturity Levels SE 652 - 2007_7_31_CMMI_Software_Quality.ppt

  15. Continuous RepresentationProcess Areas Engineering • Requirements Management (REQM-2) • Requirements Development (RD-3) • Technical Solution (TS-3) • Product Integration (PI-3) • Verification (VER-3) • Validation (VAL-3) Support • Configuration Management (CM-2) • Process & Product Quality Assurance (PPQA – 2) • Measurement and Analysis (MA-2) • Decision Analysis and Resolution (DAR-3) • Organizational Environment for Integration (OEI-3) • Causal Analysis and Resolution (CAR-5) Process Management • Organizational Process Focus (OPF-3) • Organizational Process Definition (OPD-3) • Organizational Training (OT-3) • Organizational Process Performance (OPP-4) • Organizational Innovation & Deployment (OID-5) Project Management • Project Planning (PP-2) • Project Monitoring & Control (PMC-2) • Supplier Agreement Management (SAM-2) • Integrated Project Management (IPM-3) • Risk Management (RSKM-3) • Integrated Teaming (IT-3) • Integrated Supplier Management (ISM-3) • Quantitative Project Management (QPM-4) SE 652 - 2007_7_31_CMMI_Software_Quality.ppt

  16. CMMI Assessment Cheat Sheet Institutionalization – Ingrained Way of Doing Business that an organization follows routinely as part of its corporate culture Specific Goals – Required model component that describes the unique characteristics that must be present to satisfy the process area Specific Practice – Expected model component that is considered important to achieving the associated specific goal. The specific practices describe the activities expected to result in achievement of the specific goals of a process area. (In continuous representation – every specific practice (SP) is associated with a CL, in staged – all SPs are treated equally) Generic Goal – Required model component that describes the characteristics that must be present to satisfy the institutionalization of the processes that implement a process area Generic Practice – Expected model component that is considered important in achieving the associated generic goal. The generic practices describe the activities that are expected to result in achievement of the generic goal and contribute to the institutionalization of the processes associated with a process area. SE 652 - 2007_7_31_CMMI_Software_Quality.ppt

  17. CMMI Assessment Cheat Sheet (continued) Managed Process: • Performed process planned & executed in accordance with policy • Employs skilled people • Adequate resources • Produces controlled outputs • Involves relevant stake holders • Monitored, controlled & reviewed • Evaluated for adherence to process description Defined Process: • Managed process tailored from the organizational standard processes • Maintained process description • Contributes work products, measures & other process info to organizational process assets Performed Process • Accomplishes needed work to produce work products • Specific goals of the process area are satisfied Establish & Maintain • Includes documentation & usage: • Planned • Documented & • Used SE 652 - 2007_7_31_CMMI_Software_Quality.ppt

  18. Configuration Management (CM) Assessment SE 652 - 2007_7_31_CMMI_Software_Quality.ppt

  19. DeMarco & Lister on Process Organizations driving to be SEI Level 5 (at least level N+1) Standards are good, but … Most success centered around standard interfaces Mandating a “best practice” is a bad practice Process improvement is good, but process improvement programs aren’t Competent people improve processes all the time (pride, growth, etc.) Formal process improvement moves responsibility from the individual to the organization Process improvement programs focus on process rather than product(making a poor product efficiently is often worse than making a good product poorly) Focus on process “level” tends to make organizations risk averse “The projects most worth doing are the ones that will move you down one full level on your process scale!” SE 652 - 2007_7_31_CMMI_Software_Quality.ppt

  20. Break  SE 652 - 2007_7_31_CMMI_Software_Quality.ppt

  21. Quality Standard Rationale Customers want & need assessments of supplier quality Means: Individually audit (i.e. qualify) vendor: Specific products Processes (e.g. manufacturing, design & development, support) Alternative: Common Quality Assurance standards & audits SE 652 - 2007_7_31_CMMI_Software_Quality.ppt

  22. Major Audit Types First Party Audit Within own company (aka internal audit) Used to measure own performance, strengths & weaknesses against internally established procedures & systems Second Party Audit Performed by customer on their supplier (aka external audit) Third Party Audit Outside, independent auditor contracted to audit on behalf of company or a supplier (e.g. ISO 9000 registration audit) Assessments (e.g. SCAMPI) Similar to first party audit, but typically performed by external assessors SE 652 - 2007_7_31_CMMI_Software_Quality.ppt

  23. Other Audit Types System Audit – examination of bigger picture of organization &/or project Typical cross organizational, cross process & cross product Process Audit – verify inputs, actions & outputs in accordance with defined requirements (e.g. software inspections) Product Audit – final product or service for “fitness for use” Customer oriented Compliance Audit Regulatory – audit to government regulations Management – audit to organizational rules, effectiveness & conformance Quality – systematic & independent of quality activities vs. established procedures SE 652 - 2007_7_31_CMMI_Software_Quality.ppt

  24. ISO 9001:2000 Objective Provide confidence that vendor can produce quality products Assumptions: good practices will produce good products Standard for assessing organization’s Quality Management System (QMS) • Processes • Activities • Behaviors • Training But, ISO focuses on Quality Assurance not Quality Control ISO-9001 certification does not guarantee quality products! SE 652 - 2007_7_31_CMMI_Software_Quality.ppt

  25. Tenants of ISO 9001 • Say what you do • Do what you say • Prove it! SE 652 - 2007_7_31_CMMI_Software_Quality.ppt

  26. ISO 9000 Audits Customers write requirements for current ISO-9001 certification into purchasing contracts Organizations apply for 3rd party audit,end result is ISO-9001 certification ISO International Accreditation Forum (IAF) board Audits national accreditation boards (i.e. one board each nation) Who register individual registrars (e.g. Lloyd’s, DNV) Who audit organization internal auditors (e.g. Lucent Optical Networking) & spot check Who audit design, development, manufacturing & supportteams within the organization SE 652 - 2007_7_31_CMMI_Software_Quality.ppt

  27. ISO Alphabet Soup ISO 9000:2000 Overall framework, fundamentals of quality management systems & terminology ISO 9001:2000 Requirements for quality management systems (qms) & what is required to demonstrate compliance ISO 90003 2004 (previously 9000-3) Guidelines for the application of ISO 9001:2000 to computer software ISO 19011 Guidelines for auditing quality and environmental management systems SE 652 - 2007_7_31_CMMI_Software_Quality.ppt

  28. What is wrong with ISO 9001? Vendors ISO-9001 certified, but quality still elusive! No visibility into supplier quality levels Not getting quality levels they wanted Solution: TL9000 (Quest forum, telecommunications) QS9000 (automotive) SE 652 - 2007_7_31_CMMI_Software_Quality.ppt

  29. TL9000 ISO on steroids Wholly subsumes ISO 9001-2000 Requires vendors prove they are actually improving Metrics focused on cost drivers of service providers: Know vendor is measuring Visibility into quality improvement results SE 652 - 2007_7_31_CMMI_Software_Quality.ppt

  30. TL9000 Top Management Requirements Monitor & improve customer satisfaction Set long & short term objectives for organization effectiveness Set targets for TL9000 product performance metrics Use an explicit life-cycle model Establish a quality improvement program Periodic management review of quality system SE 652 - 2007_7_31_CMMI_Software_Quality.ppt

  31. TL9000 Metrics Cross-discipline metrics • # of problem reports • Problem report fix response time • Overdue problem report fix responsiveness • On-time delivery Hardware & Software measurements • System Outages Hardware measurements • Return rates Software measurements • Software installation & release application aborts • Corrective patch quality • Feature patch quality • Software update quality SE 652 - 2007_7_31_CMMI_Software_Quality.ppt

  32. TL9000 Common Audit Questions • Do you know how to find your Quality Policy, QMS and the processes you should be using for your work? • Do you know your organization’s product delivery & improvement goals and what you must do to support them? • Do you know what skills you should have? • Do you know what you have to do to approve/baseline/finalize your documents, designs & code? • Do you know how to store & find records of reviews, inspections, key decisions, etc.? • Do you know what to do if a problem is found with the product or process? • Do you know your organization’s performance with respect to customer satisfaction, quality of delivered products & process execution? SE 652 - 2007_7_31_CMMI_Software_Quality.ppt

  33. TL9000 Sample Requirements Planning • Must have methods for estimating & tracking • Determine where you will do reviews & tests • Risk management plans, customer, user & supplier involvement in reviews & evaluation Software Outputs • Requires architecture, detailed designs, code & user documentation • Each design thread must be reviewed at some point prior to integration or system test Software Testing • All testing must have test plans; test process must be documented • Plans must include test cases with inputs, output & test success criteria • Plans must include types of testing, requirements traceability, coverage definition & measurement, test environment, defect handling, et.al. • Integration testing specifically required SE 652 - 2007_7_31_CMMI_Software_Quality.ppt

  34. Team Project Postmortem Tracking process improvements during project Process Improvement Proposals (PIP) Port-Mortem Areas to consider Better personal practices Improved tools Process changes SE 652 - 2007_7_31_CMMI_Software_Quality.ppt

  35. Postmortem process Team discussion of project data Review & critique of roles SE 652 - 2007_7_31_CMMI_Software_Quality.ppt

  36. Postmortem process Review Process Data Review of cycle data including SUMP & SUMQ forms Examine data on team & team member activities & accomplishments Identify where process worked & where it didn’t Quality Review Analysis of team’s defect data Actual performance vs. plan Lessons learned Opportunities for improvement Problems to be corrected in future PIP forms for all improvement suggestions Role Evaluations What worked? Problems? Improvement areas? Improvement goals for next cycle / project? SE 652 - 2007_7_31_CMMI_Software_Quality.ppt

  37. Cycle Report Table of contents Summary Role Reports Leadership – leadership perspective Motivational & commitment issues, meeting facilitation, req’d instructor support Development Effectiveness of development strategy, design & implementation issues Planning Team’s performance vs. plan, improvements to planning process Quality / Process Process discipline, adherence, documentation, PIPs & analysis, inspections Cross-team system testing planning & execution Support Facilities, CM & Change Control, change activity data & change handling, ITL Engineer Reports – individual assessments SE 652 - 2007_7_31_CMMI_Software_Quality.ppt

  38. Role Evaluations & Peer Forms Consider & fill out PEER forms Ratings (1-5) on work, team & project performance, roles & team members Additional role evaluations suggestions Constructive feedback Discuss behaviors or product, not person SE 652 - 2007_7_31_CMMI_Software_Quality.ppt

  39. Project Notebook Updated Requirements & Design documents Conceptual Design, SRS, SDS, System Test Plan, User Documentation* Updated Process descriptions Baseline processes, continuous process improvement, CM Tracking forms ITL, LOGD, Inspection forms, LOGTEST Planning & actual performance Team Task, Schedule, SUMP, SUMQ, SUMS, SUMTASK, CCR* SE 652 - 2007_7_31_CMMI_Software_Quality.ppt

  40. August 4 Class CMMI Introduction & Configuration Management Appraisal ISO 9000/TL-9000 Due July 31: Cycle 2 Design & Code, hand off to System Tester System Test Plan Inspected & Baselined Project notebook updates including inspection records, meeting minutes, etc. Deliverables for August 7 Project Postmortem (cycle report) Cycle 2 presentations Peer Feedback forms Completed project notebooks Cycle Exit Completed project (source, documents & all quality records) SE 652 - 2007_7_31_CMMI_Software_Quality.ppt

More Related