1 / 41

Ludmila Matveeva ISS SQE Leader

METHODOLOGY. Ludmila Matveeva ISS SQE Leader. TOPICS COVERED. ISS Process Brief Overview Software (SW) Process Improvement SW Process Quality Models and Standards CMMI Overview The Benefits from SW Process Deployment How to Get Things Started.

kato
Download Presentation

Ludmila Matveeva ISS SQE Leader

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. METHODOLOGY Ludmila Matveeva ISS SQE Leader

  2. TOPICS COVERED • ISS Process Brief Overview • Software (SW) Process Improvement • SW Process Quality Models and Standards • CMMI Overview • The Benefits from SW Process Deployment • How to Get Things Started

  3. ISS – Information Software Systems Ltd., was founded in 2000 as a company for Motorola outsourced engineering projects in Ukraine. ISS partners with different Motorola and Freescale (former Motorola Semiconductor sector) affiliates, and works on various projects for these affiliates. In July 2003 ISS became the first software company in Ukraine formally assessed for CMM, joining the ranks of 1,500 companies from 60 countries that meet this highest Quality Standard. ABOUT US

  4. ISS SOFTWARE DEVELOPMENT OVERVIEW 2007 CMMI Appraisal SCAMPI Class C 2006 Tailoring CMMI process to the Customer’s Agile process 2005 Turned to CMMI 2004 2004 Implementation of Implementation of CMM Level 4 and 5 practices CMM Level 4 and 5 practices 2003 2003 Institutionalization of Institutionalization of ISS SW Process ISS SW Process CMM level 3 CMM level 3 2002 2002 ISS SW Process ISS SW Process Establishment Establishment 2001 2001 2001 Planning foundations and basic metrics gathering foundations

  5. ISS Software Engineering Handbook SEPG and regular Process Improvement Tailoring of the project’ Process according the project needs Periodic Process Gap analysis and self-assessments Everyone participates in Process activities Use of quantitative methods ISS concept of process management is founded on the principles of statistical process control Establishing admissible levels of processes variability Controlled process Stable process Predictable results Prepare achievable plans, meeting cost and scheduling commitments & Deliver required product functionality and quality Defect prevention practices ISS SW PROCESS BEST PRACTICES

  6. Quality and Test Groups are independent Defect Tracking System established. DTS is the tool for tracking of different process and project activities Audit Program Quality Assurance activities planned on organizational and project levels, audits conductedon regular basis Audits results stored in Audit Database Metrics Program Projects and process metricscollected and analysed Historical Database maintained Quality engineer participates in: project reviews, postmortems, and causal analysis Checklists used across all process areas ISS SW PROCESS BEST PRACTICES (cont.)

  7. Training Program Developed Training activities planned on organizational and project levels Training records stored in Training Database Training Champion identified Training Standards and Procedures Defined Training Program Implemented Local training regularly conducted by local instructors. Training materials reviewed Training feedback collected and analyzed Permanent Process training and consultations are provided by Motorola specialists ISS SW PROCESS BEST PRACTICES (cont.)

  8. A Definition of SW Process. Process is the basis of industrial software production SW Process Improvement foundation – quality of product is governed by quality of process; quality cannot be added later to the product. Most of the SW products are extremely complex systems. To build quality into such products, engineering discipline is required Understanding existing processes and introducing process changes to improve product quality, reduce costs or accelerate schedules. Surviving in the competitive business world that is constantly changing requires constant adaptation and learning SOFTWARE PROCESS DEFINITION. PROCESS IMPROVEMENT

  9. The process improvement cycle involves process measurement, process analysis and process change Process measurement Attributes of the current process are measured to quantify its status and performance. These are a baseline for assessing improvements and information for decision making. Process measurement should be used to answer specific process questions, based on organizational improvement goals (Goal-Question-Metric Paradigm) Process analysis The current process is assessed and bottlenecks and weaknesses are identified Study existing processes to understand their activities and the relationships between parts of the process and to compare them with other processes Process modeling is the documentation of a process which records the tasks, the roles and the entities used Produce an abstract model of the process and analyze it to discover process problems This involves discussing process activities with stakeholders and discovering problems and possible process changes Process change Changes to the process that have been identified during the analysis are introduced: plan, justify, implement, and train Changes should be driven by measurable goals The impact and benefits gained should be assessed and compared to the costs of changes made to the processes PROCESS IMPROVEMENT

  10. What is a Process Model? A model is a structured collection of elements that describes characteristics of effective process A model is used to help set process improvement objectives and priorities as a guide for improvement of project and organizational processes with an assessment method to diagnose the state of improvement efforts Why is a Model Important? A model provides a starting place, a framework for prioritizing actions, a common language and a shared vision, the benefit of a community’s prior experiences It is always best to start process analysis with an existing model. People then may extend and change this What are the risks of model-based improvement? Models are not comprehensive Interpretation and tailoring must be aligned to business objectives Judgment is necessary to use models correctly and with insight Potential for dysfunctional behavior to get a maturity level rating “All models are wrong; some are useful” A PROCESS MODEL

  11. Many possible choices for improvement framework… SEI’s Capability Maturity Models ISO 9001 (Quality Management System) ISO/IEC 15504 (SPICE) …or ISO 12207 (Software Life Cycle Processes), Bootstrap (Europe), domain-specific model Trillium (Bell): Telecom domain… … or improvement “philosophies”/approaches such as SixSigma, GQM (Goal-Question-Metrics), IDEAL model, SPC (Statistical Process Control), PDCA (Plan-Do-Check-Act) cycle… SW PROCESS QUALITY MODELS AND STANDARDS

  12. Six Sigma originally developed by Motorola to systematically improve processes by eliminating defects Bill Smith 1986 SixSigma is: Quality management philosophy Reduce variation in your business and make customer-focused, data driven decisions Quality improvement methodology Methodology and tools to improve business processes Pareto charts Histograms Cause-and-effect diagrams (Ishikawa) Run charts and Control charts, etc. Uses SPC methods, for ex.: DMAIC – Define, Measure, Analyze, Improve, Control Measurement of product and process quality 3.4 defects per million opportunities SW PROCESS IMPROVEMENT APPROACHES. SixSigma

  13. GQM paradigm aims to provide information needed for understanding, guiding, and changing the software processes of a software development project Basili & Weiss 1984 Goals What is the organization trying to achieve? The objective of process improvement is to satisfy these goals Questions Questions about areas of uncertainty related to the goals. You need process knowledge to derive these Metrics Measurements to be collected to answer the questions GQM translates Goals into Measures by means of Questions and Indicators = GQ(I)M GQ(I)M operates with: G - Entities (Objects) QI - Attributes (Characteristics of Objects) M - Measures (Values of Attributes), Measurements (Rules to assign Values of Attributes) SW PROCESS IMPROVEMENT APPROACHES. GQM Paradigm

  14. Two-dimensional view of the IDEAL model McFeeley 1996 SW PROCESS IMPROVEMENT APPROACHES. IDEAL model

  15. SPC (Statistical Process Control) measurement process Florac & Carleton 1999 SW PROCESS IMPROVEMENT APPROACHES. SPC

  16. PDCA (Plan-Do-Check-Act) cycle W. Edwards Deming 1986 Control Circle within the Total Quality Control (TQC) improvement model Ishikawa 1985 SW PROCESS IMPROVEMENT APPROACHES. PDCA cycle

  17. SW Process best practices Assessment-based approaches SPICE CMM(I) Bootstrap: the European counterpart to CMM SW Process standards ISO 9000 series. ISO 9001(2000) provides a model for quality assurance in design, development, production, installation, and servicing. ISO 9000-3(1997) provides guidelines for the application of ISO 9001 in SW development. SWEBOK (international project promoted by IEEE Computer Society and the ACM) SW PROCESS QUALITY MODELS AND STANDARDS (cont.)

  18. Other Process frameworks… RUP (Rational Unified Process) – SW engineering process based on the best practices in modern SW development with an interactive knowledge base integrated with tools Agile methods: Agile Modeling, eXtreme Programming, Scrum, Feature Driven Development, Crystal methods… Individuals and Interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan …and product quality standards ISO 9126 – standard for SW product quality IEEE Std. 1061:1998 – standard for SW quality metrics methodology SW PROCESS QUALITY MODELS AND STANDARDS (cont.)

  19. Process used should depend on the type of product being developed For large systems or long life-time products, management is usually the principal problem so a strictly managed process is needed; For smaller systems or short life-time products, more informality is possible There is no uniformly applicable process which should be standardized within an organization High costs may be incurred if you force an inappropriate process on a development team; Inappropriate methods can also increase costs and lead to reduced quality A fundamental factor for successes is the capability to focus on the right processes in improvement lest resources be wasted for nothing SW PROCESS QUALITY MODELS AND STANDARDS (cont.)

  20. The CMMI framework is the current stage of work on process assessment and improvement that started at the Software Engineering Institute (SEI) in the 1980s. Software Engineering Institute (SEI) USA Established in 1984 Sponsored by DoD (Department of Defense) Under Carnegie Mellon University The SEI’s mission is to promote software technology transfer particularly to US defense contractors SEI has had a profound influence on process improvement Capability Maturity Model introduced in the early 1990s. Revised maturity framework (CMMI) introduced in 2001. SEI CMM(I) HISTORY

  21. Process FOUNDATIONS OF CMMI

  22. A framework of the key process elements for system development Structured collection of processes proven through experience The CMMI tells you what to do but not how to do it An integrated view of process improvement across multiple disciplines Set process improvement goals and priorities Provides guidance for quality processes Provides a yardstick for assessing current practices Based on concepts and approached pioneered by Crosby, Deming, Juran, Humphrey, et al. FOUNDATIONS OF CMMI. WHAT IS CMMI?

  23. Two approaches to process improvement Process capability – pertains to an individual process Knowing the process capabilities of a collection of processes has implications for organizational maturity Organizational maturity – pertains to a set of processes Begins at a particular level of organizational maturity has process capability implications for multiple processes Two representations exist: staged and continuous CMMI support each approach with a representation Process capability approach  continuous representation Organizational maturity approach  staged representation The type of representation to use is based on the purpose of the improvement task FOUNDATIONS OF CMMI (cont.)

  24. Initial FOUNDATIONS OF CMMI (cont.)

  25. FOUNDATIONS OF CMMI (cont.)

  26. Process areas 24 process areas that are relevant to process capability and improvement are identified. These are organized into 4 groups. Goals Goals are descriptions of desirable organizational states. Each process area has associated goals. Practices Practices are ways of achieving a goal - however, they are advisory and other approaches to achieve the goal may be used. Process improvement in the CMMI model is based on reaching a set of goals related to good software engineering practice THE CMMI MODEL COMPONENTS

  27. THE CMMI MODEL COMPONENTS (cont.) Process areas

  28. Assessment requirements for CMMIdefine: Responsibilities of the Sponsor and Leader Assessment Method Documentation Planning and Preparing for Assessment Assessment Data collection Data Consolidation and Validation Rating Reporting results SCAMPI (Standard CMMI-based Assessment Method for Process Improvement) principles include: Senior management sponsorship Focus on organization’s business goals Use of documented assessment method Use of a process reference model (a CMMI model) Confidentiality of interviewees Collaborative team approach Focus on follow-up improvement actions CMMI APPRAISALS

  29. 3 SCAMPI classes: class C – approach class B –deployment class A – institutionalization Appraisal Team qualification CMMI APPRAISALS (cont.)

  30. Typical benefits of software process improvement (SPI) include: Higher quality (fewer defects) Lower maintenance (less rework) Higher productivity (lower development cost) Faster cycle times (quick time-to-market) Greater value (more product features)and variety (more product variations) Higher customer satisfaction (more contracts) THE BENEFITS OF PROCESS DEPLOYMENT

  31. SW Process Improvement results from Motorola GSG (now – Motorola Software Group): THE BENEFITS OF PROCESS DEPLOYMENT (cont.)

  32. ROI (Return On Investment) is the quantification of the benefits received or financial return of any given investment ROI numbers ease decision making How should managers estimate the costs and benefits of different software process improvement (SPI) methods in order to select SPI method with the greatest ROI from the alternatives? THE BENEFITS OF PROCESS DEPLOYMENT. ROI

  33. ModelPhillips J.L. (1997). Return on investment in training and performance improvement programs. Houston, TX: Gulf Publishing Company THE BENEFITS OF PROCESS DEPLOYMENT. ROI

  34. Motorola reported total ROI 677% when raising their capability from SW-CMM level 2 to 5. The calculation is based on required SPI investment rework originating from defects, and differences in defects rates between the SW-CMM level 2 and level 5 projects Boeing Space Transportation Systems reported total ROI 775% measured after CMM and inspections implementation THE BENEFITS OF PROCESS DEPLOYMENT. ROI

  35. COQ = (A + T + DP) + COPQ A-auditing efforts; T-testing efforts; DP-defect prevention activity efforts; COPQ- time for defects fixing COQ introduced by J.M. Juran in 1951 as a means of providing justification to management for investments in process improvements Efforts spent for quality reduce COQ COPQ is increased exponentially during project lifecycle EXAMPLE THE BENEFITS OF PROCESS DEPLOYMENT. Cost of SW Quality (COQ)/Cost of Poor Quality (COPQ)

  36. Knox’s theoretical model for cost of software quality Stephen T. Knox 1993 THE BENEFITS OF PROCESS DEPLOYMENT. Cost of SW Quality (COQ)/Cost of Poor Quality (COPQ)

  37. Our Sigma (DKNCSS)value: from 4.68 before CMM level 3 formal assessment to 5.16 after it Our SEA: THE BENEFITS OF PROCESS DEPLOYMENT (cont.)

  38. Basic steps to improve process capability: Understand the current status of development process and practices (software process assessment) Develop a vision of the desired process (improvement objectives) Establish a list of required process improvement actions in order of priority (high level roadmap) Produce a plan to accomplish the required actions (action plan) Commit the resources to execute the plan (improvement schedule) Start over at step 1(continuous loop) HOW TO GET THINGS STARTED

  39. IDEAL (SPI Model) HOW TO GET THINGS STARTED (cont.)

  40. Quality Control (QC) vs Quality Assurance (QA) QC (for ex., testing) is about detecting defects; the QA is about avoiding them QA analyst identifies improvement opportunities Searches for patterns of inefficiencies, defects, or other problems QA engineers effective processes Any initial process description or procedure must be seen as only the starting point for the real process engineering work. Only monitoring a process as it is used to perform real work can we see where its inefficiencies lie QA ensures that people are doing their work consistently This part of QA is the closest to QC, but it is still different in its focus. QC evaluates the product of actions; QA evaluates the actions themselves QA is an investment in improving our organization Start by using your data to find the most troublesome problems, either problems that keep repeating themselves, or those that are most costly QC indicates the level of quality that we produced on a project. When we supplement QC with QA, we improve the level of quality we will produce on all future projects HOW TO GET THINGS STARTED (cont.)

  41. The EndThank You For Attention

More Related