1 / 42

Software Quality Assurance

Software Quality Assurance. Introduction. What is software quality? How can it be measured? How can it be measured before the software is delivered? Some key quality factors Some measurable indicators of software quality. Introduction. Think of an everyday object e.g. a chair

mea
Download Presentation

Software Quality Assurance

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Software Quality Assurance

  2. Introduction • What is software quality? • How can it be measured? • How can it be measured before the software is delivered? • Some key quality factors • Some measurable indicators of software quality

  3. Introduction • Think of an everyday object • e.g. a chair • How would you measure it’s “quality”? • construction quality? (e.g. strength of the joints,…) • aesthetic value? (e.g. elegance,…) • fit for purpose? (e.g. comfortable,…) • All quality measures are relative • there is no absolute scale • we can say A is better than B but it is usually hard to say how much better • For software: • construction quality(建造的品質)? • software is not manufactured • aesthetic value (美學上的價值)? • but most of the software is invisible • aesthetic value matters for the user interface, but is only a marginal concern • fit for purpose? • Need to understand the purpose

  4. McCall’s Quality Factors Maintainability Flexibility Testability Portability Reusability Interoperability revision transition operation Correctness reliability usability integrity efficiency

  5. 軟體品質因素

  6. Measuring Quality examples... The Quality Concepts (abstract notions of quality properties) reliability complexity usability information flow between modules? time taken to learn how to use? Measurable Quantities (define some metrics) mean time to failure? minutes taken for some user task??? Counts taken from Design Representations (realization of the metrics) run it and count crashes per hour??? count procedure calls???

  7. SQA: What is SQA? • SQA is the process of assuring people that every effort has been made to ensure that software products have the desired level of reliability, maintainability, usability, and salability. • SQA是一種執行軟體評估與衡量的活動 (Baker and Fisher) • Set of systematic activities providing evidence of the ability of the software process to produce a software product that is fit to use (G. Schulmeyer and J. McManus, Software Quality Handbook, Prentice Hall, 1998.)

  8. SQA and CMMI • Software (Process and Product ) quality assurance is the key process in Level 2, Managed. • The CMMI lists four goals that must be achieved to satisfy this key process area: • Goal 1: Objectively Evaluate Processes and Work Products • Adherence of the performed process and associated work products and services to applicable process descriptions, standards, and procedures is objectively evaluated. • Goal 2: Provide Objective Insight • Noncompliance issues are objectively tracked and communicated, and resolution is ensured. • Goal 3: Institutionalize a Managed Process • Goal 4: Institutionalize a Defined Process

  9. Software Quality Assurance • How do you assure the quality of your software? • Good processes • Good documentation and artifacts • Accountability (可說明性) • Learn and improve

  10. The Objectives of SQA • Monitoring processes and products throughout the software development lifecycle to ensure the quality of the delivered product(s) • Monitoring the processes • Provides management with objective feedback regarding process compliance to approved plans, procedures, standards, and analyses • Monitoring the products • Focus on the quality of product within each phase of the SDLC • e.g., requirements, test plan, architecture, etc. • Objective: identify and remove defects throughout the lifecycle, as early as possible

  11. SQA: An SQA Program • Minimizing number of defects in delivered s/w • Creating mechanisms for controlling software development and maintenance so that costs and schedules can be met • Making certain that the delivered product can be used in its intended marketplace • Improving the quality of future products

  12. Software defects • Mistakes made at any point in the software process • Requirements, design, coding, maintenance • Consequences • Inconvenience, loss of service, financial loss, equipment damage, injury, death

  13. The percentage of defects found by various methods

  14. The truth of defects 1. The later in the life cycle that an error is detected the more expensive it is to repair. 2. Errors remain latent and are not detected until well after the stage at which they are made. • 54% of errors detected after coding and unit testing. • 45% of these errors were requirements and design errors. 3. There are numerous requirements errors. • Estimates indicate that 56% of all errors are errors during the requirements stage.

  15. Relative Cost to Repair based on when it was found • Requirements - 1 time • Design - 3-6 times • Code - 10 times • Unit test - 15-40 times • System test - 30-70 times • Field operation - 40-1000 times

  16. When should quality assurance be done? At every stage in the software process

  17. Testing • Process of exercising or evaluating a system by manual or automatic means to verify that it satisfies specified requirements or to identity differences between expected and actual results.

  18. Testing • Begins as soon as requirements and specifications are written • >= 40% of total project effort • Constructive activity • Includes formal reviews and walkthroughs

  19. Testing Objectives • Testing is a process of executing a program with the intent of finding an error. • A good test case is one that has a high probability of finding an as yet undiscovered error. • A successful test is one that uncovers an as yet undiscovered error.

  20. Testing Limitations • Cannot determine whether software meets user’s needs; only whether it appears to conform to user’s requirements • Cannot show that a system is free of defects; only that it contains defects • Does not correct defects; only finds defects

  21. The Process of SQA 功能與完整性 需求評估 定義品質需求 制定SQA計畫 需求分析 評估客戶滿意需求程度 feedback 設計評估 設計 在需求分析階段完成 feedback 測試評估 測試 系統完整性與一致性 measurement 執行效率與正確性

  22. SQA Planning • IEEE Std 730-2002 • Standard for Software Quality Assurance Plans • 12 pages • IEEE Guide for Software Quality Assurance Planning • draft P730.2 • 87 pages

  23. SQA Planning • IEEE SQAP • Purpose (Section 1 of the SQAP) • Reference documents (Section 2 of the SQAP) • Management (Section 3 of the SQAP) • Documentation (Section 4 of the SQAP) • Standards, practices, conventions, and metrics (Section 5 of the SQAP) • Reviews and audits (Section 6 of the SQAP) • Test (Section 7 of the SQAP) • Problem reporting and corrective action (Section 8 of the SQAP) • Tools, techniques, and methodologies (Section 9 of the SQAP) • Code control (Section 10 of the SQAP) • Media control (Section 11 of the SQAP) • Supplier control (Section 12 of the SQAP) • Records collection, maintenance, and retention (Section 13 of the SQAP) • Training (Section 14 of the SQAP) • Risk management (Section 15 of the SQAP)

  24. Contents of SQA Plan (sect 1&2) • Purpose • list software covered • state portion of software life cycle covered • Reference Documents • complete list of documents referenced elsewhere

  25. Sect 3 - Management • organization - depict structure of org. • responsibilities • tasks • tasks to be performed • relationship between tasks and checkpoints • sequence of tasks • responsibilities • of each organizational unit

  26. Sect 4 - Documentation • identifyrequired documents • state how documents will be evaluated • minimum documents • SRS - Software Requirements Specification • SDD - Software Design Description • SVVP – S. Verification and Validation Plan • SVVR - S. Verification and Validation Report • User documentation - manual, guide • SCMP – S. Configuration Management Plan

  27. Verification and Validation • 驗證(Verification): "我們是否正確的開發了產品？" • 軟體應該與規格相符 • 確認(Validation): "我們是否開發了對的產品？" • 軟體應該執行使用者真實的需求 • 驗證與確認著重於讓程式的缺失得以浮現 • 除錯著重於找出缺失並加以改正

  28. S. Verification and Validation Plan • 測試規劃與建立測試程序的標準有關，而非著重於描述產品如何測試

  29. Sect 5- Standards, Practices, Conventions and Metrics • Identify S,P,C,and M to be applied • How compliance is to be monitored and assured • Minimum • documentation standards, logic structure standards, coding standards, testing standards • selected SQA product and process metrics

  30. Sect 6 - Reviews and Audits • purpose • define what reviews/audits will be done • how they will be accomplished • what further actions are required • Minimum • Software Requirements Reviews • Preliminary Design Review • evaluate technical adequacy of top-level design

  31. Min Set of Reviews/Audits (cont) • Detailed Design Review • acceptability of detailed designs • Software Verification and Validation Plan Review • adequacy of planned verification and validation • Functional Audit • all requirements in SRS have been met • Physical Audit • software and documents are consistent and ready • In-Process Audit • Managerial Reviews

  32. Sect 7 - Test • All tests that are not included in SVVP

  33. Sect 8 - Problem Reporting • Practices and Procedures for reporting, tracking, and resolving problems • Organizational responsibilities

  34. Sect 9 - Tools, Techniques and Methodologies • identify the special software tools, techniques and methodologies • purpose • describe use

  35. SQA: 6 SIGMA QUALITY • Sigma = “Standard Deviation” • Typical software has 3 to 4 defects per KLOC • 6 Sigma = 3 to 4 defects per million lines of code • Average companies accept 99.98% quality = 4S • 6 Sigma = 99.9999998% level of quality

  36. SQA: 6 SIGMA QUALITY • Quality Improvements: • 3Sigma to 4Sigma = 10 fold • 4Sigma to 5Ssigma = 30 fold • 5Sigma to 6Sigma = 70 fold • Best-in-Class companies in some industries operate at 6Sigma (Airline = 6.4Sigma; 2.5M:1) • Software organizations need to assess this

  37. SQA: Quality Software Process People Management Discipline SQA

  38. SQA: Pursuing SQA What organizations are doing • Nothing (42%) • Slogans(口號) - “Quality is Job One!” (4%) • Improved testing (24%) • Focus on defect prevention (20%) • Process Improvements (9%) • Other... (1%)

  39. SQA: Software Reliability (MTBF) Putnam’s Software Reliability Model Operational Capability N U M B E R E R R O R S O F 0 0 T I M E

  40. Purpose includes improvement Quality philosophy Eliminate mass inspections Award business based on more than price Continuous improvement Institute OJT(On the job training) Institute Leadership SQA: Pursuing SQA the Deming Way • Drive out fear • Break down barriers • Eliminate slogans • Eliminate numerical quotas & goals • Remove barriers to pride of workmanship • Institute education and self-improvement • Get everyone involved

  41. SQA: Strategy by Yourdon If management or customer says... • Speed up testing...just say NO! • Don’t worry about a few bugs...just say NO! • We’ll pin down the specs later...just say NO! • Don’t worry, its just a beta version...just say NO! • I don’t care if there are bugs, get it out the door...just say NO!

  42. Summary • Software quality generally means fitness for purpose • need to know what that purpose is… • …what functions must it perform • …what other properties must it have (e.g. modifiability, reliability, usability…) • Not all quality attributes can be measured during design • because quality is not an attribute of software in isolation • but we can look for predictors • Reliability, efficiency, maintainability, usability • are usually the four most important quality factors • …although different authors give different lists • Modularity is often a good predictor of quality • measure it by looking at cohesion and coupling

More Related