1 / 23

Software Quality Assurance

Software Quality Assurance. Industry statistics for 1994 (WW. Gibb): For every 6 systems that work, 2 are cancelled It is worse for large systems: 3 out of every 4 large systems were reduced in scope or did not work at all (e.g. Denver airport)

india-keith
Download Presentation

Software Quality Assurance

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Software Quality Assurance • Industry statistics for 1994 (WW. Gibb): • For every 6 systems that work, 2 are cancelled • It is worse for large systems: 3 out of every 4 large systems were reduced in scope or did not work at all (e.g. Denver airport) • Still bad today for low maturity software development • Good quality software doesn’t just happen, however good the team producing the software and however good the development techniques they use. • Good software development needs procedures and management. That is what QA is all about

  2. Why are procedures and management necessary for quality? • Software development is complex: much to remember • Hence standards, procedures and guidelines which embody best practice • These also help training of new staff • When the pressure’s on… something has to give... • Hence defined points at which quality of work so far is assured • Hence being explicit with the customer about cost-time-scope-quality trade-off: ‘if you want software of this quality...’ • Some defects may go unnoticed by the author • Hence reviews… which wouldn’t happen without management! • Helps assure the customer that the software is of sufficient quality

  3. This lecture • What is quality? • Quality factors • What is quality assurance? • Quality manuals and plans • Standards for quality manuals: the ISO 9000 family • Group Project Standards • Reviews • Kinds of review: particularly the Formal Technical Review • How to do a formal review

  4. What is quality? • ISO definition (ISO 8402:1994) • “The totality of features and characteristics of a product, process or service that bears on its ability to satisfy stated and implied needs.” • Or… • Software quality is meeting the requirements of the user • With the caveats that… • The user requirements may not be precisely stated • Some user requirements may not be stated at all, because of ‘shared assumptions’ between user and developer • The developer may also have requirements which may not be part of the user requirements (e.g. testability, maintainability)

  5. Quality factors (ISO 9126:1992) • Functionality • Satisfying stated or implied needs (regarding the software’s functions) • Reliability • Maintaining performance under stated conditions for stated period of time • Usability • The effort needed for use, and the individual assessment of such use by a stated or implied set of users • Efficiency • The relationship between performance and resources required • Maintainability • The effort needed to make specified modifications • Portability • The ability of software to be transferred from one environment to another

  6. What is Quality Assurance? • Quality Assurance is the overall management of quality. • Within a company, it includes: • Specifying a quality policy: General quality objectives and commitments • Maintaining a quality manual, which contains details regarding: • What constitutes good quality: Standards, procedures, guidelines • Quality controls: methods of assuring that quality is good • Producing quality plans for each project • The parts of the quality manual which apply to this project • Assigning specific people to quality assurance, often including a project-independent quality assurance group • Planning (schedule/budget), performing, evaluating quality assurance activities

  7. Recognised standards ofQuality Assurance Increases customer’s confidence that a company produces quality products General Quality Standards e.g. ISO 9000 Quality Manual Project 1 Quality Plan Project 2 Quality Plan Project n Quality Plan

  8. The ISO 9000 family • ISO 9000: Guidelines for selection and use of the other documents in the family. • ISO 9001: Model for QA in design, development, production, installation and servicing. • ISO 9002: Model for QA in production, installation and servicing. • ISO 9003: Model for QA in final inspection and test. • ISO 9004: Guidelines on quality management and quality system elements. • Also ISO 9000-3: Application of ISO 9001 to the development, supply and maintenance of software. Direct correspondence between clauses of ISO 9001 and ISO 9000-3.

  9. Management Responsibility. Quality System. Internal quality system audits. Corrective action. Contract review. Purchaser’s requirements spec. Development planning. Quality planning. Design and implementation. Testing & Validation. Acceptance. Maintenance. Configuration Management. Document Control. Measurement. Rules, practices, conventions. Training. Purchasing. Some example sections from ISO 9000-3 ISO 9001/9000-3 are short (10-15 pages): open to interpretation

  10. ISO 9001 … Quality planning shall be consistent with all other requirements of a supplier’s quality system and shall be documented in a format to suit the supplier's method of operation. The supplier shall give consideration to... ISO 9000-3 Quality planning should address… quality requirements, expressed in measurable terms, where appropriate; the life cycle model to be used for software development defined criteria for starting and ending each project phase identification of types of reviews, tests and other verification and validation activities to be carried out identification of configuration management procs to be carried out ISO 9001 versus ISO 9000-3:‘quality planning’ section

  11. ISO 9000:A standard for quality manuals • Companies can, and do, get certification that: • Their quality manual meets the ISO 9000 standards • They run their projects with adherence to their quality manual • A company is ISO certified by an independent body • The company first receives training in ISO 9000 etc. • The quality manual is sent to the body, which assesses it and reports back • An assessment team visits the company to ensure that it follows its manual • The body then awards the company ISO 9000 status unconditionally, conditionally, or not at all • After certification, an assessment team visits unannounced 2-4 times per year - and conducts a full reassessment every 3 years

  12. Organisational commitment to continuous process improvement; defect prevention Level 5 Optimizing Management by measurement: quantitative targets for each project; productivity and quality measures; measurement database Level 4 Managed Stable process model including entry/exit criteria; reviews, quality tracking, full process training program Level 3 Defined Stable project planning, tracking of cost/schedule/ functionality, problem identification, project standards; can ‘repeat the success of previous projects’ Level 2 Repeatable Level 1 Initial Unstable, ad-hoc processes; project success depends on staff skills/experience (‘heroics’) Software Engineering Institute’s Capability-Maturity Model (SEI CMM)

  13. Comments on the CMM • CMM versus ISO 9000 • CMM much more prescriptive (500 pages versus 10-15 pages) • Companies rated ISO 9000 are probably partly level 2, partly level 3; but they may only be at level 1 (because ISO open to interpretation) • CMM is improvement-oriented, ISO 9000 is ‘minimum quality’ • Disadvantages of the CMM • Nothing about technology or risk analysis • Originally to help DoD select contractors: large, slow-moving systems • Key areas a bit arbitrary: 80% level 2 + 70% level 3 = level 1! • Too bureaucratic for small companies? • At least it collects together some best practices

  14. A project’s quality plan • Should be defined at an early stage of the project • Can use quality factors to determine which parts of the quality manual are particularly appropriate to this project • Is portability important? • How long will this product live? • Define activities to be performed. Answer questions for each activity: • What is checked and against what? • Who performs the check and who approves the checks? • Who takes the final decision if there is controversy? • How are results recorded? • How are corrective actions controlled?

  15. Quality controls: The general process 1 Define things which are considered ‘good quality’. 2 Define the procedure for checking quality. 3 Carry out the procedure on a specified item. 4 Record the result. 5 Take and record any corrective action • Regarding both the item and the quality system

  16. Group Project Standards • A cut down version of what you might see in industry • They cover the following areas: • QA.01 Quality plan - see previous slide on ISO-9001 Quality planning • QA.02 Project management - how we will run the project • QA.03 Documentation - what they will cover/look like • QA.05 Design Spec standards • QA.06. Test Spec standards • QA.07 Review standards - will cover now... • QA.08 Configuration management • QA.09/10 Coding standards for Java/C# • QA.11 Final report / maintenance documentation

  17. An important quality control:the review • Bugs are caught more efficiently by review than by testing • A review is… • A meeting at which some aspect(s) of the project are inspected and assessed by a group of people • All reviews have an element of formality, and their outcome is minuted • Formal Technical Review • To identify defects in a workproduct (design document, code listing, …) • Walkthrough • To identify defects in a workproduct, but with the emphasis on learning and evaluating alternatives - less formal

  18. Formal Technical Review: process • Author submits document or code to the review convenor • Briefly examined before accepting it for review • Review preparation • Review team formed • Criteria formed by which the document/code will be examined • All necessary documents sent to reviewers, who read them • The review meeting (2 hours maximum) • Issues (possible defects) are detected, but not corrected • Follow-up: document changed in response to issues highlighted in review meeting, and resubmitted if necessary

  19. Documents needed for a review • The item under review! • The document(s) from which the item is derived • For test spec: requirements, user interface design, design? • For code: detailed design (or for us, list of classes/attributes/methods) • The relevant quality plan documents • A review checklist: questions to answer • Summarises what to look for when reviewing • Derive from the quality plan documents and (to an extent) from the documents from which the item is derived • Try to fit on one sheet of paper, to concentrate on major issues

  20. Example review checklist: design • Do the reviewers understand the design? • Will the design meet the requirements? • Does the design satisfy the relevant standards? • Can the design be simplified? • Will the design lead to a maintainable system? • Could the design be modified so as to offer more opportunity for reusing existing software? The more specific the quality plan, the easier to create checklists SE.QA.07 has a more specific design checklist for group project.

  21. Example review checklist: code • Are all variables initialised before their values are used? • Have all constants been named? • Is each loop certain to terminate? • Are upper and lower bounds of arrays sensible? • Have pointers been correctly reassigned after modification? • Does code conform to QA code standards for the language?

  22. The review meeting • The item being reviewed is ‘walked through’ from start to end • Issues are highlighted as they occur • Includes issues found before, as well as during, the meeting • The author evaluates each issue, and… • Accepts as a defect, or explains why not a defect, or answers a question which a reviewer has • Minutes are taken by the scribe (QA Manager for us!)

  23. Conclusions • QA is the management of doing things in such a way to ensure that end results meet the users requirements. • Steps must be appropriate & economically realistic. • Aim is not highest possible quality, but is a compromise between • Time • Cost • Quality • Scope

More Related