Quality Management Fall 2005

1. Quality ManagementFall 2005 by Eva Trosborg Agenda What is quality ? Process-, product- and need based quality Definitions of quality Quality review: What and how? Example: Review of a requirement specification Measurement from DK and US

2. After this lesson you should : Be able to discuss quality management as well as plan and implement quality reviews in a smaller project.

3. Quality Definition • Quality is fitness for use – J.M.Juran • Quality is meeting or exceeding customer expectations at a cost that represent a value to them – H. James Harrington • Quality should be defined as surpassing customer needs and expectations throughout the life of the product – Howard Gitlow and Shelley Gitlow • Quality is conformance with requirements Philip Crossby

4. We want quality! • Because we want to make a good piece of work • Because we are tired of making errors • Because it is boring to search for errors • Because it is cheaper and quicker to do it right first time • Because quality make our customers happy

5. Four types of quality Process based Quality Product based Quality Need based kvalitet Value based Quality

6. time1 time2 Wish New requirement Need Development process Core Interpretation Commu nication Product Need Wish Requirement specification Need Wish

7. Product based quality time1 time2 Wish New need Need Development process Core Interpretation Commu- nication Product Need Wish Requirement specification Need Wish Need based quality Process based quality

8. What is the quality work in a project ? • Use of defined methods and tools (process- quality) • Running reviews (process- and product-quality) • Quality Measurements (non-functional requirements) • Managing changes from a quality measurement point of view • Measurement on the way (process) and final (product and need) • Systematic reporting of quality (based on measurements)

9. Do quality payoff?

10. Pareto Analysis This is based on the notion that, in many situations, 80% of incidents are accounted for by 20% of the causes. It gets its name from an Italian statistician who studied the phenomenon in detail. • Also known as ‘The 80:20 Rule’, it is not always exactly split 80 to 20, but the general principle is remarkably common. What it does is encourage the circle members to find out which of the causes have the greatest effect. • For example, you could find in your organisation that: • 80% of the turnover comes from 20% of the products • 80% of the complaints come from 20% of the customers • 80% of the absenteeism is accounted for by 20% of the workforce • 80% of the errors are produced on 20% of the machines As you probably found, it is not a clean 80:20 split in each case, but there is frequently a relationship between the majority of the problems and a small minority of the causes.

11. Quality Management • Managing quality from the first idea through development, marketing and implementation to the final product • Quality management is a top-management responsibility. It consists of a decision of Quality Level and features – and assuring that the IT products lives up to the agreed

12. Quality Management Quality Management includes: • Agreement of quality goals / quality policy (strategic decision) and • Which are fulfilled in practice by means of Quality Management Kilde:(DS/EN ISO 9004-1: 1994, Dansk introduktion, side 2)

13. PLAN - DO – CHECK – ACT Plan, specify & decide evaluate do registre/ measure

14. Quality Management = Quality Planning + Quality measuring + Quality control

15. Quality Planning • Specification of requirement for the required quality for the IT-products • Decision of IT-processes, plans, procedures concerning time, place, phases, people etc.

16. Quality control Activities aiming at evaluating if the requirement specification conforms to specifications. The Quality Control consists of: • Measuring and registration • Evaluating aiming at corrective actions

17. Quality Assurance • Measurements and registration • Evaluation, compare and work to secure corrections are performed if needed Activities performed with the objective to evaluate if IT-process are performed with respect to the decided plans and procedures. It consist of:

18. Review A review is a meeting where the quality of a product are evaluated. The goal is: • To point out improvements • To accept a product • To secure equal quality in each part • To educate the participants The purpose is to point out problems not to solve them

19. Some says about Review’s • A method involving a structured encounter in which a group of technical personnel analyzes or improves the quality of the original work product as well as the quality of the method (Johnson 1998) • An inspection intended to expose defects in the product (Kendall 1992) • A “filter” for the software engineering process (Pressman 2000) • Gives the analyst or programmer an honest appraisal of the system (Edwards 1993)

20. Phase’ and roles in a review Roles: • Chairman • writer • note taker • Critics Six Phases: • Planning • Information meeting • Preparation • Review meeting • Correction • Ending

21. Plan Inform Prepare Review meeting Correct End 6 phases in a general review process • Choose a group, material and date • Present the product, process and target • Check product, note findings • Present and consolidate findings • Correct the finding • Verify corrections, end the process

22. Planning Inform Prepare Review Plan Correct End • Objectives • Put together the package to be reviewed: product, checklist, references, data, standards • Decides who to attend • Set a date and place • Procedure • Chair • Decides group and review package • Fits in checklists (if needed) • Plans for dates and meetings (information and review) • Checks if the product is ready for review? • Helps the writer to prepare information

23. Information Inform Prepare Review Plan Correct End • Objectives • Writer gives an overview • The ones to review gets the “package” • Objective and preparation fixed • The ones to review accepts • Procedure • The chair gives out the review package • The writer informs if needed • The role a secretary (who) decided • Chair informs how the preparation should happen

24. Prepare Inform Prepare Review Plan Correct End • Objectives • Find as many errors a possible • Procedure • Allocate time for preparation • Each participant makes a review individually Use checklists, references and standards as focus • Note everything found

25. The review meeting Inform Prepare Review Plan Correct End • Objective • To create a list containing all findings • To create a forum where(2+2=5) • To improve the ability to review by watching what others do • Create a understanding of the product • Procedure • Chairman leads the meeting page for page paragraph for paragraph • Findings presented • Secretary notes findings – if possible in a manner so participants can follow the process. • At the end the findings are categorized on type and how serious

26. Examples of categories • Critical • Defects which gives some wrong results or dangerous. Hang-ups or break downs, no known work around. • Serious • Defect which leads to ambiguous results or wrong behavior. Several requirements not fulfilled but required a reprogramming. Something in a program behaving wrongly but where there is a (”work around”) to omit the error. • Moderate • Defects only concerning a single requirements or a small part of the system. The defect can be ignored and do not have much consequences • Minor • Defects which makes it difficult to understand small thing or cosmetics

27. Example categorizing depending on TYPE • Example (need) – Missing example to understand • Figure (need) - Missing example to understand • Logical (error) – non conformance to previous work • Missing – Something missing in the document/program • To much – Something which is written twice • Reference (need) – Source • Standard (not respected) – A standard is not respected • back – Defect in previous document/program, which is a prerequisite for what being reviewed. • Not exact enough – can be misunderstood

28. Correct Inform Prepare Review Plan Correct End • Objective • Evaluate each findings (if this was not done on the review meeting). If an important finding correct or remove • For less important findings (moderate or minor) notes that is was found, but it might wait for a letter release.

29. End Inform Prepare Review Plan Correct End • Objective • Evaluate the quality of the corrected product • Evaluate the review process • Accept or discard the product • Procedure • Receive the corrected product • Check • Accept or discard • Inform the participant from the review meeting • Keep data regarding #, types and time (quality data)

30. Example: Review of requirement specification • To validate goal for the change (Why should the project be developed) • To validate that the non-functional requirements are well-defined and documented. • To validate that the described functions and data correspond to the requirements.

31. SMART Specific Measurable Achievable Reasonable Time constrained RUMBA Reasonable Understandable Measurable Believable Achievable Are the customers requirements? There can be invalid requirements

32. FURPS+The FURPS+ can be used as checklist for requirement coverage, to reduce the risk of not considering some important facets of the system • Functional – features, capabilities, security • Usability – human factors, help, documentation • Reliability – frequency of failure, recoverability, predictability • Performance – response times, throughput, accuracy, availability, resource usage • Supportability – adaptability, maintability, internationalization, configurability • The + • Implementation – resource limitations, languages and tools, hardware, … • Interface – constraints imposed by interfacing with external systems • Operations – systemmanagement in its operational setting • Packaging – for example, physical box • Legal – licensing and so forth Some of these requirements are collectively called quality attributes / quality requirements Larman p56 +

33. Supplementary Specifications • Non-functional requirements should after the use case be moved to the Special Requirement Section – to keep all non-functional requirements in one place, and not duplicated. • Quality Attributes • URPS+ usability, reliability, performance, supportability … these are qualities of the system, not of the attributes themselves. I.e supportability intended low due to … • Architectural analysis and design largely concerned with identification and resolution of quality attributes in the context of functional requirements Larman p 107 +

34. Quality Scenarios • Recommended as they define measurable (or at least observable) responses to be verified. (easy to modify needs some measure of what it means) • Quantifying some things, a performance goals and mean time between failure, are well known practices, quality scenarios extend this idea to recording most factors as measurable statements.

35. Pick your battles • Quality scenarios are short statements <stimulus> <measurable response> • When the completed sale is sent to the remote tax calculator to add the taxes, the result is returned within 2 seconds “most” of the time, measured in a production environment under “average” load conditions. “most” and “average” need further investigation and definition. A quality scenario is not really valid untill it is testable, which implies fully specified. Will anyone really test them? How and by whom?

36. Sample factor table Factor Measures and Quality scenarios Variability (current flexibility and future evaluation) Impact of factor (and its variability) on stakeholders, architecture and other factors Priority for success Difficulty or Risk Larman 547

37. Critical make-or-break quality scenario • Which are the important battles in your project? • Write those quality scenarios and follow through with a plan for their evaluation.

38. Service Level Agreement (SLA) / Service Level Management (SLM) • A Service Level Agreement (SLA) is a formal written agreement made between two parties: the service provider and the service recipient. The SLA itself defines the basis of understanding between the two parties for delivery of the service itself. The document can be quite complex, and sometimes underpins a formal contract. The contents will vary according to the nature of the service itself, but usually includes a number of core elements, or clauses. • Generally, an SLA should contain clauses that define a specified level of service, support options, incentive awards for service levels exceeded and/or penalty provisions for services not provided. Before having such agreements with customers the IT services need to have a good quality of these services, Quality management will try to improve the Quality of Service, whereas the SLAs will try to keep the quality and guarantee the quality to the customer. http://en.wikipedia.org/wiki/Service_Level_Agreement

39. SLA Management is the key for making sure you get what you paid for. • SLO (service level objectives) and KPI (key performance indicators). • SLO is then valued against the agreed-upon service levels, and the result is reported to the service provider and/or consumer.

40. Information Technology Infrastructure Library (ITIL) • is a set of best practices used to deliver high quality IT services. The best practices described in ITIL represent the consensus derived from over a decade of work by thousands of IT and data processing professionals’ world-wide, including hundreds of years of collective experience. Because of its depth and breadth, the ITIL has become the defacto world standard for IT best practices. http://www.itilsurvival.com/

41. Example: Review of requirement specification – Sources of errors • Misunderstandings in the communication • Requirement and measurements are not quantified • Missing formal information or documentation • Lack in understanding of where are we going • Misunderstanding of existing data • Missing understanding of relationships in data • Misunderstanding of examples and diagrams

42. Example - Review of requirement specification - documentation • Revised requirement specification • Descriptions of use (Use Cases) • Minutes of the meetings • Documentation of conflicts and solutions • List of demands and wishes being refused • List of riscs

43. Measurements from Lyngsø in Denmark • One manhour a page – including preparation, free-meeting and planning included. It is possible to review 7-9 pages an hour • Review meetings in the morging is most effective – this means more errors and ommisions are found an hour   Kilde: Bjørn Runge: Erfa-møde i Datateknisk forum om ”Lyngsøs erfaringer med reviews”

44. Measurements from IBM, USA • Review meeting lasting more than 2 hours are less effective • If more than 20 pages are to be reviewed then it is better to split the meeting. Kilde: Michael Fagan: ”Design og Code Inspections to reduce Errors in program development” IBM Systems Jn., vol. 15, nr. 3, 1976, side 182-211

45. Review of requirements with customer the 7th October in room 3A18 if review material received before the 4th October Two presentations: one for us as customers and one for us a part of the organization

46. See you again the 28th. OctoberObject oriented design and more patterns chapter 22 - 26Eva Trosborg