1 / 78

SW Quality

SW Quality. Aimed at Technical Level Software Engineers, Univessity Students 50 Minute session 1500-1540. HOW TO DEFINE SOFTWARE QUALITY. 8. Quality Quantification. 8. Quantify. They Say Al Said. “Not everything that can be counted counts, and not everything that counts can be counted.”

sandra_john
Download Presentation

SW Quality

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. SW Quality Aimed at Technical Level Software Engineers, Univessity Students 50 Minute session 1500-1540 www.Gilb.com Tom & Kai Gilb

  2. HOW TO DEFINE SOFTWARE QUALITY www.Gilb.com Tom & Kai Gilb

  3. 8. Quality Quantification

  4. 8. Quantify They Say Al Said “Not everything that can be counted counts, and not everything that counts can be counted.” Falsely attributed to Albert Einstein (according to Calaprice, Expanded Quotable Einstein, 2000) I agree. But this does not include system qualities. Qualities can be quantified. Tom

  5. How to Quantify Quality(how to find a Scale) Goal Use known quantification ideas Do 8. Quantify Modify known quantification ideas to suit your current problems Study Use your common sense and powers of observation to work out new measures Act Learn early, learn often, adjust early definitions • CROSBY97: Alfred W. Crosby, The Measure of Reality. • Quantification and Western Society 1250-1600, ISBN 0-521-63990-5 (paperback), 1997, Cambridge University Press, 245 pages

  6. ‘Environmentally Friendly’ Quantification Example Give the quality a stable name tag Environmentally Friendly Define approximately the target Ambition:A degree of protection ……. 8. Quantify Define a scale of measure: Scale: % change in environment Decide a way to measure in practice. Meter: {scientific data…} Define benchmarks. Past[2001] +50% <-intuitive Record [2001, ….] 0% Trend [2003,…] -30% Define constraint & targets. Fail[next year] +0% <-not worse Goal +5 years, ….] +30%<-TG Wish [2004,…] +50%<-Marketing

  7. Exercise: Aspects of Love, orLove is a many splendored thing! 8. Quantify • Make inventory of love’s many aspects • Quantify one requirements for love • Duration: 6 minutes

  8. Love Attributes 8. Quantify • Kissed-ness • Care • Sharing • Respect • Comfort • Friendship • Sex • Understanding • Trust Support Attention Passion <-Matheus Satisfaction<-Mick Jagger ... ... ...

  9. Trust [Caroline] 8. Quantify • Love.Trust.Truthfulness • Ambition: NO LIES • Scale: Average Black lies/month from [defined sources]. • Meter: independent confidential log from sample of the defined sources. • Past Lie Level: • Past [My Old Mate, 1999] 42 <-Bart • Goal [My Current Mate, Year = 2000] Past Lie Level/2 • Black: Defined: Non White Lies • Other aspects of Trust: • Broken Agreements • Late Appointments • Late delivery • Gossiping to Others Go to the Camaraderie Example

  10. Devices to help quantifying quality ideas:Standard Hierarchy of Concepts (& Scales)from Gilb: Principles of Software Engineering Management. 8. Quantify QUALITY ADAPT- ABILITY WORK- CAPACITY USABILITY AVAIL-- ABILITY MAINTAINABILITY RELIABILITY 1. PROBLEM RECOGNITION 6. QUALITY CONTROL 7. DO THE CHANGE 2. ADMINISTRATIVE DELAY 3. TOOLS COLLECTION 8. TEST THE CHANGE 4. PROBLEM ANALYSIS 9. RECOVER FROM FAULT 5. CHANGE SPECIFICATION

  11. Devices to help quantifying quality ideas:ISO - 9126, Software Engineering - Product QualityInternal Metrics, External Metrics, Quality in Use Metrics 8. Quantify • Usability • Understandability • Learnability • Operability • Attractiveness • Compliance • Maintainability • Analyzability • Changeability • Stability • Testability • Compliance • Functionality • Suitability • Accuracy • Interoperability • Security • Compliance • Portability • Adaptability • Installability • Replaceability • Co-existence • Compliance • Reliability • Maturity • Fault Tolerance • Recoverability • Compliance • Efficiency • Time behaviour • Resource Utilization • Compliance

  12. Devices to help quantifying quality ideas:Using ‘Parameters’ when defining a Scale of Measure.The Scale is more flexible and re-usable! 8. Quantify • Using [qualifiers] in the Scale definition gives flexibility of detailed specification later. • Example • Scale: the % of • [defined Users] • using [defined system Components] • who can successfully accomplish [defined Tasks] CE: "Competitive Engineering" result-planning.com & Addison-Wesley (2000) Goal [ Users =NOVICES, Components = USER MANUAL, Tasks = ERROR CORRECTION ] 60%

  13. Quality Quantification

  14. How toQuantify any Qualitative Requirement 8. Quantify Diagram courtesy of Lindsey Brodie, Editor of Competitive Engineering. May 2000

  15. A ‘Quality Quantification’ Principle 8. Quantify He had a lot of hats. He wants to be best in hatmanship. • 0. THE PRINCIPLE OF • 'BAD NUMBERS BEAT GOOD WORDS' • Poor quantification is more useful than none; at least it can be improved systematically. Scale: hats on his head. Past:3 Goal: 13 General Hatmanship: Ambition: improve ability to have hats on head and nearby Hatmanship On Head: Scale: hats on top of persons head Past [Me, This year] 10 <- Guess Record [Guinness 96, UK] 15 <- GB Record Wish [Guinness Record, April] 20 <- Tom Hatmanship Nearby: Scale: hats not on head, but on, or near, body; within 10 meter radius. Past…. Goal……..etc.

  16. 10. Principles for Quality Quantification 8. Quantify10PRIN • Some hopefully deep and useful guidelines • to help you quantify quality ideas 0. Bad numbers beat good words 1. Quality Quantification: all qualities can be expressed quantitatively 2. ‘Many splendor things’ 3. Scalar definition 4. Threats are measurable 5. Limits to detail 6. Meters matter 7. Horses for courses 8. Benchmarks 9.Numeric future

  17. 0. THE PRINCIPLE OF 'BAD NUMBERS BEAT GOOD WORDS’ (re-visited!) 8. Quantify10PRIN • Poor quantification is more useful than none; • at least it can be improved systematically. State of the Art Flexibility Enhanced Usability Improved Performance Not Clear!

  18. 1. THE PRINCIPLE OF 'QUALITY QUANTIFICATION' 8. Quantify10PRIN • All qualities can be expressed quantitatively, • 'qualitative' does not mean unmeasurable. “If you think you know something about a subject, try to put a number on it. If you can, then maybe you know something about the subject. If you cannot then perhaps you should admit to yourself that your knowledge is of a meager and unsatisfactory kind.” Lord Kelvin, 1893

  19. 2. THE PRINCIPLE OF 'MANY SPLENDORED THINGS' 8. Quantify10PRIN • Most quality ideas are usefully broken into several measures of goodness. Usability: Entry Qualification: Scale IQ, ……. Learning Effort: Scale: Hours to learn, ….. Productivity: Scale: Tasks per hour,……. Error Rate: Faults per 100 tasks, ….. Like-ability: % Users who like the system, ….

  20. 3. THE PRINCIPLE OF 'SCALAR DEFINITION' 8. Quantify10PRIN • A Scale of measure is a powerful practical definition of a quality Flexibility:Scale: Speed of Conversion to New Computer Platform

  21. 4. THE PRINCIPLE OF 'THREATS ARE MEASURABLE' 8. Quantify10PRIN • If lack of quality can destroy your project • then you can measure it sometime; • the only discussion will be 'how early?'.

  22. 5. THE PRINCIPLE OF 'LIMITS TO DETAIL' 8. Quantify10PRIN • There is a practical limit to the number of facets of quality you can define and control, • which is far less than the number of facets that you can imagine might be relevant.

  23. 6. THE PRINCIPLE OF 'METERS MATTER’ 8. Quantify10PRIN • Practical measuring instruments • improve • the practical understanding • and applicationof ‘Scales of measure’. Portability: Scale: Cost to convert/Module Meter [Data] measure/1,000 words converted Meter [Logic] measure/1,000 Function Points Converted

  24. Why Meters in Requirements? 8. Quantify10PRIN • Defines in practice how we intend or propose to measure and get feedback • Not absolutely necessary to state a requirement • Does help stakeholders ‘buy in’ to this level and cost of ‘testing’ • Allows us to better estimate cost and time and people necessary for the necessary testing • Sends a message: ‘take this requirement seriously (in design etc) because we are really going to measure that it is delivered.

  25. 7. THE PRINCIPLE OF 'HORSES FOR COURSES' 8. Quantify10PRIN • Different quality-Scale measuring processes • will be necessary for • different points in time, • different events and • different places. Availability: Scale: % Uptime for System Meter [USA, 2001] Test X Meter [UK, 2002] Test Y

  26. 8. THE PRINCIPLE OF 'BENCHMARKS' 8. Quantify10PRIN • Past history and future trends help define words like "improve" and "reduce". Reliability Scale: MTTF Past [Brighton, 1999] 30,000 Hours Trend [Nato Allies, 2003] 50,000 Hours Goal [UK MOD, 2003] 60,000 Hours

  27. 9. THE PRINCIPLE OF 'NUMERIC FUTURE' 8. Quantify10PRIN • Numeric future requirement levels complete the quality definition of relative terms like 'improved'. Usability: Scale: Time to learn average task. Past [Old product, 1999] 20 minutes Wish [New product, 2002] 2 minutes Fail [End 2000, Students] 15 minutes Goal [End 2003, Teachers] 5 minutes

  28. 8. Quantify A Corporate Quality Policy should include quantification (Ericsson) 1. QUANTIFY QUALITY 7. CONTINUOUS WORK PROCESS IMPROVEMENT 2. CONTROL MULTIPLE DIMENSIONS Ericsson Policy 3. EVALUATE RISK 6. EVOLUTIONARY DELIVERY CONTROL 5. DOCUMENT QUALITY EVALUATION 4. CONFIGURATION MANAGEMENT - TRACEABILITY

  29. 8. Quantify (Real) Policy on QUANTIFICATION, CLARIFICATION AND TESTABILITY OF CRITICAL OBJECTIVES: “All critical factors or objectives (quality, benefit, resource) for any activity (planning, engineering, management) shall be expressed clearly, measurably, testably and unambiguously at all stages of consideration, presentation, evaluation, construction and validation. “ <- (Quality Manual Source is) 5.2.2, 4.1.2, 4.1.5, 5.1.1, 6.1, 6.4.1, 7.1.1, 7.3 and many others.

  30. SOFTWARE TEMPLATES www.Gilb.com Tom & Kai Gilb

  31. Requirement Hierarchy and Scale Templates • These templates serve these purposes: Instant Teaching More intelligible than ‘Rules’ Reminds us of rules for specific spec types • Example Scale/Benchmark/Targets/Constraints for Performance/Quality/Resource requirements Reminds us of opportunities (the Parameters) Reminds us of exactly what specification is required (<the hints> ) Reminds us of what we have not specified yet Gives us a standard recognizable set of parameters and parameter sequence

  32. 1. Performance: useful variable value deliverable to stakeholders Performance Attributes Resources Qualities (How well) System Mission Capacities (How Much) Savings (How much resource saved) • 1.1 Qualities. ‘How well a system performs’ • 1.2 Capacity. ‘How much a system performs’ • 1.3 Savings. ‘How relatively cheaply a system performs’

  33. Performance. Concept *434. September 13, 2001 • Performance is the set of attributes, for a function, which describe its output characteristics before consequential effects on other systems are considered. • Related Concepts. •  Performance Levels: are the numeric levels, specified or observed, for any defined scalar performance attribute. These are classed as Benchmarks or Targets. •  Measures of Performance: MOPs: are the defined Scales of measure, we have defined, to describe the effectiveness of a system or function. • Workload (Synonym: Work Capacity) *459: ability of a system to do work in defined time or space. • Savings *429 • Quantity *437: quantified measures of Workload and Saving, excluding Qualities *125. They describe ‘how much’ a system can do. • Qualities, *125: measures of performance which describe ‘how well’ they system performs. They include what most people would characterize as qualities, such as reliability, usability, security, adaptability. • Effectiveness *053

  34. 1.1 Qualities: 1st level subset(all are potentially Complex Attributes) • 1.1.1 Availability: • Scale: • % of defined [Time Period] • a defined [System] • is [Available] • for its defined [Tasks] • 1.1.2 Adaptability: • Scale: • time needed to [Adapt] • a defined [System] • from a defined [Initial State] • to another defined [Final State] • using defined [Means]. • 1.1.3 Usability: • Scale: • Speed for defined [Users] • to correctly accomplish defined [Tasks] • when given defined [Instruction] • under defined [Circumstances].

  35. 1.1.1 Availability: % ‘Up’ time. The readiness of a system to do its work. • 1.1.1.1 Reliability: • Scale: • Mean time • for a defined [System] • to experience defined [Failure Type] • under defined [Conditions]. • 1.1.1.2 Maintainability: • Scale: • Mean time • to do a defined type of [Repair] • to a defined [System] • using defined [Repair Means] • under given [Conditions]. • 1.1.1.3 Integrity: • Scale: • Probability • for a defined [System] • to [Cope] with defined [Attacks] Cope= {e.g. detect, prevent, capture}. • under defined [Conditions].

  36. 1.1.2 Adaptability:‘The ease with which a system can be changed’ • 1.1.2.1 Portability: • Scale: • the cost to transport • a defined [System] • from a defined [Initial Environment] • to a [Target Environment] • using defined [Means]. • 1.1.2.2 Extendability: • Scale: • the cost to add • to a defined [System] • a defined [Extension Class] • and defined [Extension Quantity] • using a defined [Extension Means]. • 1.1.2.3 Improvability: • Scale: • the cost to add • to a defined [System] • a defined [Improvement] and • defined [Quantity] • using a defined [Means].

  37. 1.1.3 Usability: ‘How easy a system is to use’ • 1.1.3.1 Entry Requirement: aka :Usability.Entry Requirement. • Scale: • the defined [Level of Knowledge] • required to receive training • or to use a defined [System]. • 1.1.3.2 Learning Requirements: • Scale: • the degree of training required • for a defined [Type of person] • to achieve a defined [Proficiency] • with a defined [System]. • 1.1.3.3 Handling Ability: • Scale: • a defined degree of [Proficiency] • with a defined [System] • by a defined class of [User]. • 1.1.3.4 Like-ability: • Scale: • the degree to which defined [Users] • declare that they are pleased with defined [Aspects] of a System.

  38. 1.2 Capacity • 1.2.1 Throughput: • Scale: • the quantity of defined [Work Units] • which can be successfully handled per [Time Unit]. • 1.2.2 Response: • Scale: • the mean speed of defined [Reaction] • to a defined [Impulse]. • 1.2.3 Space: • Scale: • the capacity to store defined [Units] • under defined [Conditions].

  39. 1.3 Savings • 1.3.1 Financial Savings: • Scale: • net Financial savings planned or achieved • compared to a defined [Benchmark Amount]. • 1.3.2 Time Savings: • Scale: • net Time savings planned or achieved • compared to a defined [Benchmark Amount]. • 1.3.2 Effort Savings: • Scale: • net Effort savings planned or achieved • compared to a defined [Benchmark Amount]. • 1.3.3 Space Savings: • Scale: • net Space savings planned or achieved • compared to a defined [Benchmark Amount].

  40. Note: • a detailed treatment of these scale templates is presented in [Gilb88] chapter 19, • a copy of which will be found at www.result-planning.com.The copy has a lot more explanatory text and examples.

  41. 1.1.1.2 Maintainability:Performance.Quality.Availability.Maintainability. • Scale: • Mean time to do a defined type of [Repair] • to a defined [System] • using defined [Repair Means] • under given [Conditions].

  42. A detailed view of Maintainability: • Maintainability: • Type: Complex Requirement. • (so the following slides will define that detail structure)

  43. Maintainability.Problem Recognition: • Type: Elementary Quality requirement. • Scale: • Clock hours from defined [Fault Occurrence: Default: Bug Occurs in Any use or test of system]] • until fault officially recognized by defined [Recognition Act: Default: Fault is Logged Electronically].

  44. Maintainability.Administrative Delay: • Type: Elementary Quality requirement. • Scale: • Clock hours from defined [Recognition Act] • until defined [Correction Action] • initiated • and assigned • to a defined [Maintenance Instance].

  45. Maintainability.Tool Collection: • Type: Elementary Quality requirement. • Scale: • Clock Hours for defined [Maintenance Instance: Default: Whoever is assigned.] • to acquire all defined [Tools: Default: all systems and information necessary to analyze, correct and QC correction]

  46. Maintainability.Problem Analysis: • Type: Elementary Quality requirement. • Scale: • Clock Time for the assigned [Maintenance Instance] • to analyze the fault symptoms • and be able to begin to formulate a correction hypothesis.

  47. Maintainability.Correction Hypothesis: • Type: Elementary Quality requirement. • Scale: • Clock hours needed by defined [Maintenance Instance] • to fully and correctly describe the necessary correction actions, • according to current applicable standards for this. • Note [Scale]: this includes any additional time for corrections after QC and Tests.

  48. Maintainability.Correction Spec Quality Control: • Type: Elementary Quality requirement. • Scale: • Clock hours for the necessary correction-hypothesis quality-control; • against available-standards for the correction-activity.

  49. Maintainability.Correction Activity: • Type: Elementary Quality requirement. • Scale: • Clock Hours from beginning to correct ending, • to actually carry out the correction activity as planned. • Including any necessary corrections as a result of SQC or testing.

  50. Maintainability.Unit test: • Type: Elementary Quality requirement. • Scale: • Clock Hours to carry out required unit tests • for a single fault correction.

More Related