Understanding Complexity and Quality Models

1. Useful Quality Models Goal of presentation • to deepenyour understanding of complexity and quality requirements including their specification, achievement, monitoring, and verification • to show that complex quality models can be useful David Gelperin david@clearspecs.com

2. Understanding Complexity • Complex means difficult to understand by people of average intelligence • Simple means easy to understand by people of average intelligence • We are biologically programmed to prefer simple, • but we often overdo it • Simplistic means too simple e.g., female identification by male jewel beetles • Simplistic sells i.e., widely-accepted may mean too-simple

3. Oversimplification Bias • Psychologists say we prefer simplistic solutions • We are mentally addicted to few, unconditional rules • Software process advice (and political rhetoric) is filled with these • Advantages of small-scale mental models may help explain our simplistic bias • In addition, thehuman (and male jewel beetle) capacity for denial is infinite

4. We often oversimplify Ideas should be made as simple as possible -- but no simpler. paraphrasing A. Einstein For every complex problem there is an answer that is clear, simple, and wrong. H. L. Mencken

5. Reality is more complex than we imagine i.e., Occam may be wrong Requirements development is much more complex than we imagine

6. Models • A model represents aspects of its subject. • H2Ois a model of a water molecule. It identifies the constituent atoms and their counts.Another model shows relative size and orientation of these atoms as well. • A simplistic model omits useful elements of the aspects it represents and/or misrepresents these aspects. For example, ISO25010 is a simplistic model of quality attributes. Functional and non-functional requirements is a simplistic model of requirements types.

7. US tax code Rube Goldberg devices Complex models are not simple Spaghetti code sufficient, but partially unnecessary Unnecessary Complexity necessary and sufficient Complexity (3 kinds) Essential Complexity Simplicity London Underground General Relativity Benzene ring DNA 7

8. Complexity of Requirement Types necessary and sufficient Essential Complexity UR Requirements Model (19) Simplicity Complexity necessary, but insufficient and/or defective SEBOK Requirements Model (2) Insufficient Complexity (Simplistic, Naive) 8

9. Complexity of Quality Models necessary and sufficient LiteRM Quality Model (> 30) Simplicity Complexity Essential Complexity necessary, but insufficient and/or defective Insufficient Complexity (Simplistic, Naive) ISO 25010 et. al. (1)

10. Complexity of Requirements Development necessary and sufficient Basic RD pattern (elicit +18) Simplicity Complexity Essential Complexity necessary, but insufficient and/or defective SEBOK Requirements Process (elicit) Insufficient Complexity (Simplistic, Naive) Requirements development is rife with simplistic models

11. Managing Risk “Risk management is project management for adults” -- Tom DeMarco & Tim Lister Risk management tactics When feasible, simplify unnecessarycomplexity e.g., refactor code When necessary, complexify the simplistic models and methods Simplistic models limit understanding You can’t manage what you don’t understand

12. You knowYou don’t know e.g., my birthdate e.g., your birthdate What you know What you don’t know simplistic ain’t so is so essentially complex, but currently simplistic e.g., number of red stripes on a US flag undiscovered ideas e.g., perceptrons

13. Quality requirements A quality goal (requirement) is a specified achievement level of a quality attribute e.g., security Since quality attributes can conflict,e.g., security and usability, tradeoffs may be necessary Quality models are embedded in requirements models Quality goals (requirements) should be derived from useful quality models

14. An essentially-complex requirements model Not all types are necessary in every set of requirements Note major role played by developers

15. Crosscutting requirements Some requirements (e.g., supplier attributes or project deadlines) impact no code. Local requirements(such as the CRUD functions for domain objects e.g., reservations) only impact a few components. Crosscutting requirements(such as most quality support functions e.g., safeguards) impact code in many components. 3

16. Types of crosscuts Universal crosscuts (e.g., coding standards) impact all code. Homogeneous crosscuts (e.g., logging functions) do the same thing wherever they appear. Heterogeneous crosscuts (e.g., security guards) do different things wherever they appear, because they are context-sensitive. 4

17. Domain functionality is just the beginning A component To minimize quality debt, crosscutting requirements should be identified early 5

18. Developing quality-support strategies - 1 • Developers must: • Selectrelevant qualities. • Identify their supporting qualities and potential conflicts. • Specifygoals for each selected quality along with levels and priorities. • Identifyup to three candidate architectures based on the software’s core functionality as well as its contextual elements, constraints, and qualities. • For each architectural candidate, outline achievement, monitoring, and verification strategies for the high-priority qualities until a single architecture emerges. 6

19. Developing quality-support strategies - 2 • For the selected architecture, designan achievement, monitoring, and • verification strategy for each selected quality and estimate their effort. • Reference stringent design and coding standards, promoting software understandability, whose compliance will be verified. • When appropriate, develop a glossary of precise definitionsfor domain concepts, verifiable quality measures, and terminology in the achievement and verification strategies. • For critical qualities and their supports, use assurance casesto verifying the adequacy of the achievement and verification strategies. • Carry outthe strategies to achieve, monitor, and verify each selected quality on each development increment. 7

20. Quality Requirements UpFront (QRUF) QRUF development means doing as much of the first six tasks as possible at the beginning of a project so the last task can be carried out on each developmental increment. This tactic mitigates the pervasive risk of significant quality debt. A tailored version of the LiteRM quality model helps with strategy development tasks 1, 2, 3, 5, and 6. If a team is NOT using a tailored version of this model,they are working too hard ornot hard enough. 8

21. What makes a quality model useful? Significant support for4quality requirement development tasks: • Selectrelevant qualities. • Identify their supporting qualities and potential conflicts. • Specifygoals for each selected quality along with levels and priorities. • For the selected architecture, designan achievement, monitoring, and verification strategy for each quality and estimate their effort.

22. BABOK quality model • Availability • Compatibility • Functionality • Maintainability • Performance Efficiency • Portability • Reliability • Scalability • Security • Usability • Certification • Compliance • Localization • Service Level Agreements • Extensibility (Almost) alphabetical list of 15 qualities

23. ISO 25010 2-Dquality model • Internal qualities (which support all others) are tucked under Maintainability • understandability – missing • reviewability – missing • verifiability - missing

24. Quality models for certification IIBA – CBAP based on list 9000 CBAP certified QAI– CSQA based on list 22000 to 25000 CSQA certified world-wide IREB – CPRE based on list, but references ISO model 56700 in 84 countries have passed the CPRE Foundation Level ASQ – CSQE based on list, but references ISO model 1500 CSQE certified world-wide 90,000 professionals certified based on simplistic quality models

25. SEI 2-D quality model PART TWO QUALITY ATTRIBUTES CHAPTER 4Understanding Quality Attributes (19) CHAPTER 5Availability (25) CHAPTER 6Interoperability (15) CHAPTER 7Modifiability (15) CHAPTER 8Performance (17) CHAPTER 9Security (13) CHAPTER 10Testability (17) CHAPTER 11Usability (11) CHAPTER 12Other Quality Attributes (19)

26. Wiegers & Beatty 3-D quality model

27. LiteRM 3-D quality model a internal qualities directly visible only to developers external qualities fully visible to users mixed qualities partially visible to users [think icebergs] a a

28. Basic internal qualities support all others Runtime self-checking Mainly verified by reviews, analysis, and measurement 3 pillars of internal quality

29. Some external & mixed qualities Primarily verified during system test Needs full range of verification tactics

30. Support relationships between types Basic internalssupportall other qualities including themselves Some externalssupport some mixed

31. Common characteristics and values [1 of 4] • DefinitionSafety: A system’s ability to do little or no harm to valuable assets • Software subfield Safety engineering • Assumptions/Rationale • Safety is a fragile quality because it depends on many other qualities and the accurate identification of safety hazards. • Leading indicators– provide preoperational evidence of quality goal achievement • Ratio of hazards added during HA technical review to hazard count after HA technical review • Ratio of safeguard defects found during a technical review to number of safeguards • Ratio of safeguard defects found during testing to number of safeguards • Operational measures • Number of “dangerous” failures or defects detected per time interval • Greatest harm from a harmful event • Ratio of actual loss to acceptable loss in a duration

32. Common characteristics and values [2 of 4] • Supported bydependability, ease of learning • Conflicts withefficiency, interoperability, adaptability qualities • Threats [identify using hazard analysis] • Mitigations [identify after identifying hazards] • Other achievement tactics • identify safety-critical users • identify valuable assets and hazards • identify safety-critical and safety-related functions and constraints • isolate and protect safety-critical functions • guard safety-critical functions with explicit conditions • alert users to dangerous actions with rotating warning messages • precede each dangerous action with a delay so users can change their minds and cancel • design interfaces that prevent and detect user errors

33. Common characteristics and values [3 of 4] • Verification tactics • verify all supporting qualities • conduct a safety audit to review hazards and mitigations for completeness and effectiveness • thoroughly test each safeguard • track time since last “dangerous” failure or defect • track number of “dangerous” failures or defects • monitor for misbehavior e.g., unanticipated acceleration • monitor system state to make sure safety-critical and safety-related functions are active • Elicitation questions • Associated tools • Resources[pointers] • Risk factors • a. Developer understanding = [superficial, limited, deep] • b. Cost of implementation, verification, & maintenance = [high, medium, low] • c. Feasibility (technical, cost, understanding) = [low, medium, high]

34. Resourcesfor qualities with large “bodies of knowledge”

35. Common characteristics and values [4 of 4] Other features a. Sources/Enterprise goals: b. Type = mixed behavior quality c. Associated scope = [system, <specific partitions>] d. Design scope = het cc [het cc, hom cc, universal, local] e. Consensus priority = [critical, important, desirable] f. Architecture-relevant = yes [yes, maybe, no] Past quality goal specs Past achievement, verification, and monitoring strategies Current quality goal spec Current achievement, verification, and monitoring strategies

36. Features of the LiteRM quality model • 3-dimensional • dynamic • tailorable • improvable • embedded quality goals & strategies • essentially complex • mildly defective • incomplete,but provides a rich template • and many examples of characteristics • supporting the quality achievement tasks

37. Fun is big business Video game industry earns over $100B USD per year A target market ? $90B USD lifetime 65% of American adults play video games

38. Example of Planguage Quality Measure

39. Comparing usefulness of quality models

40. Resources http://www.quality-aware.com/software-quality-KB.php 1. Downloadable LiteRM quality model (pdf) with > 60 qualities > 30 characteristics per quality 2. Downloadable chapter on quality goals (pdf)

41. Thanks For Listening Contact Dave with questions, defects, and comments E-mail: david@clearspecs.com Phone: +1 480-296-3559