Sesam Seminarium 2000-10-19 Claes Norelöv claes.norelov@esavionics.se

1. Metoder och tekniker för granskning av OO-modeller - del som berör litteraturstudier och inhämtning av information inom Ericsson Sesam Seminarium 2000-10-19 Claes Norelöv claes.norelov@esavionics.se

2. Disposition • Frågeställningar • Klassificering av granskningstekniker • Experiment 1: Perspective Based Reading vs Checklist Based Reading (Laitenberger et al) • ”Reading Techniques” för granskning av OO-modeller (Travassos et al) • Experiment 2: Vertical & Horizontal Reading (Travassos et al) • ”Guided Inspection” (Melissa L. Major)

3. Frågeställningar • Vilka problem/brister finns idag genom att man granskar dokument? Dessa är ju enbart en vy av en OO-orienterad modell. • Hur skall man rent arbetsmässigt gå tillväga när man granskar modeller? Skiljer sig förfarandet vid granskning under olika faser av utvecklingen, vilka aspekter av modellen granskas? • Vilka krav ställs på konfigurationsstyrning? Vad utgör Configuration Items i modellen?

4. Frågeställningar, forts • Givet att man har tekniker för granskning av OO-modeller och arbetar direkt på modellunderlag i stället för dokument, hur hanterar man då kvalitetsaspekterna som man förmodligen åtagit sig enligt någon standard (ISO 9000-3, MIL-STD 498, ISO 12207, RTCA-DO 178B etc).

5. Frågeställningar, forts • Man har alltså att ta ställning till frågeställningar kopplade till följande processteg i en en granskningsprocess: • Planering av granskning • Genomförande av granskning (Lämpliga tekniker att granska modeller) • Återkoppling av granskningsresultat

6. Informationsinhämtning #1 • Frågeställningarna har varit vidarebefordrade i två nätverk • Ericsson Review and Inspection Academy • TTG Programvara inom Saab

7. Informationsinhämtning #2 • Litteratursökning har gjorts på webben. och ett antal träffar angående “reading techniques” har studerats: • University of Maryland Depatment of Computer Science (http://www.cs.umd.edu/projects/SoftEng/ESEG/) • Fraunhofer Center for Experimental Software Engineering, Maryland (http://fc-md.umd.edu/), Kaiserslautern (http://www.iese.fhg.de)

8. Classification of Inspection Techniques

9. Roles Activities Products Experiment #1: How to conduct an Inspection? Software Product Reading Techniques 1 PlanningForm Planning organizer Defect Report Form 2 Detection inspector 3 Defect Collection Form moderator inspectors author Collection 4 Corrected Software Product Defect Correction Form Software Inspection Correction author

10. Classification of Inspection Techniques #1 • Ad-hoc • Inspectors intuition & experience • Checklist-based reading (CBR) • Shortcomings • Based on past defect information (new defect types?) • Too many questions & rare to find concrete instructions (HOW?) • Does not require the inspector to document his or hers analysis • Inspector often get swamped with unnecessary details

11. Classification of Inspection Techniques #2 • Scenario-based readingextend the work of Parnas and Weiss on Active Design Reviews and allocate specific responsibilities to inspectors and provide guidance in the form of so-called scenarios (what to check & how to perform the required check)

12. Classification of Inspection Techniques #3 • Scenario-based reading • Defect-based Reading • Reading scenarios based on a defect taxonomy for inspection of functional requirements documents <Porter et al.> • FP-based Reading • A FP scenario consist of questions about a specific FP-element (input, output, file, inquiries)

13. Classification of Inspection Techniques #4 • Scenario-based reading (Continued) • Perspective-based Reading (PBR) • Checking from a particular stakeholders perspective (customer, requirements, design, test, maintenance) • A PBR-scenario consists of activities an inspector is to perform to extract information or build abstractions from the inspected document/artifact and questions to analyze the extracted information. • Generalizing Perspective-based Inspection to handle Object Oriented Development Artifacts <Laitenberger, Atkinson>

14. Classification of Inspection Techniques #5 • Scenario-based reading (Continued) • Traceability-based Reading <Travassos et al> • The scenarios describe how to perform correctness and consistency checks among various UML-models.

16. Experiment #1(Laitenberger et al) An Experimental Comparison of Reading Techniques for Detection in UML Design Documents

17. Experiment#1: Software Inspection Process • Planning • Defect Detection • Empirical findings reveal that the synergy effect of inspection meetings is rather low in terms of impact of defects detected. Therefore it can be considered as an individual rather than a group activity. • Defect Collection • Often performed in a team meeting led by an inspection moderator • Defect Correction • Performed by the author

18. Experiment#1: Rationale for a new Inspection Technique • Increased EffectivenessThe proportion of defects found during an inspection • Decreased CostCost is defined as the effort involved in finding a single defect • In the experiment the cost has been separated for the two phases • individual defect detection • defect collection in a meeting

19. Planing Overview Follow-up Detection Correction Collection Experiment #1: Compare Reading Techniques Process • Organizer • Moderator • Inspector • Author • (Recorder) • (Presenter) • to be inspected • to conduct an inspection Roles Products Inspection Reading Techniques • Defect-based Reading • Perspective-based Reading • Ad-hoc • Checklists • RSA

20. Experiment#1: Experimental Hyphoteses • The Effectiveness of PBR is larger than the Effectiveness of CBR for Teams • Per-defect Detection Cost for Teams is Lower with PBR than with CBR • Meeting Cost for Teams is Lower with PBR than with CBR • Overall Inspection Cost is Lower with PBR than with CBR

21. Experiment#1: Subjects • Experiment performed in the context of a course in object-oriented development • 18 practitioners with various backgrounds and experience • Already written UML Documents (median 3 on a 5p scale) • Developing Design Documents (some experience) • Programming (1-16 years, median 2 years) • Inspections (none had participated in an inspection)

22. Experiment#1: Materials • Checklist (CBR) • Scenarios (PBR) • Designer Scenario • Tester Scenario • Implementor Scenario

23. Results of the Experiment • Defect DetectionPBR exhibits a 41% effectiveness improvement over CBR • Cost per Defect(Detection Phase + Meeting Phase) The average relative improvement of all inspection teams shows that PBR exhibits 58% cost per defect improvement over CBR.

28. Reading Technique - Definition • Reading Technique can be defined as a series of steps for the individual analysis of a textual software product to achieve the understanding needed for a particular task.

29. Reading Technique 1 • First, the series of stepsgives the reader guidance on how to achieve the goal for the technique. By defining a concrete set of steps, we give all readers a common process to work from which we can later improve based on experience.

30. Reading Technique 2 • Secondly, a Reading Technique is for individual analysis, meaning that the aim of the technique is to support the understanding process within an individual reader.

31. Reading Technique 3 • Finally, the techniques strive to give the reader the understanding that they need for a particular task, meaning that the reading techniques have a particular goal and they try to produce a certain level of understanding related to that goal.

33. Object Oriented Reading Techniques • Two main questions drive all of the Object Oriented reading techniques. • Do all of the design artifacts describe the same system, i.e. are the individual artifacts consistent with each other?Horizontal Reading • Do the design artifacts, as a whole, accurately describe the same system that the requirements describe, i.e. are the requirements and the design consistent?Vertical Reading

34. One reading technique (Scenario) for each pair of or group of diagrams that could usefully be compared to each other.

35. Horisontal Reading Scenario Version #1

36. Experiment #2 (Travassos et al) Techniques for OO Inspection

37. Experiment #2 - Goal • To evaluate feasibility of applying reading techniques • To receive feedback to create a second version of the techniques

38. Experiment #2 - Subjects • 44 undergraduates from a SW Engineering Course with a mix of SW experience • 32% had some previous industry experience in SW design from requirements and use cases • 9% had no experience in SW design • 59% had classroom experience • All students were trained in OO development, UML and OO SW design activities as part of the course

40. Experiment #2 - Procedure and Result • Subjects randomly assigned to 15 team • One subject performed vertical reading • the other two divided the horizontal techniques between them • Teams detected 11 defects on average • 11,7 for horizontal (defects of ambiguities and inconsistency between design artifacts) • 10,4 for vertical (omitted and incorrect functionality)

41. Experiment #2 - Lessons Learned • The first lesson learned was:OO reading techniques should concentrate on semantic, not syntactic, issues.

42. Experiment #2 - Lessons Learned • Second lesson learned was:Reading techniques need to include not only instructions for the reader, but some motivation as to why those instructions are necessary. The information in the qualitative data suggested that subjects were interested in why the steps of the technique were important and useful, as opposed to just being told what to do.

43. Experiment #2 - Lessons Learned • The third lesson that was learned was:The level of granularity of the instructions needs to be precisely described. Discussing functionality is a difficult but necessary part of the reading techniques. The difficulty comes from the many different levels of granularity at which system behavior can be described. To solve this problem we decided to define 3 distinct ways of discussing functionality, ranging from very specific systemmessages (the communications between objects that work together to implement system behavior) to system services (the steps by which some larger goal or functionality is achieved) and finally, at the top level, system functionalities (the behavior of the system, from the user’s point of view).

44. Horisontal Reading Scenario Version #2

45. Experiment #2 - Observational Technique • Employing techniques (observational and inquisitive) to observe subjects while working with the reading techniques may be a way to capture more complete and accurate information about what is really going on (e.g. the subject’s thought process)

46. Experiment #2 - Observational Technique • Observations • Having applied the techniques once, the time spent to apply the techniques diminished • The complexity of the artifact being inspected is an ifmportant actor (noticed especially when reading classes belonging to inheritance hierarchies) • Some domain knowledge seemed to be necessary to support horizontal reading (Readers suggested that the entire set of requirements was not needed but some description of the system, its main concepts and objectives)

47. Experiment #2 - Observational Technique • Observations • Necessary level of object-oriented development expertise.When different levels of abstraction were used (e.g., references to objects versus classes), readers without a more detailed knowledge of the OO paradigm encountered difficulty. This does not imply that readers need experience in OO development, just some knowledge of the OO concepts such as class, object, attribute and behavior

50. Guided by Testcases