Do Maintainers Utilize Deployed Design Patterns Effectively?

1. Do Maintainers Utilize Deployed Design Patterns Effectively? ACM International Conference on Software Engineering (ICSE) T.H. Ng, S.C. Cheung, W.K. Chan, Y.T. Yu Presented by Jason R. Beck and Enrique G. Ortiz

2. Outline • Research Questions • Participant Descriptions • Hotel Example • Question Formalization and Null Hypothesis • Target Programs • Required Changes • Experiment / Subjects / Procedure • Results • Pros / Cons

3. Research Questions • Will maintainers use design patterns? • How likely are maintainers to perform specific tasks (i.e. T1, T2, and T3)? • Is the maintainer more likely to use specific tasks over others? • Is the code produced of higher quality using the design patterns?

4. T1 – Concrete Participants • Participant that is subclass of another participant

5. T2 – Abstract Participants • Participant that is superclass of another participant

6. T3 – Client Partcipants • Participant that is not a subclass or superclass, but statically binds to abstract participant (superclass)

7. Hotel Example • Focus on Perfective Maintenance (adding new capabilities to program)

8. T1 – Adding new class as a concrete participant • New Local Setting: Hong Kong • Must implement new class that implements locale settings to inherit Locale (abstract) class

9. T2 – Modifying the existing interfaces of a participant • Display currently selected country in the user interface • Add a new method to the Locale class hierarchy

10. T3 – Introducing a new client • Add support for console-based user interface • Introduce new client codes for Locale inheritance hierarchy

11. Question Formalization • Q1: What proportion of maintainers utilize the relevant design patterns in completing the required anticipated change? • What proportion of them perform T1? • What proportion of them perform T2? • What proportion of them perform T3? • Are the above proportions significantly different?

12. Question Formalization • Q2: Comparing the codes delivered by maintainers who utilize the relevant design patterns in completing the required anticipated change to those delivered by maintainers who do not utilize patterns, are they equally faulty? • Comparing those who perform T1 with those who do not, are their codes equally faulty? • Comparing those who perform T2 with those who do not, are their codes equally faulty? • Comparing those who perform T3 with those who do not, are their codes equally faulty?

13. Null Hypothesis • HA0 • Maintainers have same tendency to perform each task • HB0 • Maintainers who perform tasks and those who do not provide same code quality • HC0 • “No strong correlation between the number of faults in the delivered codes and whether or not a task is performed to complete the anticipated change”

14. Target Programs • JHotDraw • Open-source drawing editor • Java based • GUI • MCM • Multiple-user calendar manager • Edit individual appointments • Console-based • HMS • Hotel management system prototype • Edit room usage (check-in and check-out) • Console-based

15. Target Program Analysis • Different applications • Different sizes (few hundred LOC to ten-thousands LOC) • Defined anticipated changes • Created design patterns

16. Required Changes • Perfective changes (most frequent maintenance)

17. Required Changes • Must spend at least three hours performing changes (ensures effort)

18. Experiment Subjects • 215 Undergraduate Java course students. • Grouped and individually assigned to tasks • 27 students  Image modification • 28 students  Language modification • 100 students  Group appointments and Undo • 60 students  GUI-based HMS and Search

19. Experimental Procedure • Briefed the students in a two hour information session. • Covered • Requirements and specification and demo of correctly operating program. • Presented the source code. • Documentation (API and UML) • Also covered general object-oriented concepts • Given a month to work on the assignment

20. Results • Subjects more likely to perform tasks T1-T3 than not use them at all. • Maintainers were more likely to perform task T1 (adding a concrete participant). • Followed by T3 (introduce a new client) and then T2 (modify existing interface).

21. Do maintainers utilize deployed design patterns? • Yes, they do use them more often than not.

22. Do maintainers utilize deployed design patterns?

23. Do maintainers who utilize deployed design patterns deliver better code? • Functional correctness measured in number of faults on the submitted programs. • Means compared between those that used one of T1 – T3 in each program and those who did not. • Fewer faults for those who used one of T1 – T3; results statistically significant at 5% level.

24. Threats to validity • Internal (Authors opinion) • Plagiarism • Wide variety of student expertise (all approaches feasible even if no design pattern used) • External • Only used a few target applications. • Design patterns used were not exhaustive • Only used finite number of perfective changes

25. Related Work • Others have researched maintenance, but not also how design patterns factor in. • Other experiments had result that conflicted between each other regarding design pattern ease of maintenance. • Experiment to see if additional design patterns in code is beneficial regardless or work experience. (By the authors)

26. Conclusion • When design patterns are initially implemented in anticipation of maintenance work, they are more often used than not by maintainers. • Conclude that T1 is most always recognized and utilized, but work is needed to “raise the maintainers’ awareness to perform” T2 and T3.

27. Pros • Good goals. Maintainers should make use of existing design. • Statistical significance of the fault results helped strengthen the argument. • Good variety of maintenance tasks and applications. Covered a good variety of design pattern tasks.

28. Cons • Are student representative of maintenance programmers? • Undergraduates in a Java class. (likely very new to programming) • Covered “general object-oriented concepts” when giving them the assignment. • Showed the students how the final results should look, showed them the requirements, and original source code. (probably not the hand holding a real maintainer gets). • A month to complete these tasks.

29. Cons • Results showed those which used patterns in the tasks had fewer faults. • Students who recognize design patterns are probably more skilled in the language. • The statistical significance is not between using patterns and not using patterns but between students who are skilled to use patterns and those who are not.

30. Cons • None of this confronted in the threats to validity, instead they thought plagiarism was more of a threat.

31. Areas to Improve / Comments • Take out the student factor and perform an experiment on professionals who perform maintenance on a regular basis. • Gain insight on decisions made by professionals. • Creates a less bias fault statistic. • Choosing not to utilize the pattern only makes things worse. • Mismatched documentation • Ruining the designers intent • Likely to cause problems for future maintainers

32. Questions?