Expert Problem Solving Strategies For Program Comprehension

1. Expert Problem Solving Strategies For Program Comprehension By Jürgen Koenemann and Scott P. Robertson Presented by Rab Harbart

2. Where are they now? • Jürgen Koenemann is currently the Director of Consulting at Amdocs, a communications software company in Germany. Aside from HCI knowledge he has a background in business and enjoys jogging • Scott P. Robertson is currently a professor at the University of Hawaii in Manoa. His area of study is Human-Computer Interaction and Digital Government

3. Notably …

4. Outline • Intro • Theory • Participants • Experiment • Results • Analysis of Task B • Conclusions • Response • Wrap up

5. Introduction • The intent is to show that program comprehension should be understood as a goal- oriented, hypothesis- driven problem- solving process • Data will show that programmers use an As- needed approach to gain knowledge of only the sections of code that are pertinent to the task at hand • Use of the As- needed can lead to miss conceptions and errors • Find criteria to establish better programming tools

6. Theory • Systemic • Extensive symbolic processing of the data and control flow between subroutines to gain a fuller understanding of the code • As- needed • Minimized approach to understanding localized to studying only the parts of code that will be pertinent to the task at hand

7. Participants • 12 subjects • Know at least 3 programming languages • Programmed for 4 to 15 years • Pascal experience for at least 4 years • Regularly used worked with large programs • Had education is either Computer Science or Electrical Engineering

8. Experiment Setup • 636 lines of uncommented Pascal code • Included a main and 39 sub- functions • Each had a code listing and separate description • 6 module decomposition charts and 3 global program descriptions • Modification Tasks • Task A - Boldfacing (Functional Addition) • Task B - Footer/Header Centering (Enhancement) • Task C - Underlining (Functional Change) • Task D - Default Footers (Default Value Change)

9. Experiment Procedure • Subjects read a summary of the program and tasks, then verbalized their method to perform each task • They were assigned one of the modification tasks and told to think out loud and verbalize their methods • Subjects were allowed only one piece of information at a time and were asked to explain the reason for requesting a different source. The sequence of requested sources was also recorded

10. Results • Completion time range was 15- 44 minutes with an average of 28.5 minutes • The main source of information was the code listings and the decomposition charts. Overall the detailed descriptions were not used as often • On average subjects looked at the information pieces only 1.5 times showing that most of the desired information was found during the first viewing

11. Table 1

12. Results as they continue • Subjects only looked at 1/5 of the code which accounted for less than 1/3 of the entire program • Almost 1/4 of the functions or sub-routines were ignored by the subjects, a little over 1/5 of the entire program was unknown to the subjects • Of the code that was not looked at most were low level functions like MAX, MIN, or ISDIGIT

13. Table 2

14. Results: an intermission of theory • Relevance Strategy • Directly - Parts of code that are going to be modified or used in a copy- edit fashion • Intermediate - Code that interacts with the relevant code and believed that surveying this code will assist in understanding the relevant code • Strategic - Code not completely understood but used as a pointer to help locate directly or intermediately relevant code

15. Results and analysis of usage • Relevant information varied depending on the task performed • All subjects used information of strategic relevance; charts and global descriptions, in the beginning of the process • Use of the code listing increased over time while use of the remaining information sources decreased

16. Figure 1

17. Analysis of Task B • Task B was to automatically center headers and footers • Required additional code into puttl • Or placing that same code in puthead and putfoot • All subjects started with a top- down approach by studying global descriptors • Subjects created hypotheses based on knowledge about programming and the task domain

18. Figure 2

19. Conclusion • This experiment shows that program comprehension desires a top- down method • No subject used the systematic method, no time was used in studying non- relevant code • Subjects found a deficiency in viewing one piece of at a time information • Descriptions should offer what is not visible in the code

20. Conclusion - Implications • With a loss of performance in switching information pieces, programmer are best served with multi- window views • Utilization of charts and code suggest a graphical browser would facilitate better navigation • Regarding copy- edit, links should be established that reference new code and offer to connect them with existing code that is analogous to shows a correspondence • Comment code instead of offering dense descriptions

21. Response • Goal oriented approach seemed more reasonable and likely in practice • Agree top- down method is important initially but later code is the dominant resource • Issues can occur from code never looked at during modification process, though through understanding of direct and intermediate code, effects might be mitigated

22. Response - second wind • In regard to relevance strategy, focus on code to be modified and work outward only enough to fully understand the code to be worked on • Experiment had an overall more practical and useful direction, less generalized and conceptually centered then previous reads

23. Source • Koenemann, J. and Robertson, S., (1991), "Expert Problem Solving Strategies for Problem Comprehension", in Proceedings of Conference on Human Factors and Computing Systems (CHI'91), New Orleans, LA, April 27 - May 2, pp. 125-130.

24. Wrap up • Questions or Parting Shots?