Test-Driven Development of Graphical User Interfaces: A Pilot Evaluation

1. Test-Driven Development of Graphical User Interfaces:A Pilot Evaluation Thedore D. Hellmann , Ali Hosseini-Khayat, Frank Maurer XP 2011

2. Outline • Capture and Replay Tools • Introduction • Difficulties • A Prototype-Driven Approach to UITDD • Pilot Evaluation • Conclusion

3. Capture and Replay Tools • Capture and Replay Tools(CRTs) • Tools that record users’ interactions with an application. • Can be used to record test scripts.

4. Introduction • Test-driven development (TDD) has proven benefits for the software engineering process • increased developer confidence • increased software quality • In the approach, user interface prototypes are created in such a way as to allow capture/replay tools to record interactions with them. • These recordings can then be replayed on the actual GUI as it is being developed in a test-driven fashion. • TDD of GUI-based applications (UITDD) Does having tests for a GUI written before the functionality of the GUI is implemented help developers?

5. Difficulty • What Makes GUI Testing and UITDD Difficult? • Complexity • Verification • Finding Widgets from Test code

6. Difficulty - Complexity • Modern GUIs are composed of a set of user interface elements, or widgets. Like most classes in object-oriented systems, widgets tend to have a long chain of inheritances. • Even a simple GUI has a huge number of possible user interactions, GUI states, and events. • For Example:

7. Difficulty - Verification • A test is separated into two parts: • test procedure • test oracle • The test procedure interacts with part or all of an application in order to generate a state that is interesting from a testing perspective. • The test oracle verifies that the system behaves as expected in response to this interaction. • The effectiveness of automated tests is directly limited by the difficulty of writing useful test oracles .

8. Difficulty - Finding Widgets from Test code • In GUI testing, it is necessary to look up widgets when a test is run based on information about the widget that is recorded when a test is created. • This process of look up widgets is in large part responsible for the fragility of GUI tests. • There are two ways of performing this search: • testing with object maps • keyword-based testing

9. A Prototype-Driven Approach to UITDD • ExpenseManager • ExpenseManager is a GUI-based application. • Vies Totals • New Report • Modify Report

10. A Prototype-Driven Approach to UITDD(2) • After defining the features of ExpenseManager, a prototype of the application was created. This prototype was created using SketchFlow. • SketchFlow • Prototypes are created by dragging-and-dropping widgets onto a canvas. • Widgets can also be assigned behaviors such as transitioning to a different state. • we used LEET (LEET Enhances Exploratory Testing) to record events from interactions with prototypes of ExpenseManager. • LEET • It is based on the Automation API • Using keyword-based identification to determine which widget to interact with for each step in the test script.

11. Pilot Evaluation • We designed a pilot evaluation to determine whether our approach is actually useful to developers. • Based on 3 particepants: • Researchers completed the development of ExpenseManager, including its GUI and event handlers, based on the tests. • The researchers removed all of the code contained in ExpenseManager’s 7 event handlers.

12. Pilot Evaluation(2) • Interestingly, all three participants entered into the same development cycle during their development of ExpenseManager. • Participants used the provided automated GUI tests as the sole indication of completeness of functionality. Observations Collected during the Study

13. Pilot Evaluation(3) • The participants asked to fill out a survey, which perception of the usefulness and usability of this approach. Observations Collected from Post-Study Surveys • The first and second questions were ranked from A (Very Useful) to E (Useless). • These responses imply that the participants saw the potential of this approach to be of some benefit in their own work.

14. Pilot Evaluation(4) • Participants were also asked to rank the resources they were provided with in terms of decreasing usefulness for their development task. • Instead of ranking features by frequency of use, participants seem to have ranked resources based on the value they provided to the development effort. Observations Collected from Post-Study Surveys • This implies that one way to immediately make this approach to UITDD provide higher value to users : • Making technical details of widgets being tested more visible so that users can understand the expected details of the system they are building.

15. Study Limitations • Only three participants took part in this study. • The time allotted for each study session was limited to one hour in order to avoid inconveniencing participants. • The application used for this study was, based on Table 2, too complicated for a one-hour study.

16. Conclusion • In this paper we present a pilot evaluation of an approach to test-driven development of GUI-based applications. • we hope to test our approach to UITDD on multi-touch and gesture-based interfaces. • we were able to determine that participants seemed to feel that UITDD had benefits, and would be a useful practice.

17. Question Time