1 / 32

Project 7: Testing Suggested Modifications to a User Interface for Source Code Search

Project 7: Testing Suggested Modifications to a User Interface for Source Code Search. Customer : Rosalva Gallardo Team members : Susan Lin Buda Chiou Jim Milewski Marcos Mercado November 23, 2010. Agenda. Primary Purpose Method Overview Usability experiments Data Collection

dolf
Download Presentation

Project 7: Testing Suggested Modifications to a User Interface for Source Code Search

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Project 7: Testing Suggested Modifications to a User Interface for Source Code Search Customer: Rosalva Gallardo Team members: Susan Lin Buda Chiou Jim Milewski Marcos Mercado November 23, 2010

  2. Agenda • Primary Purpose • Method Overview • Usability experiments • Data Collection • Preliminary Results • Current Status and Updated Project Plan • Q&A

  3. Primary Purpose • Does the new “treatment” interface improve the usability of the metrics?

  4. Primary Purpose • Does the new “treatment” interface improve the usability of the metrics? Introduced sort functionality for metrics

  5. Primary Purpose • Does the new “treatment” interface improve the usability of the metrics? Changed the presentation of the metrics

  6. Primary Purpose • Does the new “treatment” interface improve the usability of the metrics? Included a mix of graphics and text for the “Reviews” metric

  7. Primary Purpose • Does the new “treatment” interface improve the usability of the metrics? Introduced Reviewer’s information

  8. Method Overview • Pilot test plus 15 usability experiments performed in the Hana Lab • Average experiment time 1.5 hours • Some experiments ran in parallel. • Monetary compensation and incentive provided to accomplish tasks.

  9. Experiment Structure • Experiments were run according to client’s Handbook for Source Code Retrieval Experiment. • Welcome and study information sheet/consent • Instructions to subject and general scenario • Practice task with scenario • 4 Experiment Tasks with scenarios and post-task questionnaires • Participant’s background questionnaire • Debriefing Interview

  10. Data Collection • Think aloud protocol • Excel file filled in while participant performed the tasks • Morae recording • Questionnaire after each task • Debriefing interview • Subject background questionnaire

  11. More data capture: Morae to Excel • Video recordings analyzed (1 video = 5.5 hours) • Before and after pages highlighting some of the changes • e.g. you needed to capture the observations of the participants and couldn’t really focus on how quickly they were switching between screens • Seconds, Observations, Candidates

  12. Preliminary Results

  13. Preliminary Results

  14. Preliminary Results

  15. Preliminary Results

  16. Preliminary Conclusions • Similar trend is observed except for Reviews where we combined graphics and text • Increase of Reviews usage • New layout for metrics presented in text only did not alter the trend.

  17. Project Plan

  18. Q&A

  19. Previous Presentation

  20. Project background & overview • Original Experiment • Customer goals were: • Understand the strategies and information software developers use when they select a source code search result from the Web • Identify the relationship among strategies, information used, and successful completion of the experiment’s tasks.

  21. Project background & overview • New Experiment • Customer goals are: • Same as in the original experiment • Analyze how an enhanced GUI will influence users’ decisions on the information they use. • Variants: • New participants • Use of an enhanced treatment GUI • Use of a log mechanism to assist in the collection of experiment data related to GUI usage.

  22. Project phases • Implement user interface enhancements to experiment’s treatment GUI. • Implement an automated log mechanism to capture certain events during GUI usage. • Conduct new experiment • Analysis of results

  23. Usability problems that we’re dealing with

  24. Usability problems that we’re dealing with

  25. Comparative Analysis • Google Code, Koders.com, Source Forge, Ohloh

  26. Prototypes

  27. http://www.jimsplace949.com/GUIFinal2.aspx

  28. Methods that the team is going to employ to address the usability problems (continued) • Number & type of users: • 16 ICS students with experience in Java (13 graduate-level and 3 undergraduate). • Tasks for users and questions we will ask them: • CSV task: Find the best source code to read a CSV file and place the data into a list of strings. • DIFF task: Find the best source code to compare two text files and show the differences. • EMAIL VALIDATION task: Find the best source code to validate email addresses. • TIMEZONE task: Find the best source code to convert date & time between different time zones.

  29. Methods that the team is going to employ to address the usability problems (continued) • Strategies for involving users • Recruitment email • Use of incentives • Where… • Will meet them in the Hana Lab • Each meeting will last 1.5-2 hrs

  30. Project Plan overview http://www.jimsplace949.com/ProjPlan_Project7_SrcCdSrchExperiment.html

More Related