Demonstrating the Viability of Automatically Generated User Interfaces

1. Demonstrating the Viability of Automatically Generated User Interfaces Jeffrey Nichols, Duen Horng Chau, Brad A. Myers IBM Almaden Research Center and Human Computer Interaction Institute, Carnegie Mellon University 25th Conference on Human Factors in Computing Systems (CHI) May 3, 2007

2. History of Automated Design UIDE and Cartoonist, Sukaviriya 1994 Humanoid, Szekely 1992 Jade, Vander Zanden 1990 ITS, Wiecha 1990

3. New Automated Design Systems INCITS/V2 Standard PUC and Uniform, Nichols 2006 iCrafter, Ponnekanti 2001 Supple, Gajos 2004 Xweb, Olsen 2000

4. Viability… • Automatically generated interfaces must improve on those available today (subject to the same constraints) • Development and manufacturing costs for supporting the automatically generated interfaces must not exceed costs of today’s technology

5. Personal Universal Controller PUC[Nichols, UIST 02] Personal MobileDevice AutomaticallyGeneratedInterface Control AbstractSpecification State Feedback

6. Uniform Copier A Copier B [Nichols, CHI 06] Original (PUC) Interfaces

7. Uniform Copier A Consistent Copier B Copier B [Nichols, CHI 06] Consistency enabled – Copier A used first Original (PUC) Interfaces OriginalCopier B

8. In This Talk • I will present a study of the PUC system with two comparisons: • Existing hand-designed interfaces with automatically generated interfaces • Automatically generated interfaces with and without consistency • Argue that cost of generating these interfaces is competitive with existing design costs

9. Outline • Introduction • Interfaces • Study Procedure • Results • Discussion and Future Work

10. Appliances for Study Canon PIXMA MP780 HPPhotosmart2610

11. Specifications HP printer Canon printer • Written by different people to simulate differences due to manufacturer • Faithful to design of the appliance • Took advantage of specification language features,e.g. multiple labels • Initial drafts produced in 2-3 days • Debugging took another 2-3 days • Similarity information for generation of consistent interfaces was created in several hours • Quite complex: • HP Printer 85 variables and commands 1924 lines of XML • Canon Printer 134 variables and commands 2949 lines of XML

12. Interfaces HP printer without consistency Canon printer without consistency Canon printer consistent with HP

13. Interfaces – Full View HP printer Consistent Canon printer Canon printer

14. Outline • Introduction • Interfaces • Study Procedure • Results • Discussion and Future Work

15. Study HP printer Canon printer • Evaluation of Generated Interfaces • Users perform 8 tasks with two all-in-one printer interfaces (HP and Canon) • Two comparisons Physical interface to PUC interface Without consistency and with consistency • 48 participants (28 male, 20 female) Divided into 6 groups, 8 per group Recruited through CMU service • Metrics Completion time Failures

16. Tasks • Send a fax to the number stored in the third speed dial • Configure the fax function so that it will always redial a number that is busy • Configure the fax function so that any document received that is larger than the default paper size will be resized to fit the default • Configure the fax function so that it will only print out an error report when it has a problem receiving a fax • Make two black-and-white copies of the document that has already been placed on the scanner of the printer • Imagine you find the copies too dark. Improve this by changing one setting of the device • Given a page with a picture, determine how to produce one page with several instances of the same picture repeated (demonstrated with actual paper copies) • The device remembers the current date and time. Determine where in the interface these values can be changed (do not change them)

17. Procedure • Subject performs all 8 tasks on appliance #1(HP or Canon) Five minutes maximum allowed for each task • Subject is instructed on the optimal way to perform each task Subject performs the task again until completed correctly. Additional instruction given as necessary • Subject performs all 8 tasks on appliance #2 (Canon or HP)

18. Three Interface Conditions • Built-In User sees only physical appliance interfaces Physical HP → Physical Canon Physical Canon → Physical HP • AutoGen User sees only PUC interfaces without consistency PUC HP → PUC Canon PUC Canon → PUC HP • Consistent User sees a PUC interface without consistency followed by a Uniform interface generated to be consistent with the previous PUC interface PUC HP → Uniform Canon generated to be consistent with HP PUC Canon → Uniform HP generated to be consistent with Canon

19. Conditions • 3 interface conditions * 2 appliance orderings = 6 groups

20. Comparing Usability • Focus primarily on the first set of tasks • First set not affected by earlier tasks or instruction

21. Usability Results * * * * * * * * * † † * * • Failures per subject also significantly less for PUC than Physical(Fisher’s Exact Test, p < 0.05) HP: 1.125 Built-In vs. 0.125 PUC Canon: 2.0 Built-In vs. 0.625 PUC * = p < 0.05 † = p < 0.1

22. Comparing Consistency • Focus only on the second set of tasks • Previous experience during study should influence results

23. Consistency Results * * * * † † * * • Very few failures in these conditions: 4 total for all tasks performed by 32 subjects (256 tasks, 1.5% failure rate) * = p < 0.05 † = p < 0.1

24. Outline • Introduction • Interfaces • Study Procedure • Results • Discussion and Future Work

25. Discussion • Fairness of Comparison • Why not compare with hand-designed interfaces on PDA? • Cost • Many users already have a device capable of acting as remote control, so cost of device should not be counted. • Main per-appliance development cost comes from writing specification, which is likely less than developing a full interface. • Manufacturing cost may increase slightly due to need for wireless communication capability, though it may be possible to compensate by removing unneeded interface hardware. • Limitations of Study • We studied only two variants of one type of appliance

26. Conclusion • Results: • PUC interfaces were more usable than existing appliance interfaces • The PUC’s consistency algorithms created interfaces that were even better when learning to use a new appliance with familiar functionality • This suggests that automated design can provide better user interfaces in situations where interfaces are constrained by external factors or individual user customization can provide substantial benefits • Future Work • What other types of auto-gen features would be useful? • Can we automatically modify existing hand-designed interfaces to add features like consistency?

27. Thesis Committee Brad A. Myers (chair) Scott Hudson John Zimmerman Dan Olsen Jr. Funding National Science Foundation Microsoft General Motors Intel Pittsburgh Digital Greenhouse Equipment Grants Mitsubishi (MERL) VividLogic Lucent Lutron Lantronix Nokia PUC Project Members Brandon Rothrock Duen Horng Chau Kevin Litwack Thomas K. Harris Michael Higgins Joseph Hughes Roni Rosenfeld Rajesh Seenichamy Pegeen Shen Htet Htet Aung Mathilde Pignol Suporn Pongnumkul Stefanie Shriver Jeffrey Stylos Peter Lucas Acknowledgements • Collaborators & Friends • Naomi Ramos • Desney Tan • Daniel Avrahami • Gaetano Borriello • Laura Dabbish • Andrew Faulring • James Fogarty • Krzysztof Gajos • Darren Gergle • Andy Ko • Amy Nichols • Mick Nichols • Sally Nichols • Trevor Pering • Fleming Seay • Irina Shklovski • Roy Want • Jake Wobbrock • and many others…

28. Thanks for listening! For more information… http://www.jeffreynichols.com/ http://www.pebbles.hcii.cmu.edu/puc/

30. Were users at all affected by a lack of spit/polish in the interface designs, or will users put up with poorer looking interfaces that are otherwise more usable? –Stina Nylander Have you done any studies to evaluate how easily people can learn to use the PUC specification language? –Unknown How do you deal with issue of interface branding? Without branding support, will manufacturers support your technology. –Nathan Freier Questions