Large Display User Interaction

1. Large Display User Interaction Mary Czerwinski, Senior Researcher Microsoft Research © 2003 Microsoft Corporation. All rights reserved.

2. Thanks to my Colleagues • Desney Tan • George Robertson • Greg Smith • Patrick Baudisch • Brian Meyers • Gary Starkweather © 2003 Microsoft Corporation. All rights reserved.

3. Ignore Science Fiction at Our Peril Workstation in the world of the Matrix © 2003 Microsoft Corporation. All rights reserved.

4. Large Display Surfaces are Here Workstation in the real world © 2003 Microsoft Corporation. All rights reserved.

5. Why A Larger Display Surface? • Productivity benefits 15-30% (despite OS issues) • Users prefer more display surface • Prices dropping fast • Footprints getting smaller © 2003 Microsoft Corporation. All rights reserved.

6. Use Multimon No Multimon 32% 30% Plan to Use Multimon 38% Multimon Trend is Growing (Jon Peddie Research Dec, 2002 N=6652) © 2003 Microsoft Corporation. All rights reserved.

7. 2004 Large Monitor ASP Projections 16:9 x 22” Diagonal 20”Diagonal 2 x 17” (30” Diagonal) $1000 Relative Pricing 2 x 15” (26” Diagonal) 17”Diagonal 15”Diagonal Single Multiple Wide © 2003 Microsoft Corporation. All rights reserved. Note: All Prices are for Liquid Crystal Displays Source for Single Panel Pricing: IDC and Display Search

8. Large Display User Experience, MSR • Large display surfaces fundamentally change user interaction • Visualization, input techniques, work management, … • Large display surfaces provide non-linear productivity increases • Additional space has different utility • E.g. Focal/peripheral displays provide different cues © 2003 Microsoft Corporation. All rights reserved.

9. Windows and Task Management Issues Emerge • Larger displays = more open windows • Multimon users arrange windows spatially • TaskBar does not scale: • Aggregation model not task-based • Users can’t operate on groups of related windows © 2003 Microsoft Corporation. All rights reserved.

10. DEMOS: High Density Cursor © 2003 Microsoft Corporation. All rights reserved.

11. INPUT: Drag ‘n Pop • Problems: • Large displays create long distance mouse movement • Drag ‘n Pop brings proxies of targets to the user from across display surfaces © 2003 Microsoft Corporation. All rights reserved.

12. DEMO: GroupBar • GroupBar joins related items in the taskbar and remembers spatial layouts of tasks • To help solve this problem, Shell needs to remember Windows’ states between sessions • Download: \\gregsmi2\dowloads\groupbar.exe © 2003 Microsoft Corporation. All rights reserved.

13. DEMO: GroupBar Layouts • Windows management time-consuming • Screen real estate often gets wasted during layout • Solution—Smart Windows Layout Options © 2003 Microsoft Corporation. All rights reserved.

14. Advanced UI: Scalable Fabric • Configurable central focus + peripheral context areas • Easy task switch from periphery to focus area • Personally meaningful layouts are preserved by system • Leverages human spatial memory • Proximity determines group membership © 2003 Microsoft Corporation. All rights reserved.

15. Women Take a Wider View (CHI 2002) • Grew from work designing and evaluating 3D virtual navigation techniques • On regular desktop display: • Men performed significantly better than women • On exploratory widescreen display: • Overall improvement for all users • Surprising finding: • Gender gap disappeared - Males and females performed equally on widescreen display © 2003 Microsoft Corporation. All rights reserved.

16. Related Work • Formation of cognitive maps while navigating 3D virtual worlds • Spatial abilities • Artifacts (maps, landmarks,…) • Gender differences in spatial ability and navigation strategies • Most report male advantages, especially in virtual environments © 2003 Microsoft Corporation. All rights reserved.

17. Related Work: Optical Flow • Changing retinal image as we move through the environment • Aids perception of environmental structure © 2003 Microsoft Corporation. All rights reserved.

18. Related Work: Optical Flow • Changing retinal image as we move through the environment • Aids perception of environmental structure © 2003 Microsoft Corporation. All rights reserved.

19. What we know about Optical Flow • Optical flow benefits heading perception in active navigation • Shown for fields of view up to 90 degrees • Hypothesized that effectiveness of optical flow depends on spatial ability • [Cutmore et al. 2000] • Gender unexplored © 2003 Microsoft Corporation. All rights reserved.

20. Our Hypotheses • Optical flow cues help all users form better cognitive maps when navigating 3D virtual environments • Better optical flow cues help women more than men in cognitive map formation • Wider displays offer even better optical flow cues © 2003 Microsoft Corporation. All rights reserved.

21. Dsharp Display 43" 11" © 2003 Microsoft Corporation. All rights reserved.

22. Task: General Description • Learning: User controls movement along path through virtual 3D maze • Testing: Remember path traveled © 2003 Microsoft Corporation. All rights reserved.

23. Virtual Maps • 14 rooms (6 straight ahead, 8 turns) • Some paths go through same room twice For example: © 2003 Microsoft Corporation. All rights reserved.

24. Cognitive Map Learning • Use arrow keys to go through green door • Determine if path crosses itself • Remember full path © 2003 Microsoft Corporation. All rights reserved.

25. Cognitive Map Memory Test • Tested on memory for maze • Forward test and backward test • Measured task time & number of correct doors opened on first attempt • Same controls as in learning phase, but without green door guides • Given feedback © 2003 Microsoft Corporation. All rights reserved.

26. Experimental Design Female Male Small FOV: 100 degrees Large FOV: 120 degrees Optical Flow Absent Optical Flow Present © 2003 Microsoft Corporation. All rights reserved.

27. Experimental Procedure • Paper folding test of spatial ability • 1 practice trial + 4 test trials • Satisfaction questionnaire © 2003 Microsoft Corporation. All rights reserved.

28. Benefits of Optical Flow © 2003 Microsoft Corporation. All rights reserved.

29. Optical Flow Helps All Users in Forward Test Forward Backward © 2003 Microsoft Corporation. All rights reserved.

30. Optical Flow Benefits Females More in the Forward Test Females Males © 2003 Microsoft Corporation. All rights reserved.

31. Other Results • No effects for field of view • No effects for spatial ability measure • Satisfaction ratings matched performance results © 2003 Microsoft Corporation. All rights reserved.

32. Conclusion • Optical flow cues help all users form better cognitive maps when navigating 3D virtual environments • Better optical flow cues help women more than men in cognitive map formation • Unexplained by biases in spatial ability • Wider displays offer even better optical flow cues • 100 degree field of view seems sufficient © 2003 Microsoft Corporation. All rights reserved.

33. Information Voyeurism: Social Impact of Large Displays • Exploit social cues induced by physical size: • Help people communicate • Increase productivity on individual tasks • Must quantify in order to exploit • Information on large displays more public • Ask user? Cannot guarantee accuracy • Video? Cannot disambiguate glance from reading © 2003 Microsoft Corporation. All rights reserved.

34. Measuring ‘Peeking’ • Implicit memory priming paradigm • Expose user to stimulus • Test user implicitly on how much they’ve processed stimulus • Word stem completion • Eg. Mon_____ • Priming measured by faster response or higher frequency of stimulus • Monkey, Money, Monster, Monday, Monopoly, … © 2003 Microsoft Corporation. All rights reserved.

35. Experiment Materials • Stimulus: 30 words embedded in: • 7 e-mail subject lines • 2 e-mail messages • Place e-mail where it can be seen by user • Priming test to see if they’ve read it • Word stem completion © 2003 Microsoft Corporation. All rights reserved.

36. 114″ 66″ 16″ 27.5″ Large Projection Screen 156″ Small Desktop Monitor 38″ Experimenter Participant Experimental Setup © 2003 Microsoft Corporation. All rights reserved.

37. Implicit Memory Results N=12 Average # of Target Words N=12 Small Display Large Display © 2003 Microsoft Corporation. All rights reserved.

38. Other Converging Data • More users admitted reading text on: • Large Screen (7/12) vs. Small Screen (3/12) • Comments indicated reading someone else’s e-mail more acceptable on large screen • Video shows users glanced more at: • Large Screen (M=19 seconds) vs. Small Screen (M=14 seconds) © 2003 Microsoft Corporation. All rights reserved.

39. Future Work • Understand why large displays more public • Physical proximity of information to owner? • Wall-mounted nature of large display? • Protect private information from prying eyes • Private information never placed on public screens • Interface conventions that convey level of privacy • Facilitate ad hoc collaboration • Display systems that make people interact more © 2003 Microsoft Corporation. All rights reserved.

40. Summary • Novel application of implicit memory priming paradigm for measuring if someone has peeked at information • People are more willing to read someone else’s e-mail on large wall-sized displays than on smaller displays • Even with identical visual angles and legibility • Future Work © 2003 Microsoft Corporation. All rights reserved.

41. Future Research Directions • Continue researching benefits of and methods for testing large displays • Task management for information workers • Reconfigurable information display surfaces • Heterogeneous devices • Social impact, collaboration and communication scenarios • Gestures, vision and tracking • Novel UI and interaction techniques © 2003 Microsoft Corporation. All rights reserved.

42. Thank you… © 2003 Microsoft Corporation. All rights reserved.

43. Large Display Surfaces are Here Workstation in the real world (InfoCockpit--CMU) © 2003 Microsoft Corporation. All rights reserved.

44. Optical Flow Benefits All Users Equally in the Backward Test © 2003 Microsoft Corporation. All rights reserved.

45. User Views of Maze Narrow field of view (100 degrees) Wide field of view (120 degrees) © 2003 Microsoft Corporation. All rights reserved.