large displays are like regular sized displays, only larger, right?

1. large displaysare like regular sized displays, only larger, right? patrick baudisch microsoft research visualization and interaction research

2. large screens and multimon

3. Use Multimon No Multimon 32% 30% Plan to Use Multimon 38% ... are coming • information mural[Guimbretière, Winograd] • on large screens optical flow helps navigation [Tan 2001] • large screens help productivity tasks [Czerwinski 2003] • multi-monitor setups: access palette windows in Photoshop, CAD… [Grudin 2001] [Jon Peddie ResearchDec, 2002 N=6652]

4. fill-in cursorscurrent frame fill-in cursorsprevious frame mouse motion building a large displayfocus plus context screens help—I can’t find my mouse cursorhigh-density cursor help—I can’t reach my stuff anymoredrag-and-pop context where there is no room for contexthalo

5. Hardware • At least one hi-res display • At least one larger low-res display • Software • scaling of the display content is preserved • resolution varies

7. setup

8. application scenarios  video

9. Subject’s task Document/view Smallest detail Ratio Static documents Web designer Page: 800 pixel Table detail: 1 pixel 800 Mechanical engineer Polybot segment: 5cm Clearance: 0.03mm 2,000 Graphic designer Poster: 1m Align: 0.5mm 2,000 Architect in remodeling Building: 50m Accuracy: 1cm 5,000 Photogrammetry (2) Highway 2 miles Accuracy: 1 inch 100,000 Geographic info. system County: 80km Land boundaries: 0.5m 160,000 Chip designers (2) Wafer: 12cm Grid: 0.5m 240,000 Dynamic Air traffic ctrl. tool builder Zone: 50km Plane distance in 25m steps 2,000 Ego shooter gamer Surrounding: 360º Aiming: 0.1º 3,000 Submarine ROV op. Surrounding: 360º Use arms: 1mm/0.05º 8,000 Strategy gamers (2) Map: 30k pixel Aiming: 1 pixel 30,000 field study: users & tasks

10. Display technology homogeneous resolution 4 VisualizationSame # of pixels fisheye overview plus detail 5 4 4 wall-size, hi-res display … andcurrentsolutions What participants used focus plus context screen Available to½ of participants

11. experiment 1: • 3 interfaces: • focus plus context screen • overview + detail • homogeneous • 2 tasks • 12 subjects from Xerox PARC • Within subjects, counter-balanced • Same number of pixels

12. task 1: closest hotel 8 maps per interface F+C screen and O+D use same magnification factor

13. task 2: verify connections Verify a different set of 24 connections on the board

14. results manually zoomingtakes time 21% faster 36% faster 700 600 500 zooming+panning 400 overview+detail Average task completion times in seconds 300 focus+context screen 200 visually switching  reorientation 100 0 Map task Board task visually more ambiguous

15. experiment 2:driving simulation 120 sec sequence 100 fields of nails; 30 rocks; tradeoff

16. results Error rate only 1/3 of two-monitor setup 25 • Sweet spot:flight simulation, unmanned vehicles… 20 overview+detail focus+context screen 15 Mean number of collisions subjects caused Subjects preferred thef+c interface 10 5 0 run-over nails rocks hit

17. but how about the computer desktop? ktop? ktop? • so it worked really well withcontent that already wasfocus & context… • …but what about the computer desktop? • how to view peripheral content in high-res • problem only because periphery is low-res? • not really… how to view a detail on a huge display wall? • it is just hard to see detail if located far away • the distinction of screen space into focusand context regions is always there(focus plus context screens only emphasize it)

18. keeping the mouse working • on a large screen, cursor isfurther away from user • longer distances à higher mouse acceleration • temporal aliasing: 500 pixels jumps • lack of visual continuity & weak stimulus àusers lose track of the cursor

19. the problem will get worse • “yes, but won’t faster computers make this problem go away?” • àNO: cursor update is limited by screen refresh rate • screen refresh rate has actually decreased (LCDs) • larger screens + lower refresh rate à status quo • future: even larger screens à problem will get worse

20. fill-in cursorscurrent frame fill-in cursorsprevious frame inserts cursor image between actual cursor positions  the mouse cursor appear more continuous how it works previous cursorposition current cursorposition mouse motion

21. this is not the mouse trail video • the windows mouse trail… • makes mouse trail last longer • drawback: cursor images lag behind • ...is not high-density cursor • hd cursor makes mouse trail denser • lag-free: mouse stops=>cursor stops

22. benefits previous cursor position current cursor position mouse motion fill-in cursorscurrent frame mouse motion fill-in cursorsprevious frame • 1. mouse cursor appear more continuous • à easier to track the cursor • 2. higher “visual weight” • à easier to re-acquire the cursor

23. designs alternatives a b c d e f g h frame acceleration • reference: exponential acceleration

24. designs alternatives a b c d e f g h frame acceleration • motion blur with higher weight

25. designs alternatives a b c d chose discreet version 1. latest cursor position is always shown blur-free and in full opacity 2. appearance that users are familiar with today 3. computationally less expensive e f g h frame acceleration • temporal super-sampling vs. motion blur

26. a b c d e f g h frame acceleration designs alternatives • density = detectability vs. intrusiveness

27. transfer function distancebetweencursor images hd cursor has no effect transfer function(configurable) cursor trail provides no speed cues onset threshold (configurable) mouse speed

28. a b c d e f g h frame acceleration designs alternatives • optional cursor growth

29. user study • conducted pre-study to define interface candidates • interfaces: control vs.high-density cursor (conservative, tripleDensity, plusScaling) • fitts’ law task • triple-mon: button located at 5” to 40” distance • participants: 7 external participants, 5 coworkers • hypotheses • high-density cursor faster • the greater the distance thegreater the effect • tripleDensity and plusScalingfaster than conservative

30. 102 100 98 short distance conservative speedupup to 7% 96 + scale +3-dense 94 92 90 results regular mouse cursor time % relative to regular cursor high-density cursors 125 250 500 750 1000 target distance (mm)

31. subjective satisfaction • most participants did not notice that cursor was different!“did that condition use a different mouse acceleration?”…

32. lesson we learned:display frame rate is not a hard limit

33. but how to see details? • so this gets the mouse to the periphery—nice • …but what if the user uses touch for input? • … or if user needs to see content in detail? let’s focus on a specificcase for a moment:extend basic actionsdrag-and-drop and picking

34. scenario 1: long distances dragging is designed for small screens…… but becomes time-consuming on large screens

35. scenario 2: dragging+bezels dragging across bezels in display wall is no problem for the mouse… …but a big problem when using pen/touch input

36. drag-and-pop: demo • users starts dragging icon towards a distant folder or application • icons of compatible type come towards mouse cursor • user drops icon with minimal motion • targets retract drag-and-pop works across bezels

37. drag-and-pop generalizes direct manipulation • bring content to the user • let the user interact with it • send content back

38. scenario 1: long distances dragging on large screens

39. scenario 2: dragging+bezels dragging acrossbezels in display wall

40. the displays we used…

41. design

42. selecting candidates • initialize • all icons are candidates • filter • eliminate icons with non-matching file types • eliminate icons that are too close • eliminate icons outside target angle • if necessary, restrict to some hard limit

43. preserving layout • snap to grid • eliminate empty rows and columns • translate back • place center of bounding box in front of user • closer for experts

44. the rubber band • animationdid not work • “frozen”motion blur • narrow midriff • suggests elasticity • clue for distance • simplified version

45. getting it out of the way • to rearrange icons on the desktop (overloaded): • any mouse motion moving away from the “popped-up” icons de-activates drag-and-pop • introduce flick gesture into mouse motion

46. pre-study • 15 single, 6 dual, and4 triple monitor users • overall resolutions 800,000 pixels to 3,900,000 pixels • (= 66% more than the display wall used in the experiment). • 3 layouts for study: sparse (11), frame (28), cluttered (35)

47. user study • participants: 2 female, 5 male • dynaWall • 3 Smartboard • 15’ long (4.5m) • 3 x 1024x768 pixels • native code not stable enough Macromedia Flash version • task: drag icons into matching folder • highlighting disappeared when started • each desktop: 11-35 icons + 10 icons to be filed

48. Control Drag-and-pop results Control Drag-and-pop 3.7 timesspeedup • faster with drag-and-pop • error rate higher with drag-and-pop • most of the effect caused by the bezels

49. subjective satisfaction • > 6 (out of 7) • “I liked using drag-and-pop” • “I always understood what was happening when drag-and-pop was on”, • “I would use drag-and-pop for large displays.” • < 3 for • “It took a long time to get used to drag-and-pop” • “It was hard to control what the targets did when drag-and-pop was on.” • drag-and-pop interface causes less manual stress and fatigue than the control interface • lesson learned: the shortest connection between two points on a display wall is not a straight line (fixed)

50. drag-and-pick • problem • launch app or open file • drag-and-pick • user drags “background” • all icons in that direction move to the cursor • user drags % releases mouse over it • target is activated