1 / 119

interacting with large screens

interacting with large screens. patrick baudisch microsoft research visualization and interaction research. large screens. building a large display focus plus context screens interacting using the mouse high-density cursor mouse ether interacting using pen & touch drag-and-pop

chelsea-levy
Download Presentation

interacting with large screens

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. interacting withlarge screens patrick baudisch microsoft research visualization and interaction research

  2. large screens building a large displayfocus plus context screens interacting using the mousehigh-density cursor mouse ether interacting using pen & touch drag-and-pop tablecloth (halo) return

  3. large screens and multimon

  4. Use Multimon No Multimon 32% 30% Plan to Use Multimon 38% ... are coming • information mural[Guimbretière, Winograd] • on large screens optical flow helps navigation [Tan 2001] • large screens help productivity tasks [Czerwinski 2003] • multi-monitor setups: access palette windows in Photoshop, CAD… [Grudin 2001] [Jon Peddie ResearchDec, 2002 N=6652]

  5. focus plus context screens mouse acrossbezels reach icons mouse acrossdistances reach anything building one conclusions

  6. Hardware • At least one hi-res display • At least one larger low-res display • Software • scaling of the display content is preserved • resolution varies

  7. skip to dnp

  8. setup

  9. application scenarios  video

  10. Subject’s task Document/view Smallest detail Ratio Static documents Web designer Page: 800 pixel Table detail: 1 pixel 800 Mechanical engineer Polybot segment: 5cm Clearance: 0.03mm 2,000 Graphic designer Poster: 1m Align: 0.5mm 2,000 Architect in remodeling Building: 50m Accuracy: 1cm 5,000 Photogrammetry (2) Highway 2 miles Accuracy: 1 inch 100,000 Geographic info. system County: 80km Land boundaries: 0.5m 160,000 Chip designers (2) Wafer: 12cm Grid: 0.5m 240,000 Dynamic Air traffic ctrl. tool builder Zone: 50km Plane distance in 25m steps 2,000 Ego shooter gamer Surrounding: 360º Aiming: 0.1º 3,000 Submarine ROV op. Surrounding: 360º Use arms: 1mm/0.05º 8,000 Strategy gamers (2) Map: 30k pixel Aiming: 1 pixel 30,000 field study: users & tasks

  11. Display technology homogeneous resolution 4 VisualizationSame # of pixels fisheye overview plus detail 5 4 4 wall-size, hi-res display … andcurrentsolutions What participants used focus plus context screen Available to½ of participants

  12. experiment 1: • 3 interfaces: • focus plus context screen • overview + detail • homogeneous • 2 tasks • 12 subjects from Xerox PARC • Within subjects, counter-balanced • Same number of pixels

  13. task 1: closest hotel 8 maps per interface F+C screen and O+D use same magnification factor

  14. task 2: verify connections Verify a different set of 24 connections on the board

  15. results manually zoomingtakes time 21% faster 36% faster 700 600 500 zooming + panning 400 overview+detail Average task completion times in seconds 300 focus+context screen 200 visually switching  reorientation 100 0 Map task Board task visually more ambiguous

  16. experiment 2:driving simulation 120 sec sequence 100 fields of nails; 30 rocks; tradeoff

  17. results Error rate only 1/3 of two-monitor setup 25 • Sweet spot:flight simulation, unmanned vehicles… 20 overview+detail focus+context screen 15 Mean number of collisions subjects caused Subjects preferred thef+c interface 10 5 0 run-over nails rocks hit

  18. observations • low-res periphery is often ok • on a wall-size display: If periphery was hi-res I still could not read it due to distance •  focus-plus-context interaction toc

  19. we observed: lots of input problems ktop? ktop? • mouse • pointer moves too fast, users lose track of it • pointer behaves in unpredicted ways when crossing screen boundaries • touch/pen input • users cannot reach their content anymore conclude

  20. high density cursor mouse acrossbezels reach icons mouse acrossdistances reach anything building one conclusions

  21. problem: mouse across long distances • large screen • longer distancesà higher mouse acceleration • temporal aliasing: 500 pixels jumps • lack of visual continuity & weak stimulusà users lose track of the pointer

  22. the problem will get worse • cursor update is limited by screen refresh rate • screen refresh rate has actually decreased (LCDs) • larger screens + lower refresh rate à status quo • future: even larger screens à problem will get worse

  23. fill-in cursorscurrent frame fill-in cursorsprevious frame inserts cursor image between actual cursor positions  the mouse cursor appear more continuous how it works previous cursorposition current cursorposition mouse motion

  24. this is not the mouse trail video • the windows mouse trail… • makes mouse trail last longer • drawback: cursor images lag behind • ...is not high-density cursor • hd cursor makes mouse trail denser • lag-free: mouse stops=>cursor stops

  25. benefits previous cursor position current cursor position mouse motion fill-in cursorscurrent frame mouse motion fill-in cursorsprevious frame • 1. mouse cursor appear more continuous • à easier to track the cursor • 2. higher “visual weight” • à easier to re-acquire the cursor

  26. designs alternatives a b c d e f g h frame acceleration • reference: exponential acceleration

  27. designs alternatives a b c d e f g h frame acceleration • motion blur with higher weight

  28. designs alternatives a b c d chose discreet version 1. latest cursor position is always shown blur-free and in full opacity 2. appearance that users are familiar with today 3. computationally less expensive e f g h frame acceleration • temporal super-sampling vs. motion blur

  29. a b c d e f g h frame acceleration designs alternatives • density = detectability vs. intrusiveness

  30. transfer function distancebetweencursor images hd cursor has no effect transfer function(configurable) cursor trail provides no speed cues onset threshold (configurable) mouse speed

  31. a b c d e f g h frame acceleration designs alternatives • optional cursor growth

  32. user study • conducted pre-study to define interface candidates • interfaces: control vs.high-density cursor (conservative, tripleDensity, plusScaling) • fitts’ law task • triple-mon: button located at 5” to 40” distance • participants: 7 external participants, 5 coworkers • hypotheses • high-density cursor faster • the greater the distance thegreater the effect • tripleDensity and plusScalingfaster than conservative

  33. 102 100 98 short distance conservative speedupup to 7% 96 + scale +3-dense 94 92 90 results regular mouse cursor time % relative to regular cursor high-density cursors 125 250 500 750 1000 target distance (mm)

  34. subjective satisfaction • most participants did not notice that cursor was different!“did that condition use a different mouse acceleration?”…

  35. lesson we learned:display frame rate is not a hard limit toc

  36. mouse ether mouse acrossbezels reach icons mouse acrossdistances reach anything building one conclusions

  37. mouse ether start target

  38. tasks with incorrectvisual feedback

  39. mouse across multimonproblem 1: different dpi start target low dpi high dpi

  40. mouse across multimonproblem 2: the gap start target

  41. contents • problem: why the cursor gets warped • eliminating warping with mouse ether • calibration • user study: up to 28% faster in acquisition task • conclusions: need to test fitts’ law

  42. system oblivious of • dpi differences • system oblivious of • gaps start target why the cursor gets warped start target low dpi high dpi

  43. now users can acquire targets on direct path with mouse ether start target high dpi low dpi start target mouse ether start target low dpi high dpi start target

  44. step 1 step 2 step 3 calibration

  45. (ex,ey,) ’ ’ the math (ex,ey,) +(dx,dy) every frame on init (cx,cy) (cx,cy) ’ ’ screen i screen j

  46. off-screen travel with mouse ether start target

  47. with mouse ether start start blocked target target same dpi same dpi start target start target blocked high dpi low dpi low dpi high dpi target target start start  name ether more off-screen travel user’s perspective system’sperspective

  48. taming off-screen travel • iteration 1:restrict to convex hull • iteration 2:allow entering ether only for the purpose of transit • towards another screen • at sufficient speed • otherwise screen boundaries “hard” target start

  49. user study • 2x 18” screens • 1280x1024 and800x600 resolution • eight participants • WinFitts software • 72 movements(=4 trials x 9 paths x 2 directions)

  50. results • with mouse ether up to 28% faster Control 1800 1600 MouseEther 1400 1200 1000 Movement Time (ms ±SEM) 800 600 400 200 0 High-Low Low-High Direction of Movement

More Related