1 / 47

The Challenges and Potential of End-User Gesture Customization

The Challenges and Potential of End-User Gesture Customization. Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information Studies University of Maryland, College Park. uranoh@cs.umd.edu | leahkf@umd.edu. Touchscreen gestures are widely used…

carr
Download Presentation

The Challenges and Potential of End-User Gesture Customization

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. The Challenges and Potential of End-User Gesture Customization UranOh1 and Leah Findlater2 1 Department of Computer Science 2 College of Information Studies University of Maryland, College Park uranoh@cs.umd.edu | leahkf@umd.edu

  2. Touchscreen gestures are widely used… Who designs these gestures? Design experts. Apple’s touchpad gestures

  3. Previous Research: Tools for supporting designers (developers) to create gestures with ease MAGIC: [Ashbrook et al. 2010] Proton++: [Kin et al. 2012] Gesture Coder: [Lü et al. 2012]

  4. Previous Research: 2) Methods for creating a gesture set that are intuitive and guessable bya wide range of users [Wobbrock et al. 2009], [Kray et al. 2010], [Ruiz et al. 2011] A figure from [Wobbrock et al. 2009]

  5. Previous research: (2) Methods for creating a gesture set that are intuitive and guessable bya wide range of users Our focus: Supporting end-users Personal gestures for a single user [Wobbrock et al. 2009], [Kray et al. 2010], [Ruiz et al. 2011] A figure from [Wobbrock et al. 2009]

  6. Previous research: (2) Methods for creating a gesture set that are intuitive and guessable bya wide range of users Our focus: Supporting end-users Personal gestures for a single user Why? [Wobbrock et al. 2009], [Kray et al. 2010], [Ruiz et al. 2011] A figure from [Wobbrock et al. 2009]

  7. Potential Advantages of Self-defined Gestures… Memorability Efficiency Accessibility

  8. Memorability Self-defined gestures improve memorability over predefined gestures [Nacentaet al. 2013]

  9. Efficiency Gestural shortcuts can be used as an efficient mean of accessing information [Ouyanget al. 2012]

  10. Accessibility Customized gestures mayimprove accessibility for people with physical disabilities [Anthony et al. 2013]

  11. Our Goal: To investigate the feasibility of end-user gesture creation

  12. Our Goal: To investigate the feasibility of end-user gesture creation How do typical users create gestures? What are the challenges therein? How can we support the process?

  13. Study With Three Tasks v Task 2 Task 1 Task 2 Task 3 Saliency of Gesture Features Action-Specific Gesture Creation Open-Ended Gesture Creation

  14. Study Method Controlled lab study - 20 participants (age from 20 to 35, M=29.3) - Prior experience with touchscreen devices - Single one-hour session with 3 tasks - Think-aloud protocol Apparatus - Samsung Galaxy Tab 2 (10.1’’ running Android 4.0.4)

  15. Are users able to create new gestures easily? If not, what are the barriers? Q. v Task 1 Task 2 Task 2 Task 3 Open-Ended Gesture Creation Saliency of Gesture Features Action-Specific Gesture Creation

  16. Task 1: Open-ended Gesture Creation “Create as many gestures as possible”

  17. Task 1: Open-ended Gesture Creation “Create as many gestures as possible” • For any purpose • Any number of strokes, fingers, hands • As long as they are: • easy to draw, easy to remember, distinguishable

  18. Task 1: Open-ended Gesture Creation “Create as many gestures as possible”

  19. Task 1: Open-ended Gesture Creation GesturesCreated 12.2 gestures created on average (SD = 8.1, range of 5 to 36) Total number of gestures and the number of arbitrary gestures are correlated (Pearson’s r=.47, p=.037) p3 p5

  20. Task 1: Open-ended Gesture Creation GesturesCreated 12.2 gestures created on average (SD = 8.1, range of 5 to 36) Total number of gestures and the number of arbitrary gestures are correlated (Pearson’s r=.47, p=.037) p3 p5 p5

  21. Task 1: Open-ended Gesture Creation Difficulties Creating Gestures Tendency to focus on the familiar “I just thought of gestures my tablet PC had.” (P1) “These gestures are all I use, I cannot be more creative” (P8) Opaque nature of gesture recognizer “Can I use all fingers?” (P2)

  22. Users felt difficulty in creating new gestures Better understanding of recognizer is needed A. v Task 1 Task 2 Task 2 Task 3 Open-Ended Gesture Creation Saliency of Gesture Features Action-Specific Gesture Creation

  23. Q. What is a “good gesture” to end-users? How is it different from recognizer’s perspective? Task 2 Task 2 Task 1 Task 3 Action-Specific Gesture Creation Saliency of Gesture Features Open-Ended Gesture Creation

  24. Task 2: Action-Specific Gesture Creation Brainstorm gestures per action 12 Specific Actions Zoom-in Zoom-out Rotate Copy Cut Paste Select-single Select-multiple Previous Next Call-Mom Launch a web-browser

  25. Task 2: Action-Specific Gesture Creation Brainstorm gestures per action 12 Specific Actions Zoom-in Zoom-out Rotate Copy Cut Paste Select-single Select-multiple Previous Next Call-Mom Launch a web-browser

  26. Task 2: Action-Specific Gesture Creation Brainstorm gestures per action Composecustom set of gestures, one per action

  27. Task 2: Action-Specific Gesture Creation Brainstorm gestures per action Composecustom set of gestures, one per action

  28. Task 2: Action-Specific Gesture Creation Brainstorm gestures per action Composecustom set of gestures, one per action

  29. Task 2: Action-Specific Gesture Creation Brainstorm gestures per action Composecustom set of gestures, one per action Create training examples (4 per selected gesture)

  30. Task 2: Action-Specific Gesture Creation Brainstorm gestures per action Composecustom set of gestures, one per action Create training examples (4 per selected gesture) Rate satisfaction with the custom gesture set

  31. Task 2: Action-Specific Gesture Creation Brainstorm gestures per action Composecustom set of gestures, one per action Initial example Test recognition accuracy with $N recognizer Create training examples (4 per selected gesture) Training examples Rate satisfaction with the custom gesture set

  32. Task 2: Action-Specific Gesture Creation Reasons for selecting a gesture for custom set Others reasons: Generally preferred, fast, consistent, easy to remember, etc.

  33. Task 2: Action-Specific Gesture Creation • Only two participants were fully satisfied • ( M=5.3, SD = 1.1 where 1=negative, 7=positive) Need for improvement Inability to improve gesture sets • Participants gave up the opportunity to edit their gesture set to make improvements

  34. Task 2: Action-Specific Gesture Creation Low Recognition Potential of the Custom Sets $N recognizer (default setting) with 5-fold cross validation 76–88% accuracydepending on amount of training

  35. Customized set can be improved for both user’s and recognizer’s perspective A. Task 2 Task 2 Task 1 Task 3 Action-Specific Gesture Creation Saliency of Gesture Features Open-Ended Gesture Creation

  36. What features do users rely on to distinguish between gestures? Q. Task 3 Task 2 Task 1 Task 2 Saliency of Gesture Features Action-Specific Gesture Creation Open-Ended Gesture Creation

  37. Task 3: Saliency of Gesture Features Gesture Features Judged 6 features from Rubine’s recognizer [Rubine. 1991] Orientation Scale Aspect Ratio Pattern Repetition Curviness Speed Very slow Very fast slow fast moderate

  38. Task 3: Saliency of Gesture Features Gesture Features Judged 6 features from Rubine’s recognizer [Rubine. 1991] 3 touchscreen features Finger Count Orientation Scale Stroke Count Aspect Ratio Pattern Repetition Stroke Order Curviness Speed Very slow Very fast slow fast moderate

  39. Task 3: Saliency of Gesture Features “Rank the distinguishability of 9 features” 6 features from Rubine’s recognizer [Rubine. 1991] 3 touchscreen features Finger Count Orientation Scale Stroke Count Aspect Ratio Pattern Repetition Stroke Order Curviness Speed Very slow Very fast slow fast moderate

  40. Task 3: Saliency of Gesture Features Objective features are more distinguishable Features that can be consistently interpreted/manipulated are considered distinguishable “Even if the same person is performing the gesture, it might not have the same speed and size” (P7) Pattern Repetition Stroke Order Finger count Aspect Ratio Stroke count Orientation Curviness Speed Scale Very fast More distinctive

  41. Task 3: Saliency of Gesture Features Objective features are more distinguishable Features that can be consistently interpreted/manipulated are considered distinguishable “Even if the same person is performing the gesture, it might not have the same speed and size” (P7) Pattern Repetition Stroke Order Finger count Aspect Ratio Stroke count Orientation Curviness Speed Scale Very fast More distinctive

  42. Task 3: Saliency of Gesture Features Objective features are more distinguishable Features that can be consistently interpreted/manipulated are considered distinguishable “Even if the same person is performing the gesture, it might not have the same speed and size” (P7) Pattern Repetition Stroke Order Finger count Aspect Ratio Stroke count Orientation Curviness Speed Scale Very fast More distinctive

  43. Number of fingers/strokes, stroke order are distinguishable than speed or size A. Task 3 Task 2 Task 1 Task 2 Saliency of Gesture Features Action-Specific Gesture Creation Open-Ended Gesture Creation

  44. Summary • Creating new gestures is hard for end-users • Tendency to focus on the familiar • Opaque nature of gesture recognizer • Quality of gesture sets can be improved • Users are not fully satisfied with their gesture sets • Low recognition potential • Objective features are more distinguishable • Finger/stroke count, stroke order are more distinguishable than speed and scale

  45. Take-away Message Potential Benefits of Allowing End-User Customization Systematic Support is Needed for End-User Customization Memorability Efficiency Accessibility

  46. Future Work Mixed-initiative support for customization Feedback Gesture set System User Train Edits

  47. Questions? The Challenges and Potential of End-User Gesture Customization Thank you for listening UranOh1 and Leah Findlater2 1 Department of Computer Science 2 College of Information Studies University of Maryland, College Park uranoh@cs.umd.edu | leahkf@umd.edu 

More Related