470 likes | 571 Views
The Challenges and Potential of End-User Gesture Customization. Uran Oh 1 and Leah Findlater 2 1 Department of Computer Science 2 College of Information Studies University of Maryland, College Park. uranoh@cs.umd.edu | leahkf@umd.edu. Touchscreen gestures are widely used…
E N D
The Challenges and Potential of End-User Gesture Customization UranOh1 and Leah Findlater2 1 Department of Computer Science 2 College of Information Studies University of Maryland, College Park uranoh@cs.umd.edu | leahkf@umd.edu
Touchscreen gestures are widely used… Who designs these gestures? Design experts. Apple’s touchpad gestures
Previous Research: Tools for supporting designers (developers) to create gestures with ease MAGIC: [Ashbrook et al. 2010] Proton++: [Kin et al. 2012] Gesture Coder: [Lü et al. 2012]
Previous Research: 2) Methods for creating a gesture set that are intuitive and guessable bya wide range of users [Wobbrock et al. 2009], [Kray et al. 2010], [Ruiz et al. 2011] A figure from [Wobbrock et al. 2009]
Previous research: (2) Methods for creating a gesture set that are intuitive and guessable bya wide range of users Our focus: Supporting end-users Personal gestures for a single user [Wobbrock et al. 2009], [Kray et al. 2010], [Ruiz et al. 2011] A figure from [Wobbrock et al. 2009]
Previous research: (2) Methods for creating a gesture set that are intuitive and guessable bya wide range of users Our focus: Supporting end-users Personal gestures for a single user Why? [Wobbrock et al. 2009], [Kray et al. 2010], [Ruiz et al. 2011] A figure from [Wobbrock et al. 2009]
Potential Advantages of Self-defined Gestures… Memorability Efficiency Accessibility
Memorability Self-defined gestures improve memorability over predefined gestures [Nacentaet al. 2013]
Efficiency Gestural shortcuts can be used as an efficient mean of accessing information [Ouyanget al. 2012]
Accessibility Customized gestures mayimprove accessibility for people with physical disabilities [Anthony et al. 2013]
Our Goal: To investigate the feasibility of end-user gesture creation
Our Goal: To investigate the feasibility of end-user gesture creation How do typical users create gestures? What are the challenges therein? How can we support the process?
Study With Three Tasks v Task 2 Task 1 Task 2 Task 3 Saliency of Gesture Features Action-Specific Gesture Creation Open-Ended Gesture Creation
Study Method Controlled lab study - 20 participants (age from 20 to 35, M=29.3) - Prior experience with touchscreen devices - Single one-hour session with 3 tasks - Think-aloud protocol Apparatus - Samsung Galaxy Tab 2 (10.1’’ running Android 4.0.4)
Are users able to create new gestures easily? If not, what are the barriers? Q. v Task 1 Task 2 Task 2 Task 3 Open-Ended Gesture Creation Saliency of Gesture Features Action-Specific Gesture Creation
Task 1: Open-ended Gesture Creation “Create as many gestures as possible”
Task 1: Open-ended Gesture Creation “Create as many gestures as possible” • For any purpose • Any number of strokes, fingers, hands • As long as they are: • easy to draw, easy to remember, distinguishable
Task 1: Open-ended Gesture Creation “Create as many gestures as possible”
Task 1: Open-ended Gesture Creation GesturesCreated 12.2 gestures created on average (SD = 8.1, range of 5 to 36) Total number of gestures and the number of arbitrary gestures are correlated (Pearson’s r=.47, p=.037) p3 p5
Task 1: Open-ended Gesture Creation GesturesCreated 12.2 gestures created on average (SD = 8.1, range of 5 to 36) Total number of gestures and the number of arbitrary gestures are correlated (Pearson’s r=.47, p=.037) p3 p5 p5
Task 1: Open-ended Gesture Creation Difficulties Creating Gestures Tendency to focus on the familiar “I just thought of gestures my tablet PC had.” (P1) “These gestures are all I use, I cannot be more creative” (P8) Opaque nature of gesture recognizer “Can I use all fingers?” (P2)
Users felt difficulty in creating new gestures Better understanding of recognizer is needed A. v Task 1 Task 2 Task 2 Task 3 Open-Ended Gesture Creation Saliency of Gesture Features Action-Specific Gesture Creation
Q. What is a “good gesture” to end-users? How is it different from recognizer’s perspective? Task 2 Task 2 Task 1 Task 3 Action-Specific Gesture Creation Saliency of Gesture Features Open-Ended Gesture Creation
Task 2: Action-Specific Gesture Creation Brainstorm gestures per action 12 Specific Actions Zoom-in Zoom-out Rotate Copy Cut Paste Select-single Select-multiple Previous Next Call-Mom Launch a web-browser
Task 2: Action-Specific Gesture Creation Brainstorm gestures per action 12 Specific Actions Zoom-in Zoom-out Rotate Copy Cut Paste Select-single Select-multiple Previous Next Call-Mom Launch a web-browser
Task 2: Action-Specific Gesture Creation Brainstorm gestures per action Composecustom set of gestures, one per action
Task 2: Action-Specific Gesture Creation Brainstorm gestures per action Composecustom set of gestures, one per action
Task 2: Action-Specific Gesture Creation Brainstorm gestures per action Composecustom set of gestures, one per action
Task 2: Action-Specific Gesture Creation Brainstorm gestures per action Composecustom set of gestures, one per action Create training examples (4 per selected gesture)
Task 2: Action-Specific Gesture Creation Brainstorm gestures per action Composecustom set of gestures, one per action Create training examples (4 per selected gesture) Rate satisfaction with the custom gesture set
Task 2: Action-Specific Gesture Creation Brainstorm gestures per action Composecustom set of gestures, one per action Initial example Test recognition accuracy with $N recognizer Create training examples (4 per selected gesture) Training examples Rate satisfaction with the custom gesture set
Task 2: Action-Specific Gesture Creation Reasons for selecting a gesture for custom set Others reasons: Generally preferred, fast, consistent, easy to remember, etc.
Task 2: Action-Specific Gesture Creation • Only two participants were fully satisfied • ( M=5.3, SD = 1.1 where 1=negative, 7=positive) Need for improvement Inability to improve gesture sets • Participants gave up the opportunity to edit their gesture set to make improvements
Task 2: Action-Specific Gesture Creation Low Recognition Potential of the Custom Sets $N recognizer (default setting) with 5-fold cross validation 76–88% accuracydepending on amount of training
Customized set can be improved for both user’s and recognizer’s perspective A. Task 2 Task 2 Task 1 Task 3 Action-Specific Gesture Creation Saliency of Gesture Features Open-Ended Gesture Creation
What features do users rely on to distinguish between gestures? Q. Task 3 Task 2 Task 1 Task 2 Saliency of Gesture Features Action-Specific Gesture Creation Open-Ended Gesture Creation
Task 3: Saliency of Gesture Features Gesture Features Judged 6 features from Rubine’s recognizer [Rubine. 1991] Orientation Scale Aspect Ratio Pattern Repetition Curviness Speed Very slow Very fast slow fast moderate
Task 3: Saliency of Gesture Features Gesture Features Judged 6 features from Rubine’s recognizer [Rubine. 1991] 3 touchscreen features Finger Count Orientation Scale Stroke Count Aspect Ratio Pattern Repetition Stroke Order Curviness Speed Very slow Very fast slow fast moderate
Task 3: Saliency of Gesture Features “Rank the distinguishability of 9 features” 6 features from Rubine’s recognizer [Rubine. 1991] 3 touchscreen features Finger Count Orientation Scale Stroke Count Aspect Ratio Pattern Repetition Stroke Order Curviness Speed Very slow Very fast slow fast moderate
Task 3: Saliency of Gesture Features Objective features are more distinguishable Features that can be consistently interpreted/manipulated are considered distinguishable “Even if the same person is performing the gesture, it might not have the same speed and size” (P7) Pattern Repetition Stroke Order Finger count Aspect Ratio Stroke count Orientation Curviness Speed Scale Very fast More distinctive
Task 3: Saliency of Gesture Features Objective features are more distinguishable Features that can be consistently interpreted/manipulated are considered distinguishable “Even if the same person is performing the gesture, it might not have the same speed and size” (P7) Pattern Repetition Stroke Order Finger count Aspect Ratio Stroke count Orientation Curviness Speed Scale Very fast More distinctive
Task 3: Saliency of Gesture Features Objective features are more distinguishable Features that can be consistently interpreted/manipulated are considered distinguishable “Even if the same person is performing the gesture, it might not have the same speed and size” (P7) Pattern Repetition Stroke Order Finger count Aspect Ratio Stroke count Orientation Curviness Speed Scale Very fast More distinctive
Number of fingers/strokes, stroke order are distinguishable than speed or size A. Task 3 Task 2 Task 1 Task 2 Saliency of Gesture Features Action-Specific Gesture Creation Open-Ended Gesture Creation
Summary • Creating new gestures is hard for end-users • Tendency to focus on the familiar • Opaque nature of gesture recognizer • Quality of gesture sets can be improved • Users are not fully satisfied with their gesture sets • Low recognition potential • Objective features are more distinguishable • Finger/stroke count, stroke order are more distinguishable than speed and scale
Take-away Message Potential Benefits of Allowing End-User Customization Systematic Support is Needed for End-User Customization Memorability Efficiency Accessibility
Future Work Mixed-initiative support for customization Feedback Gesture set System User Train Edits
Questions? The Challenges and Potential of End-User Gesture Customization Thank you for listening UranOh1 and Leah Findlater2 1 Department of Computer Science 2 College of Information Studies University of Maryland, College Park uranoh@cs.umd.edu | leahkf@umd.edu