220 likes | 310 Views
Making Touchscreen Keyboards Adaptive to Keys , Hand Postures , and Individuals – A Hierarchical Spatial Backoff Model Approach. Ying Yin 1,2 , Tom Ouyang 1 , Kurt Partridge 1 , and Shumin Zhai 1. 1 Google Logo here. 2 MIT Logo here. Foundations to current methods. Language modeling
E N D
Making Touchscreen Keyboards Adaptive to Keys, Hand Postures, and Individuals– A Hierarchical Spatial Backoff Model Approach Ying Yin1,2, Tom Ouyang1, Kurt Partridge1, and Shumin Zhai1 1 Google Logo here 2MIT Logo here
Foundations to current methods • Language modeling • vocabulary • 1-gram, 2-gram … N gram frequencies • Spatial models • converting input touch points into probabilities of letters • Edit distance correction • assigning cost to insertion, deletion, and other spelling errors User and posture independent
Research questions • One promising area for improvement is by making them adapt to the user • What types of adaption are possible? • How do they affect performance?
Contributions • A novel hierarchical adaptive model • Show benefits of posture and user adaptation • Online posture classification method • 13.2% reduction in character error rate • compared to base model • without language model
Types of adaptation • Individual differences (cf. Findlater & Wobbrock, 2012) • Furthermore, people use different hand postures to type (cf. Azenkot and Zhai, 2012)
Different typing postures: two thumbs, one finger, or one thumb
Types of adaptation • Different postures different touch patterns • Touch patterns also depend on letter keys Need adaptation (Azenkot & Zhai, 2012)
Challenges of adaptation • Complexity • three adaptive factors: key, posture, individual • large number of submodels • need sufficient data to build each submodel • Model selection • wrong selection may hurt keyboard quality • uncertainty in posture classification
Hierarchical spatial backoff model (SBM) • Combinatorial and fine grained adaptation • Conservative • Does not require an extra training phase • Updates the model continuously online
Research method • “Pepper” dataset (Azenkot & Zhai, 2012) • 30 right-handed participants • given random phrases to type • between-subject: each person uses one posture • 84,292 touch points in total • 10-fold cross validation
Effective key areas Two-thumb One-finger
Posture classification • SVM-based classifier • Based on correlation between time and distance between consecutive touch points • no additional sensors required • speed independent • 86.4% accuracy • Real-time
Prototype implementation of SBM • 13.2% reduction in character error rate • compared to base model • without language model • Integrated with real keyboard • combined with language model • runs on Android phone in real-time
Future work • Weighted average of submodels instead of making binary decisions • More data: real-use logging and game playing • User studies • validate the accuracy and speed improvement • how users adapt their behavior to SBM • Combine spatial and language models
Contributions • A novel hierarchical adaptive model • Show benefits of posture and user adaptation • Online posture classification method • Opens up many more interesting HCI questions
Prototype implementation of SBM • Posture & key adaptation models • supervised and batch learning • Individual adaptation models • unsupervised and online learning • Backs-off to more basic models when • posture estimation is uncertain (conf. < 0.94) • there is insufficient user data (< 50 data points)