1 / 22

Ying Yin 1,2 , Tom Ouyang 1 , Kurt Partridge 1 , and Shumin Zhai 1

Making Touchscreen Keyboards Adaptive to Keys , Hand Postures , and Individuals – A Hierarchical Spatial Backoff Model Approach. Ying Yin 1,2 , Tom Ouyang 1 , Kurt Partridge 1 , and Shumin Zhai 1. 1 Google Logo here. 2 MIT Logo here. Foundations to current methods. Language modeling

tovi
Download Presentation

Ying Yin 1,2 , Tom Ouyang 1 , Kurt Partridge 1 , and Shumin Zhai 1

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Making Touchscreen Keyboards Adaptive to Keys, Hand Postures, and Individuals– A Hierarchical Spatial Backoff Model Approach Ying Yin1,2, Tom Ouyang1, Kurt Partridge1, and Shumin Zhai1 1 Google Logo here 2MIT Logo here

  2. Foundations to current methods • Language modeling • vocabulary • 1-gram, 2-gram … N gram frequencies • Spatial models • converting input touch points into probabilities of letters • Edit distance correction • assigning cost to insertion, deletion, and other spelling errors User and posture independent

  3. Research questions • One promising area for improvement is by making them adapt to the user • What types of adaption are possible? • How do they affect performance?

  4. Contributions • A novel hierarchical adaptive model • Show benefits of posture and user adaptation • Online posture classification method • 13.2% reduction in character error rate • compared to base model • without language model

  5. Types of adaptation • Individual differences (cf. Findlater & Wobbrock, 2012) • Furthermore, people use different hand postures to type (cf. Azenkot and Zhai, 2012)

  6. Different typing postures: two thumbs, one finger, or one thumb

  7. Types of adaptation • Different postures  different touch patterns • Touch patterns also depend on letter keys Need adaptation (Azenkot & Zhai, 2012)

  8. Challenges of adaptation • Complexity • three adaptive factors: key, posture, individual • large number of submodels • need sufficient data to build each submodel • Model selection • wrong selection may hurt keyboard quality • uncertainty in posture classification

  9. Hierarchical spatial backoff model (SBM)

  10. Hierarchical spatial backoff model (SBM) • Combinatorial and fine grained adaptation • Conservative • Does not require an extra training phase • Updates the model continuously online

  11. Research method • “Pepper” dataset (Azenkot & Zhai, 2012) • 30 right-handed participants • given random phrases to type • between-subject: each person uses one posture • 84,292 touch points in total • 10-fold cross validation

  12. Comparison of spatial models

  13. Effective key areas Two-thumb One-finger

  14. Posture classification • SVM-based classifier • Based on correlation between time and distance between consecutive touch points • no additional sensors required • speed independent • 86.4% accuracy • Real-time

  15. Posture adaptation

  16. Individual adaptation

  17. Prototype implementation of SBM • 13.2% reduction in character error rate • compared to base model • without language model • Integrated with real keyboard • combined with language model • runs on Android phone in real-time

  18. Future work • Weighted average of submodels instead of making binary decisions • More data: real-use logging and game playing • User studies • validate the accuracy and speed improvement • how users adapt their behavior to SBM • Combine spatial and language models

  19. Contributions • A novel hierarchical adaptive model • Show benefits of posture and user adaptation • Online posture classification method • Opens up many more interesting HCI questions

  20. Q & A

  21. Prototype implementation of SBM • Posture & key adaptation models • supervised and batch learning • Individual adaptation models • unsupervised and online learning • Backs-off to more basic models when • posture estimation is uncertain (conf. < 0.94) • there is insufficient user data (< 50 data points)

More Related