Security problems of your keyboard Authentication based on key strokes Compromising emanations consist of electrical, me

1. Security problems of your keyboard • Authentication based on key strokes • Compromising emanations consist of electrical, mechanical, or acoustical • Supply chain attack (Bluetooth, SD card) • Power usage?

2. Key stroke biometrics with number-pad input (DSN 2010) • 28 users typed the same 10 digit number • Use statistical machine learning techniques • Detection rate 99.97% • False alarm rate 1.51% • Can be used for real life two-factor authentication

3. Keyboard Acoustic Emanations Revisited Li Zhuang, Feng Zhou and J. D. Tygar U. C. Berkeley

4. Motivation • Emanations of electronic devices leak information • How much information is leaked by emanations? • Apply statistical learning methods to security • What is learned from recordings of typing on a keyboard?

5. Alicepassword Keyboard Acoustic Emanations • Leaking information by acoustic emanations

6. Acoustic Information in Typing • Frequency information in sound of each typed key • Why do keystrokes make different sounds? • Different locations on the supporting plate • Each key is slightly different • [Asonov and Agrawal 2004]

7. Timing Information in Typing • Time between two keystrokes • Lasting time of a keystroke • E.g. [Song, Wagner and Tian, 2001]

8. Previous Work vs. Our Approach

9. Key Observation • Build acoustic model for keyboard & typist • Non-random typed text (English) • Limited number of words • Limited letter sequences (spelling) • Limited word sequences (grammar) • Build language model • Statistical learning theory • Natural language processing

10. Language Model Correction keystroke classifierrecovered keystrokes Overview Initial training Subsequent recognition wave signal wave signal Feature Extraction Feature Extraction Unsupervised Learning Keystroke Classifier Language Model Correction (optional) Sample Collector Classifier Builder recovered keystrokes

11. Language Model Correction keystroke classifierrecovered keystrokes Feature Extraction Initial training Subsequent recognition wave signal wave signal Feature Extraction Feature Extraction Unsupervised Learning Keystroke Classifier Language Model Correction (optional) Sample Collector Classifier Builder recovered keystrokes

12. Sound of a Keystroke • How to represent each keystroke? • Vector of features: FFT, Cepstrum • Cepstrum features used in speech recognition

13. Linear Classification Neural Networks Gaussian Mixtures Training Test 1 Test 2 Training Test 1 Test 2 Training Test 1 Test 2 Cepstrum vs. FFT • Repeat experiments from [Asonov and Agrawal 2004] 1 accuracy 0

14. Language Model Correction keystroke classifierrecovered keystrokes Unsupervised Learning Initial training Subsequent recognition wave signal wave signal Feature Extraction Feature Extraction Unsupervised Learning Keystroke Classifier Language Model Correction (optional) Sample Collector Classifier Builder recovered keystrokes

15. Unsupervised Learning • Group keystrokes into N clusters • Assign keystroke a label, 1, …, N • Find best mapping from cluster labels to characters • Some character combinations are more common • “th” vs. “tj” • Hidden Markov Models (HMMs)

16. “t” “h” “e” Bi-grams of Characters • Colored circles: cluster labels • Empty circles: typed characters • Arrows: dependency 5 11 2 EM

17. keystroke classifierrecovered keystrokes Language Model Correction Initial training Subsequent recognition wave signal wave signal Feature Extraction Feature Extraction Unsupervised Learning Keystroke Classifier Language Model Correction Language Model Correction (optional) Sample Collector Classifier Builder recovered keystrokes

18. Word Tri-grams • Spelling correction • Simple statistical model of English grammar • Use HMMs again to model

19. Two Copies of Recovered Text Before spelling and grammar correction After spelling and grammar correction _____ = errors in recovery = errors in corrected by grammar

20. keystroke classifierrecovered keystrokes Sample Collector Initial training Subsequent recognition wave signal wave signal Feature Extraction Feature Extraction Unsupervised Learning Keystroke Classifier Language Model Correction Language Model Correction (optional) Sample Collector Classifier Builder recovered keystrokes

21. keystroke classifierrecovered keystrokes Feedback-based Training Initial training Subsequent recognition wave signal wave signal Feature Extraction Feature Extraction Unsupervised Learning Keystroke Classifier Language Model Correction Language Model Correction (optional) Sample Collector Classifier Builder recovered keystrokes

22. Feedback-based Training • Recovered characters • Language correction • Feedback for more rounds of training • Output: keystroke classifier • Language independent • Can be used to recognize random sequence of keys • E.g. passwords • Representation of keystroke classifier • Neural networks, linear classification, Gaussian mixtures

23. keystroke classifierrecovered keystrokes Keystroke Classifier Initial training Subsequent recognition wave signal wave signal Feature Extraction Feature Extraction Unsupervised Learning Keystroke Classifier Language Model Correction Language Model Correction (optional) Sample Collector Classifier Builder recovered keystrokes

24. Experiment (1) • Single keyboard • Logitech Elite Duo wireless keyboard • 4 data sets recorded in two settings • Quiet & noisy • Keystrokes are clearly separable from consecutive keys • Automatically extract keystroke positions in the signal with some manual error correction

25. Data sets Initial & final recognition rate

26. Experiment (2) • Multiple Keyboards • Keyboard 1: DELL QuietKey PS/2, P/N: 2P121 • In use for about 6 months • Keyboard 2: DELL QuietKey PS/2, P/N: 035KKW • In use for more than 5 years • Keyboard 3: DELL Wireless Keyboard, P/N: W0147 • New

27. 12-minute recording with ~2300 characters

28. Experiment (3) • Classification methods in feedback-based training • Neural Networks (NN) • Linear Classification (LC) • Gaussian Mixtures (GM)

29. Limitations of Our Experiments • Considered letters, period, comma, space, enter • Did not consider numbers, other punctuation, backspace, shift, etc. • Easily separable keystrokes • Only considered white noise (e.g. fans)

30. Defenses • Physical security • Two-factor authentication • Masking noise • Keyboards with uniform sound (?)

31. Summary • Recover keys from only the sound • Using typing of English text for training • Apply statistical learning theory to security • Clustering, HMMs, supervised classification, feedback incremental learning • Recover 96% of typed characters