Decoding Vibrations from Nearby Keyboards Using Mobile Phone Accelometers

1. Decoding Vibrations from Nearby Keyboards Using Mobile Phone Accelometers Philip Marquardt(MIT Lincoln Laboratory) Arunabh VermaHenry CarterPatrick Traynor(Georgia Institute of Technology)

2. INTRODUCTION • In 90s - • No Web browser • No Email Client • Low Processing Power • No Sensors • In 2012 - • Competent with Desktops • Browsers with Flash support • Sophisticated Sensors

3. ACCELEROMETERS • Accelerometers can be used to leak unintended significant information from user’s environment. • Keypresses made on a nearby keyboard can be recorded and reconstructed on the basis of vibrations. • Neural Network used to develop profiles for keypress events. • Dictionaries used to to recover words from the translated content.

4. MOTIVATION • Use of emanations of electrical and mechanical devices to expose information. • Electro-magnetic emanations enabled Data recovery from CRT and LCDs. • Similar attacks on emanations from Smart Cards, CMOS Chips, Serial port cables and keyboards. • Acoustic emanations made by devices easily captured by less capable adversaries.

5. EXPERIMENTAL SETUP • Can keypresses be detected by present Sensors? • Can previously developed methodology be applied to identify keystrokes? • iPhone 4 • Apple A1255 Wireless Bluetooth keyboard • Phone and Accelerometer • Signal Processing (Matlab’s FFT) • RapidMiner/Machine Learning

6. MODELLING • Keypress Event Modeling • Low Sampling rate of accelerometer • Difficult to characterize individual keys • Characterize pairs of key presses by defining a relation • Relation b/w two successive keypresses Pi, Pj using two features • Horizontal Orientation : loc(Pi) • Distance between Consecutive KeyPresses : dist (Pi, Pj)

7. Contd.. • We define consecutive keypress events as rel(Pi, Pj) = loc(Pi)||loc(Pj)||dist(Pi, Pj), where || represents feature concatenation. • Eg: “canoe” • ca : LLN • An : LRF • no : RRF • oe : RLF “canoe” = LLN.LRF.RRF.RLF

8. LEARNING PHASE • Data Collection • Keys A to Z pressed 150 times randomly • 3900 distinct key press events • Feature Extraction • Define a feature vector for every key stroke containing time-domain and frequency domain features • FV (Pi)= <mean, kurtosis, variance, min, max, energy, rms, mfccs, fft> • Word Labeling • Train the model using a dictionary • Each word in the training dictionary is broken down into its constituent characters and character-pairs.

9. ATTACK PHASE • Data Collection • Feature extraction • Key-Press Classification • Word matching • The word matcher takes each predicted word profile of length n-1 and assigns a score against each word in the dictionary with length n.

10. EXPERIMENTALRESULTS 1

11. EXPERIMENTAL RESULTS 2 Test Sentence – First ten sentences from “Harvard Sentences” Dictionary– Same ten sentences.

12. CHALLENGES and LIMITATIONS • Recognition versus Distance – Small increase in distance drastically decreases the effectiveness of attack. • Orientation of Monitoring Device • Possible ambient vibrations (including typing speed) • Characteristics of the desk surface

13. CONCLUSION • The first level of security within smart phone is at the level of Operating System. • Powerful sensors present in Mobile Phones can be utilized for recovering data from nearby keyboard. • Thus, access to sensors must be carefully monitored and regulated to overcome the security hazards. • Concrete protocols must be enforced for authenticated access of sensors by applications.

14. THANK YOU !! Ravikanth Safina Srinivas