TapLogger: Inferring User Inputs on Smartphone Touchscreens

1. Introduction • Sensors equipped on a smartphone bring potential risks of leaking user’s private information • We observe the correlations between the tapped position on the touchscreen and the motion changes of smartphones; • Attack Workflow • Step 1: The user is tricked to install the TapLogger app; • Step 2: TapLogger learns the motion change patterns of tap events when the user is interacting with it; • Step 3: TapLogger runs in the background, stealthily monitor the motion changes, and uses the learnt tap event pattern to infer user inputs on touchscreens. • Note that, monitoring the readings of motion sensors requires no security permissions. TapLogger: Inferring User Inputs On Smartphone Touchscreens Using On-board Motion SensorsZhi Xu, Kun Bai, and Sencun Zhu Tap Event Detection • TapLogger detects tap events by monitoring the acceleration changes (i.e. SqSum = Ax2 + Ay2 + Az2) • Unique pattern of tap events • Pattern is user specific and device specific • Experimental results of tap event detection Tap Position Inference • TapLogger infers the position tapped by monitoring the gesture changes (i.e. the readings of Roll and Pitch) • Observed correlations • Use extracted features to distinguish tap events • The training layout and target layout Proposed Applications With TapLogger • Number Pad Logging Attack during the call • Password Stealing Attack when unlocking the phone • The distribution of inferred labels after entering the passwords “5 7 6 8” for 32 rounds • An example of inference • Evaluation with 20 sequences of tap inputs with length of 16 • Evaluation with different passwords (30 rounds each) Reference: This poster is based on the paper “TapLogger: Inferring User Inputs On Smartphone Touchscreens Using On-board Motion Sensors," in Proc. o ACM Conference on Security and Privacy in Wireless and Mobile Networks (WiSec’12)