Li Zhuang, Feng Zhou, and J. D. Tygar
University of California, Berkeley
Abstract: We examine the problem of keyboard acoustic emanations. We present a novel attack taking as input a 10-minute sound recording of a user typing English text using a keyboard, and then recovering up to 96% of typed characters. There is no need for a labeled training corpus. Our attack employs a combination of machine learning and speech recognition techniques, including cepstrum features and hidden Markov models, and uses an English language model as well as spelling and grammar checking to correct decoding errors. Our results suggest that using microphones to recover typed text is a serious threat, not only for keyboards but also for other devices with mechanical parts.
Overview
We show that using a generic microphone, we can successfully recover almost all text typed on standard keyboards. Unlike previous research, our method works even if we have no information about the typist, the keyboard, and no "training data" (examples of the typist typing known text). Simply put a microphone in a room with a typist, record 10 minutes of data, and our algorithms recover the typed text - including arbitrary text, such as passwords.
Our work breaks even "quiet" keyboards that are designed not to make sounds. Our results suggest that recovery is possible even if microphones are outside the room (using parabolic microphones).
Key findings
Up to 96% of typed characters can be recovered from a 10-minute recording
No prior knowledge of the typist or keyboard is required
The attack uses unsupervised learning - no labeled training data needed
Standard "quiet" keyboards are also vulnerable
Recovery may be possible even with microphones placed outside the room
Methodology
Our approach combines several techniques:
Feature extraction using cepstrum coefficients from keystroke sounds
Unsupervised clustering to group similar-sounding keystrokes
Hidden Markov Models to decode the sequence of key presses
Language modeling with English text statistics to improve accuracy
Spelling and grammar correction as a post-processing step
Publication
This paper appeared at the 12th ACM Conference on Computer and Communications Security (CCS 2005), November 2005, Alexandria, Virginia.
Li Zhuang, Feng Zhou, and J. D. Tygar. "Keyboard Acoustic Emanations Revisited." Proceedings of the 12th ACM Conference on Computer and Communications Security, pp. 373-382, 2005.