Multi-modal Biometric Verification for Small and Very Small Devices

1. Multi-modal Biometric Verification for Small and Very Small Devices

2. Overview • Background and application: SecurePhone • Multimodal biometric recognition • face, voice, signature: natural • For small devices: PDA • Good performance, short verification time • Security problem • For very small devices: SIM card • Global features to run on slow CPU • Short verification time, acceptable performance • Conclusion • Further improvements by glottal feature fusion? • Relevance for COST2102 Int’l Workshop on Verbal and Nonverbal Communication Behaviours, Vietri sul Mare, 29-31 March 2007, slide

3. Financing: EU 6th framework IST Background: SecurePhone project • Duration: 01.01.2004 – 30.11.2006 • Aim: “a mobile phone with biometric authentication and e-signature support for dealing secure transactions on the fly” • SecurePhone consortium: • Management • Research • Implementation • Exploitation Int’l Workshop on Verbal and Nonverbal Communication Behaviours, Vietri sul Mare, 29-31 March 2007, slide

4. video camera datacapture touch screen datacapture biometric recogniser datacapture biometric preprocessor microphone SecurePhone e-signature manager SIM card PINnumber GPRS/UMTS Int’l Workshop on Verbal and Nonverbal Communication Behaviours, Vietri sul Mare, 29-31 March 2007, slide

5. Later: HL4 LL HL LH HH Multimodal biometric recogniser Voice Signature Face Haar LL4 wavelets MFCCs geometric features GMM GMM GMM “biometricrecogniser” user profile world model accept userrelease private key reject user Int’l Workshop on Verbal and Nonverbal Communication Behaviours, Vietri sul Mare, 29-31 March 2007, slide

6. PDA: fusion results for PDAtabase Marcos Faundez-Zanuy:Face recognition:an unsolved problem DET curves/result table for 5-digit (left), 10-digit (middle) and phrase prompts (right) Int’l Workshop on Verbal and Nonverbal Communication Behaviours, Vietri sul Mare, 29-31 March 2007, slide

7. From small to very small devices: problem • Biometric data cannot be stored or processed on the PDA, because impostors could steal biometric data. • Therefore storage and processing must be on SIMcard, which self-destroys when tampered with physically. • Instead of a few seconds on the PDA, verification on the SIMcard takes one hour! • Bottleneck: large number of comparisons in voice and signature verification (for client model and UBM) • for large number of frames per prompt • for large number of Gaussian mixtures in GMM Int’l Workshop on Verbal and Nonverbal Communication Behaviours, Vietri sul Mare, 29-31 March 2007, slide

8. From small to very small devices: solution • Reducing the frame rate or the number of GMM mixtures cannot reduce the processing time in a sufficient order of magnitude • Drastic solution: globalised features (idea taken from static signature representations) • Means (cf. Long-Term Average Spectrum for voice) and standard deviations per vector parameter across all frames; also greatly reduced number of Gaussians required for modelling the vectors • To counteract the effect of averaging, compute globalised features for subparts of the signal Marcos Faundez-Zanuy:Open your mind: sometimes a simple solution can give a good result“ (and sometimes you cannot get around it) Int’l Workshop on Verbal and Nonverbal Communication Behaviours, Vietri sul Mare, 29-31 March 2007, slide

9. PDA results EER (percent) for globalised means (columns 2-5) and means plus standard deviations (columns 6-9) Int’l Workshop on Verbal and Nonverbal Communication Behaviours, Vietri sul Mare, 29-31 March 2007, slide

10. SIM card results EER (percent) for globalised means (columns 2-5) and means plus standard deviations (columns 6-9) for voice and signature divided into two equal subparts Int’l Workshop on Verbal and Nonverbal Communication Behaviours, Vietri sul Mare, 29-31 March 2007, slide

11. Improvement needed • Performance drop: • PDA EER 2.39% (meanwhile improved to 0.9%) • SIM EER 10.05% (8.28 for two equal subparts) • Performance can be improved if we do not restrain the GMM models to be the same across all modalities • Otherwise: Use of complementary features within a modality • Face: simple face geometric variables • Voice: parameter values of LF model fitted to glottal flow derivative, obtained from inverse filtering of mic signal Int’l Workshop on Verbal and Nonverbal Communication Behaviours, Vietri sul Mare, 29-31 March 2007, slide

12. Interest to this COST action • Interest in glottal flow derivative for speaker recognition stems from • expected complementarity to MFCC representation of spectrum • applicability in applications which use very little training data (as in SecurePhone, for user-friendliness) • But can also be useful for other classification problems, like “the recognition of emotional states, gesture, speech and facial expressions, in anticipation of the implementation of useful application such as intelligent avatars and interactive dialog systems”(quote from aims website of this workshop) Int’l Workshop on Verbal and Nonverbal Communication Behaviours, Vietri sul Mare, 29-31 March 2007, slide

13. Last night’s addendum: speech & gestures • Source signal parameters can also be used together with other spectral parameters as well as F0, duration, loudness measures to signal prominence. • In speech, these signals can be used differently across languages (syllable-timed vs. stress-timed) and speakers (German Research Council “rhythm project” led by Bill Barry, Saarland University, to which NTNU contributes with Norwegian database recordings and analyses). • Prominence also signalled by extent/size as well as acceleration of gestures. • In how far do gestures and speech signal parameters correlate? When are they used as complementary/ alternative strategies for signalling prominence? Int’l Workshop on Verbal and Nonverbal Communication Behaviours, Vietri sul Mare, 29-31 March 2007, slide

14. Thank you for your attention. Int’l Workshop on Verbal and Nonverbal Communication Behaviours, Vietri sul Mare, 29-31 March 2007, slide