Evaluation of Biometric Identification Systems - PowerPoint PPT Presentation

evaluation of biometric identification systems n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Evaluation of Biometric Identification Systems PowerPoint Presentation
Download Presentation
Evaluation of Biometric Identification Systems

play fullscreen
1 / 52
Evaluation of Biometric Identification Systems
295 Views
Download Presentation
ganya
Download Presentation

Evaluation of Biometric Identification Systems

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Evaluation of Biometric Identification Systems Dr. Bill Barrett, CISE department and US National Biometric Test Center San Jose State University email: wbarrett@email.sjsu.edu

  2. The Biometric Test Center • Funded by several federal agencies • Centered in a disinterested university setting • Provide objective evaluations of commercial biometric instruments • Provide consulting services to sponsors regarding the most effective application of biometric instruments • No funds accepted from vendors • No independent competing research

  3. Summary of Presentation • Three Basic Biometric Operations • Measures of effectiveness - the ROC curve • Comparison Rate Measures • Collection Variables • Evaluation strategies • Testing issues • Some results • Conclusion

  4. Three Basic Biometric Operations

  5. Enrollment: first time in. Verification: does this credit card belong to this person? Identification: who is this person, anyway? The Three Biometric Operations

  6. Enrollment

  7. Verification

  8. Identification

  9. COMPARE operation • Yields a DISTANCE measure between candidate C and template T • d = distance(C, T) • d LARGE: C probably is NOT T • d SMALL: C probably IS T • NOTE: is reversed for fingerprints

  10. Distance Measures • Euclidean • Hamming • Mahalonobis

  11. Variations on the basic measurement plan • 3 strikes and you’re out • Multiple templates of same person • Template replacement over time • Template averaging • Binning

  12. Binning • Find some way to segment the templates, e.g. • male/female • particular finger • loop vs. whorl vs. arch • May have to include the same template in different bins • Improves search performance, may reduce search accuracy (more false non-matches)

  13. Measures of Effectiveness

  14. Distributions of Large Ensemble of Candidates

  15. Integrated Distributions

  16. Cross-over Threshold • tc = cross-over threshold • where probability of false match: I = probability of false rejection: A

  17. Changing the Device Threshold • td > tc : reduces false rejection: A increases false match: I (bank ATM choice) • td < tc : increases false rejection: A reduces false match: I (prison guard choice)

  18. The d-prime Measure • Measures the overall quality of a biometric instrument. • d’ usually in the range of 2 to 10, logarithmic, like the Richter Scale. • Assumes normal distribution.

  19. Comparison Rate Measures

  20. Penetration Rate • Percentage of templates that must be individually compared to a candidate, given some binning. • Search problem: usually exhaustive search, with some comparison algorithm, no reliable tree or hash classification. • Low penetration rate implies faster searching

  21. Example: fingerprints AFIS (FBI automated classification system) classifies by: • Left loop/ right loop • Arch/whorl • Unknown Then • Exhaustive search of the subset of prints

  22. Jain, Hong, Pankanti & Bolle, An Identity-Authentication System Using Fingerprints, Proc. IEEE vol. 85, No. 9, Sept. 1997

  23. Bin Error Rate • Probability that a search for a matching template will fail owing to an incorrect bin placement • Related to confidence in the binning strategy • AFIS Bin error typically < 1%

  24. Collection Variables

  25. Collection Variables • Physical variations during biometric collection that may change the measurement • Translation/scaling/rotation usually compensated in software • Tend to increase the width of the authentics distribution, and thus • ...make it easier to get a false rejection • ...cause a smaller d’

  26. Liveness Issue Can the device detect that the subject is live? • Fake face recognition with a photograph? • ...or a rubber print image (fingerprint)? • ...or a glass eye (iris encoding)?

  27. Collection Variables -- Fingerprints • Pressure • Angle of contact • Stray fluids, film buildup • Liveness

  28. Collection Variables - Hand Geometry • Finger positioning (usually constrained by pins) • Rings • Aging • Liveness

  29. Collection Variables -Iris Identification • Lateral angle of head • Focus quality • Some people have very dark irises; hard to distinguish from pupil • Outer diameter of iris difficult to establish • Eyelids, lashes may interfere • NO sunglasses • Liveness can be established from live video

  30. Collection Variables -Palm Print • Pressure • Stray fluids, film buildup • Liveness

  31. Collection Variables -Face Recognition • 3D angles • Lighting • Background • Expression • Hairline • Artifacts (beard, glasses) • Aging • Liveness: smiling, blinking

  32. Collection Variables -Voice Recognition • Speed of delivery • Articulation • Nervousness • Aging • Laryngitis • Liveness: choose speech segments for the user to repeat, i.e. “Say 8. Say Q. Say X”

  33. Example - Miros Face Recognition System • Lighting is specified • Static background, subtracted from candidate image to segment face • Camera mounted to a wall - standing candidate • Height of eyes above floor used as an auxiliary measure • Verification only recommended • Liveness - can be fooled with a color photograph

  34. Example - FaceitTM Face Recognition System • No particular lighting specified; it expects similar lighting & expression of candidate and template • Face segmented from background using live video • Face lateral angles not well tolerated • Liveness: blinking, smiling test

  35. Evaluation Strategies

  36. Common Factors • Bio capture: easy to capture the full image • Bio encoding algorithm: often proprietary • Bio encoding: usually proprietary • Database distance: may be proprietary

  37. Convenience Factors • Many are concerned about intrusiveness • Some are concerned about touching • What is the candidate’s learning curve? ...device may require some training

  38. Collecting a Template Database for Testing • Precise identity : code registration • Getting plenty of variety: gender, age, race • Getting many images of same identity • Getting many different images • Significant time frame

  39. Practical Databases • Many large template databases with unique identities & single images available • Many large databases with inaccurate identity correlation • Many databases with limited diversity • Difficult to collect data over time

  40. Some Results

  41. Hand Geometry for INSPASS • INSPASS: INS Passenger Accelerated Service System • Collected 3,000 raw transaction records • Unique individuals in database (separate magnetic identity card) • ...from three international airports • Statistical modelling is suspect for this data • Experimental d’ is 2.1; equal error rate ~2.5%

  42. J. L. Wayman, Evaluation of the Inspass Hand Geometry Data, 1997

  43. FaceitTM General Comments • Supported by a flexible Software Development Kit (SDK), using Microsoft Visual C++TM • Several example applications • Well documented • Can use any video camera • Segments a face with motion video • Liveness: smile or eye blink

  44. FaceitTM Face Recognition • No lighting conditions specified • No background conditions specified • Multiple faces can be segmented • The database includes full images, with default limit of 100 templates • Image conversion and code comparison is separated, therefore is testable

  45. FaceitTM Face Recognition

  46. FaceitTM Study Summary • Done by senior computer engineering students • Not a fully diversified, controlled experiment • 50 different persons, 10 images each • Overall time frame ~ 2 months • Equal error rate crossover point ~5.5%

  47. MirosTM Face Recognition • Designed for verification • Lighting conditions specified • Static background - system takes snapshot of background, uses it to segment a face • Keyboard code plus two images • Double image helps liveness • Software Development Kit similar to Faceit • Equal crossover error rate ~5%

  48. IriscanTM Recognition System • Based on a patent by John Daugman, US 5,291,560, Mar. 1, 1994 • Uses iris patterns laid down a few months after birth. Claims no significant aging over lifespan. • Claims high d’, yielding cross-over error rate < 1 in 1.2 million • Claims high rate of code comparison. Hamming distance. ~100,000 IrisCodes/second on a PC.

  49. IrisScanTM Performance Face Recognition, Spring-Verlag 1998

  50. IriscanTM Observations • Capture equipment more expensive: zoom / telephoto / swivel robotics / autofocus • Question of conversion and code standardization: most of the system is proprietary • Liveness • Promises to have the highest discrimination of all