Face recognition: Opportunities and Challenges - PowerPoint PPT Presentation

Face recognition opportunities and challenges l.jpg
Download
1 / 41

Microsoft Research March 17, 2006 Face recognition: Opportunities and Challenges Yu Hen Hu University of Wisconsin – Madison Dept. Electrical and Computer Enginerring Madison, WI 53706 yhhu@wisc.edu Collaborators: Nigel Boston, Charles Dyer, Weiyang Lin, Ryan Wong, Goudong Guo Agenda

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Presentation

Face recognition: Opportunities and Challenges

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Face recognition opportunities and challenges l.jpg

Microsoft Research

March 17, 2006

Face recognition: Opportunities and Challenges

Yu Hen Hu

University of Wisconsin – Madison

Dept. Electrical and Computer Enginerring

Madison, WI 53706

yhhu@wisc.edu

Collaborators: Nigel Boston, Charles Dyer,

Weiyang Lin, Ryan Wong, Goudong Guo


Agenda l.jpg

Agenda

  • Human Face Recognition Problem

  • Recent Progress and Findings

  • Research at UW-Madison

  • Invariant of transformation group

    • Existing Invariants

    • Integral invariant

    • Summation invariant

  • Experiment Environment

    • Face Recognition Grand Challenge

    • BEE: Biometric Experimentation Environment

  • Preliminary FRGC Results

(c) 2004-2006 by Yu Hen Hu


Face recognition problem l.jpg

Face Recognition Problem

gallery

probe image

Probe

(c) 2004-2006 by Yu Hen Hu


Face recognition by human l.jpg

Face Recognition by Human

A. Adler and J. Maclean,

“Performance comparison of human and automatic

face recognition”

Biometrics Consortium Conference 2004

(c) 2004-2006 by Yu Hen Hu


Understanding human face recognition l.jpg

Understanding Human Face Recognition

  • Many physiological and psychological studies have been conducted. Eg. Pawan Sinha @MIT. For example, we learned

  • facial configuration plays an important role in human judgments of identity [Sinha05]

Celebrity faces created using identikits

[Sinha05]

C:\users\yuhen\1Research\face\sinha\

(c) 2004-2006 by Yu Hen Hu


Human face recognition limitations l.jpg

Human Face Recognition Limitations

Harmon and Julesz, 1973

Degraded images

Human can handle low resolution celebrity images quite well.

[Sinha05]

(c) 2004-2006 by Yu Hen Hu


Face recognition by machine l.jpg

Face Recognition by Machine

Pre-processing

Feature Extraction

Pattern Classification

(c) 2004-2006 by Yu Hen Hu


Face recognition process l.jpg

Face Recognition Process

  • A pattern classification problem.

  • Two steps:

    • Feature extraction

    • Classification

  • Feature extraction properties

    • Invariant

    • Discriminant

  • Classification

    • Often use distance based method

Query image

Feature Extraction

Query image feature vector

Gallery face

images

feature vectors

Pattern Classification

Recognition results

Wei-Yang Lin

(c) 2004-2006 by Yu Hen Hu


Challenges l.jpg

Challenges

  • Sinha et al [2005] use this example to illustrate the difficulty of finding a suitable “similarity” measure to gauge similarity between a pair of faces.

  • In this example, the outer two faces actually belong to the same person while the middle one does not. But conventional pixel-based measures who say otherwise.

  • Common variations in pose (this case), lighting, expression, distance, aging remain challenges to face recognition.

(c) 2004-2006 by Yu Hen Hu


Face recognition challenges l.jpg

Face Recognition Challenges

Table 1 Face Recognition Technology Evaluation Size

Note that the MCINT portion of FRVT 2002 is the only test in this chart that included “video” signatures.Signatures in all other tests were a single still image.

http://www.frvt.org/FRVT2002/default.htm

(c) 2004-2006 by Yu Hen Hu


Face recognition grand challenge l.jpg

Face Recognition Grand Challenge

  • Goal: to advance performance of face recognition by 10-fold (20%  2% verification rate @0.1% false alarm rate)

  • Focus on five different scenarios.

  • Status: on-going to be concluded by the end of 2005

(c) 2004-2006 by Yu Hen Hu


Object recognition l.jpg

Object Recognition

  • Three Approaches for object recognition (Powen Sinha)

    • Transformationist approach

      • Requires normalization

      • Computationally expensive.

    • View-based approach

      • Store all possible views of the same object

      • Expensive on storage.

    • Invariant-based approach

      • Different views  same invariant features

  • Desired properties of features

    • Invariant to variations of the same object

    • Discriminate to separate similar objects

(c) 2004-2006 by Yu Hen Hu


Geometric transformation groups l.jpg

Euclidean

Similarity

Affine

Projective

Geometric Transformation Groups

(c) 2004-2006 by Yu Hen Hu


Moment invariants l.jpg

Moment Invariants

  • Introduced by M. K. Hu in 1962

  • Advantages

    • Do NOT require parameterization.

    • Not sensitive to noise.

  • Limitations

    • Low discriminating power.

    • Local characteristics can NOT

      be extracted.

(c) 2004-2006 by Yu Hen Hu


Differential invariants l.jpg

Differential Invariants

  • Two examples in 2D

  • Limitation

    • sensitive to noise

(c) 2004-2006 by Yu Hen Hu


Integral invariants l.jpg

Integral Invariants

  • Hann and Hickman [2002] extend transformation to integrals

  • Advantages

    • do NOT require derivatives

    • local characteristics can be extracted

    • Invariants can be systematically generated

  • Limitation

    • Require analytical expression of shape

(c) 2004-2006 by Yu Hen Hu


Summation invariants l.jpg

Summation Invariants

  • Introduced by Lin et al. [2005]

  • Advantages

    • systematical approach

    • robustness

    • high discriminating power

  • Limitation

    • require parameterization

(c) 2004-2006 by Yu Hen Hu


Method of moving frame l.jpg

Method of Moving Frame

  • The method of moving frame, introduce by Elie Cartan, is a tool for finding invariants under group actions.

  • Definition: A moving frame is a smooth, G-equivariant map

(c) 2004-2006 by Yu Hen Hu


Example differential invariants of e 2 l.jpg

Example: Differential Invariants of E(2)

(c) 2004-2006 by Yu Hen Hu


Example integral invariants of e 2 l.jpg

Example: Integral Invariants of E(2)

(c) 2004-2006 by Yu Hen Hu


Example summation invariants of e 2 l.jpg

Example: Summation Invariants of E(2)

(c) 2004-2006 by Yu Hen Hu


Euclidean summation invariants of curves l.jpg

Euclidean Summation Invariants of Curves

  • Given a curve under Euclidean transformation

  • We can find a moving frame by solving

(c) 2004-2006 by Yu Hen Hu


Euclidean summation invariants of curves23 l.jpg

Euclidean Summation Invariants of Curves

  • From the moving frame, a family of invariant functions can be derived

(c) 2004-2006 by Yu Hen Hu


Euclidean summation invariants of curves24 l.jpg

Euclidean Summation Invariants of Curves

  • The first summation invariants are explicitly shown below

    where

(c) 2004-2006 by Yu Hen Hu


Euclidean summation invariants of surfaces l.jpg

Euclidean Summation Invariants of Surfaces

  • Similarly, the family of surface invariants

    where

(c) 2004-2006 by Yu Hen Hu


Face recognition grand challenge frgc l.jpg

Face Recognition Grand Challenge (FRGC)

  • Organized by NIST to facilitate the development of FR technology

  • Provide challenging problems and facial images

  • FRGC v2.0 dataset contains 50,000 recordings, including

    • high resolution still images

    • 3D images

(c) 2004-2006 by Yu Hen Hu


Four frgc experiments l.jpg

Four FRGC Experiments

  • Controlled indoor

  • Multiple images

  • 3D images

  • Controlled vs. uncontrolled

(c) 2004-2006 by Yu Hen Hu


Baseline @ far 0 1 l.jpg

Baseline @ FAR = 0.1%

(c) 2004-2006 by Yu Hen Hu


Biometric experimentation environment bee l.jpg

Biometric Experimentation Environment (BEE)

Image Preprocessing

BioBox

Sub-similarity Generation

Similarity Normalization

Analysis

(c) 2004-2006 by Yu Hen Hu


Sub similarity generation l.jpg

Sub-similarity Generation

(c) 2004-2006 by Yu Hen Hu


Frgc 3d experiment l.jpg

FRGC 3D Experiment

(c) 2004-2006 by Yu Hen Hu


Frgc 3d baseline algorithm l.jpg

FRGC 3D Baseline Algorithm

(c) 2004-2006 by Yu Hen Hu


Proposed algorithm l.jpg

Proposed Algorithm

shape

curve

SI

PCA

similarity

score

similarity

score

+

PCA

similarity

score

shape

surface

SI

(c) 2004-2006 by Yu Hen Hu


Experiment setup l.jpg

Experiment Setup

  • Use only 3D shape. Note 2D texture is not utilized in our experiment.

  • Specified the region of interest

  • At each pixel, compute summation invariants from a specified window region

  • Perform PCA to reduce dimensionality

(c) 2004-2006 by Yu Hen Hu


3 d facial surface and its summation invariants l.jpg

3D Facial Surface and its Summation Invariants

(c) 2004-2006 by Yu Hen Hu


3 d facial surface and its summation invariants36 l.jpg

3D Facial Surface and its Summation Invariants

(c) 2004-2006 by Yu Hen Hu


Roc performance of curve invariants l.jpg

ROC Performance of Curve Invariants

(c) 2004-2006 by Yu Hen Hu


Roc performance of surface invariants l.jpg

ROC Performance of Surface Invariants

(c) 2004-2006 by Yu Hen Hu


Fusion of summation invariants l.jpg

Fusion of Summation Invariants

(c) 2004-2006 by Yu Hen Hu


Comparison with baseline algorithm l.jpg

Comparison with Baseline Algorithm

(c) 2004-2006 by Yu Hen Hu


Conclusion l.jpg

Conclusion

  • Summation invariants

    • novel geometric feature

    • provide useful shape information

    • fusion further improve recognition performance

  • In principle, one can apply summation invariants to unnormalized images

(c) 2004-2006 by Yu Hen Hu


  • Login