Similarity based matching for face authentication
This presentation is the property of its rightful owner.
Sponsored Links
1 / 22

Similarity-based matching for face authentication PowerPoint PPT Presentation

  • Uploaded on
  • Presentation posted in: General

Similarity-based matching for face authentication. Christophe Rosenberger Luc Brun ICPR 2008. outline. Introduction Image description Dissimilarity measure Pattern modeling Experimental results. Introduction.

Download Presentation

Similarity-based matching for face authentication

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Similarity-based matching for face authentication

Christophe Rosenberger

Luc Brun

ICPR 2008


  • Introduction

  • Image description

  • Dissimilarity measure

  • Pattern modeling

  • Experimental results


  • Propose in this paper a face authentication method based on a similarity measure

  • Find interest key points

    • SIFT descriptor is used

  • A graph is then built where nodes correspond to these keypoints

    • Model the authentication problem as a graph matching process

Image description

  • SIFT (Scale Invariant Feature Transform)

    • Applied locally on key-points

    • Based upon the image gradients in a local neighborhood

    • SIFT detector

      • Scale-space extrema detection

      • Keypoint localization

    • SIFT descriptor

      • Orientation assignment

      • Keypoint descriptor

SIFT detector

  • Detection of scale-space extrema

    • DoG(Difference-of-Gaussian) filter for scale space

    • Keypoint localization (local extrema)

X is selected if it is larger or

smaller than all 26 neighbors

SIFT detector & descriptor

  • Accurate keypoint localization

    • Reject points with low contrast (flat)

    • Elimination edge responses

  • Orientation assignment

    • To make the feature

      invariant to rotation

    • Find local major


      • Orientation histogram for a window size

  • Local image descriptor

    • 16x16 array of locations in scale space

    • Create array of orientation histograms

    • 8 orientations x 4x4 histogram array (128D)

Gradient magnitude

Gradient orientation

Dissimilarity measure

  • Each individual is described by a set of invariant features Y (I) = {ki = (si, xi, yi)}, i = 1 : N(I)

    • Si is the SIFT invariant descriptor

    • (xi, yi) its position in the original image I

    • N(I) the number of detected keypoints in the image I

  • Database image

    • {Y (Ii), i = 1 : M}

  • Dissimilarity between two sets of points such as Y (I1) and Y (I2)

    • Often modeled as a weighted bipartite graph matching problem

      • Greedy algorithm

      • Hungarian method

      • Hybrid method

Greedy algorithm

  • Weighted bipartite graph G(U,V,E,W)

  • Given the graph G , scan the set of edges and find an edge (u,v) with the smallest weight

  • Match u with v and remove the edges incident on u and those incident on v

  • Repeat the above step until every node in U is matched to some node in V













Greedy algorithm

  • The greedy algorithm does not always produce a matching of minimum weight

  • Hungarian method

    • A well-known algorithm for finding a minimum weight matching in a bipartite graph

    • Node in U by u1, u2,… un

    • In V by v1, v2,… vn

    • x ij= 1 if ui is matched with vjand 0 otherwise

      subject to constraints















Each node in U is

matched to exactly one node in V and vice versa

Hungarian method - Example

Step 1

First for each row we subtract the row minimum from the rest of the row









Step 2

Then for each column we subtract the column minimum from the rest of the column

Hungarian method - Example



Step 4

1.Find the min value not covered (B3-R1)

2.subtracted from each uncovered entry and added to each entry covered by both a vertical and horizontal line (B1-R2 & B1-R4)

3. Step 3 is then repeated

Step 3

Cover the min number of rows or columns required to cover all of the zeros in the matrix







If the # of line = the # of rows/columns in the matrix then we are finished

Otherwise, we have to go to the next step



Blue1 - Red3Blue2 - Red2Blue3 - Red1Blue4 – Red4




Hybrid method

  • We formulated the point matching problem in terms of minimum weight matching in a bipartite graph

  • In a real world situation, this is not always appropriate

    • Feature points extracted from two corresponding images

    • First image are denoted by a and b

    • Second image are denoted by c and d

One might argue that since b and c both have the same feature value of 100,we should match them together and completely ignore a and d

Hybrid method

  • Feature points forming tight clusters in the feature space can be avoided by a simple combination of the greedy method and the Hungarian method

  • Combined algorithm

    • We first apply the Hungarian method to get a matching M1

    • Apply the greedy algorithm to get a matching M2

    • form their intersection M=M1∩M2

Greedy algorithm



A bipartite graph

Hungarian method

Dissimilarity measure

  • Such a weighted bipartite graph matching method

    • Optimizes a global matching score

    • May thus be sensible to outliers

  • We thus rather use the following matching method

    • Given two points and , we say that x is associated to y iff :

      • C is an arbitrary threshold

      • d(., .) denotes the Euclidean distance between SIFT descriptors

      • y’ denotes any point of Y (I2) whose distance to x is minimal but greater than d(x,y)

Dissimilarity measure

  • The significance of the necessary gap between d(x, y) and d(x, y’) is encoded by the constant C

    • y is associated to x

      • x is the closest point from y among Y (I1)

      • second smallest value of this distance d(y, x’) satisfies d(y, x) < Cd(y, x’)

  • X is matched to Y

    • iff x is associated to y and y is associated to x

  • If Y (I1) and Y (I2) do not define any matching point

    • Both sets have an infinite distance

  • Otherwise

    • The set of matching points M1 of Y1 and M2 of Y2 may be computed sequentially

  • If we cannot find two couples of points within Y (I1) × Y (I2) with a similar distance, the relationship “is matched to “ induces a bijective mapping Φ from M1 to M2

Dissimilarity measure

  • First step

    • The above procedure classify both sets Y (I1) and Y (I2)

      • Dissimilar (case M1 = M2 =ø)

      • A set of N = |M1| = |M2| couples (x, Φ(x)) of matched points

  • Second step

    • Take into account the spatial relationship of the points among M1 & M2

    • Use one of the four pattern representations

      • Using the locations of each matched keypoint

      • Weighted graph

        • Edges are weighted by the similarity of the adjacent points

    • The matching between M1 and M2being encoded by the application Φ

      • Encode the adjacency matrices of both graphs

      • A line or a column encoding the adjacency relationships

        • Point x of the first matrix

        • Φ(x) in the second matrix

Graph Adjacency matrix

Pattern modeling

  • Spatial relationship between keypoints

    • Using graph representations

    • Connect keypoints by considering their ”local neighborhoods”

    • Each keypoint corresponds to a vertex in the neighborhood graph which connects “close” vertices

      • Epsilon-neighborhood graph (ENG)

        • Two vertices are connected if the distance of the corresponding keypoints is less than epsilon

      • A completely connected graph (CCG)

        • epsilon greater than the maximal distance between keypoints.

Pattern modeling

  • Symmetric k-nearest neighbor graph (SKNG)

    • Two vertices V1, V2are connected if V1 is among the k-nearest neighbors of V2 or vice versa

  • Mutual k-nearest neighbor graph (MKNG)

    • Two vertices V1, V2 are connected if V1 is among the k-nearest neighbors of V2 and vice versa

  • Given both adjacency matrices

    • The quality of the match is estimated

      using the squared Euclidean

      distance which provides a global

      measure of the dissimilarity between both graphs

  • Experimental results

    • Database

      • AR face database

      • This face database was created by Aleix Martinez and Robert Benavente in the Computer Vision Center (CVC) at the U.A.B

      • It contains over 4,000 color images

      • 126 people's faces (70 men and 56 women)

        • 26 image per individual

      • Images feature frontal view faces

        • Different facial expressions, illumination conditions, and occlusions (sun glasses and scarf).

    Experimental results

    • Tested the four graph representations

    • Two parameters have to be set

      • C is used in the first step of the keypoints matching

        • C induce a restrictive selection of matching points

      • K for the three graph representations

        • It corresponds to the number of neighbors considered in the graph definition

    • The dissimilarity measure gives a coefficient that can be interpreted as a confidence criterion on the fact that two faces belong to different individuals

    • Gives the experimental results on the AR face database

      on a ROC curve

    Experimental results

    • We use the FAR (False Acceptance Rate) and FRR (False Rejection Rate) criteria to analyse the behavior of an algorithm

    The representation based on a

    Completely connected graph (CCG) gives the worst result

    This means that the selection of edges is necessary

    Mutual k-nearest neighbor graph (MKNG) with k=5 and C=0.8 provides


    • We proposed in this paper a face authentication method based on a similarity measure

    • A graph representation based on SIFT invariant keypoints is used

    • The graph definition allows us to take into account their spatial relationships

  • Login