- 42 Views
- Uploaded on
- Presentation posted in: General

Similarity-based matching for face authentication

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Similarity-based matching for face authentication

Christophe Rosenberger

Luc Brun

ICPR 2008

- Introduction
- Image description
- Dissimilarity measure
- Pattern modeling
- Experimental results

- Propose in this paper a face authentication method based on a similarity measure
- Find interest key points
- SIFT descriptor is used

- A graph is then built where nodes correspond to these keypoints
- Model the authentication problem as a graph matching process

- SIFT (Scale Invariant Feature Transform)
- Applied locally on key-points
- Based upon the image gradients in a local neighborhood
- SIFT detector
- Scale-space extrema detection
- Keypoint localization

- SIFT descriptor
- Orientation assignment
- Keypoint descriptor

- Detection of scale-space extrema
- DoG(Difference-of-Gaussian) filter for scale space
- Keypoint localization (local extrema)

X is selected if it is larger or

smaller than all 26 neighbors

- Accurate keypoint localization
- Reject points with low contrast (flat)
- Elimination edge responses

- Orientation assignment
- To make the feature
invariant to rotation

- Find local major
orientation

- Orientation histogram for a window size

- To make the feature
- Local image descriptor
- 16x16 array of locations in scale space
- Create array of orientation histograms
- 8 orientations x 4x4 histogram array (128D)

Gradient magnitude

Gradient orientation

- Each individual is described by a set of invariant features Y (I) = {ki = (si, xi, yi)}, i = 1 : N(I)
- Si is the SIFT invariant descriptor
- (xi, yi) its position in the original image I
- N(I) the number of detected keypoints in the image I

- Database image
- {Y (Ii), i = 1 : M}

- Dissimilarity between two sets of points such as Y (I1) and Y (I2)
- Often modeled as a weighted bipartite graph matching problem
- Greedy algorithm
- Hungarian method
- Hybrid method

- Often modeled as a weighted bipartite graph matching problem

- Weighted bipartite graph G(U,V,E,W)
- Given the graph G , scan the set of edges and find an edge (u,v) with the smallest weight
- Match u with v and remove the edges incident on u and those incident on v
- Repeat the above step until every node in U is matched to some node in V

3

3

10

12

5

5

20

25

6

4

9

4

- The greedy algorithm does not always produce a matching of minimum weight
- Hungarian method
- A well-known algorithm for finding a minimum weight matching in a bipartite graph
- Node in U by u1, u2,… un
- In V by v1, v2,… vn
- x ij= 1 if ui is matched with vjand 0 otherwise
subject to constraints

1

A

A

C

C

16

16

0

0

B

2

D

B

D

Each node in U is

matched to exactly one node in V and vice versa

Step 1

First for each row we subtract the row minimum from the rest of the row

R1

B1

R2

B2

R3

B3

R4

B4

Step 2

Then for each column we subtract the column minimum from the rest of the column

R1

B1

Step 4

1.Find the min value not covered (B3-R1)

2.subtracted from each uncovered entry and added to each entry covered by both a vertical and horizontal line (B1-R2 & B1-R4)

3. Step 3 is then repeated

Step 3

Cover the min number of rows or columns required to cover all of the zeros in the matrix

R2

B2

R3

B3

R4

B4

If the # of line = the # of rows/columns in the matrix then we are finished

Otherwise, we have to go to the next step

3

Result:

Blue1 - Red3Blue2 - Red2Blue3 - Red1Blue4 – Red4

4

13

2

- We formulated the point matching problem in terms of minimum weight matching in a bipartite graph
- In a real world situation, this is not always appropriate
- Feature points extracted from two corresponding images
- First image are denoted by a and b
- Second image are denoted by c and d

One might argue that since b and c both have the same feature value of 100,we should match them together and completely ignore a and d

- Feature points forming tight clusters in the feature space can be avoided by a simple combination of the greedy method and the Hungarian method
- Combined algorithm
- We first apply the Hungarian method to get a matching M1
- Apply the greedy algorithm to get a matching M2
- form their intersection M=M1∩M2

Greedy algorithm

Combined

matching

A bipartite graph

Hungarian method

- Such a weighted bipartite graph matching method
- Optimizes a global matching score
- May thus be sensible to outliers

- We thus rather use the following matching method
- Given two points and , we say that x is associated to y iff :
- C is an arbitrary threshold
- d(., .) denotes the Euclidean distance between SIFT descriptors
- y’ denotes any point of Y (I2) whose distance to x is minimal but greater than d(x,y)

- Given two points and , we say that x is associated to y iff :

- The significance of the necessary gap between d(x, y) and d(x, y’) is encoded by the constant C
- y is associated to x
- x is the closest point from y among Y (I1)
- second smallest value of this distance d(y, x’) satisfies d(y, x) < Cd(y, x’)

- y is associated to x
- X is matched to Y
- iff x is associated to y and y is associated to x

- If Y (I1) and Y (I2) do not define any matching point
- Both sets have an infinite distance

- Otherwise
- The set of matching points M1 of Y1 and M2 of Y2 may be computed sequentially

- If we cannot find two couples of points within Y (I1) × Y (I2) with a similar distance, the relationship “is matched to “ induces a bijective mapping Φ from M1 to M2

- First step
- The above procedure classify both sets Y (I1) and Y (I2)
- Dissimilar (case M1 = M2 =ø)
- A set of N = |M1| = |M2| couples (x, Φ(x)) of matched points

- The above procedure classify both sets Y (I1) and Y (I2)
- Second step
- Take into account the spatial relationship of the points among M1 & M2
- Use one of the four pattern representations
- Using the locations of each matched keypoint
- Weighted graph
- Edges are weighted by the similarity of the adjacent points

- The matching between M1 and M2being encoded by the application Φ
- Encode the adjacency matrices of both graphs
- A line or a column encoding the adjacency relationships
- Point x of the first matrix
- Φ(x) in the second matrix

Graph Adjacency matrix

- Spatial relationship between keypoints
- Using graph representations
- Connect keypoints by considering their ”local neighborhoods”
- Each keypoint corresponds to a vertex in the neighborhood graph which connects “close” vertices
- Epsilon-neighborhood graph (ENG)
- Two vertices are connected if the distance of the corresponding keypoints is less than epsilon

- A completely connected graph (CCG)
- epsilon greater than the maximal distance between keypoints.

- Epsilon-neighborhood graph (ENG)

- Symmetric k-nearest neighbor graph (SKNG)
- Two vertices V1, V2are connected if V1 is among the k-nearest neighbors of V2 or vice versa

- Mutual k-nearest neighbor graph (MKNG)
- Two vertices V1, V2 are connected if V1 is among the k-nearest neighbors of V2 and vice versa

- The quality of the match is estimated
using the squared Euclidean

distance which provides a global

measure of the dissimilarity between both graphs

- Database
- AR face database
- This face database was created by Aleix Martinez and Robert Benavente in the Computer Vision Center (CVC) at the U.A.B
- It contains over 4,000 color images
- 126 people's faces (70 men and 56 women)
- 26 image per individual

- Images feature frontal view faces
- Different facial expressions, illumination conditions, and occlusions (sun glasses and scarf).

- Tested the four graph representations
- Two parameters have to be set
- C is used in the first step of the keypoints matching
- C induce a restrictive selection of matching points

- K for the three graph representations
- It corresponds to the number of neighbors considered in the graph definition

- C is used in the first step of the keypoints matching
- The dissimilarity measure gives a coefficient that can be interpreted as a confidence criterion on the fact that two faces belong to different individuals
- Gives the experimental results on the AR face database
on a ROC curve

- We use the FAR (False Acceptance Rate) and FRR (False Rejection Rate) criteria to analyse the behavior of an algorithm

The representation based on a

Completely connected graph (CCG) gives the worst result

This means that the selection of edges is necessary

Mutual k-nearest neighbor graph (MKNG) with k=5 and C=0.8 provides

- We proposed in this paper a face authentication method based on a similarity measure
- A graph representation based on SIFT invariant keypoints is used
- The graph definition allows us to take into account their spatial relationships