1 / 29

Face Recognition and Retrieval in Video

Face Recognition and Retrieval in Video. Basic concept of Face Recog . & retrieval And their basic methods. C.S.E. Kwon Min Hyuk. True? False?. Q1 : in recently, face recognition researches focus on video-based rather than still image-based (O / X) Q2 : There is three approaches; (O/ X)

salvador
Download Presentation

Face Recognition and Retrieval in Video

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Face Recognition and Retrieval in Video Basic concept of Face Recog. & retrieval And their basic methods. C.S.E. Kwon Min Hyuk

  2. True? False? • Q1 : in recently, face recognition researches focus on video-based rather than still image-based (O / X) • Q2 : There is three approaches; (O/ X) • 1. key-frame based • 2. Temporal model based • 3. image set based • Face variation and expression make face recognition difficult. (O/X)

  3. Answers • All of statement is true.

  4. Intro • Q1 : Why do we need Face recognition system? • Increasing request to search specific people related video contents • Can be applied at security, human-computer inter action etc.

  5. Intro • Q2: What is recent trend of approach to Face recog.? • Traditionally ,focused on Still image-basedappr. • Recently , focused on Video-based appr. • We can extract more information from video than that of still image.

  6. General steps for Face recognition • Where is face located in video frame? • We should look for which part of the frame is face.  Face detecting and Tracking. • Recognizing face • There is some basic approach for Face Recog. • Key fame-based approach • Temporal Model-based approach • Image set-based approach

  7. Face detection • Using statistical geometric model • From the frame Extract appearance features such as edge, intensity, color(histogram) • To evolve the face detector by using machine learning tech. • Adaboost • Neural Network • Support Vector Machine

  8. Face tracking • Face detection’s limit. • It detect only frontal or near frontal view. • Tracking face is needed to handle large head motions. • Face tracking • Difficulties • Method to solve

  9. Face tracking • Difficulty 1 : There is • Face appearance variation • 3D motion • Background change • Method to solve: face online boosting • Using tracked images in previous frames. • Applying current result to tracking seq. for next frame. (real-time updating feedback) example in next slide 

  10. Example for online boosting algorithm. So-called “adaptive tracker.”

  11. Face tracking(con’d) • Difficulty 2: The adaptive tracker can adapt to non-targets. • Method to solve : add basal appearance of target • Teaching the tracker about some basal appearance of the target. • Basal appearance : Image set of various target condition (face expression, pose etc)

  12. After face detecting and tracking • We can determine the part of frame where face is located.  Now, we can get to face recognition.

  13. Face Recognition • Basic steps for Face Recog. • 1. get weak evidence in individual frame. • 2. collect that evidence over time. • 3. lead(determine) reliable result. • Three approaches • 1. key-frame approach • 2. temporal model-based approach • 3. image set-based approach

  14. Key-frame based approach • Treat each video as a collection of images. • Basic steps of the approach. • 1. input data(still images, video) • 2. from data, extract images of the target. • Extracted images are called key-frames or examplars. • 3. matching them with all or subset of other video sequence(where the target is).

  15. Key-frame based approach(con’d) • How can we get some ‘good’ key-frame from input data? • By image-based recognition • In each frame, probe the nose and eyes’ triangular structure. If it is in the frame, then face recognition is performed. And key-frame is extracted.

  16. Key-frame based approach(con’d) • Applying K-Means clustering • Cluster means a group whose elements have some common property. • This algorithm is grouping some data observations into one of cluster which has nearest mean.

  17. Key-frame based approach(con’d) • Other algorithms • Isomap algorithm • Combination of majority and probabilistic voting. • And so on. (I’ll skip the details.) • Finally, all or subset of video sequence will be compared(matched) with extracted ‘good’ key-frame to determine recognition.

  18. Temporal Model Based approach • To handle face dynamics • Ex: face expression(non-rigid) or head movement(rigid) • Using temporal sequence(continuous coherent) • Ex> Using whole sequence of changing face dynamics as a image set.

  19. Temporal Model Based approach(con’d) • Basic methods • Matching the face Trajectory. • Trajectory means the moving face’s path(orbit) through in surfaces. • Two(model and object) trajectory distance accumulates recognition evidence over time.

  20. Temporal Model Based approach(con’d) • Other method • Trained statistical face model • Using density estimation. • Probabilistic approach • Using time-series state space variables • Hidden Markov Model • Fusing pose and person-discriminant features. I’ll skip all of details.

  21. Image set based approach • This approach uses • both image collected over consecutive time (similar with temporal image set) • And independent still image set (similar with key-frame) • Combination of both temporal and key-frame based approaches. • Two major approaches. • Statistical modal-based • Mutual subspace-based

  22. Image set based approach(con’d) • Image set classification • Non-parametric sample based • Compare representative images of each image sets • Parametric model-based • In terms of probabilistic, compare two distributions of each image set.

  23. Image set based approach(con’d) • Statistical Model-based • To determine recognition, consider similarity of two manifolds • manifold is large group (more than cluster) which contains several cluster. • Drawback • Need to solve the difficult parameter estimation problem.

  24. Image set based approach(con’d) • Mutual Subspace-Based Model(MSM) • To determine Similarity between image sets • measure by the smallest principal angles between subspaces. • CMSM is expansion of MSM • Assume more constraints.

  25. Face Retrieval • It is difficult to recognize face in the uncontrolled condition like face dynamics , light intensity, hair styles • There is two applications 1. Person Retrieval 2. Cast listing

  26. Person Retrieval • Face recognition tech. are applied. • Basic method • Using head model (with multiple texture map) • Step1. rendering(extract or generate) face images • Step2. identifying target face. • Step3. updating the texture map of the model.

  27. Cast listing (cast : actors or characters in film) • Automatic cast listing is interesting problem. • Based on face recognition • Because face is repeatable cue in the film • Using image clustering method • For accuracy, Treat clothing appearance additional cues for clustering.

  28. Challenges and Future direction • Databases. • Constructed in lab. enviro. Not a real world. • Limited face appearance of variation. • Low-quality Video data • Exist lots of noise hard to filter out. • Computational Cost • Face recognition requires quite high power devices.

  29. Conclusion • Face recog. can be applied in various area. • Face detecting and Tracking. • Three general Methods • Key-frame based • Temporal model based • Image set based • Person Retrieval and Cast listing • Challenges to evolve Face recog. system.

More Related