Paper Reading

1. Paper Reading DalongDu Nov.27, 2009

2. Papers • Leon Gu and Takeo Kanade. A Generative Shape Regularization Model for Robust Face Alignment. ECCV08. • Yan Li, Leon Gu, Takeo Kanade. A Robust Shape Model for Multi-view Car Alignment. CVPR09.

3. A Generative Shape Regularization Model for Robust Face Alignment Leon Gu and Takeo Kanade

4. Outline • AuthorIntroduction. • Problem Introduction. • How to do? • Discussion.

5. Outline • AuthorIntroduction. • Problem Introduction. • How to do? • Discussion.

6. Author Introduction (1/3) Takeo Kanade(金出武雄) Leon Gu

7. Author Introduction (2/3) • Leon Gu • Ph.D. candidate in the Computer Science Department at Carnegie Mellon University, advised by Professor Takeo Kanade. • His main research interest is in developing robust and efficient algorithms for object recognition. A common thread has been the focus on reasoning theshapeof visual objects from noisy, real-world images, where the uncertainties over image appearance and imaging conditions are prevalent.

8. Author Introduction (3/3) • Takeo Kanade • Director of the Robotics Institute ofCarnegie Mellon University • Wisdom：像外行一样思考，像专家一样实践

9. Outline • AuthorIntroduction. • Problem Introduction. • How to do? • Discussion.

10. Problem Introduction (1/12) • Face Q consists of N landmark points: • The geometry information of Q decouples into two parts: • A canonical shape S • b • e.g.Or other linear or nonlinear methods • A transformation • θ • e.g. similarity s, R, t Or Affine or others. b θ

11. Problem Introduction (2/12) • Probabilistic Formulation • Generic alignment problem Where • Pose space Θ is free • Shape space is constrained • A solution maximizes the posterior • A chicken and egg problem • A best solution Amaxposterior

12. Problem Introduction (3/12) • In the Eyes of Computer • On the basis of such “noisy observation”, how can make the best hypothesis (b, θ) ? Reflectance, Occlusion, Image blur, ….. Noisy feature map

13. Deformation Problem Introduction (4/12) Transformation • AGenerativehierarchicalmodel • Deformation • The magnitude of deformation is controlled byb. • The canonical shape S is generated from b through, a processthat could be linear or nonlinear. • Transformation • The transform could be similarity/affine. • Image Likelihood • Varies with the type of image local feature • Profile, local image patch, Haar-like feature… … Image Likelihood

14. Deformation Problem Introduction (5/12) Transformation • Baseline Model • Linear Deformation Where • Shape prior , Λ is diagonal. • Isotropic shape noise (Probabilistic PCA) • The average residual variance outside of the subspace , where N is the number of landmark points, M is the subspace dimension. • {Φ, μ, σ} are learned from training samples. Image Likelihood

15. Deformation Problem Introduction (6/12) Transformation • Baseline Model • Similarity Transform Where • θ={s, R, t} are scale, Rotation, translation coefficients respectively. • Diagonal observation noise • measures the noise level of the observation of n-th landmark point. • Σ is also learned from training samples. Image Likelihood

16. Deformation Problem Introduction (7/12) Transformation • Baseline Model • Observed shape Y is generated from feature point detector. • EM • Q-function: • E-step: compute the statistics that are required to evaluate Q-function. • M-step: maximize Q-function to find the updated shape and pose. Image Likelihood

17. Problem Introduction (8/12) • Alignment algorithm

18. Problem Introduction (9/12) • Problems? • Linear deformation model • Cannot handle faces of rare shapes (babies, etc) • Cannot handle extreme expressions • Single candidate position for each feature point • Best position may be the one with second strongest response • This paper extends the generative framework to handle • Large face shape deformation including extreme expressions • Multiple candidate positions for each feature point • Identify outliers, like occluded feature points.

19. Problem Introduction (10/12) • Handling Extreme Expressions

20. Problem Introduction (11/12) • Handling Large Occlusion

21. Problem Introduction (12/12) • Handling Real World Images

22. Outline • AuthorIntroduction. • Problem Introduction. • How to do? • Discussion.

23. How to do? (1/12) • Face Q consists of N landmark points: • The geometry information of Q decouples into two parts: • A canonical shape S • A similarity transformation • Map S from a common reference frame to the coordinate plane of the image I b θ

24. How to do? (2/12) • Make a mixture of constrained Gaussian • Multiple subspace

25. How to do? (3/12) • Allow generate multiple candidate • For n-th landmark • K candidate positions • denote the whole set of N × K candidates • Set a binary N × K matrix hto specify the “true” candidate e.g.

26. Deformation How to do? (4/12) Transformation • A new generativehierarchicalmodel Image Likelihood Deformation Transformation Image Likelihood

27. How to do? (5/12) • Deformation • Define prior distribution over the shape S as a mixture of Gaussian • Introduce a multinomial distribution z • Model parameters learned from training samples e.g.

28. How to do? (6/12) • Similarity Transform Where • θ={s, R, t} are scale, Rotation, translation coefficients respectively. • Diagonal observation noise • measures the noise level of the observation of n-th landmark point. • Σ is also learned from training samples and can updated on fitting phase. • So

29. How to do? (7/12) • Image Likelihood • The image likelihood of seeing a landmark atone particular position Qnk is measured by • isgenerated by feature detector.

30. How to do? (8/12) • Goal: • Solve b and θ on the basis of the candidate point set Q. • MAP problem which can be solved by EM • Posterior with latent variables S, h, z • Take the expectation of the log over the posterior of the latent variables S, h, z • Q function:

31. How to do? (9/12) • Alignment Algorithm

32. How to do? (10/12) • Update Canonical Shape

33. How to do? (11/12) • Update Shape Parameters Shrinkby:

34. How to do? (12/12) • Identifying Outliers: • Use observation noise model • Observation noises are unpredictable • Update online • Change it according to the fitting error between the model prediction and the averaged candidate position • Define weights to update Canonical Shape • A smaller leads a larger weight to the canonical shape and less to the observed candidate.

35. Outline • AuthorIntroduction. • Problem Introduction. • How to do? • Discussion.

36. Discussion (1/6) • Evaluation

37. Discussion (2/6) • Handling Extreme Expressions • Number of mixture components is L = 3.

38. Discussion (3/6) • Handling Large Occlusion

39. Discussion (4/6) • Handling Real World Images

40. Discussion (5/6) • Similarity Transform Where • Diagonal observation noise • measures the noise level of the observation of n-th landmark point. • The independence assumption to each landmark is not reasonable. • Markov Network • …

41. Discussion (6/6) • The regularization step does not consider the image information anymore

42. A Robust Shape Model for Multi-view Car Alignment Yan Li, Leon Gu and Takeo Kanade

43. Outline • Problem Introduction. • How to do? • Discussion.

44. Outline • Problem Introduction. • How to do? • Discussion.

45. Problem Introduction • Previous shape alignment model • A hypothesisofGaussianobservationnoise. • Usealltheobserveddatatofitaregularizedshape. • ThisGaussianassumptionisvulnerabletogrossfeaturedetectionerror. Partial occlusions and spurious background features

46. Outline • Problem Introduction. • How to do? • Discussion.

47. How to do? (1/3) • A hypothesis-and-test approach. • Hypothesis: Bayesian Partial Shape Inference (BPSI) algorithm • Test: The hypotheses are then evaluatedto find the one that minimizes the shape prediction error.

48. How to do? (2/3) • The observed data • Random sample from Y • used to generate hypothesis—shapebandposeθ(s, R, t). • used to test hypothesis. • Bayesian Partial Shape Inference (BPSI) algorithm • A MAP problem: • A typical missing data problem can be solved by EM.

49. How to do? (3/3) Generate Hypothesis Test Hypothesis is the residual between the i-th Corresponding point of

50. Discussion