Point Distribution Models Active Appearance Models

1. Point Distribution Models Active Appearance Models Compilation based on: Dhruv Batra ECE CMU Tim Cootes Machester

2. Essence of the Idea (cont.) • Explain a new example in terms of the model parameters

3. So what’s a model Model “texture” “Shape”

4. Active Shape Models training set

5. Texture Models warp to mean shape

6. Intensity Normalisation • Allow for global lighting variations • Common linear approach • Shift and scale so that • Mean of elements is zero • Variance of elements is 1 • Alternative non-linear approach • Histogram equalization • Transforms so similar numbers of each grey-scale value

7. Shape: Review of Construction Mark face region on training set Sample region Normalise The Fun Step Statistical Analysis

8. Multivariate Statistical Analysis • Need to model the distribution of normalised vectors • Generate plausible new examples • Test if new region similar to training set • Classify region

9. Fitting a gaussian • Mean and covariance matrix of data define a gaussian model

10. Principal Component Analysis • Compute eigenvectors of covariance, S • Eigenvectors : main directions • Eigenvalue : variance along eigenvector

11. Eigenvector Decomposition • If A is a square matrix then an eigenvector of A is a vector, p, such that • Usually p is scaled to have unit length,|p|=1

12. Eigenvector Decomposition • If K is an n x n covariance matrix, there exist n linearly independent eigenvectors, and all the corresponding eigenvalues are non-negative. • We can decompose K as

13. Eigenvector Decomposition • Recall that a normal pdf has • The inverse of the covariance matrix is

14. Fun with Eigenvectors • The normal distribution has form

15. Fun with Eigenvectors • Consider the transformation

16. Fun with Eigenvectors • The exponent of the distribution becomes

17. Normal distribution • Thus by applying the transformation • The normal distribution is simplified to

18. Dimensionality Reduction • Co-ords often correllated • Nearby points move together

19. Dimensionality Reduction • Data lies in subspace of reduced dim. • However, for some t,

20. Approximation • Each element of the data can be written

21. Normal PDF

22. Useful Trick • If x of high dimension, S huge • If No. samples, N<dim(x) use

23. Building Eigen-Models • Given examples • Compute mean and eigenvectors of covar. • Model is then • P – First t eigenvectors of covar. matrix • b – Shape model parameters

24. Eigen-Face models • Model of variation in a region

25. Applications: Locating objects • Scan window over target region • At each position: • Sample, normalise, evaluate p(g) • Select position with largest p(g)

26. Multi-Resolution Search • Train models at each level of pyramid • Gaussian pyramid with step size 2 • Use same points but different local models • Start search at coarse resolution • Refine at finer resolution

27. Application: Object Detection • Scan image to find points with largest p(g) • If p(g)>pmin then object is present • Strictly should use a background model: • This only works if the PDFs are good approximations – often not the case

28. Back (sadly) to Texture Models raster scan Normalizations

29. PCA Galore Reduce Dimensions of shape vector Reduce Dimension of “texture” vector They are still correlated; repeat..

30. Object/Image to Parameters modeling ~80

31. Playing with the Parameters First two modes of shape variation First two modes of gray-level variation First four modes of appearance variation

32. Active Appearance Model Search • Given: Full training model set, new image to be interpreted, “reasonable” starting approximation • Goal: Find model with least approximation error • High Dimensional Search: Curse of the dimensions strikes again

33. Active Appearance Model Search • Trick: Each optimization is a similar problem, can be learnt • Assumption: Linearity • Perturb model parameters with known amount • Generate perturbed image and sample error • Learn multivariate regression for many such perterbuations

34. Active Appearance Model Search • Algorithm: • current estimate of model parameters: • normalized image sample at current estimate

35. Active Appearance Model Search • Slightly different modeling: • Error term: • Taylor expansion (with linear assumption) • Min (RMS sense) error: • Systematically perturb and estimate by numerical differentiation

36. Active Appearance Model Search (Results)

37. Sub-cortical Structures Initial Position Converged

38. Random Aside • Shape Vector provides alignment = 43 Alexei (Alyosha) Efros, 15-463 (15-862): Computational Photography, http://graphics.cs.cmu.edu/courses/15-463/2005_fall/www/Lectures/faces.ppt

39. Random Aside • Alignment is the key 1. Warp to mean shape 2. Average pixels Alexei (Alyosha) Efros, 15-463 (15-862): Computational Photography, http://graphics.cs.cmu.edu/courses/15-463/2005_fall/www/Lectures/faces.ppt

40. Random Aside • Enhancing Gender more same original androgynous more opposite D. Rowland, D. Perrett. “Manipulating Facial Appearance through Shape and Color”, IEEE Computer Graphics and Applications, Vol. 15, No. 5: September 1995, pp. 70-76

41. Random Aside (can’t escape structure!) Antonio Torralba & Aude Oliva (2002) Averages: Hundreds of images containing a person are averaged to reveal regularities in the intensity patterns across all the images. Alexei (Alyosha) Efros, 15-463 (15-862): Computational Photography, http://graphics.cs.cmu.edu/courses/15-463/2005_fall/www/Lectures/faces.ppt

42. Random Aside (can’t escape structure!) “100 Special Moments” by Jason Salavon Jason Salavon, http://salavon.com/PlayboyDecades/PlayboyDecades.shtml

43. Random Aside (can’t escape structure!) “Every Playboy Centerfold, The Decades (normalized)” by Jason Salavon 1960s 1970s 1980s Jason Salavon, http://salavon.com/PlayboyDecades/PlayboyDecades.shtml