1 / 55

Active Shape Models: Their Training and Applications Cootes, Taylor, et al.

Active Shape Models: Their Training and Applications Cootes, Taylor, et al. Based on Slides of Robert Tamburo, July 6, 2000. Other Deformable Models. “Hand Crafted” Models Articulated Models Active Contour Models – “Snakes” Fourier Series Shape Models Statistical Models of Shape

addison-hyde
Download Presentation

Active Shape Models: Their Training and Applications Cootes, Taylor, et al.

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Active Shape Models:Their Training and ApplicationsCootes, Taylor, et al. Based on Slides of Robert Tamburo, July 6, 2000

  2. Other Deformable Models • “Hand Crafted” Models • Articulated Models • Active Contour Models – “Snakes” • Fourier Series Shape Models • Statistical Models of Shape • Finite Element Models

  3. “Hand Crafted” Models • Built from subcomponents (circles, lines, arcs) • Some degree of freedom • May change scale, orientation, size, and position • Lacks generality • Detailed knowledge of expected shapes • Application specific back

  4. Articulated Models • Built from rigid components connected by sliding or rotating joints • Uses generalized Hough transform • Limited to a restricted class of variable shapes back

  5. Active Contours – “Snakes” • Energy minimizing spline curves • Attracted toward lines and edges • Constraints on stiffness and elastic parameters ensure smoothness and limit degree to which they can bend • Fit using image evidence and applying force to the model and minimize energy function • Uses only local information • Vulnerable to initial position and noise back

  6. Spline Curves • Splines are piecewise polynomial functions of order d • Sum of basis functions with applied weights • Spans joined by knots back

  7. Fourier Series Shape Models • Models formed from Fourier series • Fit by minimizing energy function (parameters) • Contains no prior information • Not suitable for describing general shapes: • Finite number of terms approximates a square corner • Relationship between variations in shape and parameters is not straightforward back

  8. Statistical Models of Shape • Register “landmark” points in N-space to estimate: • Mean shape • Covariance between coordinates • Depends on point sequence back

  9. Finite Element Models • Model variable objects as physical entities with internal stiffness and elasticity • Build shapes from different modes of vibration • Easy to construct compact parameterized shapes next

  10. Motivation – Prior Models • Lack of practicality • Lack of specificity • Lack of generality • Nonspecific class deformation • Local shape constraints

  11. Goals of Active Shape Model (ASM) • Automated • Searches images for represented structures • Classify shapes • Specific to ranges of variation • Robust (noisy, cluttered, and occluded image) • Deform to characteristics of the class represented • “Learn” specific patterns of variability from a training set

  12. Goals of ASM (cont’d.) • Utilize iterative refinement algorithm • Apply global shape constraints • Uncorrelated shape parameters • Better test for dependence?

  13. Point Distribution Model (PDM) • Captures variability of training set by calculating mean shape and main modes of variation • Each mode changes the shape by moving landmarks along straight lines through mean positions • New shapes created by modifying mean shape with weighted sums of modes

  14. Manual Labeling Alignment Statistical Analysis Point Distribution Model PDM Construction

  15. Labeling the Training Set • Represent example shapes by points • Point correspondence between shapes

  16. Aligning the Training Set • xi is a vector of n points describing the ith shape in the set: xi=(xi0, yi0, xi1, yi1,……, xik, yik,……,xin-1, yin-1)T • Minimize: Ej = (xi – M(sj, j)[xk] – tj)TW(xi – M(sj, j)[xk] – tj) • Weight matrix used:

  17. Alignment Algorithm • Align each shape to first shape by rotation, scaling, and translation • Repeat • Calculate the mean shape • Normalize the orientation, scale, and origin of the current mean to suitable defaults • Realign every shape with the current mean • Until the process converges

  18. Mean Normalization • Ensures 4N constraints on 4N variables • Equations have unique solutions • Guarantees convergence • Independent of initial shape aligned to • Iterative method vs. direct solution

  19. Aligned Shape Statistics • PDM models “cloud” variation in 2n space • Assumptions: • Points lie within “Allowable Shape Domain” • Cloud is hyper-ellipsoid (2n-D)

  20. Statistics (cont’d.) • Center of hyper-ellipsoid is mean shape • Axes are found using PCA • Each axis yields a mode of variation • Defined as , the eigenvectors of covariance matrix , such that ,where is the ktheigenvalue of S

  21. Approximation of 2n-D Ellipsoid • Most variation described by t-modes • Choose t such that a small number of modes accounts for most of the total variance • If total variance = and the approximated variance = , then

  22. Generating New Example Shapes • Shapes of training set approximated by: , where is the matrix of the first t eigenvectors and is a vector of weights • Vary bkwithin suitable limits for similar shapes

  23. Application of PDMs • Applied to: • Resistors • “Heart” • Hand • “Worm” model

  24. Resistor Example • 32 points • 3 parameters capture variability

  25. Resistor Example (cont.’d) • Lacks structure • Independence of parameters b1and b2 • Will generate “legal” shapes

  26. Resistor Example (cont.’d)

  27. Resistor Example (cont.’d)

  28. Resistor Example (cont.’d)

  29. “Heart” Example • 66 examples • 96 points • Left ventricle • Right ventricle • Left atrium • Traced by cardiologists

  30. “Heart” Example (cont.’d)

  31. “Heart” Example (cont.’d) • Varies Width • Varies Septum • Vary LV • Vary Atrium

  32. Hand Example • 18 shapes • 72 points • 12 landmarks at fingertips and joints

  33. Hand Example (cont.’d) • 96% of variability due to first 6 modes • First 3 modes • Vary finger movements

  34. “Worm” Example • 84 shapes • Fixed width • Varying curvature and length

  35. “Worm” Example (cont.’d) • Represented by 12 point • Breakdown of PDM

  36. “Worm” Example (cont.’d) • Curved cloud • Mean shape: • Varying width • Improper length

  37. “Worm” Example (cont.’d) • Linearly independent • Nonlinear dependence

  38. “Worm” Example • Effects of varying first 3 parameters: • 1st mode is linear approximation to curvature • 2nd mode is correction to poor linear approximation • 3rd approximates 2nd order bending

  39. PDM Improvements • Automated labeling • 3D PDMs • Nonlinear PDM • Polynomial Regression PDMs • Multi-layered PDMs • Hybrid PDMs • Chord Length Distribution Model • Approximation problem

  40. PDMs to Search an Image - ASMs • Estimate initial position of model • Displace points of model to “better fit” data • Adjust model parameters • Apply global constraints to keep model “legal”

  41. Adjusting Model Points • Along normal to model boundary proportional to edge strength • Vector of adjustments:

  42. where Calculating Changes in Parameters • Initial position: • Move Xas close to new position (X + dX) • Calculate dxto move X to X + dX • Update parameters to better fit image • Not usually consistent with model constraints • Residual adjustments made by deformation

  43. Model Parameter Space • Transforms dx to parameter space giving allowable changes in parameters, db • Recall: • Find db such that • - yields • Update model parameters within limits

  44. Applications • Medical • Industrial • Surveillance • Biometrics

  45. ASM Application to Resistor • 64 points (32 type III) • Adjustments made finding strongest edge • Profile 20 pixels long • 5 degrees of freedom • 30, 60, 90, 120 iterations

  46. ASM Application to “Heart” • Echocardiogram • 96 points • 12 degrees of freedom • Adjustments made finding strongest edge • Profile 40 pixels long • Infers missing data (top of ventricle)

  47. ASM Application to Hand • 72 points • Clutter and occlusions • 8 degrees of freedom • Adjustments made finding strongest edge • Profile 35 pixels long • 100, 200, 350 iterations

  48. Conclusions • Sensitivity to orientation of object in image to model • Sensitivity to large changes in scale? • Sensitive to outliers (reject or accept) • Sensitivity to occlusion • Quantitative measures of fit • Overtraining • Occlusion, cluttering, and noise • Dependent on boundary strength • Real time • Extension to 3rd dimension • Gray level PDM

  49. MR of Brain2 • Improves ASM • Tests several model hypotheses • Outlier detection adjustment/removal • 114 landmark points • 8 training images • Model structures of brain together • Model brain structures

  50. MR Brain (cont.’d)

More Related