Medial Object Shape Representations for Image Analysis & Object Synthesis

1. Medial Object Shape Representationsfor Image Analysis & Object Synthesis Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group University of North Carolina, USA Credits: Many on MIDAG, especially Daniel Fritsch, Andrew Thall, George Stetten, Paul Yushkevich

2. Medial Object Shape Representationsfor Image Analysis & Object Synthesis

3. Whatshape representation is for • Analysis from images • Extract the kidney-shaped object • Register based on the pelvic bone shapes • Extract shape information w/o model • Synthesis • Design the object • Deform the object, with physical realism • Shape science • Shape and biology • Shape-based diagnosis

4. Whatshape representation is for • Analysis from images • Extract the kidney-shaped object • Register based on the pelvic bone shapes

5. Whatshape representation is for • Synthesis • Design the object • Deform the object, with physical realism

6. Whatshape representation is for • Shape science • Shape and biology • Shape-based diagnosis Brain structures (Gerig)

7. Shape Sciences • Geometry • The spatial layout: via primitives • Landmarks • Boundary places and orientations • Medial places, figural sizes and orientations • Space itself • Statistics • The average shape • Modes of variation from the average • Computer Graphics • Image Analysis

8. Options for Primitives • Space: xi for grid elements • Landmarks: xi described by local geometry • Boundary: (xi ,normali) spaced along boundary • Figural: nets of diatoms sampling figures

9. Primitives for shape representation: Landmarks • Sets of points of special geometry

10. Primitives for shape representation: Boundaries • Boundary points with normals

11. Object Representation by M-Reps

12. Each M-figure Represented by Net of Medial Primitives

13. Each M-figure Represented by Net of Medial Primitives

14. Figural Models • Figures: successive medial involution • Main figure • Protrusions • Indentations • Separate figures • Hierarchy of figures • Relative position • Relative width • Relative orientation

15. Primitives’ Desired Properties • Geometry • Intuitive: simple, global + local • Efficiently deformable • Easily extracted or created • Spatial tolerance inherent • Statistics • Unimodality: normally distributed • Via geometrical, tolerance-sensitive metric

16. Figural Models with Boundary Deviations • Hypothesis • At a global level, a figural model is the most intuitive • At a local level, boundary deviations are most intuitive

17. Union and Difference of M-figures

18. Medial Primitives • x, (b,n) frame, r, q (object angle) • Imply boundary segments with tolerance • Similarity transform equivariant • Zoom invariance implies width-proportionality of • tolerance of implied boundary • boundary curvature distribution • spacing along net • interrogation aperture for image n

19. 3D kidney model extracted from CT Four figure model of the kidneys Red represents indentation figures

20. Need for Special End Primitives • Represent • non-blobby objects • angulated edges, corners, creases • still allow rounded edges , corners, creases • allow bent edges • But • Avoid infinitely fine medial sampling • Maintain tangency, symmetry principles

21. End Primitives Corner primitive in cross-section Rounded end primitive in cross-section

22. Displacements from Figurally Implied Boundary Boundary implied by figural model Boundary after displacements

23. Coarse-to-fine representation • For each of three levels • Figural hierarchy • For each figure, net chain, successively smaller tolerance • For each net tile, boundary displacement chain

24. Multiscale Medial Model • From larger scale medial net • Coarsely sampled • Smooother figurally implied boundary • Larger tolerance • Interpolate smaller scale medial net • Finer sampled • More detail in figurally implied boundary • Smaller tolerance • Represent medial displacements

25. Multiscale Medial Model • From larger scale medial net, interpolate smaller scale medial net and represent medial displacements b.

26. Multiscale Medial/Boundary Model • From medial net • Coarsely sampled, smoother implied boundary • Larger tolerance • Represent boundary displacements along implied normals • Finer sampled, more detail in boundary • Smaller tolerance

27. Shape Rep’n in Image Analysis • Segmentation • Extract an object from image • Registration • Find geometric transformation that brings two images into alignment • 3D/3D • 3D/2D • Shape Measurement • Find how probable a shape is

28. Shape Repres’n in Image Analysis • Segmentation • Find the most probable deformed mean model, given the image • Probability involves • Probability of the deformed model (prior) • Probability of the image, given the deformed model (likelihood)

29. Probability of a deformed model • From training set • via principal components analysis, coarse-to-fine • -C * Geometric difference from typical shape

30. Medialness: medial strength of a medial primitive in an image • Probability of image | deformed model • Sum of boundariness values • at implied boundary positions • in implied normal directions • with apertures proportional to tolerance • Boundariness value • Intensity profile distance from mean (at scale) • statistical, based on training set • Intensity differences • via Gaussian derivatives

31. Figurally implied boundaries and rendering, via 4-figure model

32. 3D DSL Model Deformation Initial Position of Model in Target Image

33. 3D DSL Model Deformation Figural DeformationIteration 3

34. 3D DSL Model Deformationwith interfigural penalties Initial position After optimization

35. Shape Repres’n in Image Analysis • Registration • Find the most probable deformation, given the image

36. Shape Rep’n in Image Analysis • Prior-free medial shape analysis • Cores: height ridges of medialness (Pizer, Fritsch, Morse, Furst) • Statistical analysis of medial diatoms (Stetten)

37. Shape Rep’n in Image Analysis • Cores: height ridges of medialness

38. M I P @ U N C

39. Shape Rep’n in Image Analysis • Statistical analysis of medial diatoms

40. sphere slab cylinder

41. sphere slab cylinder

42. sphere slab cylinder

44. sphere slab cylinder

45. sphere slab cylinder

47. Shape Rep’n in CAD/CAM • Stock figural models • Deformation tools: large scale • Coarse-to-fine specification • Figural connection tools • Direct rendering, according to display needs

48. Deformation in CAD/CAM

49. Shape Rep’n in CAD/CAM • Design models for image analysis

50. Medial Object Shape Representationsfor Image Analysis & Object Synthesis • Figural models, at successive levels of tolerance • Boundary displacements • Work in progress • Segmentation and registration tools • Statistical analysis of object populations • CAD tools, incl. direct rendering • Connection relative critical manifolds • …