1 / 28

A Highly Deformable Object Model for Tracking

A Highly Deformable Object Model for Tracking. Nilanjan Ray Department of Computing Science University of Alberta. Overview of Presentation. Tracking deformable objects Motivations: desirable properties of a deformable object model An example application (mouse heart tracking)

holland
Download Presentation

A Highly Deformable Object Model for Tracking

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A Highly Deformable Object Model for Tracking Nilanjan Ray Department of Computing Science University of Alberta

  2. Overview of Presentation • Tracking deformable objects • Motivations: desirable properties of a deformable object model • An example application (mouse heart tracking) • Some technical background • Level set function and its application in image processing • Non-parametric probability density function (pdf) estimation • Similarity/dissimilarity measures for pdfs • Proposed deformable object model and tracking technique • Results, comparisons and demos • Ongoing investigations • Incorporating color cues, and other features • Adding constraints on object shape • Application in morphing (?) • Summary • Acknowledgements

  3. Tracking Deformable Objects • Motivations for a new deformable model– desirable properties: • Adapt with deformations (sometimes drastic deformations, depending on applications) • Ability to learn object and background: • Ability to separate foreground and background • Ability to recognize object from one image frame to the next, in an image sequence

  4. Motivations: Some Existing Deformable Models • Deformable models: • Highly deformable • Examples: snake or active contour, B-spline snakes, … • Good deformation, but poor recognition (learning) ability • Not-so-deformable • Examples • Active shape and appearance models • G-snake • … • Good recognition (learning) capability, but of course poor deformation ability So, how about good deformation and good recognition capabilities?

  5. An Example Application Cardiac video tracking

  6. Technical Background: Level Set Function • A level set function represents a contour or a front geometrically • Consider a single-valued function (x, y) over the image domain; intersection of the x-y plane and  represents a contour: (X(x, y), Y(x, y)) is the point on the curve that is closest to the (x, y) point • Matlab demo (lev_demo.m)

  7. Applications of Level Set: Image Segmentation • Matlab segmentation demo (yezzi.m) • Vessel segmentation • Brain reconstruction • Virtual endoscopy • Trachea fly through • …tons out there Show videos

  8. Level Set Applications: Image Denoising • Two example videos Show video

  9. Level Set Applications: Robotics • Finding shortest path Show video

  10. Level Set Applications: Computer Graphics • Morphing • Simulation • Animation • …. http://www.sci.utah.edu/stories/2004/fall_levelset.html Go to http://graphics.stanford.edu/~fedkiw/ for amazing videos

  11. More Applications of Level Set Methods • Go to http://math.berkeley.edu/~sethian/2006/Applications/Menu_Expanded_Applications.html

  12. Technical Background: Non-Parametric Density Estimation Normalized image intensity histogram: I(x, y) is the image intensity at (x, y) i is the standard deviation of the Gaussian kernel C is a normalization factor that forces H(i) to integrate to unity

  13. Technical Background: Similarity and Dissimilarity Measures for PDFs Kullback-Leibler (KL) divergence (a dissimilarity measure): Bhattacharya coefficient (a similarity measure): P(z) and Q(z) are two PDFs being compared

  14. Proposed Method: Deformable Object Model and Tracking • Deformable Object model: • From the first frame learn the joint pdf of level set function and image intensity (image feature) • Tracking: • From second frame onward search for similar joint pdf

  15. Proposed Deformable Object Model • Joint probability density estimation with Gaussian kernels: Level set function value: l Image intensity: i J(x, y) is the image intensity at (x, y) point on the first image frame (x, y) is the value of level set function at (x, y) on the first image frame C is a normalization factor We learn Q on the first video frame given the object contour (represented by the level set function)

  16. Proposed Object Tracking • On the second (or subsequent) frame compute the density: • Match the densities P and Q by KL-divergence: • Minimize KL-divergence by varying the level set function (x, y) Note that here only P is a function of (x, y) I(x, y) is the image intensity at (x, y) on the second/subsequent frame (x, y) is the level set function at on the second/subsequent frame

  17. Minimizing KL-divergence • In order to minimize KL-divergence we use Calculus of variations • After applying Calculus of variations the rule of update (gradient descent rule) for the level set function becomes: t : iteration number t : timestep size

  18. Minimizing KL-divergence: Implementation • There is a compact way of expressing the update rule: convolution is a function defined simply as: Where g1 is a convolution kernel:

  19. Minimizing KL-divergence: A Stable Implementation • The previous implementation is called explicit scheme and is unstable for large time steps; if small time step is used then the convergence will be extremely slow • One remedy is a semi-implicit scheme of numerical implementation: Where g is a convolution kernel: is a function defined simply as: In this numerical scheme t can be large and still the solution will be convergent; So very quick convergence is achieved in this scheme

  20. Results: Tracking Cardiac Motion A few cine MRI frames and delineated boundaries on them Show videos

  21. Numerical Results and Comparison Sequence with slow heart motion Sequence with rapid heart motion Comparison of mean performance measures

  22. Extensions: Tracking Objects in Color Video • If we want to learn joint distribution of level set function and color channels (say, r, g, b), then non-parametric density estimation suffers from: • Slowness • Curse of dimensionality • Another important theme is to combine edge information and region information for objects • One remedy sometimes is to take a linear combination of r, g, and b channels • Fisher’s linear discriminant can be used to learn the coefficients of linear combination

  23. Extensions: Adding Object Shape Constraint • Can we constrain the object shape in this computational framework? Minimize: where

  24. Application in Computer Graphics: Morphing Initial object Shape and intensity/texture Final object Shape and intensity/texture (J1, 1) (I2, 2) (I1, 1), (I2, 2), ….. Morphing: generate realistic intermediate tuples (It, t)

  25. Morphing: Formulation • Generate intermediate shapes, i.e., level set function t (say, via interpolation): • Next, generate intermediate intensity It by maximizing: • Once again we get a similar PDE for It

  26. Morphing: Preliminary Results Show videos

  27. Summary • Contributions • A novel deformable object model for tracking • Related efficient and stable computation • Several exciting extensions • Application in morphing

  28. Acknowledgements • Baidyanath Saha • CIMS lab and Prof. Hong Zhang • Prof. Dipti P. Mukherjee, Indian Statistical Institute • Department of Computing Science, UofA

More Related