Create Presentation
Download Presentation

Download Presentation
## A Highly Deformable Object Model for Tracking

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**A Highly Deformable Object Model for Tracking**Nilanjan Ray Department of Computing Science University of Alberta**Overview of Presentation**• Tracking deformable objects • Motivations: desirable properties of a deformable object model • An example application (mouse heart tracking) • Some technical background • Level set function and its application in image processing • Non-parametric probability density function (pdf) estimation • Similarity/dissimilarity measures for pdfs • Proposed deformable object model and tracking technique • Results, comparisons and demos • Ongoing investigations • Incorporating color cues, and other features • Adding constraints on object shape • Application in morphing (?) • Summary • Acknowledgements**Tracking Deformable Objects**• Motivations for a new deformable model– desirable properties: • Adapt with deformations (sometimes drastic deformations, depending on applications) • Ability to learn object and background: • Ability to separate foreground and background • Ability to recognize object from one image frame to the next, in an image sequence**Motivations: Some Existing Deformable Models**• Deformable models: • Highly deformable • Examples: snake or active contour, B-spline snakes, … • Good deformation, but poor recognition (learning) ability • Not-so-deformable • Examples • Active shape and appearance models • G-snake • … • Good recognition (learning) capability, but of course poor deformation ability So, how about good deformation and good recognition capabilities?**An Example Application**Cardiac video tracking**Technical Background: Level Set Function**• A level set function represents a contour or a front geometrically • Consider a single-valued function (x, y) over the image domain; intersection of the x-y plane and represents a contour: (X(x, y), Y(x, y)) is the point on the curve that is closest to the (x, y) point • Matlab demo (lev_demo.m)**Applications of Level Set: Image Segmentation**• Matlab segmentation demo (yezzi.m) • Vessel segmentation • Brain reconstruction • Virtual endoscopy • Trachea fly through • …tons out there Show videos**Level Set Applications: Image Denoising**• Two example videos Show video**Level Set Applications: Robotics**• Finding shortest path Show video**Level Set Applications: Computer Graphics**• Morphing • Simulation • Animation • …. http://www.sci.utah.edu/stories/2004/fall_levelset.html Go to http://graphics.stanford.edu/~fedkiw/ for amazing videos**More Applications of Level Set Methods**• Go to http://math.berkeley.edu/~sethian/2006/Applications/Menu_Expanded_Applications.html**Technical Background: Non-Parametric Density Estimation**Normalized image intensity histogram: I(x, y) is the image intensity at (x, y) i is the standard deviation of the Gaussian kernel C is a normalization factor that forces H(i) to integrate to unity**Technical Background: Similarity and Dissimilarity Measures**for PDFs Kullback-Leibler (KL) divergence (a dissimilarity measure): Bhattacharya coefficient (a similarity measure): P(z) and Q(z) are two PDFs being compared**Proposed Method: Deformable Object Model and Tracking**• Deformable Object model: • From the first frame learn the joint pdf of level set function and image intensity (image feature) • Tracking: • From second frame onward search for similar joint pdf**Proposed Deformable Object Model**• Joint probability density estimation with Gaussian kernels: Level set function value: l Image intensity: i J(x, y) is the image intensity at (x, y) point on the first image frame (x, y) is the value of level set function at (x, y) on the first image frame C is a normalization factor We learn Q on the first video frame given the object contour (represented by the level set function)**Proposed Object Tracking**• On the second (or subsequent) frame compute the density: • Match the densities P and Q by KL-divergence: • Minimize KL-divergence by varying the level set function (x, y) Note that here only P is a function of (x, y) I(x, y) is the image intensity at (x, y) on the second/subsequent frame (x, y) is the level set function at on the second/subsequent frame**Minimizing KL-divergence**• In order to minimize KL-divergence we use Calculus of variations • After applying Calculus of variations the rule of update (gradient descent rule) for the level set function becomes: t : iteration number t : timestep size**Minimizing KL-divergence: Implementation**• There is a compact way of expressing the update rule: convolution is a function defined simply as: Where g1 is a convolution kernel:**Minimizing KL-divergence: A Stable Implementation**• The previous implementation is called explicit scheme and is unstable for large time steps; if small time step is used then the convergence will be extremely slow • One remedy is a semi-implicit scheme of numerical implementation: Where g is a convolution kernel: is a function defined simply as: In this numerical scheme t can be large and still the solution will be convergent; So very quick convergence is achieved in this scheme**Results: Tracking Cardiac Motion**A few cine MRI frames and delineated boundaries on them Show videos**Numerical Results and Comparison**Sequence with slow heart motion Sequence with rapid heart motion Comparison of mean performance measures**Extensions: Tracking Objects in Color Video**• If we want to learn joint distribution of level set function and color channels (say, r, g, b), then non-parametric density estimation suffers from: • Slowness • Curse of dimensionality • Another important theme is to combine edge information and region information for objects • One remedy sometimes is to take a linear combination of r, g, and b channels • Fisher’s linear discriminant can be used to learn the coefficients of linear combination**Extensions: Adding Object Shape Constraint**• Can we constrain the object shape in this computational framework? Minimize: where**Application in Computer Graphics: Morphing**Initial object Shape and intensity/texture Final object Shape and intensity/texture (J1, 1) (I2, 2) (I1, 1), (I2, 2), ….. Morphing: generate realistic intermediate tuples (It, t)**Morphing: Formulation**• Generate intermediate shapes, i.e., level set function t (say, via interpolation): • Next, generate intermediate intensity It by maximizing: • Once again we get a similar PDE for It**Morphing: Preliminary Results**Show videos**Summary**• Contributions • A novel deformable object model for tracking • Related efficient and stable computation • Several exciting extensions • Application in morphing**Acknowledgements**• Baidyanath Saha • CIMS lab and Prof. Hong Zhang • Prof. Dipti P. Mukherjee, Indian Statistical Institute • Department of Computing Science, UofA