Tracking using a highly deformable object model
Download
1 / 27

Tracking Using A Highly Deformable Object Model - PowerPoint PPT Presentation


  • 211 Views
  • Uploaded on

Tracking Using A Highly Deformable Object Model. Nilanjan Ray Department of Computing Science University of Alberta. Overview of Presentation. Tracking deformable objects Motivations: desirable properties of a deformable object model An example application (mouse heart tracking)

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Tracking Using A Highly Deformable Object Model' - brenna


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Tracking using a highly deformable object model

Tracking Using A Highly Deformable Object Model

Nilanjan Ray

Department of Computing Science

University of Alberta


Overview of presentation
Overview of Presentation

  • Tracking deformable objects

    • Motivations: desirable properties of a deformable object model

    • An example application (mouse heart tracking)

  • Some technical background

    • Level set function and its application in image processing

    • Non-parametric probability density function (pdf) estimation

    • Similarity/dissimilarity measures for pdfs

  • Proposed tracking technique

  • Results, comparisons and demos

  • Ongoing investigations

    • Incorporating color cues, and other features

    • Adding constraints on object shape

    • Application in morphing (?)

    • Incorporating object motion information (??)

  • Summary

  • Acknowledgements


Tracking deformable objects
Tracking Deformable Objects

  • Desirable properties of deformable models:

    • Adapt with deformations (sometimes drastic deformations, depending on applications)

    • Ability to learn object and background:

      • Ability to separate foreground and background

      • Ability to recognize object from one image frame to the next, in an image sequence

Show cine MRI video


Some existing deformable models
Some Existing Deformable Models

  • Deformable models:

    • Highly deformable

      • Examples: snake or active contour, B-spline snakes, …

      • Good deformation, but poor recognition (learning) ability

    • Not-so-deformable

      • Examples

        • Active shape and appearance models

        • G-snake

      • Good recognition (learning) capability, but of course poor deformation ability

So, how about good deformation and good recognition capabilities?


Technical background level set function
Technical Background: Level Set Function

  • A level set function represents a contour or a front geometrically

  • Consider a single-valued function (x, y) over the image domain; intersection of the x-y plane and  represents a contour:

    (X(x, y), Y(x, y)) is the point on the curve that is closest to the (x, y) point

  • Matlab demo (lev_demo.m)


Applications of level set image segmentation
Applications of Level Set: Image Segmentation

  • Matlab segmentation demo (yezzi.m)

  • Vessel segmentation

  • Brain reconstruction

  • Virtual endoscopy

  • Trachea fly through

  • …tons out there

Show videos


Level set applications image denoising
Level Set Applications: Image Denoising

  • Two example videos

Show video


Level set applications robotics
Level Set Applications: Robotics

  • Finding shortest path

Show video


Level set applications computer graphics
Level Set Applications: Computer Graphics

  • Morphing

  • Simulation

  • Animation

  • ….

http://www.sci.utah.edu/stories/2004/fall_levelset.html

Go to http://graphics.stanford.edu/~fedkiw/

for amazing videos


More applications of level set methods
More Applications of Level Set Methods

  • Go to http://math.berkeley.edu/~sethian/2006/Applications/Menu_Expanded_Applications.html


Technical background non parametric density estimation
Technical Background: Non-Parametric Density Estimation

Normalized image intensity histogram:

I(x, y) is the image intensity at (x, y)

i is the standard deviation of the Gaussian kernel

C is a normalization factor that forces H(i) to integrate to unity


Technical background similarity and dissimilarity measures for pdfs
Technical Background: Similarity and Dissimilarity Measures for PDFs

Kullback-Leibler (KL) divergence (a dissimilarity measure):

Bhattacharya coefficient (a similarity measure):

P(z) and Q(z) are two PDFs being compared


Proposed method tracking deformable object
Proposed Method: Tracking Deformable Object for PDFs

  • Deformable Object model (due to Leventon [1]):

    • From the first frame learn the joint pdf of level set function and image intensity (image feature)

  • Tracking:

    • From second frame onward search for similar joint pdf

[1] M. Leventon, Statistical Models for Medical Image Analysis, Ph.D. Thesis, MIT, 2000.


Deformable object model
Deformable Object Model for PDFs

  • Joint probability density estimation with Gaussian kernels:

Level set function value: l

Image intensity: i

J(x, y) is the image intensity at (x, y) point on the first image frame

(x, y) is the value of level set function at (x, y) on the first image frame

C is a normalization factor

We learn Q on the first video frame given the object contour (represented

by the level set function)


Proposed object tracking
Proposed Object Tracking for PDFs

  • On the second (or subsequent) frame compute the density:

  • Match the densities P and Q by KL-divergence:

  • Minimize KL-divergence by varying the level set function (x, y)

Note that here only P is

a function of (x, y)

I(x, y) is the image intensity at (x, y) on the second/subsequent frame

(x, y) is the level set function at on the second/subsequent frame


Minimizing kl divergence
Minimizing KL-divergence for PDFs

  • In order to minimize KL-divergence we use Calculus of variations

  • After applying Calculus of variations the rule of update (gradient descent rule) for the level set function becomes:

t : iteration number

t : timestep size


Minimizing kl divergence implementation
Minimizing KL-divergence: Implementation for PDFs

  • There is a compact way of expressing the update rule:

convolution

is a function defined simply as:

Where g1 is a convolution kernel:


Minimizing kl divergence a stable implementation
Minimizing KL-divergence: A Stable Implementation for PDFs

  • The previous implementation is called explicit scheme and is unstable for large time steps; if small time step is used then the convergence will be extremely slow

  • One remedy is a semi-implicit scheme of numerical implementation:

Where g is a convolution kernel:

is a function defined simply as:

In this numerical scheme t can be large and still the solution will

be convergent; So very quick convergence is achieved in this scheme


Results tracking cardiac motion
Results: Tracking Cardiac Motion for PDFs

A few cine MRI frames and delineated boundaries on them

Show videos


Numerical results and comparison
Numerical Results and Comparison for PDFs

Sequence with slow heart motion

Sequence with rapid heart motion

Comparison of mean performance measures


Extensions tracking objects in color video
Extensions: Tracking Objects in Color Video for PDFs

  • If we want to learn joint distribution of level set function and color channels (say, r, g, b), then non-parametric density estimation suffers from:

    • Slowness

    • Curse of dimensionality

  • Another important theme is combine edge information and region information of objects

  • One remedy sometimes is to take a linear combination of r, g, and b channels

    • Fisher’s linear discriminant can be used to learn the coefficients of linear combination

  • A demo


Extensions adding object shape constraint
Extensions: Adding Object Shape Constraint for PDFs

  • Can we constrain the object shape in this computational framework?

Minimize:

where


Application in computer graphics morphing
Application in Computer Graphics: Morphing for PDFs

Initial object Shape and

intensity/texture

Final object Shape and

intensity/texture

(J1, 1)

(I2, 2)

(I1, 1),

(I2, 2),

…..

Morphing: generate realistic intermediate tuples (It, t)


Morphing formulation
Morphing: Formulation for PDFs

  • Generate intermediate shapes, i.e., level set function t (say, via interpolation):

  • Next, generate intermediate intensity It by maximizing:

  • Once again we get a similar PDE for It


Morphing preliminary results
Morphing: Preliminary Results for PDFs

Show videos


Summary
Summary for PDFs

  • Highly deformable object tracking: Variational minimization of KL-divergence leading to fast and stable partial differential equations

  • Several exciting extensions

  • Application in morphing


Acknowledgements
Acknowledgements for PDFs

  • Baidyanath Saha

  • CIMS lab and Prof. Hong Zhang

  • Prof. Dipti P. Mukherjee, Indian Statistical Institute

  • Department of Computing Science, UofA


ad