Nalin pradeep senthamil masters student ece dept
1 / 32

Nalin Pradeep Senthamil Masters Student, ECE Dept. - PowerPoint PPT Presentation

  • Uploaded on

Nalin Pradeep Senthamil Masters Student, ECE Dept. Advisor, Dr Stan Birchfield Committee Members, Dr Adam Hoover, Dr Brian Dean . Accurate Tracking of Non-Rigid Objects using Level Sets. Clemson University, Clemson, SC USA Accepted in ICCV, 2009. Outline. Tracking Overview Literature

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Nalin Pradeep Senthamil Masters Student, ECE Dept.' - javier

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Nalin pradeep senthamil masters student ece dept l.jpg

Nalin Pradeep Senthamil Masters Student, ECE Dept.


Dr Stan Birchfield

Committee Members,

Dr Adam Hoover, Dr Brian Dean

Accurate tracking of non rigid objects using level sets l.jpg

Accurate Tracking of Non-Rigid Objects using Level Sets

Clemson University, Clemson, SC USA

Accepted in ICCV, 2009

Outline l.jpg

  • Tracking Overview

  • Literature

  • Proposed Approach

    • Object Fragmentation

      • Region Growing Mechanism

      • GMM modeling (feature-spatial)

    • Level Set Framework

    • Fragment Motion using Joint-KLT

  • Results

  • Conclusion

Tracking overview l.jpg
Tracking Overview

  • Idea: Obtain Trajectories over time to locate object

  • Three Main Categories

    • Point Tracking – Kalman, Particle filters

    • Kernel Tracking – Collins et al (linear RGB), Comaniciu (Mean-Shift)

    • Contour Tracking – Shah et al, Cremers et al

  • Applied to Surveillance – Vessel, human, vehicle etc

  • Why not internet videos ? – 65,000 videos get uploaded in YouTube everyday (rich market)

Literature l.jpg

  • Linear RGB [Collins et al. 2003]

  • Ada-boost classifier [Avidan 2005]

  • Fragments based fixed size [Adam et al. 2006]

  • Key-point Feature learning [Grabner et al. 2007]

  • Shape priors [Cremers et al. 2006]

  • Contour tracking using texture [Shah et al 2005]

  • Limitations

    • Ignore secondary cues such as multimodality

    • Lack in determining accurate object shape

    • Usually non-contour based techniques drift during occlusion

    • Often ignore spatial arrangement of pixels

Algorithm block diagram l.jpg

Tracker Initialization

User clicked ROI around object

Object Fragmentation

Each object as set of fragments

Object Modeling

Strength Map Computation

Level-Set Formulation

Estimate Fragment Motion

Algorithm Block Diagram

Update made at each frame

Object fragmentation l.jpg
Object Fragmentation

Region Growing Mechanism

  • Random pixel selected from mask – fragment (f)

  • Neighboring pixels added to (f) within Γ (std deviation)

  • Gaussian Model of (f) updated

  • Each (f) represents a Gaussian ellipsoid

  • Both Object and background are fragmented

Object modeling gmm l.jpg
Object Modeling (GMM)

Joint feature-spatial space,

Strength map l.jpg
Strength Map

+ve for FGND

-ve for BKGND

Level set framework l.jpg
Level Set Framework

  • Level Set is numerical technique for fitting contour

  • Level Set on 2D image is viewed as 3D function

  • Contour in level set identified at zero level

Level set for strength map l.jpg
Level Set for strength map



  • In general, Level set evolution defined by

  • Gradient Descent Iteration

Strength Image

Contour (zero level set)

Strength Image

Divergence operator

Level set evolution l.jpg

Iterations using “Elmo” strength map

Curve can grow inward and outward

Figure shows for first frame as example

Curve evolves from previous contours in subsequent tracking

Level-Set Evolution

Fragment motion l.jpg
Fragment Motion

  • Joint-KLT: Combines algorithms of KLT and HS

  • Hence,

  • Used to align coordinate system of object and model fragments

  • Increases accuracy of strength map

data term

smoothness term

Fragment motion contd l.jpg
Fragment Motion (contd.)

  • ‘N’ features tracked in each fragment are averaged

  • Motion of each fragment gives ‘prior’ information before computing strength map

  • Drastic motion can be addressed



Shape matching l.jpg
Shape Matching

  • Hausdorff metric is mathematical measure to compare two sets of points

  • Application in Occlusion Handling and Shape recognition

‘a’ and ‘b’ are two point sets

Occlusion handling l.jpg
Occlusion Handling

  • Rate of decrease in object size determines occlusion

  • Contour shapes learnt online is used to hallucinate during occlusion

  • Best shape is identified using Hausdorff distance metric

  • Previously learnt subsequent shapes are hallucinated during occlusion

Quantitative comparison l.jpg
Quantitative Comparison

Walk Behind

Elmo Doll

Girl Circle

Average Normalized error obtained against ground-truth of sequences at every 5 frames.

Conclusion l.jpg

  • Tracking algorithm based on modeling object and background with mixture of Gaussians

  • Simple and efficient region growing mechanism to achieve fast computation

  • Embedding “strength map” into Level-Set Framework

  • Joint KLT introduced in the framework to improve accuracy

  • Future Work:

    • Robust shape prior learning and matching

    • Self-occlusion handling for unknown fragments

Alternative tracking framework outline l.jpg
Alternative Tracking Framework (outline)

  • Overview

  • Proposed Approach

    • Vessel Detection

      • Saliency Map

      • Thresholding

    • Vessel Tracking

      • Strength Map using Linear RGB

      • ML Framework for Search

  • Results

Object detection using saliency map l.jpg
Object Detection Using Saliency Map

  • Saliency: Property of objects standing out relative to their neighbors.

  • There is a statistical relationship between backgrounds of all natural images similar to pre-attentive search done by human visual system.

  • Zhang et al (CVPR 2007) observed redundancies in log Fourier spectra of natural images. Hence, any statistical singularities in the spectrum can be treated as anomalies.

Saliency map computation l.jpg

Smoothing in frequency domain

Smoothing in spatial domain

Saliency Map Computation

  • Algorithm

    • Letbe the image.

    • Real part of Fourier Spectrum

    • Phase

    • Log Spectrum

    • Spectral Residual

    • Saliency Map , j=sqrt(-1)

Object tracking l.jpg
Object Tracking

  • Objects detected through saliency used as FGND

  • Immediate surrounding used as BKGND

  • Strength Model Computed similar to Collins Linear RGB

  • 49 features selected from linear combination used to identify strength map

  • Maximum Likelihood Framework based search used to localize objects in each frame

  • Region search was identified based on object velocity

Object tracking strength model l.jpg
Object Tracking – Strength Model



Small value – 0.01

Variance of L(i) with respect to a distribution a(i)

  • 49 features of RGB are normalized into 0-255 and discretized into 0-32 histogram bins

  • For each feature,

  • Variance Ratio of Log-likelihood is identified that best discriminates object from background

Object tracking ml framework l.jpg
Object Tracking – ML Framework



Strength Map

Prevent pixel locations farther from object

  • Objective was to recover tight bound around object

  • ML Framework is like EM algorithm

  • Search objective is to maximize the function (Mean, Covariance)

Object tracking ml framework30 l.jpg
Object Tracking – ML Framework

Mean and covariance of current estimate

  • To maximize the function, Mean and Covariance are computed iteratively

    • E-Step

    • M-Step

Iterated for 2-3 times to get optimal values

Conclusion31 l.jpg

  • Algorithm was real time and supported around 25-30 fps in speed

  • Saliency map based detection was introduced

  • Concept of “strength map” from adaptive-fragmentation is applied here

  • Depends only on color (linearRGB), and combination with KLT features would add robustness to the system. Good way to combine is explored.