1 / 29

JetStream : Probabilistic Contour Extraction with Particles

JetStream : Probabilistic Contour Extraction with Particles. Patrick Perez, Andrew Blake, and Michel Gangnet, Microsoft Research, St George House, 1 Guildhall Street, Cambridge, CB2 3NH, UK http://research.microsoft.com/vision Presented by: Vladan Radosavljevic. Outline. Introduction

tnall
Download Presentation

JetStream : Probabilistic Contour Extraction with Particles

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. JetStream: Probabilistic Contour Extraction with Particles Patrick Perez, Andrew Blake, and Michel Gangnet, Microsoft Research, St George House, 1 Guildhall Street, Cambridge, CB2 3NH, UK http://research.microsoft.com/vision Presented by: Vladan Radosavljevic

  2. Outline • Introduction • Image Gradient • Related Work • Probabilistic contour tracking • Tracking framework • Dynamics • Measurement • Iterative computation of posterior - particles • Model Ingredients • Likelihood ratio • Dynamics • Proposal Sampling Function • Experimental Results • Conclusion

  3. Motivation • Contour extraction • from segmenting images with closed contours • to the extraction of linear structures of particular interest such as roads

  4. The gradient direction is given by: • how does this relate to the direction of the edge? • The edge strength is given by the gradient magnitude Introduction - image gradient • The gradient of an image: • The gradient points in the direction of most rapid change in intensity

  5. Introduction • Most approaches to contour extraction rely on some minimal cost principle • k is the curvature, s is the arc-length • y(r(s)) scalar or vector derived at location r(s) from the raw image I • Often y(r(s)) is the gradient norm • This function captures some kind of regularity on candidate curves, for example - rewarding, by a lower cost, the presence of large gradients along the contour

  6. Introduction – related work • First approach - dynamic programming • optimal curve in the form of chain pixels • unless optimality is abandoned, and huge storage resources are available, there are tight restrictions on the form of cost function • Second approach - growing a contour from the seed point according to cost function • given the current contour, a new segment is appended to it according both to a shape prior (mainly smoothness) and to the evidence provided by the data at the location under concern • deterministic: complete discontinuous curves provided by edge detectors

  7. Introduction – related work • A probabilistic point of view: • the contours are seen as the paths of a stochastic process • tracking problem • We are interesting in the method that is able: • to avoid spurious distracting contours • to track the multiple off-springs starting at branching contours • to interpolate over transient evidence “gaps” • JetStream – a method with particle filtering

  8. Probabilistic contour tracking • Tracking contours in still images is an unconventional tracking problem because of the absence of a real notion of time • The “time” is only associated with the growing of the contour in the image plane – consider a spatial chain as a temporal chain • Contrary to standard tracking problems where data arrive one bit after another as time passes by, the whole set of data y is standing there at once • There is no straightforward way of tuning the “speed”, or equivalently the length of successive moves

  9. Tracking framework - dynamics • xi – points in the plane R2 • (x0, x1,...,xn) - curve in some standard way, e.g., the xi’s are the vertices of a polyline, starting from x0then moving along the contour to xn • dynamics: • assuming second order dynamics with some kernel q: • then a priori density is:

  10. Tracking framework - measurement • measurement y (observed image) conditioned on x0:n is considered as independent spatial process which is not true in reality but it is a reasonable approximation: • where Ώ is a discrete set of measurements locations in the image plane • the conditional distributions depend only on whether or not point u is contained in contour x0:n • in particular, for any two points u along the contour the corresponding conditional distributions are identical, similarly for any two points in the background

  11. Tracking framework - measurement • Therefore, each is either ponif u belongs to the contour x0:n, or poffif not: • Finally: • where

  12. Iterative computation of posterior • Function can be considered as a cost function • Expressed as the minimization of this function, the contour extraction problem then amounts to seeking the maximum a posteriori (MAP) estimate • Posterior densities can be computed recursively: • but there is no closed form of the pi • pican be approximatedby a finite set of M sample paths (the ‘particles’) • Best path at step i can be which is a Monte Carlo approximation of posterior expectation

  13. Iterative computation of posterior • Prediction: each path is grown one step by sampling from the proposal density function f • If the paths are samples from pithen the extended paths are samples from f*pi • Since we want samples from distribution pi+1, extended paths are weighted according to ratio • The resulting weighted path set now provides an approximation of the target distribution pi+1 • M paths are drawn with replacement from the weighted set • The weights are:

  14. Likelihood ratio l • Measurement: the norm of the luminance (or color) gradient • poffpon • poff – exponential distribution • pon - complex mixture, better keep as less informative as possible

  15. Likelihood ratio l • The direction of the gradient also retains precious information that a data model based only on gradient norm neglects: the distribution of Ψ is symmetric, and it becomes tighter as the norm of the gradient increases

  16. Likelihood ratio l • However, at corners, the norm of the gradient is usually large but its direction cannot be accurately measured • Using a standard corner detector, each pixel u is associated with a label c(u) = 1 if a corner is detected, and 0 otherwise • Where a corner has been detected assumption is that distribution is uniform

  17. Dynamics q • Because of the absence of natural time, it is better to consider a dynamics with fixed step length d. The definition of second order dynamics then amounts to specifying an a priori probability distribution on direction change Θi • Finally: • To allow for abrupt direction changes at the locations where corners have been detected, the normal distribution is mixed with a small proportion of uniform distribution

  18. Proposal sampling function f • With choice f = q, corners will be mostly ignored since the expected number of particles undertaking drastic direction changes is vM, where typically ν= 0.01 and M = 100 • At locations where no corners are detected, the proposal density is the normal component of the dynamics. • If location lies on a detected corner, the next proposed location is obtained by turning of an angle picked uniformly • Therefore:

  19. Proposal sampling function f

  20. JetStream - iteration

  21. Experimental Results – Interactive cut-out • The extraction of a region of interest from one image • In practice, JetStream is run for a fixed number n of steps (100 in our experiments) from initial conditions x0:1 chosen by the user. • If the result is satisfactory, n more steps are undertaken. • If not, a restart region within the particle flow, and an associated restart direction, can be chosen by the user

  22. Experimental Results – Interactive cut-out • More sophisticated user interaction • Provide the user with the facility to place one or more dams, defined as regions Rk where:

  23. Experimental Results – Interactive cut-out

  24. Experimental Results – Road extraction • In the specific context of road extraction, one is in fact interested in recovering “ribbons” • Using JetStream as defined previously results in paths jumping from one side of the ribbon to the other

  25. Experimental Results – Road extraction • The state space is extended to include a width variable mi, which indicates the distance at step i between the two sides of the ribbon • It expresses that xi+ and xi- defined as • are on a contour while xi is not:

  26. Experimental Results – Road extraction • A simple first order dynamics is chosen for m:

  27. Experimental Results – Road extraction

  28. Conclusion • JetStream is applicable if there are not sharp corners in the images • If sharp corners exist or the image’s intensity distribution is complex JetStream doesn’t provide good results

  29. References • [1] M. Pérez, A. Blake, and M. Gangnet. JetStream : Probabilistic contour extraction with particles. In Int. Conf. on Computer Vision, ICCV’ 2001, Vancouver, Canada, July 2001.

More Related