Create Presentation
Download Presentation

Download Presentation
## Counting Crowded Moving Objects

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**Counting Crowded Moving Objects**Vincent Rabaud and Serge Belongie Department of Computer Science and Engineering University of California, San Diego {vrabaud,sjb}@cs.ucsd.edu Presentation by: Yaron Koral IDC, Herzlia, ISRAEL**AGENDA**• Motivation • Challenges • Algorithm • Experimental Results**AGENDA**• Motivation • Challenges • Algorithm • Experimental Results**Motivation**• Counting crowds of people • Counting herds of animals • Counting migrating cells • Everything goes as long as the crowdis homogeneous!!**AGENDA**• Motivation • Challenges • Algorithm • Experimental Results**Challenges**• The problem of occlusion • Inter-object • Self occlusion • Large number of independent motions • Dozens of erratically moving objects • Require more than two successive frames Surveillance camera viewing a crowd from a distant viewpoint, but zoomed in, such that the effects of perspective are minimized.**AGENDA**• Motivation • Challenges • Algorithm • Experimental Results**Algorithm Highlights**• Feature Tracking with KLT • Increased Efficiency • Feature Re-Spawning • Trajectory Conditioning • Trajectory Clustering**Algorithm Highlights**• Feature Tracking with KLT • Increased Efficiency • Feature Re-Spawning • Trajectory Conditioning • Trajectory Clustering**Harris Corner Detector – What are Good Features?C.Harris,**M.Stephens. “A Combined Corner and Edge Detector”. 1988 • We should easily recognize a corner by looking through a small window • Shifting a window in anydirection should give a large change in intensity**Harris Detector: Basic Idea**“flat” region:no change in all directions “edge”:no change along the edge direction “corner”:significant change in all directions**Window function**Shifted intensity Intensity Window function w(x,y) = or 1 in window, 0 outside Gaussian Harris Detector: Mathematics Change of intensity for shift in [u,v] direction:**Harris Detector: Mathematics**For small [u,v]: We have:**Harris Detector: Mathematics**For small shifts [u,v] we have a bilinear approximation: where M is a 22 matrix computed from image derivatives:**Harris Detector: Mathematics**Denotes by ei the ith eigen-vactor of M whose eigen-value is i: Conclusions:**Harris Detector: Mathematics**Intensity change in shifting window: eigenvalue analysis 1, 2 – eigenvalues of M direction of the fastest change Ellipse E(u,v) = const direction of the slowest change (max)-1/2 (min)-1/2**Harris Detector: Mathematics**2 Classification of image points using eigenvalues of M: “Edge” 2 >> 1 “Corner”1 and 2 are large,1 ~ 2;E increases in all directions 1 and 2 are small;E is almost constant in all directions “Edge” 1 >> 2 “Flat” region 1**Sum of Squared Differences – Tracking Features**• SSD is optimal in the sense of ML when • Constant brightness assumption • i.i.d. additive Gaussian noise**Exhaustive Search**• Loop over all parameter space • No realistic in most cases • Computationally expensive • E.g. to search 100X100 image in 1000X1000 image using only translation ~1010 operations! • Explodes with number of parameters • Precision limited to step size**The Problem**Find (u,v) that minimizes the SSD over region A. Assume that (u,v) are constant over all A**Iterative Solution**• Lucas Kanade (1981) • Use Taylor expansion of I (the optical flow equation) • Find**Feature Tracking with KLT(We’re back to crowd counting…)**• KLT is a feature tracking algorithm • Driving Principle: • Determine the motion parameter oflocal window W from image I to consecutive image J • The center of the window defines the tracked feature**Feature Tracking with KLT**• Given a window W • the affine motion parameters A and d are chosen to minimize the dissimilarity Affine Motion = Monitoring**Feature Tracking with KLT**• It is assumed that only d matters between 2 frames. Therefore a variation of SSD is used Pure Translation -> Tracking min(λ1,λ2) > t • A window is accepted as a candidate feature if in the center of the window, both eigenvalues exceed a predefined threshold t**Algorithm Highlights**• Feature Tracking with KLT • Increased Efficiency • Feature Re-Spawning • Trajectory Conditioning • Trajectory Clustering**Increased Efficiency #1**• Associating only one window with each feature • Giving a uniform weight function that depends on 1/(window area |w|) • Determining quality by comparing: • Computation of different Z matrices is accelerated by “integral image”[1] [1] Viola & Jones 2004**Increased Efficiency #2**• Run on sample training frames first • Determine parameters that lead to the optimal windows sizes • Reduces to less than 5% of the possible parameter set • All objects are from the same class Surveillance camera viewing a crowd from a distant viewpoint, but zoomed in, such that the effects of perspective are minimized.**Algorithm Highlights**• Feature Tracking with KLT • Increased Efficiency • Feature Re-Spawning • Trajectory Conditioning • Trajectory Clustering**Feature Re-Spawning**• Along time, KLT looses track: • Inter-object occlusion • Self occlusion • Exit from picture • Appearance change due to perspective and articulation • KLT recreates features all the time • Computationally intensive • Weak features are renewed**Feature Re-Spawning**• Re-Spawn features only at specific locations in space and time • Propagate them forward and backward in time • Find the biggest “holes” • Re-spawn featuresin frame with theweighted average oftimes**Algorithm Highlights**• Feature Tracking with KLT • Increased Efficiency • Feature Re-Spawning • Trajectory Conditioning • Trajectory Clustering**Trajectory Conditioning**• KLT tracker gives a set of trajectories with poor homogeneity • Don’t begin and end at the same times • Occlusions can result in trajectory fragmentation • Feature can lose its strength resulting in less precise tracks • Solution: condition the data • Spatially and temporally**Trajectory Conditioning**• Each trajectory is influenced by its spatial neighbors • Apply a box to each raw trajectory • Follow all neighbor trajectories from the time the trajectory started**Algorithm Highlights**• Feature Tracking with KLT • Increased Efficiency • Feature Re-Spawning • Trajectory Conditioning • Trajectory Clustering**Trajectory Clustering**• Determine number of object at time t by clustering trajectories • Since at time t objects may be close, focus attention on a time interval (half-width of 200 frames) • Build connectivity graph • At each time step, the present features form the nodes of a connectivity graph G • Edges indicate possible membership to a common object.**Surveillance camera viewing a crowd from a distant**viewpoint, but zoomed in, such that the effects of perspective are minimized. Trajectory Clustering • Connectivity Graph • Bounding Box: as small as possible, able to contain every possible instance of the object • If two features do not stay in a certain box, they do not belong to the same object. • The 3 parameters of this box are learned from training data. Articulation factor**Trajectory Clustering**• Rigid parts merging • Features share similar movement during whole life span, belong to a rigid part of an object, and consequently to a common object • RANSAC is applied to sets of trajectories • Within time window • Connected in graph G**Trajectory Clustering**• Agglomerative Clustering • At each iteration, the two closest sets are considered • If all features are linked to each other in the connectivity graph, they are merged together. • Otherwise, the next closest sets are considered • Proceed until all possible pairs are analyzed**AGENDA**• Motivation • Challenges • Algorithm • Experimental Results**Experimental results**• Datasets • USC: elevated view of a crowd consisting of zero to twelve persons • LIBRARY: elevated view of a crowd of twenty to fifty persons • CELLS: red blood cell dataset consisting of fifty to hundred blood cells**Experimental results**Estimated Ground Truth**Conclusion**• A new way for segmenting motions generated by multiple objects in crowd • Enhancements to KLT tracker • Conditioning and Clustering techniques