Chaotic Invariants of Lagrangian Particle Trajectories for Anomaly Detection in Crowded Scenes

1. Chaotic Invariants of Lagrangian Particle Trajectories for Anomaly Detection in Crowded Scenes Shandong Wu, Brian E. Moore, Mubarak Shah Computer Vision Lab, University of Central Florida, USA sdwu@eecs.ucf.edu, bmoore@math.ucf.edu, shah@eecs.ucf.edu Crowd Scene Analysis • Challenges • Very high density of objects • Diverse level of coherency of motions • Traditional methods • Only suitable for sparse scenes • Suffer from severe occlusions, small object sizes, similar appearance Experiment 1: Anomaly Detection Video clip Optical flow Particle trajectory Particle advection Clustering particle trajectory Calculating optical flow Representative trajectory Chaotic description Scene model Anomaly Calculating chaotic invariants Detection &Localization Learning model Query clip Particle trajectory Clustering Chaotic Description • To identify the dynamics of representative trajectories • Chaotic dynamics by measurable chaotic invariants • Chaotic feature set • : Largest Lyapunov exponent • : Correlation dimension • : Mean of representative trajectories • Only necessary for position-caused anomalies • 2D trajectory embedding: in terms of two 1D time series Particle Advection The Algorithm X time series of a representative trajectory Video clip Embedding (m: embedding dimension; J: time delay) Orbit Optical flow Normality model For , find its nearest neighbor Representative trajectory Sub-pixel level 2D optical flow interpolation p: mean period of orbit Pairs of nearest neighbors Calculate mean rate of Separation of the nearest neighbors Use the distances of nearest neighbors Euler’s method ROC curve and comparison with previous work Detection example Dataset: Unusual crowd activity dataset from University of Minnesota d(t): average divergence Calculating L Particle’s position for jth pair of nearest neighbors : distance between the jth pair of nearest neighbors Correlation sum Calculating D Experiment 2: Anomaly Localization Particle trajectory A set of approximately parallel lines each with a slope roughly proportional to L : Heaviside step function : threshold distance : number of points Fit the average line Cluster Particle Trajectories • Observation: • A bunch of adjacent particle trajectories may belong to a single sub-object • Method: clustering • Step 1: Remove relatively motionless particles and trajectories that carry minor information • Step 2: Cluster by k-means according to position information • Output: Representative trajectories (crucial for small and noisy data set) Likelihood of representative trajectories w.r.t normality model Clip 29 Clip 30 (Anomaly source GT) Advantages of the Algorithms • The above algorithms proven to be insensitive to the changes in • time delay • embedding dimension • size of data set and • to some extent noise • More importantly, ensure L>0 for the condition of chaotic analysis (a) All representative trajectories (b) Clustering of trajectories with low probability (c) Localized major anomaly sources A more efficient and manageable crowd scene representation ** Experiment 3: Position-caused Anomaly Localization Anomaly Detection • Definition of anomaly • Spatiotemporal change of system dynamics (chaotic or/and positions) • Global anomaly: entire change of dynamics • Local anomaly: dynamic changes near particular spatial points • Normality model • Multi-variate GMM: 4D ( ) or 6D ( plus ) • Learning by: EM + IPRA algorithm Synthesized anomaly Detected anomaly Chaotic features Position feature Anomaly Localization • Localize the anomaly in terms of position & size • Calculate the likelihoods contributed by each representative trajectory • Localize those trajectories with low likelihoods • Cluster them according to position information • Filter out the clusters with fewer number of trajectories • The remaining clusters reveal the major abnormal regions • Novelties / Contributions • A novel combination of Lagrangian particle dynamics approach together with chaotic modeling • Unique utilization of clustering of particle trajectories for modeling crowded scenes (novel modeling element: representative trajectory) • Chaotic dynamics are introduced into the crowd context with effective algorithms for calculating chaotic invariants • Our methods are able to deal with both coherent and incoherent flows