Stable Multi-Target Tracking in Real-Time Surveillance Video

1. Stable Multi-Target Tracking in Real-Time Surveillance Video Active Vision Group Department of Engineering Science University of Oxford Ben Benfold Ian Reid CVPR 2011

2. OUTLINE • Introduction • Sliding Window Tracking • Observations • Data Association • Output Generation • Evaluation • Conclusions

3. OUTLINE • Introduction • Sliding Window Tracking • Observations • Data Association • Output Generation • Evaluation • Conclusions

4. Introduction • Tradition method vs Paper method • Maintain targets vs Stable location estimates • ad-HOG vs MDL • Approach • Multi-threaded • Combines asynchronous HOG detection • Simultaneous KLT tracking • MCMCDA

5. Introduction • Recent Work • Feed-forward systems which use only current and past observations to estimate the current state • Data association based methods which also use future information to estimate the current state

6. Introduction • MCMC • Tracking a single or fixed number of targets • Multi-target tracking • MCMCDA tracking systems • Associating object detections resulting from background subtraction • A boosted Haar classifier cascade • Object detections and Motion estimations

7. Introduction • Main contribution • The development of a tracking model • Treatment of false positive detections • A comprehensive evaluation of the tracker on multiple datasets using the standard CLEAR MOT evaluation criteria

8. OUTLINE • Introduction • Sliding Window Tracking • Observations • Data Association • Output Generation • Evaluation • Conclusions

9. Observation • Make the tracking algorithm robust • the most recent six seconds of video • Object detections • HOG • Trained a detector using head images • Interval • 200 milliseconds for PAL video • 1200 milliseconds for 1080p video

10. Observation • Motion estimates • Pyramidal Kanade-Lucas-Tomasi(KLT) tracking

11. Observation • Pyramidal KLT tracking • Provide robustness • Up to four corner features • Tracked both forwards and backwards • in time from detections for up to s seconds • S = 4 sec • more precise than mean-shift

12. OUTLINE • Introduction • Sliding Window Tracking • Observations • Data Association • Output Generation • Evaluation • Conclusions

13. Data Association • Hypothesis Hi • Divides the set of detections D • Disjoint subsets T1, T2 . . . TJ • Tjcorresponding to a single person • Not every detection that occurs is a true positive • ,represent Tjbeing a genuine pedestrian track • ,if we believe Tj is a track of false positives

14. Data Association • MCMC sampling to efficiently explore the space of data associations by generating a sequence H0,H1,H2, . . .

15. Data Association

16. Data Association • Likelihood Function p(Hί) • conditional probability function • An approach based on the principles of MDL

17. Data Association • 1 • Gibbs sampling • is a prior over the different track types • be the n-th detection in a track Tj

18. Data Association • Sn : the scale of the detection • Xn: the location within the frame • ₥n: an approximation to the KLT motion

19. Data Association • Detection Scales • Initial Step: • Iteration Step:

20. Data Association • Image Location • Initial Step: • assumed that the locations of both pedestrians and false-positives are uniformly distributed around the image • the probability density of xndepends on the image area a relative to the object size in pixels

21. Data Association • Image Location • First make a prediction based on a constant velocity model • the error p is still large partly due to the cyclic gait motion • humans often change direction when in crowds

22. Data Association • Image Location • The full KLT motion estimates • Kalman filter

23. Data Association • Image Location • α : The possibility that a tracked KLT feature fails completely • : The possibility of failure after for seconds • A mixture of the prior and posterior distributions

24. Data Association • Motion Magnitude • The last observation considered • the motion magnitude histogram • distinguish between false positives and true positives • Bins with boundaries • 1/8, 1/4 , 1/2 pixels per frame

25. Data Association • Sampling • Three types of move which can be made during the sampling process • the first two moves effect the state of the DA • the third has the potential to change the type of a track • First : Swap • Second : Switch • Third : M-H acceptance function

26. Data Association • First , Second

27. Data Association • Third • Although Metropolis-Hastings is good • We prefer stable output rather than samples • Keep track of the most likely hypothesis • Output the local maximum

28. Data Association • Parameter Estimation • Learned automatically • based on that of Ge and Collins • Interleaving the MCMCDA sampling with additional Metropolis-Hastings updates of the parameters • Provided the parameters are initialised • allowing some tracks to be correctly associated • converge to a maximum of the likelihood function

29. Data Association • Parameter Estimation • Longer than data association • over an hour or two • most datasets are too short for this • slow down the video used for training

30. OUTLINE • Introduction • Sliding Window Tracking • Observations • Data Association • Output Generation • Evaluation • Conclusions

31. Output Generation • The final stage • generate estimates for the object location in each frame • stimate the true image locations • Detection do not occur in every frame

32. OUTLINE • Introduction • Sliding Window Tracking • Observations • Data Association • Output Generation • Evaluation • Conclusions

33. Evaluation • The Multiple Object Tracking Precision (MOTP) • objects are located using the intersection of the estimated region with the ground truth region • The Multiple Object Tracking Accuracy (MOTA) • takes into account false positives,falsenegatives and identity switches

34. Evaluation

35. Evaluation

36. Evaluation

37. Evaluation

38. OUTLINE • Introduction • Sliding Window Tracking • Observations • Data Association • Output Generation • Evaluation • Conclusions

39. Conclusion • Described and demonstrated a scalable real-time system • The use of MCMCDA makes the system robust • Our efficient approach provides general tracking performance comparable to that of similar systems

40. Thanks For Your Listening