Emerging Topics in Video Surveillance

1. Emerging Topics in Video Surveillance Rogerio Feris IBM TJ Watson Research Center rsferis@us.ibm.com http://rogerioferis.com

2. Outline • Video Surveillance in Crowded Scenarios • Online Learning – Self-adaptation in Surveillance • Other Recent Topics

3. Simple Scenarios • Few Objects – Background Subtraction + Tracking + High-level Event/Alert Detection • Current systems work well

4. Crowded Scenarios • Many objects, occlusions, shadows, etc. Object Segmentation, Tracking and Event Analysis in crowded scenarios: Open Problem!

5. Parts-based Detectors [Pedro et al, A discriminatively trained, multiscale, deformable part model, CVPR’08] • Occlusion Handling • Root filter (low-res) + Parts filters (high-res)

6. Parts-based Detectors • Score of a window: score of root + score of parts • Score of Parts: Appearance + Geometry • Efficient localization of parts through Dynamic Programming • SVM Classification (Structured prediction)

7. Detecting Pedestrians in Crowds [Leibe et al, Pedestrian Detection in Crowded Scenes, CVPR’05] • Combination of different models: bag of features, segmentation, and chamfer matching

8. Tracking in Crowds [Andriluka et al, People-tracking-by-detection and people-detection-by-tracking, CVPR’08] • Extends [Leibe et al, CVPR’05] to temporal-domain and person articulation (parts) estimation • Click for Video Demo

9. Crowd Segmentation [Dong et al, Fast Crowd Segmentation Using Shape Indexing, ICCV’07]

10. Crowd Analysis [Ali & Shah, A Lagrangian Particle Dynamics Approach for Crowd Flow Segmentation and Stability Analysis, CVPR’07]

11. Online Learning

12. Offline Adaboost Learning • Adaboost ensembles many weak classifiers into one single strong classifier • Initialize sample weights • For each cycle: • Find a classifier/rectangle feature that performs well on the weighted samples • Increase weights of misclassified examples • Return a weighted combination of classifiers

13. Offline Adaboost Learning Major Problems: • Large number of examples required to train a robust classifier • time consuming to label data • slow training (may take several days) • No Adaptation to particular surveillance scenarios

14. Learning from Small Sets • Choice of Features (Levi & Weiss, CVPR’04) • Co-Training (Levin & Viola, ICCV’2003) Online Adaptation: • Online Boosting (Oza’01, Javed’05, Bischof’06, Pham’07)

15. Online Boosting [Oza,2001] • Train a generic strong classifier (set of weak classifiers, # of weak classifiers fixed) on a small training set. • Online Process: • Given one single example with known label: • “Slide” the example over each weak classifier • When the weak classifier receives the example • update the weak classifier online • update the weight of the example and pass to the next weak classifier

16. Online Boosting [Oza,2001]

17. Online Boosting [Oza,2001]

18. Online Boosting Car and People Detection [Omar Javed, CVPR’05] • Train a generic strong classifier (set of weak classifiers, # of weak classifiers fixed) on a small training set. • While running the classifier on unlabeled data, if an example is confidently predicted by a subset of weak classifiers  use it for online learning • “Co-training framework” • BGS used for efficiency, for using more expensive features, and for balancing the number of positive and negative examples

19. Online Boosting Car and People Detection [Omar Javed, CVPR’05]

20. More Recent Work • [Helmut & Hurst, Online Boosting and Vision, CVPR’06] • [Bo Wu & Nevatia, Improving Part-based Object Detection by Unsupervised, Online Boosting, CVPR’07] • [Pham & Cham, Online Learning Asymmetric Boosted Classifiers for Object Detection, CVPR’07] • [Huang et al.,Incremental Learning of Boosted Face Detector, ICCV’07] – Boosting Adaptation • IEEE Online Learning for Classification Workshop (CVPR’08)

21. Other Recent Topics

22. High-Resolution Imagery [Kopf et al, Capturing and Viewing Gigapixel Images, SIGGRAPH’07] • How can we make use of high-resolution in video analytics? • Much more info – e.g., in face reco: skin texture, iris, etc.

23. Next Generation Neural Networks [Hinton, Reducing the dimensionality of data with neural networks, Science 2006] • New algorithm for learning deep belief nets • State-of-the art results in MNIST digit dataset (better than SVMs) • Youtube talk at Google: http://www.youtube.com/watch?v=AyzOUbkUf3M • Matlab Code: http://www.cs.toronto.edu/~hinton/

24. Learning with lots of data • How can we recognize thousands of products in a retail store for loss prevention? • 80 Million Tiny Images (http://www.cs.nyu.edu/~fergus/) Surveillance with Moving Cameras • Cameras in vehicles, or even wearable cameras. New challenges: object detection, etc. • [Leibe et al, Dynamic 3D Scene Analysis from a Moving Vehicle, CVPR 2007]

25. Many more recent topics: • Check for papers in recent computer vision conferences (like CVPR, ICCV, and ECCV) and also specialized workshops/conferences such as AVSS and PETS