1 / 30

How to make working vision-based interaction systems

How to make working vision-based interaction systems. Zoran Zivkovic ISLA LAB University of Amsterdam. Overview. Sensors (geometry + other properties) Theory (computer vision + machine learning) Building and testing the system (user studies). Sensors. Camera calibration

alton
Download Presentation

How to make working vision-based interaction systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. How to make working vision-based interaction systems Zoran Zivkovic ISLA LAB University of Amsterdam

  2. Overview • Sensors (geometry + other properties) • Theory (computer vision + machine learning) • Building and testing the system (user studies)

  3. Sensors • Camera calibration • First time in my MSc thesis • Calibrated camera often used in my further work E.g. used in: "A stabilized adaptive appearance changes model for 3D head tracking"Z.Zivkovic, F.van der Heijden2nd IEEE International Workshop on Recognition, Analysis and Tracking of Faces and Gestures in Real-time Systems (ICCV2001), Vancouver, Canada, July 2001

  4. Sensors • Camera calibration • Building and calibrating an omnidirectional camera "How did we built our hyperbolic mirror omnidirectional camera - practical issues and basic geometry"Zivkovic Z., Booij O.Technical Report IAS-UVA-05-04, Informatics Institute, University of Amsterdam, 2005.

  5. Sensors • Building the whole sensory system (Cogniron project) E.g. used in: "Towards foveated part based people detection"Z. Zivkovic and B. KröseIEEE International Conference on Computer Vision Systems , 2007.

  6. Overview • Sensors (geometry + other properties) • camera calibration, multi view geometry • Theory (computer vision + machine learning) • Building and testing the system (user studies)

  7. Theory • EM using a prior approximating MML for selecting number of mixture components on-line "Recursive unsupervised learning of finite mixture models "Z.Zivkovic, F.van der Heijden IEEE Trans. on Pattern Analysis and Machine Intelligence, vol.26, no.5, pages 651-656, 2004

  8. Theory • EM using a prior approximating MML for selecting number of mixture components on-line "Efficient adaptive density estimation per image pixel for the task of background subtraction"Z. Zivkovic , F. van der HeijdenPattern Recognition Letters, vol. 27, no. 7, pages 773-780, 2006.

  9. Theory-layered image model • Generative model for binary images and probability maps using Bernoulli distributions (inspired by Fray and Jojic and, NIPS 1999) "Transformation invariant component analysis for binary images"Z. Zivkovic and J.J. VerbeekIEEE Conference on Computer Vision and Pattern Recognition, pages 254-259, 2006.

  10. Theory-layered image model • Generative model for multiple overlapping objects (similar to Jojic and Fray, CVPR 1999) Submitted to BMVC 2007

  11. Theory - part based model • History: • Fischler & Elschlager 1973 • Yuille ‘91 • Brunelli & Poggio ‘93 • Lades, v.d. Malsburg et al. ‘93 • Cootes, Lanitis, Taylor et al. ‘95 • Amit & Geman ‘95, ‘99 • Perona et al. ‘95, ‘96, ’98, ’00,’03 • Felzenszwalb & Huttenlocher ’00 • … Figure from [Fischler73]

  12. Theory - part based model Number of missing parts Number of background parts Observed parts Missing parts Background parts Unknown part assignment Gauss Uniform or wide Gauss learned from data Probability table Uniform Poisson Inspired by Perona et al. ‘95, ‘96, ’98, ’00,’03

  13. Theory - part based model Inspired by Perona et al. ‘95, ‘96, ’98, ’00,’03

  14. Theory - part based model + geometry • Detected parts (no plane constraint) • Prune detected parts

  15. Theory - part based model + geometry • Pruned detected parts • Ground plane positions • Detect person using the model

  16. Theory - part based model + geometry

  17. Theory - part based model + geometry

  18. Experiments - using multiple sensors

  19. Experiments - using multiple sensors • Image: • Laser:

  20. Overview • Sensors (geometry + other properties) • camera calibration, multi view geometry • Theory (computer vision + machine learning) • part based model, layered image representation • EM (HMM), approximate inference – ADF, variational methods • Building and testing the system (user studies)

  21. Building and testing systems • Face tracking demo • Shown at ICCV, and many national conferences • 25 frames/s on 500MHz Pentium2 "A stabilized adaptive appearance changes model for 3D head tracking"Z.Zivkovic, F.van der Heijden2nd IEEE International Workshop on Recognition, Analysis and Tracking of Faces and Gestures in Real-time Systems (ICCV2001), Vancouver, Canada, July 2001

  22. Building and testing systems • Recognising tennis strokes • within DMW project "Recognizing Strokes in Tennis Videos Using Hidden Markov Models"M.Petkovic, Z.Zivkovic, W.JonkerIASTED International Conference Visualization, Imaging and Image Processing, Marbella, Spain, September 2001

  23. Building and testing systems • Traffic monitoring • demo with The Dutch Ministry of Transport "Two video analysis applications using background/foreground segmentation"Z.Zivkovic, M.Petkovic, R.van Mierlo, M.van Keulen, F.van der Heijden, W.Jonker, E. Rijnierse IEE Visual Information Engineering Conference, UK, 2003

  24. Building and testing systems – home robot (EU project-Cogniron) • Localization, navigation, people following Some publications: -Robotics and Autonomous Systems Journal Special Issue: From Sensors to Human Spatial ConceptsEditors: Z.Zivkovic, B. Krose, vol.55, no.5,2007 -”Navigation Using an Appearance Based Topological Map"O. Booij, B. Terwijn, Z. Zivkovic, B. KroseIEEE International Conference on Robotics and Automation, 2007. -"Keeping track of humans: have I seen this person before?"W.Zajdel, Z.Zivkovic and B.Krose IEEE Int. Conf. on Robotics and Automation 2005, Barcelona, Spain,2005 -"Hierarchical Map Building Using Visual Landmarks and Geometric Constraints"Z.Zivkovic, B. Bakker and B. KröseIEEE/RSJ International Conference on Intelligent Robots and Systems, 2005.

  25. Building and testing systems – home robot (EU project-Cogniron) • Localization • Graph based space representation • Local 3D reconstruction -"Hierarchical Map Building Using Visual Landmarks and Geometric Constraints"Z.Zivkovic, B. Bakker and B. KröseIEEE/RSJ International Conference on Intelligent Robots and Systems, 2005.

  26. Building and testing systems – home robot (EU project-Cogniron) • Navigation -”Navigation Using an Appearance Based Topological Map"O. Booij, B. Terwijn, Z. Zivkovic, B. KroseIEEE International Conference on Robotics and Automation, 2007.

  27. Building and testing systems – webcam games • www.webcamgames.net • Shown at University Twente and University of Amsterdam open days,… "Optical-flow-driven Gadgets for Gaming User Interface"Z.Zivkovic International Conference on Entertainment Computing, 2004

  28. Building and testing systems – webcam games • User study – using optical flow to detect movements "Optical-flow-driven Gadgets for Gaming User Interface"Z.Zivkovic International Conference on Entertainment Computing, 2004

  29. Overview • Sensors (geometry + other properties) • camera calibration, multi view geometry • Theory (computer vision + machine learning) • part based model, layered image representation • EM (HMM), approximate inference – ADF, variational methods • Building and testing the system (user studies) • demos + user studies (face tracking, tennis analysis, traffic monitoring, home robot, webcam games)

More Related