Detection & Identification of People & Animals by Exploration Robots

1. Craig Hahn April 13, 2009 Detection & Identification of People & Animals by Exploration Robots

2. Importance • robots must interact safely with humans and animals • Image: http://depletedcranium.com/riman

3. Importance • many “exploration” applications involve close interaction with humans (such as search-and-rescue operations) • people and animals are often the most dynamic part of a real-world environment • humans and animals must be identified quickly in all such situations

4. Basic Methods • Stationary face detection using basic pattern recognition techniques • Image: http://www.motorauthority.com/wpcontent/uploads/odds_and_ends/2008/3/2/face_pic.jpg

5. Facial Recognition • Easily done but of little use to mobile robots: • robot and target likely moving • faces are not always visible • Image: http://www-robotics.jpl.nasa.gov/roboticImages/img840-192-browse.jpg

6. Basic Methods • Tracking using stereo vision cameras • problem: • distinguishing people from their surrounding environment, especially moving people

7. Basic Methods • Use laser rangefinders to detect larger moving objects (not just looking at people’s faces) • But again you can’t always distinguish people from their surrounding environment using only laser rangefinders • possible problems: • people too close to other objects in environment (walls) • other objects (tables, chairs, etc.) mistaken for people

8. Better Idea • Combine data from multiple sources • optical sensors • laser rangfinders • auditory systems • cameras • pattern recognition • sonar

9. Kyoto University • Created a robot that combines facial recognition and auditory data • detect people by looking for faces • track them using stereo vision and sound source detection

10. Autonomous Systems Lab Swiss Federal Institute of Technology Lausanne • Mobile robot Robox finds (moving) objects by detecting changes in a static environment with a laser rangefinder...

11. Robox • …then checks such objects using visual cues to determine if they are people

12. Department of Neuroinformatics and Cognitive Robotics Ilmenau Technical University • HOROS • laser-rangefinder • sonar system • omnidirectional camera

13. sensory input from each sensing system

14. - assigns each set of data a Gaussian probability distribution(,C) - mean  represents the position of the detection - covariance matrix C represents the uncertainty about the position

15. reasonably successful in a real-world environment

16. Advantages • HOROS doesn’t rely on just one sensing system • easy to add more sensing systems • just factor in another Gaussian distribution

17. Centre for Applied Autonomous Sensor Systems Örebro University • Mobile robot identifies people in an indoor environment • laser rangefinder • camera • Learns a color model of a person • hair • skin • clothes • shoes

18. laser rangefinder scans the environment • detects and tracks people • local minima correspond to the legs of a person

19. image from camera is broken down into three parts

20. color data is extracted from each part of the image • hue-saturation-value color model • color data is put into self-organizing maps (SOMs) • color model of a person is learned based on the best fit

21. Assignment • Come up with another robotic system that combines three or more different sensing methods to identify humans or animals, and tell the difference between a human and a humanoid robot • explain: • why you chose the methods that you did • how each method can be used to identify humans or animals and differentiate humans from humanoid robots • how the data from each can be integrated together in a meaningful way • if applicable, identify specific equipment that could be used to implement your system

22. Supply Companies • SICK laser rangefinders: • Image: http://www.pages.drexel.edu/~kws23/tutorials/sick/sickLMS200.jpg • widely used

23. Omnidirectional Cameras • Nikon • Sony • Remote Reality • Image: http://blog.wired.com/photos/uncategorized/2008/07/01/hal9000_focus_jpg.jpg

24. Other Important Companies • JAXY Optical Instrument Company (laser radar systems, sonar systems • Opnext, Inc. (laser rangefinders) • Instruments, Inc. (amplifiers, sonar systems, range sensors)

25. Cutting-Edge Researchers • Ilmenau Technical University Department of Neuroinformatics and Cognitive Robotics • Örebro University Dept. of Technology, Centre for Applied Autonomous Sensor Systems • Kyoto University Department of Intelligence Science and Technology • Swiss Federal Institute of Technology LausanneAutonomous Systems Lab • Albert-Ludwigs-Universität Freiburg Social Robotics Laboratory, Institut für Informatik

26. Future Improvements • Improvements are expected in: • identifying more dynamic objects (people and animals) • differentiating between multiple objects of the same type • differentiating between many different kinds of moving people and objects • being worked on at Albert-Ludwigs-University in Freiburg, Germany

27. Other Improvements • identifying and classifying the behaviors of humans and animals • same accuracy of prediction and detection with: • less data • smaller periods of observation • can reach conclusions more quickly (less computational time)

28. References • Sensor Fusion Using a Probabilistic Aggregation Scheme For People Detection and Tracking Department of Neuroinformatics and Cognitive Robotics Ilmenau Technical University <http://www.tu-ilmenau.de/fakia/fileadmin/template/startIA/neuroinformatik/publications/conferences_int/Martin-ECMR-05b.pdf> • Person Identification by Mobile Robots in Indoor Environments Centre for Applied Autonomous Sensor Systems Dept. of Technology, O¨ rebro University <ftp://aass.oru.se/pub/tdt/rose03.pdf> • Unsupervised Learning and Classification of Dynamic Objects Social Robotics Laboratory Institut für Informatik Albert-Ludwigs-Universität Freiburg <http://srl.informatik.uni-freiburg.de/papers/luberRSS08.pdf>

29. References • Auditory fovea based speech separation and its application to dialog system Kyoto University, Department of Intelligence Science and Technology <http://thamakau.usc.edu/Proceedings/ICSLP%202002/ICSLP/PDF/AUTHOR/SL021615.PDF> • Robox at expo.02: A large scale installation of personal robots Swiss Federal Institute of Technology Lausanne,Autonomous Systems Lab <http://infoscience.epfl.ch/record/97516>