Strategic Research Plan for Computer Vision and Market Opportunities at NICTA

1. A research strategic plan for computer vision at NICTA

2. Market Drivers • Safety and Security: • SAFE: est video surveillance software in US – $1.8-4 billion • Intelligent Transportation Systems: • STARSense: • SCATS installed in 1300 cities in 20 countries • More than 9 million vehicle trips daily in Sydney Urban corridor • Aiming for 5% improvement in traffic flow • Dramatic decrease in cost of traffic sensing deployment • Automap • Map development cost for TeleAtlas and Navteq est > US$1 billion • Potential price for fresh data for Aust capitals is $10m p/a

3. Market Drivers (cont) • Biomedical and Life Sciences • Bionic eye and low vision devices: • 3,300-4,000 people / million legally blind (AMD, retinopathy, cataract) • 480,000 people in Australia with visual impairment • Cost of cochlea implant in US > US$60,000 • Environmental Management • GREEnMan: • Agricultural industries in Australia approximately 135,000 farms • 65,000 crop or horticultural • Plant industries value $15 billion at farm gate – contribution to the economy of $10 billion

4. Problems

5. Problems

6. Problems

7. Problems

8. Research Approach • Fusing key theory from • Multi-view Geometry • statistical pattern recognition • Into key algorithms such as: • Detection • Tracking • Recognition • To introduce novel theory driven approaches, that provide robust solutions to complex dynamic scene analysis • Delivered as a NICTA-wide platform for computer vision

9. Approaches – Building in geometry: Night-time Traffic Surveillance • Objectives • Count the numbers of vehicles for each lane • Estimate the speed of the vehicles • Classify the vehicles as car / truck or bus • Detect the changes of lane • Make the installation easy, automatic • Avoid the manual thresholds (scene dependent)

10. Approaches – Bionic eye – a running example Wireless link Implant Camera+processor

11. Approaches – Bionic eye – a running example VIsion processing for the Bionic Eye Dynamic scene understanding: Real-time structure and motion recovery object identification Signals on electrodes Reprojection to RGC array

12. Approaches – Bionic eye (cont) • Based on visual processing, context, motion and user feedback, we may choose what information to present at each retinal ganglion cell in each cycle.

13. Approaches – Bionic eye (cont) • Centre for Eye Research Australia clinicians running focus groups with low vision subjects (Results June) • Identified ‘friend recogniser’ as possible demonstrator • From wearable computing • Identify 20 close ‘friends’ in close proximity • Output via audio • Problem: from wearable cameras/computers • Pedestrian detection/tracking • Face detection/tracking • Face recognition from small database

14. Approaches – Hardware enhanced detection Towards Safer Roads by Integration of Road Scene Monitoring and Vehicle Control, L Petersson, L Fletcher, et. al. The International Journal of Robotics Research, 2006

15. Approaches – pedestrian detection Sakrapee Paisitkriangkrai, Chunhua Shen, and Jian Zhang. An experimental evaluation of local features for pedestrian classification in DICTA'07, Dec 2007. IEEE Computer Society [Best Paper Award].

16. Approaches – fusing statistical approaches to tracking Kernel-based tracking from a probabilistic viewpoint, Quang Nguyen, Antonio Robles-Kelly, Chunhua Shen, IEEE CVPR'07. Minnesota, USA, June, 2007.

17. Face detection and recognition

18. Approaches – Multi-body multi-motion recovery Hongdong Li, Two-view Motion Segmentation from Linear Programming Relaxation,  in Proceedings of CVPR, 2007

19. Approaches – Spherical optical flow time to collision – real-time motion understanding C.MacCarthy, N. Barnes and R. Mahony, “A Robust Docking Strategy for a Mobile Robot using Flow Field Divergence”, IEEE Trans Robotics, in press 2008. C. McMcarthy, N. Barnes and M. Srinivasan"Real Time Biologically-Inspired Depth Maps from Spherical Flow", Proc. IEEE ICRA2007, Rome, Italy, May, 2007.

20. Approaches – Spherical egomotion recovery – statistical Geometry, for robust parallel implementation John Lim and Nick Barnes, “Directions of Egomotion from Antipodal Points”, accepted, IEEE-CVPR, 2008

21. Approaches and complexities

22. NICTA-wide computer vision platform • Parallel hardware: • Nvidia CUDA GPU • Blackfin DSP • Spartan 3 FPGA • Regular Intel SIMD multicore • Software • implementations on NICTA Intel PXA270 Microprocessor-based embedded platform

23. NICTA-wide platform project

24. Projects to be proposed • VIBE (Vision Processing for the Bionic Eye) • Vision platform: • Dynamic Scene understanding • Parallel embedded platform QRL + CRL • Serial embedded platform • Code contributions from all Labs • Focus on Detection/Tracking/Recognition • GREEnMan

25. Why CV@NICTA is the best group to do this • One of the strongest CV research groups in the world in terms of A+ and A publications at recent CV conferences (see appendices) • One of the strongest CV research groups in the world in terms of citations (see appendices) • One of the strongest groups in real-time structure from motion (see appendices) • Dominant world group in Multi-body structure from motion • Strong mix across research approaches in CV and pattern recognition • World-class parallel real-time implementations (PMS) • World-class traffic surveillance systems • Outstanding linkages in safety and security

26. Contributors and acknowledgements • Contributors • Nick Barnes, Abbas, Bigdeli, Terry Caelli, Richard Hartley, Bernhard Hengst, Brian Lovell, Chris, Nicol, John Parker, Lars Petersson, Antonio Robles-Kelly, Jian Zhang • Discussions with the following people greatly contributed to this plan: • Emma Barron, David Everitt, Terry Percival, Phil Robertson, David Skellern, Chris Scott, Bob Williamson