USAR Challenge

1. Virtual Robots RoboCupRescue Competition: Contributions to Infrastructure and Science

2. USAR Challenge

3. Rapid Advancement in USAR2001-present Simulation League: communication models to cooperation & learning [Kitano and Tadokoro, 2001] H. Kitano and S. Tadokoro, Robocup rescue: A grand challenge for multiagent and intelligent systems. AI Magazine, 22(1):39–52, 2001. Robot Rescue League:video driven teleoperation to 3D scanning & autonomous exploration [Jacoff et al., 2001] A. Jacoff, E. Messina, J. Evans, Experiences in deploying test arenas for autonomous mobile robots, Proceedings of the 2001 Performance Metrics for Intelligent Systems (PerMIS) Workshop, Mexico City, Mexico, 2001.

5. Our first team Pioneer Tarantula PERs

6. 2004 USAR winners in Lisbon

7. Mobility comes to dominate Rescue Robot competition by 2005 INSERT VIDEO HERE RAPTOR, CARNEGIE MELLON and UNIV. OF PITTSBURGH, USA RED ARENA with Random Step Fields and other difficult mobility Obstacles is for very agile robots, all control modes are allowed.

8. Virtual Robots as a Bridge VR  Physical League • Continually improving simulation quality and validation VR Simulation League • Expanding team size & problem complexity

9. USARsim Architecture Simulation Desiderata • Expense and availability of simulation hardware and software to USAR robotics community • Ease of programming to reflect targeted aspects of design • Fidelity of simulation w.r.t. aspects of design to be tested

10. USARsim Architecture SimulationRequirements Video feedfor teleoperation and visual search and identification Sensor simulation- for autonomous control and fused displays Simulated robot dynamics- for teleoperation and autonomous control Multiple entity simulation- to allow interaction and cooperation among teams of robots

11. Image server USARsim Architecture The image server captures images from video memory so they can be subjected to visual processing just like input from a real camera. COTS game engine supplies best available graphics & physics engines Standard tools like 3D studio max or Maya are available Robots are controlled and sensor data gathered from sockets into the game

12. Brief history 2003 • Developed USARsim simulation • Limited to our own robots • Limited to our own (RETSINA) control architecture • Demo’d • USAR workshop at USF • US Open RoboCup 2003 2002

13. Brief history 2004 • Extended simulator for general access & added features such as sensor models & image server needed for research • Modeled robots commonly used robots • Made control architecture agnostic • Added plug-in/API for popular • middleware • Player/(Stage) • Pyro • Presented to USAR participants at • Robocup 2004 in Lisbon 2002 2003 2004

14. Brief history 2005 Demo approved at Robocup Rescue Camp in Rome Rule: robots must model real robots being used by team in USAR 6 teams from 4 countries participated in demo competition at Robocup in Osaka University of Rome, International University of Bremen, University of Osnabruck, University of Freiburg, Meijo University, University of Pittsburgh Virtual Robots USAR competition approved to become new competition within RobocupRescue League start for RoboCup 2006 in Bremen June 14-20 USARSim moved to Source Forge 2003 2002 2005 2004

15. USARSim Aibo model presented by Marco Zaratti at 2005 RoboCup Symposium

16. Brief history 2006 USARSim Units regularlized by NIST Mission Package designed to accommodate extensions to simulation First RoboCup Rescue VR competition held in Bremen8 teams from 6 countries 1st Freiburg, 2nd I U Bremen, 3rd Amsterdam 2003 2002 2004 2005 2006 2007

17. 2006 Competition World

18. Brief history 2007 Operator penalty repealed (as in RR league) Communications server added Second RoboCup Rescue VR competition held in Atlanta8 teams from 5 countries 1st Pitt/CMU, 2nd Jacobs, 3rd Rome Continuing work in validation and new platforms 2003 2002 2004 2005 2006 2007 2008

19. Brief history 2008 Third RoboCup Rescue VR competition held in Sizhou, China 10 teams from 8 countries UAVs added 1st SEU, 2nd UC Merced, 3rd CMU/Pitt German Open 3 teams, Iranian Open 4 teams 2003 2002 2004 2005 2006 2007 2008

20. Brief history 2009 Fourth RoboCup Rescue VR competition held in Graz, Austria11 teams from 8 countries 1st UC Merced, 2nd SEU, 3rd Amsterdam-Oxford German Open 3 teams, Iranian Open 4 teams Continuing work in validation and new platforms 2003 2002 2004 2005 2006 2007 2008

21. USARSim – Sensors

22. KRobot Legged Robot Ground Vehicle Aerial Vehicle Nautic Vehicle Skid Steered Robot Ackerman Steered Robot Rotary Wing Robot Underwater Robot AIBO P2AT Kurt2D AirRobot Submarine QRIO P2DX Kurt3D Cooper Hummer Sedan SnowStorm Zerg ATRVJr Lisa Talon Soryu TeleMax USARSim – Robots Jointefforts

23. 11 USARSim Validation Studies • Synthetic video • Carpin, S., Stoyanov, T., Nevatia, Y., Lewis, M. and Wang, J. (2006a). Quantitative assessments of USARSim accuracy". Proceedings of PerMIS 2006 • Hokuyo laser range finder • Carpin, S., Wang, J., Lewis, M., Birk, A., and Jacoff, A. (2005). High fidelity tools for rescue robotics: Results and perspectives, Robocup 2005 Symposium. • Platform physics & behavior • Sven Albrecht, Joachim Hertzberg, Kai Lingemann, Andreas N¨uchter, Jochen Sprickerhof, Stefan Stiene (2006). Device Level Simulation of Kurt3D Rescue Robots, Third International Workshop on Synthetic Simulation and Robotics to Mitigate Earthquake Disaster, 2006. • Carpin, S., Lewis, M., Wang, J., Balakirsky, S. and Scrapper, C. (2006b). Bridging the gap between simulation and reality in urban search and rescue. Robocup 2006: Robot Soccer World Cup X, Springer, Lecture Notes in Artificial Intelligence • Nicola Greggio, Gianluca Silvestri, Emanuele Menegatti, Enrico Pagello (2007). A realistic simulation of a humanoid robot in USARSim, Proceeding of the 4th International Symposium on Mechatronics and its Applications (ISMA07) , 2007 • S. Okamoto, A. Jacoff, S. Balakirsky, and S. Tadokoro (2007). Qualitative validation of a serpentine robot in USARSim Proceedings of the 2007 JSME Conference on Robotics and Mechatronics, 2007. • Okamoto, S.   Kurose, K.   Saga, S.   Ohno, K.   Tadokoro, S.   Validation of Simulated Robots with Realistically Modeled Dimensions and Mass in USARSim, IEEE International Workshop on Safety, Security and Rescue Robotics, 2008. (SSRR 2008), 77-82, 2008. • Lewis, M., Hughes, S., Wang, J., Koes, M. and Carpin, S., Validating USARsim for use in HRI research, Proceedings of the 49th Annual Meeting of the Human Factors and Ergonomics Society, Orlando, FL, 457-461, 2005. • Pepper, C., Balakirsky, S. and Scrapper, C., Robot Simulation Physics Validation, Proceedings of PerMIS’07, 2007. • Taylor, B., Balakirsky, S., Messina, E. and Quinn, R., Design and Validation of a Whegs Robot in USARSim, Proceedings of PerMIS’07. • Zaratti, M., Fratarcangeli, M., and Iocchi, L., A 3D Simulator of Multiple Legged Robots based on USARSim. Robocup 2006: Robot Soccer World Cup X, Springer, LNAI, 2006. www.sourceforge.net/project/usarsim

24. Validation: simulation & real P3-AT run from same input

25. USARSim Downloads

26. Contributions to Scientific Infrastructure • Competition provided critical mass of users to benefit from network externalities • Association with competition provided justification for NIST development & support • Involving more parties led to greater standardization & more general utility

27. Reported Studies Using USARSim • 14 Human-Robot Interaction studies-9 groups • Dialog management – 2 groups • Machine learning- 2 • Testing control algorithms • Driving behavior- 2 groups • Social interaction • Service composition for robots • Self diagnosis

28. Theses & Projects

29. Project Infrastructure • Developed under NSF ITR • Used in MURIs • CMU • Berkeley • MIT • ONR Science of Autonomy • DARPA SyNAPSE

30. Multi-Robot Mapping & Evaluating Map Quality • Direct contribution of competition • Upcoming Special issue of Autonomous Robots • special sessions on mapping and map quality at PerMIS’08 and RSS’08 workshops • Other venues Luca Iocchi and Stefano Pellegrini (2007). Building 3D maps with semantic elements integrating 2D laser, stereo vision and IMU on a mobile robot, Proceedings of the 2nd ISPRS International Workshop on 3D-ARCH, 2007. Max Pfingsthorn, Bayu Slamet and Arnoud Visser,(2007). A Scalable Hybrid Multi-robot SLAM Method for Highly Detailed Maps, Lecture Notes in Computer Science, RoboCup 2007: Robot Soccer World Cup XI, 385-392, 2008. V. Sakenas, O.  Kosuchinas, M. Pfingsthorn,  A. Birk,(2007).  Extraction of Semantic Floor Plans from 3D Point Cloud Maps, IEEE International Workshop on Safety, Security and Rescue Robotics, 2007. SSRR 2007, 1-6, 2007. D. Sun, A. Kleiner, and T. M. Wendt (2008). "Multi-Robot Range-Only SLAM by Active Sensor Nodes for Urban Search and Rescue", in In Robocup 2008: Robot Soccer World Cup XII, 2008. I. Varsadan, A. Birk, and M. Pfingsthorn (2008). "Determining Map Quality through an Image Similarity Metric", Proceedings CD of the 12th RoboCup International Symposium, Suzhou, China.

31. Elemental Tests Because contests reward composite performance they tend to promote teams with the strongest “weakest link” rather than promoting the strongest solutions. Solutions: • Sharing winning code (Agent simulation & VR) • Elemental tests as part of competition

32. Competition updates 2009 • Preliminary rounds based on automatically scored elemental tests • Rationale: • Identify “best in class” abilities • Push teams to attack new challenges • Move towards objectively measurable performance metrics

33. First elemental test • Mapping • Reward the ability to produce a map that allows a first responder to reach a set of random points in the disaster scenario • Ignore metric quality, but focus on topological utility • Automatically scored

34. Second elemental test • Radio network deployment challenge • Reward teams able to identify deployment points yielding the maximum coverage for a given environment • A priori data partially wrong • Reward planning and the ability to navigate to target points • Automatically scored (score is the covered area) • Fully autonomous challenge • Uses a newly developed Wireless simulator taking into account walls, attenuation, etc..

35. Third elemental test • Teleoperation • Reward teams able to develop an HRI where a single operator can drive a team of robots to a set of goal locations • Automatically scored • Very different target locations impose the use of heterogeneous robot teams (flying, wheeled, tracked) • Semiautonomous test

36. Next Challenge • Can contest & simulator survive change in platform? • UE2 engine cannot support large numbers of robots (~8) with high fidelity • UE2 engine cannot support physics intensive dynamics such as tracks • Moving to UE3 requires re-doing most of the infrastructure

37. Performance for tracked robots Modeling something with many constraints such as tracks is extremely difficult. In the case of this Tarantula, for example, simplifying tracks to 5 wheels/flipper yields: 20 x 5 + 4x6 = 124 constrained dof and is just about at the limit of the Karma engine. This simplification of a tracked robot is about 5 times as costly to simulate as a 4 wheeled platform.