Discovery, localization, and recognition of smart objects by a mobile robot

1. UNIVERSITY OF PADUA Dept. of Information Engineering Discovery, localization, and recognition of smart objects by a mobile robot E. Menegatti M. Danieletto, M. Mina, A. Pretto, A. Bardella, A. Zanella, P. Zanuttigh SIGNET Intelligent Autonomous Systems Lab University of Padua Special Interest Group on NETworking

2. Recognition of smart objects Goal: A robotic system exploiting Wireless Sensor Network (WSN) technologies for implementing an ambient intelligence scenario. We address the problems of object discovery, localization, and recognition in a fully distributed way. The robot does not have any a priori information! Neither on the number nor on the kind of objects in the environment.

3. RAMSES2 - ProjectRAMSES2: integRation of Autonomous Mobile robots and wireless SEnsor networks for Surveillance and reScue eyesIFX motesfrom Infineon Wireless networkchannel 802.15.4 Laptop 802.11b wireless channel WirelessSensorsNetwork AutonomousMobileRobot

4. EyesIFX connected to ATX via USB + EyesService class added to Miro Experimental Set up • EyesIFX sensor nodes • Infineon Technologies. • 19.2 kbps bit rate @ 868 MHz • Light, temperature, RSSI sensors • AMR Bender • self-made, based on Pioneer 2 ActivMedia platform • Linux OS with Miro middleware • ATX motherboard • 1,6 GHz Intel Pentium 4, 256 MB RAM, 160 GB HD • Omnidirectional camera, odometers

5. Step 1 - DISCOVERY Input/Output Functions • Physical connection between robot and mote • Serial port emulation over USB (VCP) • Standard commands for eyesIFX sensor • Predefined actions to access to the WSN Allow robot’s applications to interact with the WSN Allow a bidirectional serial communication(ASCII chars)

6. Middleware Miro Range-only SLAM with a Mobile Robot and a Wireless Sensor Networks • The robot is programmed exploiting the framework Miro • Miro is a framework for mobile robot programming developed by Gerd Mayer and Gerhard Kraetzschmar at Ulm University • Miro is a middleware based on CORBA architecture for creating and managing distributed services. • Miro is based on TAO libraries of the ACE framework. • We interact with the eyesIFX mote on board of the robot through a Miro service we created, called EyesService. 6

7. Step 2 - LOCALIZATION Why Localization? • WSN deploying is an annoying and time consuming task. • Moreover, motes can be attached to objects that are moved around • First goal of the project • localize WSN nodes spread in unknown positions inside a building using a mobile robot.

8. Localization Approaches • Three main ranging approaches: • Angle of Arrival • Time of Arrival • Received Signal Strength Indicator (RSSI) • Focus on RSSI: • No specific Hardware required  • Poor outdoor ranging performance  • Very poor indoor ranging performance 

9. Our Solution Step 2 - LOCALIZATION • SLAM (Simultaneous Localization And Mapping), for a mobile robot moving in an unknown environment in which there is a WSN (Wireless Sensor Network). We use only: • robot’s odometry; • range measurements from the nodes to the robot

10. How harsh is the indoor radio channel? • Random variations due to shadowing and fading obscure the log-decreasing law for the received power vs distance • For the same range, we can measure very different RSSIWe measure the RSSI to estimate the range, then... RSSI based ranging is VERY noisy!

11. Experiments EyesIFX v2 Mote 10 x 6 m environment Robot “Bender”

12. Results (1/4) - SLAM • Much better that classical static WSN localization algorithm • Large variance on residual error for motes locations • Slightly better results taking only highest RSSI measurements (Elab 2) Fig. 1 residual mean error on robot and motes position

13. Results (2/4) - SLAM • Much better that classical static WSN localization algorithm • Large variance on residual error for motes locations

14. Where does the error come from? • If we correctly initialize the mote position in the EKF...(Elab 5 & 6) • Results: • Slight improvements on robot residual error • Large improvements on mote residual error Fig. 2 Residual mean error on robot and motes position

15. Results (4/4) - SLAM

16. Localization with delayed particle filter Odometry RSSI Measures Motes pose and robot position estimation Particle Filter Initialization Localization Algorithm: Extended Kalman Filter

17. Delayed Initialization based on Particle Filter 15

18. Step 3 - RECOGNITION Goal: the robot should be able to recognize the objects from a description of the objects’ appearance. The object appearance is stored inside the object (in the motes). The appearances is coded by scale invariant feature descriptors robust also to motion blur (MoBIF) Feature matching under motion blur (Pretto et al. ICRA 2009)

19. MoBIF Matching • The object in the robot’s camera image are identified matching the MoBIF descriptors (robust to motion blur) transmitted by the smart objects with the MoBIF descriptor extracted from the robot image; • A minimum number of matching is needed to correctly identify an object; • matching is robust to scale, occlusion, illumination and rotation;

20. Storing the appearance • To achieve robustness to scale: • the object is imaged at distances: near, medium, far and the MoBIF descriptor are merged • To achieve robustness to rotation: • the object is imaged every 20 deg. and the MoBIF descriptor are merged • The merging of the MoBIF removes redundancy! 1 meter 2 meters 5 meters

21. Step 4 - APPROACHING The robot uses a simple visual-servoing image map to approach the object. An approaching direction is identified for each of the 9 regions of the image (the red arrow). Future work: IF the robot camera is calibrated the object location in the image can be feed to the SLAM algorithm to further refine the object pose estimation.

22. UNIVERSITY OF PADUA Dept. of Information Engineering Discovery, localization, and recognition of smart objects by a mobile robot E. Menegatti M. Danieletto, M. Mina, A. Pretto, A. Bardella, A. Zanella, P. Zanuttigh SIGNET Intelligent Autonomous Systems Lab University of Padua Special Interest Group on NETworking