1 / 59

WP1: Action Development

WP1: Action Development. V. Tikhanoff 1,2 , F Nori 2 ,G Metta 2 , A. Gijsberts 2 , A. Cangelosi 1 D. Marocco 1 , S. Nolfi 3 1 Adaptive Behaviour and Cognition, University of Plymouth (UK) 2 Robotics Brain & Cognitive Sciences, Italian Institute of Technology (Italy)

dchristy
Download Presentation

WP1: Action Development

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. WP1: Action Development V. Tikhanoff1,2, F Nori2,G Metta2, A. Gijsberts2, A. Cangelosi1 D. Marocco1, S. Nolfi3 1 Adaptive Behaviour and Cognition, University of Plymouth (UK) 2 Robotics Brain & Cognitive Sciences, Italian Institute of Technology (Italy) 3 Institute for Cognitive Science and Technology, CNR, Rome (Italy) ITALK Year 1 ReviewDüsseldorf, 30 June 2009

  2. WP1, goals • To design and develop an enhanced iCub robotic platform • To develop an iCub simulation application • To carry out robotic experiments on the development of object manipulation • To develop a motor controller “compatible” with language development, i.e.: • Compositional • General • Hierarchical • To investigate how to develop complex compositional and hierarchical actions ITALK Year 1 ReviewDüsseldorf, 30 June 2009

  3. WP1, tasks • Task 1.1: Development of extended iCub platform for action manipulation and language learning experiments • Task 1.2: Development and test of iCub simulation software for multi-agent scenarios • Task 1.3: Development of object manipulation capabilities • Task 1.4: Experiments on compositional and hierarchical actions • Task 1.5: Interaction and integration between multiple motor skills ITALK Year 1 ReviewDüsseldorf, 30 June 2009

  4. Task 1.1, iCub and the construction of multiple copies ITALK Year 1 ReviewDüsseldorf, 30 June 2009

  5. The iCub: quick summary • The iCub is the humanoid baby-robot designed as part of the RobotCub project • The iCub is a full humanoid robot sized as a three and half year-old child. • The total height is 104cm. • It has 53 degrees of freedom, including articulated hands to be used for manipulation and gesturing. • The robot will be able to crawl and sit and autonomously transition from crawling to sitting and vice-versa. • The robot is GPL/FDL: software, hardware, drawings, documentation, etc. ITALK Year 1 ReviewDüsseldorf, 30 June 2009

  6. iCub “size” • Three large assemblies • Head • Upper body • Lower body • More than 120 sub-assemblies • 230 commercial codes for a total of 2967 parts • 394 custom-made codes for a total of 1110 parts • Electronics, 200 components per card (on average): approximately 5000 additional parts • Each robot is made of more than 9000 parts ITALK Year 1 ReviewDüsseldorf, 30 June 2009

  7. Degrees of freedom • Head: vergence, common tilt + 3 dof neck • Arms: 7 dof each • Shoulder (3), elbow (1), wrist (3) • Hands: 9 dof each ► 19 joints • 5 fingers ► underactuated • Legs: 6 dof each • Hip (3), knee (1), ankle (2) • Waist: 3 dof Σ= 53 dof (not counting the facial expressions) ITALK Year 1 ReviewDüsseldorf, 30 June 2009

  8. Sensorization • Absolute position • On most joints, AMS magnetic encoder • Cameras • Pointgrey Dragonfly firewire cameras • Microphones, speaker • Standard condenser electrect miniature microphones • Pinnae • Gyroscopes, linear accelerometers • Xsense: Mtx ITALK Year 1 ReviewDüsseldorf, 30 June 2009

  9. Custom electronics • ADC card • Special connectors (40 pins < 1cm length) • 200μm stainless steel wires, coated in Teflon • Motor control • C programmable DSP 40 MIPS • Up to 4A DC motor 80x30mm Motorola DSP56F807 (5680x family) MAC instructions PWM generation ADC Digital I/O Can bus C programmable 58x42mm ITALK Year 1 ReviewDüsseldorf, 30 June 2009

  10. Wiki CVS Part lists Drawings

  11. By spring 2010 we will have 18 working platforms ITALK Year 1 ReviewDüsseldorf, 30 June 2009

  12. Promoting the iCub • RobotCub Open Call • 31 participants, 7 winners will receive a copy of the iCub free of charge • UPMC Paris, Imperial London, Inserm Lyon, TU Munich, METU Ankara, Pompeu Fabra Barcelona, Urbana-Champaign USA, IST Lisbon, EPFL Lausanne • Further development… • EU project ITALK: 4 iCub’s have been built • EU project ImClever: 3 iCub’s will be built • EU project RoboSkin: a skin system compatible with iCub • EU project CHRIS: safety features for the iCub • Collaborations • Univ. of Karlsruhe: new and longer legs • Simulator: • Open Source simulator based on ODE and Newton and as a model in Webots ITALK Year 1 ReviewDüsseldorf, 30 June 2009

  13. In the pipeline… • Force control: joint level sensors, SEA or stain gauges based sensing • Skin/tactile sensors: almost everywhere on the robot surface • Robot general improvements: e.g. zero-backlash everywhere, better control electronics, higher resolution position sensors ITALK Year 1 ReviewDüsseldorf, 30 June 2009

  14. The skin Principle Lot of sensing points Structure of the skin

  15. ITALK Year 1 ReviewDüsseldorf, 30 June 2009

  16. Fingertips • Capacitive pressure sensor with 12 sensitive zones • 14.5 mm long and 13 mm wide, sized for iCub • Embedded electronics: twelve 16 bit measurements of capacitance • either all 12 taxels independently at 50 Hz or an average of the 12 taxels at about 500 Hz ITALK Year 1 ReviewDüsseldorf, 30 June 2009

  17. ITALK Year 1 ReviewDüsseldorf, 30 June 2009

  18. 6-axial force/torque sensor • Semiconductor strain gauges • On board signal conditioning, sampling, and calibration • Digital output: CAN bus • Design: Nikos Tsagarakis and Darwin Caldwell (IIT) • Electronics: Claudio Lorini (IIT) ITALK Year 1 Review Düsseldorf, 30 June 2009

  19. The Software • Goals: • Foster collaboration in “space” and “time”… • … since we’re a relatively large community and we don’t want to re-invent the wheel too often • Manage the complexity of the hardware… • … since humanoid robots (like the iCub) are complicated • Consequence: improved on existing Open Source libraries supporting a major overhaul of YARP to the iCub ITALK Year 1 ReviewDüsseldorf, 30 June 2009

  20. Sensors Cluster Relay station (head) DSP PC1 DSP Ethernet HUB DSP Gbit PC104 DSP PCN Actuators iCub API Low-level control Yarp Embedded iCub structure higher level capabilities control of attention reaching grasping learning imitation … … ITALK Year 1 ReviewDüsseldorf, 30 June 2009

  21. YARP is an open-source middleware for humanoid robotics • History • An MIT / Univ. of Genoa collaboration • Born on Kismet, grew on COG • With a major overhaul, now used by RobotCub consortium • Exists as an independent open source project • C++ source code • In short: it’s the plumbing ITALK Year 1 ReviewDüsseldorf, 30 June 2009

  22. It factors out: • the data flow: inter-process communication • it is often useful to keep algorithms away from the plumbing • the hardware: device drivers model • it is useful to avoid references to the hardware in the source code • …while being portable: • across OS and development tools • across languages • libs in C++, bindings for many other languages ITALK Year 1 ReviewDüsseldorf, 30 June 2009

  23. The iCub With Peter Ford-Dominey (INSERM, Lyon) With Auke Ijspeert, Ludovic Righetti, Sarah Degallier (EPFL) With a lot of students @ RobotCub summer school 2008 With VisLab (IST Lisbon)

  24. T1.1 iCub extensions

  25. iCub platform extensions • Software: • definition of a yarp-portable audio data format; • design of a portaudio library wrapper; • design of a ASR (automatic speech recognition) interface; • integration of ASR software (Sphinx and Esmeralda). • Hardware: • facial expressions hardware standardization and production (installed in Bielefeld); • microphones integration (installed in Bielefeld); • stereo line-in amplifier integration (installed in Bielefeld). ITALK Year 1 ReviewDüsseldorf, 30 June 2009

  26. Hardware extensions • Face expressions: • testing; • hw standardization; • production. • Integrated microphones: • electret condenser; • functional pinnae; • custom amplifiers. • Enhanced PCI card: • 4 CAN buses interfaces; • 2 firewire; • stereo line-in amplifiers. ITALK Year 1 Review Düsseldorf, 30 June 2009

  27. Software extensions • yarp-portable audio format: • yarp:sig:sound • yarp-wrapper for libportaudio: • yarp:dev:PortAudioDeviceDriver ITALK Year 1 ReviewDüsseldorf, 30 June 2009

  28. Speech Recognition Engine Integration • CMU SPHINX: • a group of open source systems related to speech recognition; • speech recognition systems developed at Carnegie Mellon University. These includes a series of speech recognizers (Sphinx 2 - 4) and an acoustic model trainer (SphinxTrain). ITALK Year 1 ReviewDüsseldorf, 30 June 2009

  29. YARP/ASR interface design Microphone (windows version) yarp:sig:sound yarp:sig:sound Microphone (linux version) Segment “Get a sound” Interface (IAudioGrabberSound) “Get a sound” Interface (IAudioGrabberSound) Decode, DAG-search PortAudio library interface Cepstral Analysis Classify According to acoustic model type Acoustic Adaptation FFMPEG Grabber library interface Cluster Decode, DAG-search N-Best Reformat & Collate N-Best rescore Words ITALK Year 1 ReviewDüsseldorf, 30 June 2009

  30. Task 1.2, simulator ITALK Year 1 ReviewDüsseldorf, 30 June 2009

  31. Cognitive Robotics & Simulators • Why do we need a robot simulator? • Not an alternative to physical robot, but an additional tool; • preliminary testing/feasibility studies; • experimenting alternative configurations; • to ease accessibility to robot and collaboration within and outside the italk consortium. • Existing robot simulation applications: • Stage / Gazebo Project ( 2D / 3D multiple-robot simulator); • Simbad Project (3D multiple-robot simulator); • Darwin 2K Project (Simulation & Automated Synthesis for Robotics); • EvoRobot (Evolutionary Robotics Simulator); ITALK Year 1 ReviewDüsseldorf, 30 June 2009

  32. Physics & rendering engines • A Physic Engine library provides: • Simulation of rigid bodies (mass, friction, sensors etc..)‏ • Collision Detection Algorithms for all the physical interactions • Advanced joint types (also customizable)‏ • Terrains and/or Meshes for complex object creation • There are many available physic engines with their pro and cons • Different solvers for rigid-body dynamic • Different spaces for collision detection • More joint types already implemented • OpenSource, GPL license • Multi-thread capability • More accurate friction simulation • API for custom joint creation and control • Free license but not OpenSource Robot Simulation – Robot Programming with Open Dynamics Engine Morikita Publishing Co. Ltd Tokyo 2007 ITALK Year 1 ReviewDüsseldorf, 30 June 2009

  33. Physics engine & rendering • WorldSim library has been created for providing • Support different physic engines • Object-Oriented design for an easy creation of a custom version of simulator • Possibility to simulate more than one iCub • 3D renderiomg classes based on OpenGL ITALK Year 1 ReviewDüsseldorf, 30 June 2009

  34. Software Architecture • Objective: control robot with same commands for both real & simulated iCub: • use the same software infrastructure and inter-process communications. • Tools: real and simulated robot software architecture is based on YARP: • motor interface: yarp::dev::controlBoard; • video interface: yarp::sig::Image; • sensor interface: yarp:os:Portable. ITALK Year 1 ReviewDüsseldorf, 30 June 2009

  35. The iCub Platforms • iCub Physical Robot and iCubSim Simulated robot • Total height 105 cm • Total weight 20.5 kg • 53 DoFs in Total • 12 for the legs • 3 for the torso • 32 for the arms • 6 for the head • Emotion interface • Touch sensors ITALK Year 1 ReviewDüsseldorf, 30 June 2009

  36. Robot Control ITALK Year 1 ReviewDüsseldorf, 30 June 2009

  37. Transferability – Simulator to Physical • Requires just switching the robot name in the configuration files or programs from “icubSim” to “icub”. • The simulated robot shares some important robot features: • kinematic specifications; • motor interface (controlBoard interface) ; • vision. • The simulated robot is not a “perfect” match of the physical robot: • rough (body) dynamical model; • rough trajectory generator; • rough actuators dynamical model. ITALK Year 1 ReviewDüsseldorf, 30 June 2009

  38. iCubSim demo • Launching the simulator. • Accessing the motor interface: • via command line (yarp rpc); • via the robotMotorGui; • via C++ code (preprogrammed sequences). • Accessing the video interface: • via the yarpview. • Object manipulation scenario. ITALK Year 1 ReviewDüsseldorf, 30 June 2009

  39. Task 1.3, motor (and cognitive) capabilities ITALK Year 1 ReviewDüsseldorf, 30 June 2009

  40. On-line life-long learning • Developmental robotics entails: • Quasi online • at least memory bounded • adapt to changes in the dynamics (e.g. tools) • Fast computation of the solution • useful in control loops • Large data-sets • perhaps collecting data for the entire “life” of the robot • Guarantees • convergence, bounds, etc. ITALK Year 1 ReviewDüsseldorf, 30 June 2009

  41. Methods • Kernel methods • Online variants • More of a research area • ANN, feedforward, recurrent • Training via backpropagation • Via reinforcement • More established ITALK Year 1 ReviewDüsseldorf, 30 June 2009

  42. Scenario: dataset • Inputs: Joint positions, velocities and accelerations for 4 joints (3 shoulder, 1 elbow) • Outputs: Forces and torques in x, y, z directions (here we only consider Fx) • 100.000 samples collected at 50 Hz during random movements • Inputs scaled “roughly” within [-1,+1] ITALK Year 1 ReviewDüsseldorf, 30 June 2009

  43. Some batch results ITALK Year 1 ReviewDüsseldorf, 30 June 2009

  44. Passive Aggressive regression • Variant of Perceptron algorithm • Margin-based (also updates with margin errors) like SVM • Easily kernelized using kernel expansion • Binary and multiclass classification, regression, etc. • Efficient and easy to implement Crammer et al., Online passive-aggressive algorithms, 2006. ITALK Year 1 ReviewDüsseldorf, 30 June 2009

  45. On-line, efficient, bounded Time vs. # of samplesexplicit polynomial Time vs. # of samples forpolynomial kernel Accuracy for randomapproximation Time vs. # of samples forrandom approximation ITALK Year 1 ReviewDüsseldorf, 30 June 2009

  46. Learning motor control • Map sensory information into appropriate behaviors: • Gazing • Reaching • Grasping ITALK Year 1 ReviewDüsseldorf, 30 June 2009

  47. Example: gazing ITALK Year 1 ReviewDüsseldorf, 30 June 2009

  48. Reaching • Model acquisition for reaching ITALK Year 1 ReviewDüsseldorf, 30 June 2009

  49. Task-space control Major drawbacks of available algorithms for IK (Pseudoinverse, DLS): need to manage singularities, need of boundary functions to embody the notion of joints limits.

  50. Example Comparison of random XYZ positions of objects, with the actual resulting position of the robots hand ITALK Year 1 ReviewDüsseldorf, 30 June 2009

More Related