Overview of Year 1 Progress

1. Overview of Year 1 Progress Angelo Cangelosi & ITALK team italkproject.org

2. Overview Project summary Vision and methodology Objectives Main results Methodological Scientific Integration of partners Collaboration space map Talks overview ITALK Year 1 ReviewDüsseldorf, 30 June 2009

3. Overview Project summary Vision and methodology Objectives Main results Methodological Scientific Integration of partners Collaboration space map Talks overview ITALK Year 1 ReviewDüsseldorf, 30 June 2009

4. ITALK Strategic Goal Project centred on two main S/T hypotheses: • The parallel development of action and social interaction permits the bootstrapping of language capabilities, which on their part enhance cognitive development (e.g. Arbib 2002, Corballis 2002, Glenberg 1997, Tomasello 2003; Cangelosi & Riga 2006) • Developmental robotic approaches based on the integration of action, social interaction and language have fundamental technological implications for designing communication in robots (e.g. Cangelosi et al. 2005; Metta et al. 2006; Weng et al. 2001; Roy 2005; Nehaniv et al. 2007)

5. ITALK Research Themes iCub Development & Extension Emergence of Language Action Learning (Manipulation) Conceptual Learning Integration & Cognitive Bootstrapping Social Learning & Interaction ITALK Research

6. ITALK Research Themes iCub Development & Extension WP1: Robot & Action (IIT) Emergence of Language WP4: Language (BIEL, USD) Action Learning (Manipulation) Conceptual Learning Integration & Cognitive Bootstrapping WP5: Integrate (PLYM, ISAB, all) WP2:Concept. (CNR) Social Learning & Interaction WP3: Social (UH) ITALK Research

7. Objectives • To provide new theoretical insights, models and scientific explanations of the integration of action, social and linguistic skills • To develop an interdisciplinary set of methods • To develop innovative and cognitively plausible engineering principles, techniques and approaches • To demonstrate the effectiveness of the above scientific and technical advances through the use of robotic experiments

8. Objective I – year 1 • To provide new theoretical insights, models and scientific explanations… • to develop roadmap for future research the identifies the main challenges and theoretical, methodological, and experimental tools in the study of language development in robots (WP5) • to consider the role of preliminary robotic experiments for action and language integration research (WP1-5)

9. Objective II – year 1 • To develop interdisciplinary set of methods… • to develop and test the iCub simulator for robotic simulation experiments (WP1) • to develop constructivist grammar classifications of linguistic constructions from robot directed speech as input to developmental robotics experiments (WP3) • to test the use of machine learning techniques (e.g. neural networks, artificial vision routines, speech recognition systems) for developmental robotics experiments on action, categorisation, social and language learning (WP1-5) • to use experimental HRI and neuroscience methods for robot learning experiments (WP3-5)

10. Objective III – year 1 • To develop innovative and cognitively plausible engineering principles, techniques and approaches… • to design and develop an enhanced iCub robotic platform for language learning and object manipulation experiments (WP1) • to use standardisation methods for action and language learning experiments (WP1, WP5)

11. Objective IV – year 1 • To demonstrate the effectiveness … through the use of robotic experiments • to plan and carry out preliminary robotic experiments on the development of object manipulation capabilities (WP1) • to plan and carry out preliminary robotic experiments of how an artificial embodied agent can develop embodied concepts (WP2) • to plan and carry out preliminary robotic experiments on socially constructed linguistic phenomena (WP3, WP4, WP5)

12. Overview Project summary Vision and methodology Objectives Main results Methodological Scientific Integration of partners Collaboration space map Talks overview ITALK Year 1 ReviewDüsseldorf, 30 June 2009

13. Main Methodological Results Objectives II (methods) and III (eng. principles) • Extended iCub platforms for linguistic experiments • Four new iCubs • Open Source iCub Simulator

14. Main Methodological Results (1) • Extended iCub platforms for linguistic experiments Microphones ESMERALDA Sphinx CFW custom board for video and audio signals acquisition

15. Main Methodological Results (2) 2. Four new iCubs

16. Main Methodological Results (3) 3. Open Source iCub Simulator • Software development (IIT, PLYM, CNR) • Single and multi-robot scenario • Preliminary experiments • Used outside consortium (e.g. projects CHRIS, IMclever)

17. Main Methodological Results (3) 3. Open Source iCub Simulator - “realistic” model of physical robot

18. Main Methodological Results (3) 3. Open Source iCub Simulator - multi-engine (ODE, Newton) - single and multi-agent

19. Main Scientific Results Objectives I (theory) and IV (experiments) • Roadmap for future research • ROSSUM architecture scaling-up • Analyses of tutoring behaviour • Cognitive linguistic analyses • Simulation experiments (Y1) • Planning experiments (Y2+)

20. Main Scientific Results (1) • Roadmap for future research • International workshop • Public roadmap document • Submitted to:

21. Main Scientific Results (2) 2. ROSSUM architecture scaling-up • Experiment with Kaspar II

22. Main Scientific Results (3) 3. Analyses of tutoring behaviour • Tutoring Spotter • Acoustic Packaging methodology

23. Main Scientific Results (3) 3. Analyses of tutoring behaviour • Tutoring Spotter

24. Main Scientific Results (3) 3. Analyses of tutoring behaviour • With automatic motion recognition

25. Main Scientific Results (4) 4. Cognitive linguistic analyses • development of psycholinguistically motivated grammar learning scenarios • based on empirical analyses of child-directed and robot-directed speech. • criteria for the creation of suitable input for embodied grammar learning experiments were devised on the basis of the empirical analyses

26. Main Scientific Results (5) 5. Simulation experiments (Y1) • Cognitive architecture for understanding simple instruction • Hybrid HRI-simulator testing • Neural oscillator model for selective attention • Active categorical perception • Pilot experiments with physical robot

27. Main Scientific Results (5) 5. Simulation experiments (Y1) • Understanding simple instructions

28. Main Scientific Results (5) 5. Simulation experiments (Y1) • Hybrid HRI-simulator testing

29. Main Scientific Results (5) 5. Simulation experiments (Y1) • Neural oscillator model for selective attention (Borisyuk et al. in press)

30. Main Scientific Results (5) 5. Simulation experiments (Y1) • Active categorical perception (Massera et al. 2009)

31. Main Scientific Results (6) 6. Planning experiments (Y2+) • Action and Language compositionality with Tani’s networks (T1.4, 4.1, 4.3) • Biases in category learning (T2.2) • HRI Experiments on gesture, negation and grammar (T3.2, 3.3) • Acoustic Packaging analyses (T4.2) • Neuroscience and gaze in HRI (T5.3) • …

32. Overview Project summary Vision and methodology Objectives Main results Methodological Scientific Integration of partners Collaboration space map Talks overview ITALK Year 1 ReviewDüsseldorf, 30 June 2009

33. Collaboration Space Map Tasks 5.1 RoadMap

34. Collaboration Space Map Tasks 1.1: iCub development, Training, ASR

35. Collaboration Space Map Task 1.2: iCub simulator

36. Collaboration Space Map Tasks 1.4, 4.1, 4.3: Action, language, compositionality

37. Collaboration Space Map Tasks 3.1 Cognitive Linguistic analysis

38. Collaboration Space Map Tasks 3.2: HRI Experiments

39. Collaboration Space Map ALL TASKS

40. Overview Project summary Vision and methodology Objectives Main results Methodological Scientific Integration of partners Collaboration space map Talks overview ITALK Year 1 ReviewDüsseldorf, 30 June 2009

41. Time Plan

42. Next presentations ITALK Year 1 ReviewDüsseldorf, 30 June 2009