LREC 2006 workshop Genova 28 May 2006

1. LREC 2006 workshop Genova 28 May 2006 EU-funded research initiatives: - Interactive Systems - New Interfaces - Language Technology Xavier Gros DG INFSO, European Commission, Luxembourg

2. Contents • What has been done so far? • Projects running so far under “FP6” • 2nd batch of proposals under negotiation • “FP7” info • What is it? • By when? • What’s in it for you?

3. Our Mission E1 Interfaces The challenge is: • To make interfaces as simple as possible, and interaction as easy as possible • For citizens to be able to access, receive and use information in their own language • We support the research, development and integration of: • Advanced technologies for interfaces & interaction • Integrated multilingual information systems and services We focus on: • Natural interaction between humans and physical or virtual environments • Multilingual communication systems for unrestricted domains, including real-time understanding of spoken and gesture input

4. So what happened? • Calls: 1st in late 2002, and 5th in mid-2005 • Proposals: 169 requesting a total funding of 804 M€ • High quality: 58 asking for a total funding of 296 M€ • Projects: 29 projects for a total funding of ~136M€ (15 start 2006) • Participants: ~320 • Type: 7 IP’s, 4 NoE’s, 17 STReP’s, 1 SSA

5. Integrated Projects 1st Call: • CHIL - Computers In the Human Interaction Loop http://chil.server.de/ • AMI - Augmented Multi-party Interaction http://www.amiproject.org/ • TC-STAR - Technology and Corpora for Speech to Speech Translation http://www.tc-star.org/ 5th Call: • Augmented Multi-party Interaction with Distance Access • Intelligent, Persistent, Personalised Multimodal Interfaces to the Internet • Conveying Affectiveness in Leading-edge Living Adaptive Systems • Multimodal Interfaces for Capturing and Transferring of Skills

6. Networks of Excellence 1st Call: • SIMILAR - Federating the communities of human-computer interaction and signal processing http://www.similar.cc • HUMAINE - Human-Machine Interaction Network on Emotion http://emotion-research.net • ENACTIVE - Enactive Interfaces http://www.enactivenetwork.org And an SSA called MWEB - Multimodal Web Interaction http://www.w3.org/2004/MWeb/

7. Targeted Research 1st call • MATRIS - Markerless Real-time Tracking for Augmented Reality Image Synthesis http://www.ist-matris.org/ • T’nD - Touch and Design http://www.kaemart.it/touch-and-design/ • TAI-CHI - Tangible Acoustic Interfaces for Computer-Human Interaction http://www.taichi.cf.ac.uk/ • DIVINES - Diagnostic and Intrinsic Variabilities in Natural Speech http://www.divines-project.org/ • TALK - Talk and Look: Tools for Ambient Linguistic Knowledge http://www.talk-project.org/ • HIWIRE - Human Input That Works In Real Environments http://www.cvsp.cs.ntua.gr/projects/bin/view/HIWIRE/WebHome

8. Targeted Research 5th call Statistical Multilingual Analysis for Retrieval and Translation • New and more effective statistical approaches to machine translation and cross-lingual textual information access Multi-modal and Multi-Sensor Zero-distraction Interaction Interface • Develop a mobile human-computer interface for eye-busy and hand-busy applications in zero-distraction mode Multimodal Interactions Analysis of Users within Controlled Environments • Techniques for analysing eye gaze/fixation, eye blink and body movement in human-controlled environment interaction 3D and Speech Interfacing Technologies • 3D information visualisation and manipulation based upon the fusion of holographic displays, 3D hand tracking and natural speech processing Sound and Tangible Interfaces for Novel Product Shaping • Develop a new type of interface for natural hands-free interaction with virtual shapes Statistical and Hybrid Machine Translation between all European Language • Combining statistical techniques with linguistic knowledge in machine translation for all EU language pairs subject to negotiation

9. Targeted Research 5th call OpenInterface • OpenInterface is an open source platform for rapid development of multimodal interactive systems Question Answering Learning Technologies in a Multilingual and Multimodal Environment • Open domain question-answering (QA) as a new way to search, integrating multilingual open domain QA, context-aware QA, and machine learning technologies Distant-talking Interfaces for Control of Interactive TV • Develop hands-free voice interaction for use with a interactive-TV remote control Action Recognition and Tracking based on time-of-flight Sensors • Algorithms and prototype for a time-of-flight (ToF) camera that captures 3D information and can track head and eye movement Spoken Language Understanding in Multilingual Communication Systems • New algorithms and tools for robust spoken language understanding in multilingual telephone services subject to negotiation

10. SSA IP STREP NoE AMI HIWIRE MWEB MATRIS TAI-CHI SIMILAR PASCAL HUMAINE ENACTIVE TALK CHIL TC-STAR T’n D E1 FP6 call-1 portfolio ~73 million EURO MULTILINGUALITY DIVINES SPEECH TECHNOLOGIES INTERACTION MULTIMODALITY – NEW INNOVATIVE INTERFACES

11. IST call 5 Closed: 21 September 2005 Budget: 63 M€ Number of proposals: 101 Under negotiation:15 new projects • 4 IP: 6-10 MEUR 3-5 years • 11 STREP: 2-3 MEUR 2.5-3 years Projects launch: around September 2006 subject to negotiation

12. FP7 (2007-2013) 4 main Specific Programmes • Cooperation Implementation of ICT in the “Cooperation” part • to gain leadership in key scientific and technological areas by supporting • cooperation between universities, industry, research centres and public • authorities across the European Union as well as the rest of the world. • Ideas • to support the truly creative scientists, engineers and scholars, whose curiosity and thirst for knowledge are most likely to make the unpredictable and spectacular discoveries that can change the course of human understanding and open up new vistas for technological progress and solving enduring social and environmental problems. • People • to strengthen, qualitatively and quantitatively, human resources in R&D in EU • Capacities • enhance research and innovation capacity throughout Europe http://cordis.europa.eu.int/ist/about/fp7.htm

13. Specific Programme: ICT Collaborative Research – Activities Nano-electronics, photonics and micro/ nano-systems Communication Networks Embedded systems, Computing and control Software, Grids, Security and dependability Knowledge, cognitive and learning systems Simulation, visualisation Interaction and mixed Realities Personal environments Trust and confidence Home environments FET Societal challenges Robotic systems Content, creativity personal development Intelligent infrastructures New perspectives in ICT Drawing on other science and technology disciplines Business and industry 1st activity: Technology pillarspushing the limits of performance, usability, dependability, cost-efficiency 2nd activity: Integration Environments integrating multi-technology sets that underlie new functionalities, services and applications 3rd activity: Applications research providing the knowledge and the means to develop a wide range of ICT-based services and applications and supporting research at the frontiers of knowledge

14. Draft FP7 Texts • FP7 Proposal by Commission to Council & Parliament „BUILDING THE EUROPE OF KNOWLEDGE” • Information & Communication Technologies • ICT Technology Pillar Simulation, visualisation, interaction and mixed realities: tools for innovative design and creativity in products, services and digital media, and for natural, language-enabled and context-rich interaction and communication. • Specific Programmes • Co-operation – ICTSimulation, visualisation, interaction and mixed realities: tools for modelling, simulation, visualisation, interaction, virtual, augmented and mixed reality and their integration in end-to-end environments; tools for innovative design and for creativity in products, services and digital audio-visual media; more natural, intuitive and easy-to-use interfaces and new ways to interact with technology, machines, devices and other artefacts; multilingual and automatic machine translation systems.

15. “Cooperation” – Collaborative Research – Themes Nano, materials, production: 3467 m€ (11%) ICT: 9080 m€ (28.5%) Energy: 2236 m€ (7%) Environment: 1886 m€ (6%) Transport: 4150 m€ (13%) Food, agri, biotech: 1936 m€ (6%) Socio-econ research: 607 m€ (2%) Space and security: 2858 m€ (9%) Health: 5984 m€ (18%)

16. Preparing for FP7 • Interactive Systems & New Interface Technologies • 30-31 January 2006, Luxembourg • Industrial Workshop on Simulation, Visualisation, Interaction, and Mixed Realities • 14-15 February 2006, Luxembourg • Machine Translation & Human Language Technologies • 23 February 2006, Luxembourg

17. Some pointers … Important issues highlighted in FP7 consultations: • Individual interaction modalities need to be made far more accurate and robust. One important application challenge is the lack of robust and cost-effective hands-free and eye-free interfaces (e.g. hand-busy and eye-busy situations). • People want interaction and information access in their own language, and we must understand how to develop interfaces that are capable of dealing more efficiently and cost-effectively with different languages. A specific challenge is machine-assisted translation. • A major research challenge involves the shift from explicit interaction to implicit human-computer interaction, wherethe computer adapts to the user, their language and context. In addition, we must develop ways to create adaptive interfaces that enable personalised interaction. The move towards personalised interaction requires us to address ethical and privacy issues. under discussion

18. Some pointers … Important issues highlighted in FP7 consultations: • We must be able to understand and model the behaviour of users in the way they interact with machines, with services, with their surrounding environment and with each other. • We must demonstrate how effective multimodal interaction helps people and we must answer the question about the benefits that multimodal interaction brings to users. One way forward is to stress-test new interaction modalities in industrial prototypes. Standards and open source software platforms are an essential prerequisite for the rapid adoption of prototype applications. • We need common evaluation and benchmarking schemes, test and training data, tools, and metrics. Open competitions are an effective way to mobilise the research community. under discussion

19. Some pointers … • Weak integration into real applications • Language resources: lack of integration and sharing • Speech understanding still difficult • MT: lack of robustness, questionable quality • No evaluation in specific real-life situations

20. Perspectives • Resource sharing, integration of applications, collaboration across scientific domains • Fully automatic end-to-end MT systems • Easy to use and robust MT systems • Text and speech recognition • Linguistic infrastructure

21. Perspective 1: Machine Translation • Fully automatic end-to-end systems and semi-automatic machine translation systems (spoken and written languages) to support a human translator/interpreter. Automatic multilingual speech-recognition toolkits. Initial objective is “high quality translation” of (semantically) domain-limited but otherwise (syntactically) unrestricted languages.

22. Perspective 2: Robustness • Easy to use machine assisted translation tools and portable and robust translation devices for text-to-text and speech-to-speech translation across multiple languages. Effective affordable, reusable systems. Interfaces and systems for on-line translation of web pages.

23. Perspective 3: Resources • Shared databases corpora and free distribution among research non-profit institutes • Open source toolkits and modules

24. Further information General FP6: http://europa.eu.int/comm/research/fp6/ IST: http://www.cordis.lu/ist FP7: http://www.cordis.lu/ist/about/fp7.htm Xavier Gros e-mail: xavier.gros@cec.eu.int tel: +352 4301 37193