interface technologies and cognitive systems
Skip this Video
Download Presentation
Interface technologies and cognitive systems

Loading in 2 Seconds...

play fullscreen
1 / 72

Interface technologies and cognitive systems - PowerPoint PPT Presentation

  • Uploaded on

Interface technologies and cognitive systems. Philippe Gelin European Commission - Information Society. Granada 16th December 2002. Content. PART I : European Research Area PART II : Interface Technologies PART III : Cognitive Systems PART IV : FP6 Instruments. PART I.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' Interface technologies and cognitive systems' - thuyet

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
interface technologies and cognitive systems
Interface technologies and cognitive systems

Philippe Gelin

European Commission - Information Society

Granada 16th December 2002

  • PART I : European Research Area
  • PART II : Interface Technologies
  • PART III : Cognitive Systems
  • PART IV : FP6 Instruments
part i


The European Research Area (ERA)


its Sixth Framework Programme (FP6)

  • The Treaty establishing the European Community provides for the creation of multiannual research and technological development
  • The Sixth Framework Programme (FP6) is the European Unions main instrument for the funding of research in Europe within 2002-2006.
  • The overall budget for FP6 is 17.5 billion euro.
  • In the past, collaborative actions have been initiated at European and Community level,
  • but now is the time to bring our endeavours together and to build a research and innovation equivalent of the "common market" for goods and services.
  • That structure is called the European Research Area (ERA) and is regrouping all Community supports for the better coordination of research activities and the convergence of research and innovation policies, at nationaland EU levels.
from era to ist
From ERA to IST

1. Focusing and integrating Community research 1.1. Priority thematic areas of research 1.1.1. Life sciences, genomics and biotechnology for health (i) Advanced genomics and its applications for health. (ii) Combating major diseases. 1.1.2. Information Society Technologies(3.625 billion Euro)1.1.3. Nano-technologies and nano-sciences, knowledge-based multifunctional materials, and new production processes and devices 1.1.4. Aeronautics and space 1.1.5. Food quality and safety 1.1.6. Sustainable development, global change and ecosystems (i) Sustainable Energy Systems (ii) Sustainable surface transport (iii) Global change and ecosystems 1.1.7. Citizens and governance in a knowledge-based society 1.2. Specific activities covering a wider field of research 1.2.1. Policy support and anticipating scientific and technological needs 1.2.2. Horizontal research activities involving SMEs 1.2.3. Specific measures in support of international cooperation 2. Strengthening the foundations of the European Research Area 2.1. Support for the coordination of activities 2.2. Support for the coherent development of policies

ist programme 2002 2006
IST Programme 2002-2006
  • Instantiate the European challenges (ERA), Initiatives
  • Trans national partnerships
  • Critical mass of skills

and adequate resources hence ...

  • Focus, Excellence, Impact and RoI
ist vision
IST Vision

IST today …………………………..The IST in FP6 vision

  • PC based ………………………………. “Our surrounding” is the interface
  • “Writing and reading”…………….…. …Use all senses, intuitive
  • “Word” based information search……….Context-based knowledge handling
  • Low bandwidth, separate networks…. ….Infinite bandwidth, convergence, ..
  • Mobile telephony (voice)………….. ……Mobile/Wireless full multimedia
  • Micro scale………………………………Nano-scale
  • Silicon based……………………………..+ new materials
  • e-Services just emerging……………….. Wide adoption (eHealth, Learning,..)
  • < 10% of world population on-line….. …World-wide adoption

Knowledge andInterface Technologies

Communication ComputingSoftware

Future andEmerging Technologies

Technologies for major economicand societal challenges



strategic objectives in 2002 2003 wp
Call 2

Advanced displays

Optical, opto-electronic, & photonic functional components

Open development platforms for software and services

Cognitive systems (25ME)

Embedded systems

Applications and services for the mobile user and worker

Cross-media content for leisure and entertainment (55ME)

GRID-based Systems for solving complex problems

Improving Risk management

Call 1

Pushing the limits of CMOS, preparing for post-CMOS

Micro and nano-systems

Broadband for all

Mobile and wireless systems beyond 3G

Towards a global dependability and security framework

Multimodal interfaces (65ME)

Semantic-based knowledge systems

Networked audiovisual systems and home platforms

Networked businesses and governments

eSafety of road and air transports

e Health

Technology-enhanced learning and access to cultural heritage (65ME)










Strategic Objectives in 2002-2003 WP

80% of budget is pre-allocated

Remainder flexible, to be allocated after evaluation

Allocation only for the first 2 years: to be revised for 2005-2006

  • Why interface technologies?
    • IST Vision
  • What research?
    • Focus of first calls in FP6
  • Where are we today?
    • Ongoing activities
  • Building the knowledge society for all
    • Allow people to use the surrounding hi-tech easily and in a way that is consonant to humans rather than to computers
    • Facilitate interpersonal communication anywhere, anytime, beyond linguistic and cultural barriers
    • Assist people and augment their abilities when interacting and communicating
  • Bring people to the foreground – build technologies for the background
  • Our surrounding is the interface: everywhere, embedded in everyday objects (e.g: clothes, furniture, vehicles, smart materials, …)
  • Exploring interfaces beyond the PC, screens & keyboard. (brake the chains…)
  • Enabling multi-sensorial dialogues supported by ubiquitous computing & networking
  • Multi-linguistics, multi-cultural interactions
known challenges
Known Challenges
  • Strongly multidisciplinary with many component technologies
  • Significant system level integration issues
  • Research resulting in autonomous (self-learning, self-organising), adaptive (time varying) systems that can work with partial and uncertain information
2003 2004 focus
Multimodal Interfaces (in call 1)

Objective: To develop natural and adaptive multimodal interfaces, that respond intelligently to speech and language, vision, gesture, haptics and other senses.

Focus 1 Multimodal interaction

Focus 2 Multilingual communication

2003-2004 FOCUS
Interaction between and among humans and the virtual and physical environment

intuitive multimodal interfaces that are autonomous and capable of learning and adapting to the user environment in dynamically changing contexts. They should recognise emotive user reaction and feature robust dialogue capability with unconstrained speech and language input.

Multimodal Interaction

      • Human-to-human: technology mediated communication
      • Human-to-things: virtual and physical
      • Human-to-self: health, well being
      • Human-to-content: information retrieval/browsing
      • Device-to-device: human mediated device communication
  • Human-to-embodied robots
facilitating translation for unrestricted inputs, especially for spontaneous (unrestricted) or ill-formed (speech) inputs, in task-oriented settings

Unrestricted spontaneous speech-to-speech translation in task-oriented settings

Statistical/mixed approaches to translation

Adaptation to task/user, learning


Multilingual Systems

basic research
Basic research, component technologies


accurate vision

gesture tracking

speech and audio processing

language technologies

affective computing

machine learning

autonomous systems

fusion of multiple channels

Basic Research
system level integration
Proof of concept in challenging application domains, including:

wearable interfaces and smart clothes

intelligent rooms and interfaces for collaborative working tools

cross-cultural communications

usability issues and evaluation

System Level Integration
shared infrastructure
Data: large amounts of multimodal data, synchronisation and IPR issues

Metrology, technology evaluation, usability

Infrastructural knowledge: machine learning, applied mathematics

Best practice and standards

Tools and platforms

Socio-economic issues (e.g. human factors)

Shared Infrastructure
ongoing activities
Collection, processing and browsing of multimodal meeting data

systems that enable recording, structuring, browsing and querying of an archive of multi-modal recordings of meetings.

Multimodal, multicultural, multilingual communication

integration of multiple communications modes - vision, speech and object manipulation - by combining the physical and virtual worlds to provide support for multi-cultural communication and problem solving

Speech-to-speech translation

development of speech-to-speech translation and its integration in automatic in e-commerce and e-service environments

data collection for speech-to-speech translation

Ongoing Activities
ongoing activities1
Preparing future multisensorial interaction research

providing technological baselines, comparative evaluations, and assessment of prospects of core technologies for speech-to-speech translation, the detection and expressions of emotional states, and technologies for children’s speech.

Automatic animation of human models

design and development of a virtual person animation system in controlled environments, which enables the modeling, analysis and simulation of human motion.

Recognition of the user’s emotional state

human computer interaction that can interpret its users\' attitude or emotional state in terms of their speech and/or their facial gestures and expressions.

See for more information

Ongoing Activities
part iii


Cognitive Systems in FP6scope and focus

  • why cognitive systems?

IST vision

  • what research?

focus of call

  • where are we starting from?

current activities


Ambient intelligence

“… ability of computationally empowered devices to interconnect with each other and with us …”

  • sensors provide a window from world of interconnected computation into real physical world
  • using sensors, these devices will sense the world around us and respond by interacting with the world or by communicating with us
  • they need to ‘see’, ‘hear’, …. ‘understand’ and ‘act’
  • Ambient Intelligence requires perceptual systems capable of cognition

To constructphysically instantiatedorembodiedsystemsthat can :

  • perceive,
  • understand(the semantics of information conveyed through their perceptual input)
  • learning (structured memory)
  • interactwith their environment, (CCI & CHI communication)
  • evolvein order to achieve human-like performance in activities requiring context-(situation and task) specific knowledge.
focus 2002 2003
Focus 2002-2003
  • Research will aim at realizing:
    • completesystems with
    • real-timeperformance and/or bounded rationality,
    • have well developedmemory capacities(e.g. short term, long term, iconic, associative) with efficient representation,
    • and thatacquire representationsas needed to realise performance goals.
    • to performgoal-directedtasks
    • The emphasis is on closing the loop in realistic test cases.
known challenges1
Known Challenges
  • A main target of this research isinterdisciplinarity:
    • computer vision,
    • natural language understanding,
    • robotics,
    • artificial intelligence,
    • mathematics and
    • cognitive neuroscience
  • and its impact on overall system design.
  • Direct Applications:
  • in image recognition, in behavioural interpretation, in video annotation, speech recognition, in automatic categorisation and classification, in goal-specification and decision support,…
  • can serve to demonstrate and measure progress but applications NOT main target of research
  • Methods & approaches for constructing robust & adaptive systems capable of cognition
ongoing activities2
Ongoing Activities
  • cognitivevisionsystems 2000 – 2002
    • robust image recognition
    • from application-specific togeneralised solutions
    • from focus on low-level processing & robustness of individual components to systems approach where every component (incl. high-level cognitive functionalities) has a role to play in assuring robust behaviour of system
    • longer-term, largely academic, interdisciplinary effort
  • 8 RTD projects & 1 Network:
related topics
Related Topics
  • Multimodal Interfaces, call 1

intuitive multimodal interfaces ... autonomous and capable of learning and adapting … in dynamically changing contexts

  • Beyond Robotics, call 1

focus on open-ended lifelong learning systems

  • Disappearing Computer, call 1

focus on open architectures allowing arbitrary combinations of ‘building blocks’ for ‘universal application’

  • Embedded Systems, call 2

focus on networked systems, distributed control

part iv



  • Schedule
  • New Instruments
  • Old Instruments
  • Applied to IFT & CS.
  • Expected results
  • Learning from EoI
  • “Do’s and don’ts”
  • Conclusions
fp6 roadmap
  • 21 February 2001 Commission decides on first proposal for Framework Programme
  • March 2001 Informal presentation to the Ministers of Research and Education during the Uppsala Informal Council (1 - 3 March)
  • May 2001 Adoption by the Commission of proposals regarding the specific programmes and the implementation of the Framework programme
  • September 2001 Adoption by the Commission of proposals for decisions on the rules for participation and dissemination of results
  • 26 June 2001 Council discusses Commision proposal
  • July/Nov. 2001 First reading by the European Parliament
  • 12/2001- 01/2002 Common position of the Research Council
  • 11 March 2002 Council adopt final text
  • 15 May 2002 The European Parliament approves the Sixth Framework Programme
  • 3 June 2002 Council of Ministers formally adopted the Sixth Framework Programme
  • 12 June 2002 Parliament approves specific programmes
  • 19 June 2002 FP6 rules of participation compromise approved by ITRE
  • 3 July 2002 FP6 rules of participation compromise approved by EP
  • September 2002 Specific Programmes has been adopted (October)
  • End 09/2002 Rules for Participation to be formally adopted by Council and Parliament
  • November 2002 Conference marking the launch of the Sixth Framework Programme
  • 12/2002 First calls for proposals
  • end 2003 Second calls for proposals
call 1 timetable










Call 1 Timetable

Publication: 2nd half December 2002

Call deadline: 24 April 2003

Evaluation: weeks of 12th and 19th May 2003

IP/NoE hearings: week of 9th June 2003

Negotiations: from July 2003

Commission Decision: from October 2003

Project start: January 2004

overview of fp6 instruments
Overview of FP6 instruments
  • Integrated Projects (IP)
  • Networks of Excellence (NoE)
  • Article 169 (joint implementation of national programmes)
  • Specific Targeted Research Projects (STReP’s)
  • Co-ordination actions (CA)
  • Specific Support Actions (SSA)



why new instruments
Why new instruments ?
  • Adapt to the size of the challenge
    • The philosophy of both instruments is to movefrom multiple projectfunding to the funding of coherent programmes of research activities,
  • Flexibility and responsibility
    • leaving the highest degree possible of autonomy and flexibility to European research consortia.
integrated projects
Integrated Projects
  • Expected to address the objectives within a holistic approach enabling,
  • where justified, competition within and across projects
  • A major target is to integrateactors and disciplines= build a community
  • An IP should be THE project in the target area
  • “critical mass”
integrated projects1
Integrated Projects
  • Core activities
    • genuine research work
    • “engineering” tasks (esp. methods & tools)
    • system integration & validation (“total system” approach)
  • along with
    • promotion & dissemination of results
    • training, awareness & best practice (researchers, integrators, launching users)
    • cooperation & exchanges with related nationaland international efforts (incl. standards bodies)
    • socio-economic impact & consequences
example of an ip

Integrated project

Current projects

Example of an IP

Area: Multi-sensorial interfaces

Knowledge & Data bases

Virtual & Augmented Realities

Intelligent Assistants & Virtual Humans

Intelligent Virtual Worlds

Physical Virtual World

Interface Layer


Gesture & Facial Recognition



Body Sensors

Haptic devices






Accepted costs

Detailed work plan


Accepted costs

Adjusted advance

Accepted costs

Detailed work plan


Adjusted advance


Adjusted advance

Accepted costs

Detailed work plan

Initial advance

Detailed work plan

0 6 12 18 24 30 36 42 48 Months

IP - payment & report schedule

Example of 4 year contract

networks of excellence
Networks of Excellence
  • Spread excellence
  • Xfertilisation among disciplines
  • Bring together the best research in constituent disciplines, integrate resources, (e.g. image, video, multimodal,…)
  • Create a NEW research space (ERA)
  • Have a lasting nature
  • Define joint activities
networks of excellence1
Networks of Excellence
  • Joint programme of activities:
    • Integrating activities
      • coordinated programming of the partners’ activities
      • sharing of tools/facilities/ infrastructures
      • managing the knowledge portfolio
      • staff mobility and exchanges
    • Joint researchactivities
      • undertaking research on topics supporting the network’s goals
    • Spreading excellence
      • training of researchers and other key staff
      • dissemination and technology transfer to industry
  • “virtual” centres of excellence

New or additionalto ongoing efforts

noe grant
NoE - grant
  • Each call will fix the founding per researcher, based on the characteristics of the field of research concerned
      • Computer science less expensive than nuclear research
  • The number of researchers implied in the integration
      • PhD or more than 4 years of research
      • Existing head count at time of proposal.
      • Table of Found Per researcher issued at each call
  • @ max the cost incurred to implement the JPA
  • @ max 25% of the existing value proposed to be integrated
specific targeted research projects
Specific targeted research projects
  • Similar to Old instrument, except for:
    • joint and several financial liability
    • costing / funding adapted to grant scheme
    • rates / activities: research (50%), demonstration (35%), combined research & demonstration
  • Activities
    • focussed RTD and / or demonstration; management
  • Partnership
    • more limited than for IPs; in terms of breadth, time, scope
coordination specific support actions
Coordination & specific support actions

Coordination Actions

  • Instrument for ad-hoc co-operation between organisations (no durable integration !), similar to FP5 thematic networks
  • Activities include expert groups, seminars, studies, staff exchanges, project definition, etc
  • Can cover up to 100% of eligible costs, …

Specific Support Actions

  • For conferences, working groups, fact finding, technology transfer, operational support and dissemination
  • Can cover up to 100% of eligible costs, and be carried out by one participant
  • Call for Grants: conferences, workshops or exhibitions
article 169
Article 169
  • EC funding to support jointly executed member state programmes
    • most powerful tool for integration
  • However, long-winded implementation:
    • to generate a proposal requires a co-initiative by national programmes and the Commission
    • Council and Parliament would have to approve


Within Multimodal Interface


Cognitive systems

expected outcome calls 1 and 2
Multimodal Interfaces and Cognitive Systems

2/3rd of funds devoted new instruments

6-10 new instruments

8-12 old instruments










Expected OutcomeCalls 1 and 2
interface technology tools
Interface Technology Tools
  • IPsare expected to address the objectives within a holistic approach enabling, where justified, competition within and across projects.
  • NoEsshould aim at lowering barriers between hitherto split communities and disciplines and advance knowledge in the field. They should help establish and reinforce shared infrastructures, including for training andevaluation, annotation standardsand appropriateusability metrics and benchmarks.
  • STREPsare expected to bootstrap research in identifiable or emerging sub-domains and to prepare associated communities.
cognitive system tools
Cognitive System tools
  • Integrated Projectsare expected to leverage these communities to integrate methods and insights towards the objective of realizing entire systems and topromote community building.
  • NoEswill provide a channel forfostering foundational research, for developing and maintaining common resources, specifically, ofopen systemsandtraining environmentsto study learning and evolving systems.
analysis of submitted eois 14 evenly populated clusters
Infrastructures, resources and basic technologies

Interface and Artefact Design and Ergonomic, Human Factors and User Acceptance

Cognitive Science for Ambient Intelligence

Improvement of Human Senses and Perception

Speech and Vision Based Intelligent Interfaces

Interfaces Based on Recognition and Synthesis of Emotion, Facial Expression and Gesture

Multilingual Access and Dialogue

Intelligent Wearable Devices and Smart Cloths

Interfaces for Mobile Devices and Services

Intelligent Rooms with Support for Collaborative Creative Processes and Virtual Environments

Multisensory interfacing with complex information spaces and virtual manipulation of information objects

Virtual Reality and Environments

Critical Interfaces for Safer Control of Vehicles and Devices, including Training for these Environments

Physical Interfaces, Haptics, Mechatronics and Robotics

Analysis of submitted EOIs14 ~evenly populated clusters
analysis of submitted eois
150+ EoIs in Interface technologies (incl. Cognitive Systems)

70% IPs, 30% NoEs

60% of submitted IPs “defective”

lack of required ambition, scope and integration

some could be part of other IPs or a STRP

30% of the submitted NoEs “defective”

lack the main characteristics of this instrument

e.g. joint activity programme, lasting integration of existing research capacity

some more appropriate as coordination actions

Analysis of Submitted EOIs
eoi shortcomings
EoI Shortcomings
  • Content
    • Narrow scope, little or no EU and ERA dimension
    • Aims too general, lack of focus (e.g. programmes instead of projects)
    • Lack of innovation, not beyond current state of art
  • Planning
    • Links missing between objectives & workplan
    • Milestones missing or too general
    • Risk factors not addressed, no contingency plans
  • Management
    • Management structure vague or inadequate
    • Weak dissemination, exploitation plans
how to partnership
Highly competent and reliable partners

Complementarity: cover all areas you need

Duplication of competence:

acceptable for IPs depending on project needs

necessary for NoEs

At least 3 of which 2 EU/NAS or associated states

Ratio between Industry/SME/NAS/Academia participation depending solely on project needs

How To: Partnership
do s and don ts
Do carefully study the evaluation criteria

relevance, excellence, ……

Do respond to all the evaluation criteria, not just the scientific & technical ones

Do remember that proposals are evaluated by peers = “no-nonsense”

Don’t mirror the workprogramme: put your own ideas in the proposal

Don’t participate in a multitude of competing proposals, select one and make it win

Do’s and Don’ts
do s and don ts1
Do pay extra attention to using the full range of activities allowable for the new instruments

Don’tcreate artificial IPs (=inflated STREPs)

Do pay due attention to co-ordination of large projects

Do give realistic cost/resource estimates good return on investment

Rememberthat50 MEUR ~ 1,200 person/years!

Do’s and Don’ts
  • FP6 is about more ambitious endeavours, in a simpler procedural environment
  • IPs should aim at breakthroughs in the target area and yield new technologies, products and services
  • NoEs are to structure and shape research areas (interdisciplinary !)
  • STREP’s are more limited in scope but must go well beyond the state of the art
ist infopack
IST Infopack
  • IST Workprogramme 2003-2004
  • Call for proposals text Call 1 (Official Journal)
    • Submission details (deadline, address...)
  • Brochure “The 6FP in Brief”
    • FP6 and Specific Programmes, instruments, proposal to contract (flowchart), contractual matters, etc
  • Guides for proposers Call 1
    • How to submit, references,..., proposal forms
    • Instrument-specific annexes
  • Evaluation Manual
    • “High level” description of ground rules for evaluation
  • Guidelines for Evaluators Call 1
    • Evaluation process, evaluation forms
proposal content
Proposal content

Part A

  • Form A1 - General information
  • Form A2 - Information on participants
  • Form A3 - Cost breakdown (cost and grant requested as per activity / participant)

Part B

  • Aspects relating to evaluation criteria
  • Other aspects (ethics, safety, gender)
  • Overall workplan of project

IPs, NOEs additionally provide

  • 18 month Implementation plan/JPA (detailed)

No anonymity

Mats Ljungqvist /











  • Process
    • Peer-review system (many experts needed !)
    • 2-step evaluation (not in Call 1)
    • Hearings of applicants (likely in Call 1)
  • Evaluation & selection criteria

Higher thresholds on partnership and management.

Higher weights on impact and excellence.

Mats Ljungqvist /

stay in touch
Stay in Touch!
  • email: [email protected]


contact info
Contact info

Express Courier & visitors:

European Commission

EUFO 0170

Euroforum, Building,

10, rue Robert Stumper

L-2557 Luxembourg

Official address:

European Commission,

EUFO 0170

Jean Monnet Building,

Rue Alcide de Gasperi,

L-2920 Luxembourg.

Tel.: +352 - 4301 - 33423,

Sec.: +352 - 4301 - 34178.

Fax : +352 - 4301 - 34999

e-mail: [email protected]