slide1 n.
Skip this Video
Download Presentation
W3C Web Technology Day

Loading in 2 Seconds...

play fullscreen
1 / 12

W3C Web Technology Day - PowerPoint PPT Presentation

  • Uploaded on

Signal Processing for Multimodal Web Irek Defée Department of Signal Processing Tampere University of Technology. W3C Web Technology Day. Current status. Web is developed for traditional data and computer I/O: text, keyboard, mouse This is simple and effective but not a natural

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'W3C Web Technology Day' - kele

Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Signal Processing for Multimodal WebIrekDefée Department of Signal Processing Tampere University of Technology

W3C Web Technology Day

current status
Current status
  • Web is developed for traditional data and computer I/O: text, keyboard, mouse
  • This is simple and effective but not a natural

way of human interaction with the world

  • Humans interact via

perceptual system

human perceptual system
Human Perceptual System
  • Human perceptual system has multiple senses: visual, acoustical, haptic

(touch, body position, temperature)

and actuators (vocal tract, muscles, motoric system)

  • The perceptual system is intrinsically MULTIMODAL: multiple senses and actuators operate in perfectly coordinated way
perceptual information technology
Perceptual Information Technology
  • Information technology is evolving towards natural MULTIMODAL human interaction:
  • Touch gestures

revolutionized mobile devices

  • Intelligent speech input

is available

  • There is more to come:

new sensors, cameras

and intelligence

signal processing role
Signal Processing Role
  • Perceptual Information Technology requires sophisticated signal processing

and it is hard due to:

- Complex input signals

- Complex information encoding

- Complex databases of knowledge

  • Highly sophisticated algorithms and

huge processing power are required

multimodal web
Multimodal Web
  • The trend towards perceptual information is noted at the W3C:

Extending the Web to allow multiple modes of interaction: GUI, Speech, Vision, Pen, Gestures, Haptic interfaces, ...

  • Multimodal Interaction Activity:

- Multimodal Architecture and Interfaces


- InkML

- EmotionML

  • Extensible Multimodal Markup Language for Annotations

- containing and annotating the interpretation

of user input

- transcription into words of a raw signal, for

instance derived from speech, pen

- interpretation is to be generated by signal

interpretation processes, such as speech and ink

recognition, semantic interpreters

ink markup language
Ink Markup Language
  • data format for representing ink
  • input and processing of handwriting,

gestures, sketches, music using

traces of pen



emotion markup language
Emotion Markup Language
  • Annotation of material involving emotionality
  • Automatic recognition of emotions from sensors
  • Generation of emotion-related system responses: speech, music, colors, gestures, synthetic faces
  • Emotion vocabularies and representations:

<emotion category-


voc/xml#big6"> <category name="surprise"

confidence="0.9 </emotion>

department of signal processing
Department of Signal Processing
  • Signal processing has a key role as a front-end for the Multimodal Web
  • Department is on the forefront of research

in the natural information processing:

- Multimedia information analysis, retrieval and


- Audio information analysis : speech and


- Media information handling: representation and