Signal Processing for
Download
1 / 12

W3C Web Technology Day - PowerPoint PPT Presentation


  • 101 Views
  • Uploaded on

Signal Processing for Multimodal Web Irek Defée Department of Signal Processing Tampere University of Technology. W3C Web Technology Day. Current status. Web is developed for traditional data and computer I/O: text, keyboard, mouse This is simple and effective but not a natural

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' W3C Web Technology Day' - kele


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Signal Processing for Multimodal WebIrekDefée Department of Signal Processing Tampere University of Technology

W3C Web Technology Day


Current status
Current status

  • Web is developed for traditional data and computer I/O: text, keyboard, mouse

  • This is simple and effective but not a natural

    way of human interaction with the world

  • Humans interact via

    perceptual system


Human perceptual system
Human Perceptual System

  • Human perceptual system has multiple senses: visual, acoustical, haptic

    (touch, body position, temperature)

    and actuators (vocal tract, muscles, motoric system)

  • The perceptual system is intrinsically MULTIMODAL: multiple senses and actuators operate in perfectly coordinated way


Perceptual information technology
Perceptual Information Technology

  • Information technology is evolving towards natural MULTIMODAL human interaction:

  • Touch gestures

    revolutionized mobile devices

  • Intelligent speech input

    is available

  • There is more to come:

    new sensors, cameras

    and intelligence


Signal processing role
Signal Processing Role

  • Perceptual Information Technology requires sophisticated signal processing

    and it is hard due to:

    - Complex input signals

    - Complex information encoding

    - Complex databases of knowledge

  • Highly sophisticated algorithms and

    huge processing power are required


Multimodal web
Multimodal Web

  • The trend towards perceptual information is noted at the W3C:

    Extending the Web to allow multiple modes of interaction: GUI, Speech, Vision, Pen, Gestures, Haptic interfaces, ...

  • Multimodal Interaction Activity:

    - Multimodal Architecture and Interfaces

    - EMMA

    - InkML

    - EmotionML



EMMA

  • Extensible Multimodal Markup Language for Annotations

    - containing and annotating the interpretation

    of user input

    - transcription into words of a raw signal, for

    instance derived from speech, pen

    - interpretation is to be generated by signal

    interpretation processes, such as speech and ink

    recognition, semantic interpreters


Ink markup language
Ink Markup Language

  • data format for representing ink

  • input and processing of handwriting,

    gestures, sketches, music using

    traces of pen

    Trace

    attributes


Emotion markup language
Emotion Markup Language

  • Annotation of material involving emotionality

  • Automatic recognition of emotions from sensors

  • Generation of emotion-related system responses: speech, music, colors, gestures, synthetic faces

  • Emotion vocabularies and representations:

    <emotion category-

    set="http://www.w3.org/TR/emotion-

    voc/xml#big6"> <category name="surprise"

    confidence="0.9 </emotion>


Department of signal processing
Department of Signal Processing

  • Signal processing has a key role as a front-end for the Multimodal Web

  • Department is on the forefront of research

    in the natural information processing:

    - Multimedia information analysis, retrieval and

    databases

    - Audio information analysis : speech and

    music

    - Media information handling: representation and

    compression



ad