Telemorph teleturas bandwidth determined mobile multimodal presentation
1 / 15

TeleMorph & TeleTuras: Bandwidth determined Mobile MultiModal Presentation - PowerPoint PPT Presentation

  • Uploaded on

TeleMorph & TeleTuras: Bandwidth determined Mobile MultiModal Presentation. Student: Anthony J. Solon Supervisors: Prof. Paul Mc Kevitt Kevin Curran School of Computing and Intelligent Systems, Faculty of Informatics, University of Ulster, Magee. Objectives of Research.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'TeleMorph & TeleTuras: Bandwidth determined Mobile MultiModal Presentation' - morley

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Telemorph teleturas bandwidth determined mobile multimodal presentation

TeleMorph & TeleTuras:Bandwidth determined Mobile MultiModal Presentation

Student: Anthony J. Solon

Supervisors: Prof. Paul Mc Kevitt

Kevin Curran

School of Computing and Intelligent Systems,

Faculty of Informatics,

University of Ulster, Magee.

Objectives of research
Objectives of Research

  • To develop a system, TeleMorph, that dynamically morphs between output modalities depending on available network bandwidth:

    • Wireless system’s output presentation (unimodal/multimodal) depending on the network bandwidth available

    • Implement TeleTuras, a tourist information guide for the city of Derry

    • Receive and interpret questions from the user

    • Map questions to multimodal semantic representation

    • Match multimodal representation to database to retrieve answer

    • Map answers to multimodal semantic representation

    • Query bandwidth status

    • Generate multimodal presentation based on bandwidth data

Wireless telecommunications
Wireless Telecommunications

  • Generations of Mobile networks:

    • 1G - Analog voice service with no data services

    • 2G - Circuit-based, digital networks, capable of data transmission speeds averaging around 9.6K bps

    • 2.5G (GPRS) - Technology upgrades to 2G, boosting data transmission speeds to around 56K bps. Allows packet based “always on” connectivity

    • 3G (UMTS) - digital multimedia, different infrastructure required, data transmission speeds from 144K-384K-2M bps

  • Positioning Systems:

    • GPS

    • DGPS


    • GSM

Mobile intelligent multimedia systems
Mobile Intelligent MultiMedia Systems

  • SmartKom

    • Mobile, Public, Home/office

    • Saarbrucken, Germany

    • Combines speech, gesture and facial expressions on input & output

    • Integrated trip planning, Internet access, communication applications, personal organising

  • VoiceLog

    • BBN technologies in Cambridge, Massachusettes

    • Views/diagrams of military vehicles and direct connection to support

    • Damage identified & ordering of parts using diagrams

  • MUST

    • MUltimodal multilingual information Services for small mobile Terminals

    • EURESCOM, Heidelberg, Germany

    • Future multimodal and multilingual services on mobile networks

Intelligent multimedia presentation
Intelligent MultiMedia Presentation

  • Flexibly generate various presentations to meet individual requirements of:

    1) users, 2) situations, 3) domains

  • Fine-grained coordination of communication media and modalities

  • Key research problems:

    • Semantic Representation

    • Fusion, integration & coordination

    • Synchronisation

  • Semantic representation - represents semantics

    • Frame-based representations:


      - REA

    • XML-based representations:

      - SmartKom

      - MUST

  • Fusion, integration & coordination of modalities

    • Integrating different media in a consistent and coherent manner

    • Multimedia coordination leads to effective integrated multiple media in output

  • Synchronisation of modalities

    • Time threshold between modalities

      E.g. Input - “What building is this?”, Output - “This is the Millenium forum”

    • Not synchronised => side effect is contradiction

Intelligent multimedia presentation systems
Intelligent MultiMedia Presentation Systems

  • Automatically generate coordinated intelligent multimedia presentations

  • User-determined presentation


    • COordinated Multimedia Explanation Testbed

    • Generates instructions for maintenance and repair of military radio receiver-transmitters

    • Coordinates text and 3D graphics of mechanical devices

  • WIP

    • Intelligent multimedia authoring system

    • presents instructions for assembling/using/maintaining/repairing devices (e.g. espresso machines, lawn mowers, modems)


    • Graphics generation system

    • constructive/parameterised graphics generation approaches

    • Uses an extensible formalism to represent a visual lexicon for graphics generation

Intelligent multimedia interfaces agents
Intelligent MultiMedia Interfaces & Agents

  • Intelligent multimedia interfaces

    • Parse integrated input and generate coordinated output


      • Calspan-UB Research center Intelligent CONversationalist

      • Air Force Command and Control

      • Generates & recognises Speech; natural language text; displays graphics; interprets gestures made with a pointing device

  • Intelligent multimedia agents

    • Embodied Conversational Agents

    • Natural human communication - speech, facial expressions, hand gestures,

      & body stance


      • COLLaborative AGENt

      • object-oriented Java middleware for building collaborative interface agents

      • MIT Media Laboratory work on embodied conversational agents

Project proposal
Project Proposal

  • Research and implement a mobile intelligent multimedia presentation system called TeleMorph

  • Dynamically generates a multimedia presentation determined by the bandwidth available

  • TeleTuras tourist navigation aid providing testbed for TeleMorph incorporating:

    • route planning, maps, spoken presentations, graphics of points of interest and animations

    • Output modalities used

    • Effectiveness of communication

  • TeleTuras examples:

    • “Where is the Millenium forum?”

    • “Take me to the GuildHall”

    • “What buildings are of interest in this area?”

    • “Is there a Chinese restaurant in this area?”

Prospective tools
Prospective Tools

  • Development language - J2ME (Java 2 Micro Edition)

  • Speech input/output - Java Speech API – IBM’s implementation of JSAPI “speech for Java”

    • US & UK English, French, German, Italian, Spanish, and Japanese

    • Java Speech API Markup Language (JSML)

    • Java Speech API Grammar Format (JSGF)

  • Positioning system - GPS (Global Positioning System) provides the accurate location information necessary for a LBS (Location Based Service)

  • Graphics input/output - The User Interface (UI) defined in J2ME is logically composed of two sets of APIs:

    • Low-level UI API

    • High-level UI API


  • A Mobile Intelligent MultiModal presentation System called TeleMorph will be developed

  • Dynamically morphing between output modalities depending on available network bandwidth

    • Bandwidth and Device determined Mobile MultiModal presentation

  • TeleTuras will be used as a testbed for TeleMorph

  • Corpora of questions to test TeleTuras (prospective users/tourists)