telemorph teleturas bandwidth determined mobile multimodal presentation
Skip this Video
Download Presentation
TeleMorph & TeleTuras: Bandwidth determined Mobile MultiModal Presentation

Loading in 2 Seconds...

play fullscreen
1 / 15

TeleMorph & TeleTuras: Bandwidth determined Mobile MultiModal Presentation - PowerPoint PPT Presentation

  • Uploaded on

TeleMorph & TeleTuras: Bandwidth determined Mobile MultiModal Presentation. Student: Anthony J. Solon Supervisors: Prof. Paul Mc Kevitt Kevin Curran School of Computing and Intelligent Systems, Faculty of Informatics, University of Ulster, Magee. Objectives of Research.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' TeleMorph & TeleTuras: Bandwidth determined Mobile MultiModal Presentation' - morley

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
telemorph teleturas bandwidth determined mobile multimodal presentation

TeleMorph & TeleTuras:Bandwidth determined Mobile MultiModal Presentation

Student: Anthony J. Solon

Supervisors: Prof. Paul Mc Kevitt

Kevin Curran

School of Computing and Intelligent Systems,

Faculty of Informatics,

University of Ulster, Magee.

objectives of research
Objectives of Research
  • To develop a system, TeleMorph, that dynamically morphs between output modalities depending on available network bandwidth:
    • Wireless system’s output presentation (unimodal/multimodal) depending on the network bandwidth available
    • Implement TeleTuras, a tourist information guide for the city of Derry
    • Receive and interpret questions from the user
    • Map questions to multimodal semantic representation
    • Match multimodal representation to database to retrieve answer
    • Map answers to multimodal semantic representation
    • Query bandwidth status
    • Generate multimodal presentation based on bandwidth data
wireless telecommunications
Wireless Telecommunications
  • Generations of Mobile networks:
    • 1G - Analog voice service with no data services
    • 2G - Circuit-based, digital networks, capable of data transmission speeds averaging around 9.6K bps
    • 2.5G (GPRS) - Technology upgrades to 2G, boosting data transmission speeds to around 56K bps. Allows packet based “always on” connectivity
    • 3G (UMTS) - digital multimedia, different infrastructure required, data transmission speeds from 144K-384K-2M bps
  • Positioning Systems:
    • GPS
    • DGPS
    • GSM
mobile intelligent multimedia systems
Mobile Intelligent MultiMedia Systems
  • SmartKom
    • Mobile, Public, Home/office
    • Saarbrucken, Germany
    • Combines speech, gesture and facial expressions on input & output
    • Integrated trip planning, Internet access, communication applications, personal organising
  • VoiceLog
    • BBN technologies in Cambridge, Massachusettes
    • Views/diagrams of military vehicles and direct connection to support
    • Damage identified & ordering of parts using diagrams
  • MUST
    • MUltimodal multilingual information Services for small mobile Terminals
    • EURESCOM, Heidelberg, Germany
    • Future multimodal and multilingual services on mobile networks
intelligent multimedia presentation
Intelligent MultiMedia Presentation
  • Flexibly generate various presentations to meet individual requirements of:

1) users, 2) situations, 3) domains

  • Fine-grained coordination of communication media and modalities
  • Key research problems:
    • Semantic Representation
    • Fusion, integration & coordination
    • Synchronisation
Semantic representation - represents semantics
    • Frame-based representations:



    • XML-based representations:

- SmartKom


  • Fusion, integration & coordination of modalities
    • Integrating different media in a consistent and coherent manner
    • Multimedia coordination leads to effective integrated multiple media in output
  • Synchronisation of modalities
    • Time threshold between modalities

E.g. Input - “What building is this?”, Output - “This is the Millenium forum”

    • Not synchronised => side effect is contradiction
intelligent multimedia presentation systems
Intelligent MultiMedia Presentation Systems
  • Automatically generate coordinated intelligent multimedia presentations
  • User-determined presentation
    • COordinated Multimedia Explanation Testbed
    • Generates instructions for maintenance and repair of military radio receiver-transmitters
    • Coordinates text and 3D graphics of mechanical devices
  • WIP
    • Intelligent multimedia authoring system
    • presents instructions for assembling/using/maintaining/repairing devices (e.g. espresso machines, lawn mowers, modems)
    • Graphics generation system
    • constructive/parameterised graphics generation approaches
    • Uses an extensible formalism to represent a visual lexicon for graphics generation
intelligent multimedia interfaces agents
Intelligent MultiMedia Interfaces & Agents
  • Intelligent multimedia interfaces
    • Parse integrated input and generate coordinated output
      • Calspan-UB Research center Intelligent CONversationalist
      • Air Force Command and Control
      • Generates & recognises Speech; natural language text; displays graphics; interprets gestures made with a pointing device
  • Intelligent multimedia agents
    • Embodied Conversational Agents
    • Natural human communication - speech, facial expressions, hand gestures,

& body stance

      • COLLaborative AGENt
      • object-oriented Java middleware for building collaborative interface agents
      • MIT Media Laboratory work on embodied conversational agents
project proposal
Project Proposal
  • Research and implement a mobile intelligent multimedia presentation system called TeleMorph
  • Dynamically generates a multimedia presentation determined by the bandwidth available
  • TeleTuras tourist navigation aid providing testbed for TeleMorph incorporating:
    • route planning, maps, spoken presentations, graphics of points of interest and animations
    • Output modalities used
    • Effectiveness of communication
  • TeleTuras examples:
    • “Where is the Millenium forum?”
    • “Take me to the GuildHall”
    • “What buildings are of interest in this area?”
    • “Is there a Chinese restaurant in this area?”
prospective tools
Prospective Tools
  • Development language - J2ME (Java 2 Micro Edition)
  • Speech input/output - Java Speech API – IBM’s implementation of JSAPI “speech for Java”
    • US & UK English, French, German, Italian, Spanish, and Japanese
    • Java Speech API Markup Language (JSML)
    • Java Speech API Grammar Format (JSGF)
  • Positioning system - GPS (Global Positioning System) provides the accurate location information necessary for a LBS (Location Based Service)
  • Graphics input/output - The User Interface (UI) defined in J2ME is logically composed of two sets of APIs:
    • Low-level UI API
    • High-level UI API
  • A Mobile Intelligent MultiModal presentation System called TeleMorph will be developed
  • Dynamically morphing between output modalities depending on available network bandwidth
    • Bandwidth and Device determined Mobile MultiModal presentation
  • TeleTuras will be used as a testbed for TeleMorph
  • Corpora of questions to test TeleTuras (prospective users/tourists)