3d mobile internet virtual operator and 3d user interaction
Download
Skip this Video
Download Presentation
3D Mobile Internet Virtual Operator and 3D User Interaction

Loading in 2 Seconds...

play fullscreen
1 / 20

3D Mobile Internet Virtual Operator and 3D User Interaction - PowerPoint PPT Presentation


  • 133 Views
  • Uploaded on

3D Mobile Internet Virtual Operator and 3D User Interaction . Lukas Kencl ( Director ) [email protected] Jiří Danihelka (Project leader ) [email protected] Roman Hák (Expert developer) Ondřej Šindelář (Junior developer) Jan Kadlec (Junior developer)

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about '3D Mobile Internet Virtual Operator and 3D User Interaction' - hayden


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
3d mobile internet virtual operator and 3d user interaction

3D Mobile InternetVirtual Operator and 3D User Interaction

Lukas Kencl (Director)[email protected]ří Danihelka (Project leader)[email protected]

Roman Hák (Expert developer)

Ondřej Šindelář (Junior developer)

Jan Kadlec (Junior developer)

R&D Center for Mobile Applications (RDC)

FEE, Czech Technical University, Prague

2009

project group 3d mobile internet
Project group:3D Mobile Internet

Virtual Customer Care

3D Mobile Internet, Danihelka, Hák, Šindelář, Kadlec, Kencl

virtual customer care center virtual store of vodafone
Virtual Customer Care Centervirtual store of Vodafone
  • user can roam around the centre, browse catalogue and products, watch adverts,
  • avatars talk, provide assistance, offer and sell services and products
  • platform independent - VRML browser
  • desktop PC app, web browser or mobile phone
  • written Virtual Reality Makeup Language
  • VRML is similar to X3D
  • models were inspired by real Vodafone stores
  • customers can choose mobile phones, see popular advertisements and many other things
  • Link: http://internet3d.rdc.cz/vodafone/index.php
  • VRML browser is required (see next slide)

3D Mobile Internet, Danihelka, Hák, Šindelář, Kadlec, Kencl

how to download a vrml browser
How to download a VRML browser
  • Download Cortona VRML browser from:http://www.parallelgraphics.com/products/downloads
  • Cortona3D Viewer 6.0 (recomended)works with Firefox and Internet Explorer
  • Cortona VRML Client 5.1works with Internet Explorer only
  • Pocket Cortonafor Windows Mobile devices

3D Mobile Internet, Danihelka, Hák, Šindelář, Kadlec, Kencl

virtual customer care center on mobile phones
Virtual Customer Care Centeron mobile phones
  • uses Pocket Cortona VRML browser
  • only minor changes from desktop version(smaller textures)
  • some functionality is not available(no video-textures)
  • user still can move in 3D environment, interact with objects,
  • Commercialization: virtual customer-care platforms for 3rd parties (banks, shops)
  • Natural, multi-modal interaction – breaks tech barriers
  • Opportunity: novel means of access to Web resources

3D Mobile Internet, Danihelka, Hák, Šindelář, Kadlec, Kencl

project group 3d mobile internet1
Project group:3D Mobile Internet

Virtual RDC

3D Mobile Internet, Danihelka, Hák, Šindelář, Kadlec, Kencl

virtual rdc 3d model of our research center on web
Virtual RDC3D model of our research center on web
  • accurate and interactive model of RDC laboratory
  • uses the same technology asVirtual Customer Care project
  • visitors can see our GSM network devices, server room, seminar room and many other things
  • Link: http://internet3d.rdc.cz/VirtualRDC/index.en.html
  • VRML browser is required

3D Mobile Internet, Danihelka, Hák, Šindelář, Kadlec, Kencl

project group 3d mobile internet2
Project group:3D Mobile Internet

Foreground Extraction on Mobile

3D Mobile Internet, Danihelka, Hák, Šindelář, Kadlec, Kencl

object 3d modeling input image foreground extraction on mobile
Object 3D-Modeling Input:Image Foreground Extraction on Mobile

9

3D Mobile Internet, Danihelka, Hák, Šindelář, Kadlec, Kencl

  • Rapid interactive and iterative image segmentation
  • Working implementation of GrabCut – state-of-the-art high-performance algorithm (Rother, Kolmogorov, Blake - Microsoft Research Cambridge - Grabcut: Interactive foreground extraction using iterated graph cuts. SIGGRAPH, 2004.)
  • Windows-Mobile based prototype implementation - feasibility
  • Essential for 3D object modeling input
project group 3d mobile internet3
Project group:3D Mobile Internet

3D Mobile Talking Head

3D Mobile Internet, Danihelka, Hák, Šindelář, Kadlec, Kencl

talking h ead
Talking Head
  • Speaking avatar - weather forecast, telling stories- voice synthesis, not interactive
  • Built on research results from the Department of Computer Graphics and Interaction (formerly CGG), CTU, in collaboration with IBM Research
  • RDC intensively cooperates with this group
  • Added platform independent architecture and mobile support

3D Mobile Internet, Danihelka, Hák, Šindelář, Kadlec, Kencl

3d virtual assistant
3D Virtual Assistant

Cross-platform Architecture

GL Benchmark, HTC Touch Pro, Nov 2008

12

12

3D Mobile Internet, Danihelka, Hák, Šindelář, Kadlec, Kencl

Presentation title in footer

Presentation title in footer

07 June 2014

07 June 2014

  • Goal: interactive automated 3D Virtual Assistant on mobile and fixed client
  • architecture: cross-platform solution
    • Qt library with graphic interface
    • rendering using OpenGL ES on top of Qt
  • interactive voice using SIP (PJSIP on mobile)
  • RT face rendering OK - current phone: up to 7000 triangles at 15 FPS with fog or point-light
  • demo Jan 2009: graphics feasibility prototype
3d mobile internet talking with an avatar on a mobile phone
3D Mobile InternetTalking with an Avatar on a Mobile Phone

Avatar enhanced application on a mobile phone

  • platform independent application design
  • can run either on desktop PC or mobile phone

Virtual worlds with talking avatar

  • Virtual Operator
  • Virtual Shop Assistant

Speech animation

  • 3D graphics rendering
  • using MPEG4 head animation
  • able to render up to 9000 polygons model

Interface for user interaction

  • voice interface for natural speech
  • classical button interface for noisy environment

3D Mobile Internet, Danihelka, Hák, Šindelář, Kadlec, Kencl

architecture of the system
Architecture of thesystem

Client

  • our system uses client-server distributed architecture
  • application logic is stored on a remote server
  • server generates voice using Text-to-speech engine (TTS)
  • then for corresponding head animation Facial Animation Parameters (FAP) are generated in FAP Generator
  • stream of the voice and the FAP stream are put together in MPEG4 Encoder
  • mobile client decodes the stream and plays the voice and the animation

Face (MPEG4-compillant)

Geometry

FAP stream

Audio stream

MPEG4-Decoder

MPEG4Stream

User action

Server

FAP stream

Animation commands

FAP Generator

MPEG-4 Encoder

Speech commands

Audio stream

TTS Engine

Application server

3D Mobile Internet, Danihelka, Hák, Šindelář, Kadlec, Kencl

creation of head models
Creation of head models
  • Tools for MPEG4 compliant heads
  • Set animation points on model
  • Important points:- center of the eyes- top of the nose- edges of the lips- ...
  • Points are used for meshchanges during animation
  • Uses only very small bandwidth

3D Mobile Internet, Danihelka, Hák, Šindelář, Kadlec, Kencl

project group 3d mobile internet4
Project group:3D Mobile Internet

Future plans

3D Mobile Internet, Danihelka, Hák, Šindelář, Kadlec, Kencl

3d mobile internet further plans
3D Mobile Internet – further plans
  • SIP Client for Virtual Operator to Voice2Web platform

Voice2Web

SIP Client

3D Mobile Internet, Danihelka, Hák, Šindelář, Kadlec, Kencl

3d mobile internet further plans1
3D Mobile Internet – further plans

Make interconnection of the projects

Talking Head Automated Call Centernon-interactive (e.g. weather forecast) non-visual interface

3D Interactive virtual operator for mobiles and PCs

3D Mobile Internet, Danihelka, Hák, Šindelář, Kadlec, Kencl

conclusion
Conclusion

R&D Center for Mobile Applications

3D Mobile Internet, Danihelka, Hák, Šindelář, Kadlec, Kencl

thank you do you see a synergy
Thank you! Do you see a synergy?

For questions please contact:

Dr. Lukas Kencl, Director

R&D Centre (RDC) for Mobile Applications

Czech Technical University

Technicka 2

166 27 Prague 6

Czech Republic

Tel.: +420-224355991

Fax: +420-224355999

[email protected]

See: http://www.rdc.cz

Or better come in person?

3D Mobile Internet, Danihelka, Hák, Šindelář, Kadlec, Kencl

ad