ui gaming in mobile handsets l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
UI & Gaming in Mobile Handsets PowerPoint Presentation
Download Presentation
UI & Gaming in Mobile Handsets

Loading in 2 Seconds...

play fullscreen
1 / 24

UI & Gaming in Mobile Handsets - PowerPoint PPT Presentation


  • 1532 Views
  • Uploaded on

UI & Gaming in Mobile Handsets Yoshio Kuniyoshi Vice President & CTO Acrodea Inc. Outline of the Presentation Background

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'UI & Gaming in Mobile Handsets' - adamdaniel


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
ui gaming in mobile handsets

UI & Gamingin Mobile Handsets

Yoshio KuniyoshiVice President & CTO Acrodea Inc.

outline of the presentation
Outline of the Presentation
  • Background
    • Recently, many handsets become equipped with GPU. However the purpose of GPU is mainly for Gaming only. On the other hand, user interface is coming to need more and more graphics processing power.
  • VIVID UI
    • As a state of the art UI platform for mobile devices, the features of VIVID UI is introduced to describe how today’s UI platform for mobile handsets needs graphics power.
    • Describe how current VIVID UI uses GPU.
  • Roadmap for Accelerating User Interface
    • Describe the ideal way of accelerating user interface.
    • Describe problems of current GPU and its API implementation.
  • Gaming
    • As a state of the art Game platform for mobile devices, the features of X-Forge is described.
    • Describe programmable shader support for X-Forge.
introduction
Introduction
  • UI and Gaming trends in Mobile Handsets
    • More and More Graphics Processing Power is Required
      • Display Resolution Gets HigherQCIF → QVGA→ WQVGA→ VGAQCIF : VGA = 1 : 12
      • Demands for Richer Media SupportFlash, SVG, PDF, 3D, Movie(MP4 → H.264 quality)
      • Demands for Advanced 3D Technologies3D Effects makes user interface become more attractive Programmable shaders makes more realistic 3D games
    • Recently, many handsets become equipped with GPU.
    • Example
      • UI Platform for PC
        • MacOS X Quartz Extreme
        • Windows Vista Avalon
      • Handset
        • 3D Menu (SHARP V602SH)
        • VGA Display (SHARP 904SH)
        • Flash Lite 1.1 and 2.0 (Nokia, Sony Ericsson, and Most of Japanese Phones for au/DoCoMo/Vodafone)
        • GPU ProcessorsNVIDIA GoForce, IMG PowerVR MBX, TOSHIBA T5G, ATI Imageon, …All these GPUs will soon/already support programmable shaders and OpenVG.
slide5

Introduction

Next-generation UI architecture framework:

UI architecture based on scripting and OTA differential updating

vivid ui features
VIVID UI Features
  • Scripting Based UI Platform --- UI is Data not CodeEasy to create arbitrary UI structure with rich visual expression through scripting. No need to write native codes. This “UI is Data” concept enables easy provisioning of UI content service by OTA.
  • Development with GUI Authoring ToolsGUI based WYSIWYG operations result in automatic script generation.
  • Separate Management of UI from LogicLook and feel, and application logic managed separately.Improves collaboration between designers and programmers.
  • Provides of Standard UI ComponentsProvides buttons, labels, checkboxes, radioboxes, textfields and so on. Other components can be easily added.
  • Enables Any Kind of Media Composition Designed as a container for any kind of graphics renderer. Various materials including 2D images, Flash, 3D, and Movies can be freely composed by alpha blending.
vivid ui requirements complete ui solution for today and tomorrows devices

Supported platforms

    • OS : Brew / Linux / Symbian / RTOS
    • CPU : ARM7 30MHz or later
    • TOOL : Windows / Macintosh (future plan)
  • ROM / RAM size
    • ROM : between 100KB and 500KB (depends on components)
    • RAM : between 200KB and more (depends on contents)
VIVID UI RequirementsComplete UI solution for today and tomorrows devices
vivid ui contents development and distribution flow

UI Concept

Planning

UI Materials

design

Authoring

with tools

VividUI Workshop

Packaging

Terminal

embedding

OTA delivery

Placed in VividUIdistribution server

VIVID UI Contents Development and Distribution Flow

Package all data such as XML scripts (compiled form), and resource files into one file

Handset with

VividUI engine

achieving diverse ui
Achieving Diverse UI

Instead of embedding a fully designed UI, loading a UI platform environment is a much more desirable direction for handset manufacturers. Then subscribers can select their desired UI from among a variety of options to suit their preferences.

By responding to diversified needs, a whole new “UI epoch” will begin.

vivid ui multi media support 2d flash 3d and movies
VIVID UI Multi Media Support2D, Flash, 3D, and Movies
  • VIVID UI can integrates any kind of graphics renderer on a handset including 2D images, Flash, 3D, and Movies.
  • Various materials including 2D images, Flash, 3D, and Movies can be freely composed by alpha blending.
  • VIVID UI has built-in 2D renderer, 3D scene graph engine based on OpenGL|ES, and interface to Flash and several movie decoders.

3D Animated Menu Icons

Flash Animation

2D Images

2D Menu on top of movie.

vivid ui workshop
VIVID UI Workshop
  • VIVID UI Workshop is an authoring tool to facilitate user interface development with VIVID UI.
  • This GUI tool enables WYSIWYG based editing of UI structure. UI designers can create UI by just putting components on the design pane of the tool then setting several properties for each component and the tool automatically generates corresponding XML scripts on the fly.
  • UI designer can verify the behavior and appearance of the UI content with built-in emulator.
  • Handset profile feature provides easy adoption of different handset models.
  • Integrates all operations needed for UI content creation.

UI Design Tool

client architecture

Plug-ins provide extensibility, and are accessible through scripting

Handset menus developed by VUL scripting

Skins can be open to the end users, who can freely customize appearances.

VUL scripts are interpreted and then drive the VividUI engine.

Yellow frame indicates VIVID UI Platform

VU API can be used directly for UI app implementation.

User Customizable Skin

Folder Menu Script

Top Menu Script

Setting Menu Script

Camera Menu Script

The abstraction layer for various native functions.

VU

app

VU

app

VUL interpreter

VU

plug-in

VividUI API

Plug-in API

VividUI engine

VU

plug-in

Key events and handset UI functions such as screen drawing are accessible through scripting also.

Native Abstraction Layer

Scripts can launch various embedded programs.

Handset

Platform

API

Flash

3D

OES

2D

SVG-T

Video

AMC

Audio

MP3

App launcher

Client Architecture
ui scripting architecture
UI Scripting Architecture
  • UI architecture through scripting
    • VividUI builds a resource file UI composed of XML files, simple scripting language, and images.
  • Separated UI components
    • UI components are written asContents: UI structureSkin: UI appearanceLocale: UI localization
  • Modular structure
    • Overlaying feature enables UI sharing and componentizing
  • Presentation script feature
    • Style sheet definitions enable pixel-level precise control.

VUL* script 3 components

Contents

Structural scripts such as menus

Action scripts

Skins

Style scripts

Sound and image resources

Locales

Localized language data

*VividUI Scripting Language.

embedded advanced functions
Embedded Advanced Functions
  • Embedded animation effect
    • Can be used to impart attractive motion on the UI.
  • Push and pull communications
    • Adapts to server “push” communications for timely data delivery to handsets.
  • Abstraction with rendering engines
    • 3D/Movie/Flash can be accessed in concert with native rendering engine by scripting.
  • Abstraction with sound functions
    • MP3 and other native sound functions can be accessed by scripting.
  • Abstraction with status displays
    • Allows acquisition of status through scripting (signal level, battery life)

News and

real-time stock prices through “push” comm.

Innovative UI possible with 3D display

Animation generates motion effect

VividUI Engine

Unique design and placement of status indicators

Introduction of realistic sound

vividui other features
VIVIDUI Other Features
  • Support multi-task environment
    • re-entrant thread safe architecture / passive library
  • Effective compression for data
    • LZ and Huffman based compression / optimization for re-use of resource files
  • Lightweight alpha blending technology
    • optimized for mobile environment / anti-aliasing feature
  • Easy to expand UI components
    • support 2D vector graphics (FLASH) / 3D graphics (OpenGL-ES)
    • component plug-in API enables to add other UI components
    • renderer plug-in API enables to support other graphics rendering engines
  • Support hardware acceleration
    • can be optimized for CPU – ARM9/ARM11 / HITACHI SH-Mobile
    • can be optimized for GPU – OpenGL-ES compatible HW
  • Easy to port
    • simple porting layer (30 – 70 functions: depends on the requirements)
    • customizable porting layer
  • Support OTA differential update
    • can be optimized for OTA differential technology
    • can share porting layer with OTA differential technology
vivid ui oes implementation
VIVID UI OES Implementation
  • Current VIVID UI OpenGL ES Implementation
    • Supports OpenGL|ES 1.0 and 1.1
    • Full OpenGL|ES acceleration for built-in 2D renderer
    • Full OpenGL|ES acceleration for 3D scene graph engine
    • Movie is supported by software decoder
    • Flash Lite is supported (w/ Macromedia Flash Lite library)
vivid ui acceleration roadmap
VIVID UI Acceleration Roadmap
  • The Ideal way of VIVID UI Accelerated Implementation
    • Supports OpenGL|ES 1.0 and higher, and/or OpenVG
    • Full OpenGL|ES and/or OpenVG acceleration for built-in 2D renderer
    • Full OpenGL|ES acceleration for 3D scene graph engine
    • Movie is supported by hardware decoder w/ OpenMAX
    • Accelerated version of Flash Lite is supported
  • Problems of current GPU and its API implementation
    • Requirements
      • Multi context support with independent surface for multi processes.
      • Multi context support with shared surface for single processes(single/multi threads).
      • Shared surface support between OpenGL|ES, OpenVG, and OpenMAX.
      • Resource management
      • Fine grained power management
    • Current GPU and API Implementation
      • Some environments does not support multi context.
      • Some environments only support multi context with independent surface, but no support for shared surface.
      • Some environments does not support resource management. E.g.: The upper limit of the texture area is the same as GPU hardware limitation.
      • No/Rare existing support for OpenMAX and OpenVG w/ shared surface support w/ OpenGL|ES.
      • No support for fine grained power management from the application.
the x forge tool chain
The X-Forge™ Tool Chain

CODE

1. 2D and 3D Content

2. Particle Effects

Executable

(EXE)

PC Emulator Version

Compressed File Library (CFL)

CONTENT

3. Music and Sound Effects

4. Scripts

Native Device Version

the x forge sdk
The X-Forge™ SDK

Additional X-Forge™ Components

Your Game Content

Graphics

Audio

Scripts

Documentation

Stand-alone Tools

Your Game Code

Programmer Tools

Artist Tools

1000+ page API reference

Tutorials & sample code

Production & tools

Optimization guidelines

Packager

Game

Specific

Code

3ds Max™ Exporter

X-Forge Online

Application

framework

Profile Analyzer

Game Graph

UI Toolkit

X-Forge™

Game

Engine

Maya™ Exporter

Particle

FXsystem

Physics

framework

Collisions

Particle FX Editor

AI Framework

3D Audio

OS and Device Abstraction

Networking

Framework

Graphics

Audio

X-Forge™

Core

Resources

Controls

File I/O

UTF8 unicode

Debugging

tools

Localization

tools

XML

SVG

Mobile

Device

OS

Device Driver

OpenGL ES 1.x/2

Hardware

CPU 3D Graphics Accelerator

multi platform support
Multi-platform Support

Acrodea works in close co-operation with all leading mobile technology companies in order to support a wide variety of different mobile operating systems, processors, 3D graphics accelerators and devices in X-Forge.

  • Symbian OS™ / Nokia Series 60
    • Nokia N-Gage™
    • Nokia 3650, 7650, 6600…
    • Siemens SX1
    • Samsung SGH-D700
  • Symbian OS™ / UIQ
    • Sony Ericsson P800
  • Palm OS (V5.x)
    • Tapwave Zodiac™
  • Mobile Linux
    • Sharp Zaurus™
  • Microsoft Windows® CE.NET
    • AlphaCell M5
  • Microsoft Smartphone
    • Orange SPV™
  • Microsoft Pocket PC
    • ARM-based devices

X-Forge can also be ported to virtually any additional operating system and hardware.

  • X-Forge is Nokia OK certified and optimized for the Nokia N-Gage™ game deck
  • X-Forge has been optimized for leading processor architectures including Intel XScale™ / StrongARM™ and Texas Instruments OMAP™
  • X-Forge supports various 2D and 3D graphics accelerators, including products from ATI, nVIDIA and Intel
  • X-Forge is compatible with standard 3D rendering engines such as OpenGL ES™
existing x forge game content
Existing X-Forge™ Game Content

X-Forge™ Delivers Superior Quality

Geopod™

Gadarka’s Delusion

Snowboarding Extreme

Red Faction™

Red Faction™

Super Monkey Ball™

Virtua Tennis™

MotoGP™

Spy Hunter™

Galactic Realms™

Stuntcar Extreme™

Phantom Strike™

Unfinished Business™

Unfinished Business™

Firehammer™

Snowboarding Extreme

Gadarka’s Delusion

Resistance™

Gadarka’s Delusion

Geopod™

Legend of Zord™

Stuntcar Extreme™

Men in Black II™: Alien Pursuit

Spy Hunter™

Stuntcar Extreme™

Resistance™

developer network
Developer Network

This is a partial list of developers who have licensed / evaluated X-Forge™.

x forge programmable shader support
X-Forge Programmable Shader Support
  • OES Version
    • Support OpenGL|ES 2.0.
  • Effects
    • Supports “Effect” which consists of a shader, and accompanying data and rendering paths.(i.e.: FX file of DirectX)
  • Built-in Effects
    • Some effects are built-in (or pre-installed) with X-Forge SDK. Effects for glow, normal map, environment map, bump map will be prepared.
  • Treat Effect as a part of a scene graph
    • Assign effects to each model or the entire scene and then export.
    • These effects will be automatically processed at runtime.
  • Effect Editing Tool
    • Effect editor will be provided.
  • Support High Level API
    • Provides high level API to manipulate effects from C++ source code.