1 / 26

Java Video and Audio in Consumer Devices: JMF and MM API

Java Video and Audio in Consumer Devices: JMF and MM API. Pablo César and J. Luc Lamadon {pcesar, lamadon}@tml.hut.fi Helsinki University of Technology. Outline. Introduction Java Overview Personal Computer Digital Television Receiver Mobile Phone Conclusions References.

stacy-poole
Download Presentation

Java Video and Audio in Consumer Devices: JMF and MM API

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Java Video and Audio in Consumer Devices: JMF and MM API Pablo César and J. Luc Lamadon {pcesar, lamadon}@tml.hut.fi Helsinki University of Technology

  2. Outline • Introduction • Java Overview • Personal Computer • Digital Television Receiver • Mobile Phone • Conclusions • References

  3. Introduction (1/3) • What is multimedia? • Multiple media (e.g., audio, video, images, graphics, text) or multimedia objects • Synchronisation of multimedia elements • Interaction between user and system • Multimedia in Consumer Devices, user expectation • Latest mobile phones with camera, MMS, Internet connection, streaming • Digital TV based on audio visual stream + interactive services • PC with 3D games, videoconference, streaming

  4. Video objects in programming tools: Synchronized Markup Integration Language (SMIL) Flash presentations Java Media Framework (JMF) Stand alone video player: Proprietary players: Real One, Windows Media Player... Open Source players: winamp, MPlayer... Introduction (2/3) Audio and Video in Consumer Devices, two alternatives:

  5. Introduction (3/3) • The focus of this presentation is video objects in a programming tool • Specifically Java, why? • Consumer devices includes Java • Digital TV = Multimedia Home Platform (MHP) and JDK 1.1.8 • PCs = Java 2 Standard Edition • Mobile phones = Connected Limited Device Configuration (CLDC) + Information Device Profile (MIDP) • Independent of Operating System

  6. Java Overview (1/2)

  7. Implements the state machine Retrieves the actual media data Decodes and plays the media data Java Overview (2/2)

  8. Personal Computer (1/4) • Services: • Internet, videoconference • Games • … • Physical Characteristics: • Pointer device (e.g., mouse) and keyboard as major input mechanisms • Screen resolution: 640x480 to 1600x1200 pixels • Runtime memory: around 128 to 256 MB • Java Environment • Java 2 Standard Edition (J2SE) including Abstract Windowing Toolkit (AWT) and Swing

  9. Personal Computer (2/4) • JMF as an optional package either version 1.0 or 2.0 • JMF relies on the mentioned classes: Manager, Player, Data Source, and Controller. • Player behaviour: • Unrealised: when it does not have all the information to acquire the needed resources • Realised: when it has all the information to acquire the needed resources • Prefetched: when it has all the needed resources, and has already prefetched enough media data to start playing immediately • Started: when it is actually playing the media

  10. Personal Computer (3/4)

  11. Personal Computer (4/4) • Time model: Clock defines the basic timing and synchronisation operations, which control the presentation of media data (e.g. forward, reward) • Data model: DataSource encapsulates both the location and protocol of media • JMF 2.0 includes as well a low level API • A Processorperforms user-defined processing on the media data using JMF plug-ins (e.g. Codec, Demultiplexer, Effects, Multiplexer, Renderer) • Render is done in an AWT Component: • Player.getVisualComponent() • Player.getControlPanelComponent()

  12. Digital Television Receiver (1/5) • Multimedia platform in the living room • Services: • Audio visual stream (normal passive watching) • Interactive services (active behaviour) • Physical Characteristics • Remote control as major input mechanism • Screen resolution: 720x576 pixels (minimal) • Runtime memory: at least 16 MB • Java Environment • MHP

  13. Digital Television Receiver (2/5) • MHP includes JMF 1.0 (obviously, not as an optional package) with some clarifications, restrictions, and extensions • The basic classes are again: • Manager, Player, DataSource, and Controller • The player behaviour is the same as in JMF 1.0: • Unrealised, Realised, Prefetched, and Started • JMF can be used both for broadcast stream and local files (only in high-end receivers) • Video drips presents I-frame and consecutive P-frames (high quality interactive graphics)

  14. Digital Television Receiver (3/5) • Locators (reference to a particular piece of media) basically three main types: • javax.media.MediaLocator(and subclasses): media clip • org.davic.net.Locator(and subclasses): DVB transport stream or service • javax.tv.locator.Locator: any type of digital TV system • Time based (Clock) is useless in broadcast • Solution to synchronise = stream events • A JMF player cannot access directly services in other transport streams (unless explicitly done) • Includes new player events (e.g.,): • Media Presented: The media actually starts showing

  15. Digital Television Receiver (4/5) • Includes a number of specific digital TV controls: • Media Select: changes the actual stream presented (e.g., change the angle of a camera) • Language: intended to control the audio and subtitles (if present) language • Television display is composed of: • Background + Video + Graphics Layers • Video media is, normally, hardware decoded and demultiplexed. So, rendered in video layer (no AWT container) • getVisualContent() returns null • getControlPanelComponent() returns null

  16. Digital Television Receiver (5/5) • Still, video can be controlled: • BackgroundVideoPresentationControl: location and size • VideoFormatControl: picture format and aspect ratio (e.g., 4:3) • Maybe in the future (high-end receivers) a real integration of media in graphics layer can be obtained. • External audio clips can be played in JMF-MHP

  17. Mobile Phone (1/5) • Mobile phones are becoming powerful Multimedia platforms • Provide a number of services: • Internet (e.g., Nokia + Opera), MMS, Video Player • Office capabilities • Physical characteristics • Key Pad as major input mechanism • Screen resolution: 84x48 to 120x130 pixels • Runtime memory: 160 to 512 KB • Java Environment • Mobile Information Device Profile (MIDP) version 1.0 or 2.0

  18. Mobile Phone (2/5) • Mobile Multimedia API (MMAPI) version 1.0 • Latest release was on 26th June, 2003 http://java.sun.com/products/mmapi/ • MMAPI Description: • It extends MIDP functionality by providing audio, video and other time-based multimedia support • It is a thin Java layer completely platform dependent • It is not JMF • It is an optional package • MIDP 2.0 includes the audio-only subset from MMAPI (i.e. Audio Building Block)

  19. Mobile Phone (3/5) • Same concepts as JMF: Player, Controller, Manager and DataSource. • Player behaviour same (Unrealised, Realised, Prefetched, Started) as JMF but one more state: • Closed: the player cannot be used again, it has released most of the resources • Controller is embedded in Player class

  20. Mobile Phone (4/5)

  21. Mobile Phone (5/5) • The types of media supported depends on the controls associated to the Player • Player.getControls() returns the supported controls • Some examples: VolumeControl, ToneControl… • The Player is content and protocol agnostic • A Player renders media data in a component dependent on the device configuration, two options: • AWT Component • MIDP Canvas or Items • VideoControl manages the location and the size of the video

  22. Conclusions (1/2) • The actual capabilities of the targeted device is the cause of the differences between standards • Low level versus high level control of the media: • In MM API and JMF (MHP) the actual control of the media is done at the native level since they are resource-constrained devices (e.g. decoding) • JMF uses two profiles, JMF plug-ins enables developers to process the data (e.g. multiplexing) • Behaviour of the player • All the standards have the same player behaviour. But MM API defines a Closed state to make explicit that all the resources are freed

  23. Conclusions (2/2) • MM API is influenced by the design of JMF, and have number of similarities: Manager, Player, Data Source, and Controller concepts. • MM API hides Controller within Player • Video render • JMF = AWT Component (complete integration) • MHP = A layer (transparency can be applied) • MM API = MIDP canvas (minimal integration) • Controls are different depending on the targeted device (e.g., subtitles language in television) • MHP includes specific television requirements (e.g., Clock not needed in broadcast, Locators)

  24. References (1/2) • Multimedia • P. Vuorimaa, Multimedia Technology Course’s Slides, Helsinki University of Technology, accessed on November 7th 2003, http://www.tml.hut.fi/Opinnot/T-111.350/index_uk.html • S. V. Raghavan, and S. K. Tripathi, Networked Multimedia Systems: Concepts, Architecture, and design, Upper Saddle River (NJ), Prentice Hall, 1998 • Digital Television • ETSI TS 102 812 v1.1.1, Digital Television Broadcasting (DVB): Multimedia Home Platform (MHP) Specification 1.1, ETSI, November 2001 • Steven Morris, The Interactive TV Web, http://www.mhp-interactive.org , 2002

  25. References (2/2) • PC • R. Gordon, and S. Talley, Essential JMF: Java Media Framework, Upper Saddle River (NJ), Prentice Hall, 1999 • Sun Microsystems, JMF Home Page, accessed on November 7th 2003, http://java.sun.com/products/java-media/jmf/index.html • Mobile Phones • Sun Microsystems, J2ME: Java 2 Platform Micro Edition, accessed November 7th 2003, http://java.sun.com/products/j2me/ • Sun Microsystems, CLDC, accessed November 7th 2003, http://java.sun.com/products/cldc/ • Sun Microsystems, Mobile Information Device Profile (MIDP), accessed November 7th 2003 http://wireless.java.sun.com/midp/ • Sun Microsystems, Mobile Media API (MMAPI), accessed November 7th 2003, http://java.sun.com/products/mmapi/

  26. Thank you! Questions? Comments?

More Related