Design and Implementation of an Efficient MPEG-4 Interactive Terminal on Embedded Devices

1. Design and Implementation of an Efficient MPEG-4 Interactive Terminal on Embedded Devices Yi-Chin Huang, Tu-Chun Yin, Kou-Shin Yang, Yan-Jun Chang, Meng-Jyi Shieh, Wen-Chin Chen IEEE International Conference on Multimedia and Expo Volume 1,  27-30 June 2004 Page(s):715 - 718 Reporter：陳憲章 Date：2006.4.11

2. Outline • Introduction • System Architecture • Synchronization of Heterogeneous media • Implementation and Optimization • Experimental Results • Conclusions

3. Introduction • This paper present an efficient MPEG-4-based interactive player for PDA-like embedded devices. • There are many technical issues need to be tackled: • lower computing power • less hardware acceleration • constrained memory size • lack of built-in multimedia APls

4. Introduction (con’t) • To overcome these constraints, two novel mechanisms were introduced: • Adaptive Frame Rate (AFR) • Scene Cache Graph Management (SCGM) • A semi-pull modelwas also designed for the synchronization of heterogeneous media objects.

5. System Architecture • The MPEG-4 Media Process Flow

6. System Architecture (con’t)

7. Scene Cache Graph Management (SCGM)

8. Synchronization of Heterogeneous media

9. Categories of Media • 1. Instant Time Media • The media data that can be computed immediately. • 2. Discrete Time Media • The media whose sampling lengths are long enough so that computers can accomplish the processing. (ex: video & animation) • 3. ContinualTime Media • Those media whose sampling time is very short. (ex: auditory)

10. Semi-Pull Model • A new decoding thread, is created for decoding. Each decoded data sample is associated with a composition timestamp. • Acomposition timestamp is created with respect to a global reference clock generated by scene graph manager.

11. Intra-media Synchronization • For instant time media, synchronization can be achieved if the Event Monitoring Procedure (EMP) execution rate is kept high enough. • For other media types, semi-pull model ensures that the presented data sample is the most updated and rendered on time.

12. Inter-media Synchronization • Synchronization between time-variant (discrete and continual) media • Forcing all media streams synchronized to the global clock. • Currently in MPEG-4, only audio streams have this problem. Audio streams are rendered by a sound card, which has a local clock.

13. Inter-media Synchronization (con’t) • Synchronize the audio stream to the global clock by adjusting the amount of waveform data.

14. Implementation and Optimization

15. Adaptive Frame Rate Mechanism • The job of visual render thread are divided into two procedures: • Event Monitoring Procedure (EMP) • Scene Render Procedure (SRP).

16. Adaptive Frame Rate Mechanism (con’t) • In this mechanism, a scene runs in either active stateor inert state. • 1. Inert state • render thread adopts lazy-render strategy. with which the only periodic routine task is to execute EMP.

17. Adaptive Frame Rate Mechanism (con’t) • 2. Active state • To continually render the active movie textures, render thread executes SRP in every specified period Pa.

18. 2D/3D Graphic Engine • Hardware acceleration for 3D rendering and floating-point number processing unit are not present on the target platform. • To overcome these problems, the engine applies fixed-point arithmetic and performs all 3D rendering functionalities by software emulation.

19. Buffer Management • This module maintains two buffers to hold decoded data. • While a decoding thread writes data to one buffer, the render thread reads data from the other one. • To avoid the race condition, both threads may lock the buffers.

20. Script Engine • To enhance the interactivity of BlFS scene, the system supports JavaScript mechanism specified in MPEG-4. • The script engine parses all the script codes in the scene and transforms them into a syntax tree for speeding up the script execution.

21. Experimental Results • When the rendering frame rate is fixed at 12fps, the average CPU load is 98.65% without SCGM. • When SCGM is enabled, the CPU load is improved toward 6.37%. The 3D model placed in the middle →

22. Experimental Results (con’t) • There are three objects, each of which is bound with a movie texture. • To measure the performance of AFR, we consider the lifetime of the battery with power consumption varies from 100% to 20%. • The experimental result shows that AFR gains 118% improvement of the lifetime in the first scene.

23. Conclusions • This paper have successfully developed a MPEG-4 player on a resource constrained PDA. • The system adopts four novel mechanisms: • Adaptive Frame Rate • Scene Cache Graph Management • Media Decoding Framework • Semi-pull Model

24. Conclusions (con’t) • Resolving the problems including: • computational resource reduction • low power consumption • heterogeneous media synchronization • For further, this system can be developed by new functionalities.

25. Seminar Thanks for comment !