1 / 92

建構於 IP 之無線多媒體技術及應用 Wireless Multi-media over IP : Technologies and Applications 主持人:李琳山 共同主持人:貝蘇章,吳家麟 參與研究教授:鄭士康,陳良基,陳志

建構於 IP 之無線多媒體技術及應用 Wireless Multi-media over IP : Technologies and Applications 主持人:李琳山 共同主持人:貝蘇章,吳家麟 參與研究教授:鄭士康,陳良基,陳志宏,馮世邁 李宇旼,歐陽明,陳文進,黃肇雄. 計劃主題及目標 本年度主 要工作項目 本年度重要成績舉例及展示 計劃產出列表 下年度工作計劃 總檢討及結論. 大綱. 90 年度教育部考評委員會評語 : “ 在原核定項目下進行良好,唯有若干核定項目與電信關連性較 低,應考慮調整計劃重心 ” 90 年 7 月決定調整方向 :

bat
Download Presentation

建構於 IP 之無線多媒體技術及應用 Wireless Multi-media over IP : Technologies and Applications 主持人:李琳山 共同主持人:貝蘇章,吳家麟 參與研究教授:鄭士康,陳良基,陳志

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. 建構於IP之無線多媒體技術及應用 • Wireless Multi-media over IP:Technologies and Applications • 主持人:李琳山 • 共同主持人:貝蘇章,吳家麟 • 參與研究教授:鄭士康,陳良基,陳志宏,馮世邁 • 李宇旼,歐陽明,陳文進,黃肇雄

  2. 計劃主題及目標 本年度主要工作項目 本年度重要成績舉例及展示 計劃產出列表 下年度工作計劃 總檢討及結論 大綱

  3. 90年度教育部考評委員會評語: “在原核定項目下進行良好,唯有若干核定項目與電信關連性較 低,應考慮調整計劃重心” 90年7月決定調整方向: “智慧型訊號處理與多媒體資訊環境” (Intelligent Signal Processing and Multi-media Information Environment) “建構於IP之無線多媒體技術及應用” (Wireless Multi-media over IP: Technologies and Applications) 以無線通訊及IP環境下之多媒體技術為主軸 方向調整

  4. Future Integrated Networks • Real–time • Information • weather, traffic • flight schedule • stock price • sports scores • Private Services • personal notebook • business databases • home appliances • network • entertainments • Intelligent Working • Environment • e–mail processors • intelligent agents • teleconferencing • distant learning • Knowledge • Archieves • digital libraries • virtual museums • Electronic • Commerce • virtual banking • on–line transactions • on–line investments Future Network Era • Multi–media, Multi–lingual, Multi–functionalities • • Cross–cultures, Cross–domains, Cross–regions • • Integrating All Knowledge/Information/Services Globally • Most Attractive Representation for Knowledge/Information/Services will be in form of Multi-media Signals

  5. Public Information and Services Future Networks Private and Personal Services Wireless Communications are Creating a Whole Variety of User Terminals and a Completely New User Environment • Network Access at Any Time, from Anywhere • Personal Notebooks, Handsets, PDA’s , Hand-held Devices … • Small in Size, Light in Weight, Ubiquitous… • Keyboard/Mouse Most Convenient for PC’s not Convenient any longer • human fingers never shrink, and application environment is changed • Multi-media Signals will be the Most Convenient Interface for ALL User Terminals at Any Time, from Anywhere

  6. Information Flow from Servers to Users via Both Wired and Wireless Networks Many Different Network Technologies for Wired and Wireless Co-existing Wireless Communications are Creating a Completely New Transmission/Networking Environment

  7. Wireless Links examples: Personal Area Networks (Bluetooth, etc.), Wireless LAN (IEEE 802.11), Cellular (GSM, GPRS, 3G), etc. time-varying fading and noise characteristics time-varying signal level and signal-to-noise ratios bursty errors with much higher error rates much smaller and dynamic bandwidth, much lower and changing bit rates highly unreliable supporting mobility : at any time, from anywhere Link Level Characteristics for Wireless vs Wired

  8. UDP/IP- Real-time, No Retransmission/Delay as in TCP/IP Errors Packet Loss Real-time Transmission with Lost Information Packets out of Sequence Problems of Wireless Multi-media multi-media applications require real-time continuous transmission of huge quantities of data smaller and dynamic bandwidth offers only limited and time-varying data rates bursty errors with higher error rates are difficult to correct errors translated to packet loss, which are not acceptable packets out of sequence also creating problems time-varying transmission quality even varies across different wireless technologies Transport Level Issues in Wireless Multi-media

  9. Improved Link Level Technologies better link quality Enhanced Transport Level Protocols to accommodate multi-media traffic and link unreliability Developing “Convergence Functionalities” in the Application Level wireless link level problems and transport level variations become transparent to “core technologies” “core technologies” shielded from and equally applicable to all different link/transport environments examples: scalability, error concealment,etc. Robust “Core Technologies” including Intelligent User Interfaces Combatting with residual errors Examples: signal verification,error resilience, etc. Intelligent user interface for different terminals Proposed Approach for Wireless Multi-media

  10. Wireless Multi-media over IP

  11. Wireless Multi-media over IP- Year 2001 • Multi-media • virtual reality • video • audio • speech • text • Basic Signal • Processing • analysis • segmentation • detection Robustness Error Concealment Scalability • Most Link/Transport Level Technologies have been Well Standardized and Widely Used • impact due to enhancement less easy • Most Efforts Concentrated on Application Level

  12. Mission Statement • Basic Research • developing fundamental technologies for multi-media considering issues created by wireless communications • link unreliability and user interface requirements • link level, transport level, convergence functionalities and core technologies • Application Tasks • multi-media information streaming and retrieval • Academic Prototype Systems • technology demonstration Academic Prototype Systems Application Tasks Basic Research

  13. satellites servers Integration with Other Subprojects • Microwave and Millimeter-wave Technologies • Wireless Multi-media over IP: Technologies and Applications Networks fiber C radio Global Knowledge, Information and Services cable ... 0110... ...1101... • Communications and Networking Technologies

  14. Wireless Multi-media over IP- Year 2001 • Multi-media • virtual reality • video • audio • speech • text • Basic Signal • Processing • analysis • segmentation • detection Robustness Error Concealment Scalability • Most Link/Transport Level Technologies have been Well Standardized and Widely Used • impact due to enhancement less easy • Most Efforts Concentrated on Application Level

  15. Improved Transmission Technologies for Wireless ― Block Time Division Multiplexing (BTDM) IFFT P/S F E Q CP insertion D/A CP Removal A/D channel FFT S/P Receiver Transmitter • Orthogonal Frequency Division Multiplexing (OFDM)- A Mainstream Transmission Technology for Wireless • Multi-carrier with Many Attractive Properties • Difficulties • suffering from performance flooring in the presence of spectral nulls • high peak-to-average power ratio (PAPR) requiring special considerations for transmitter amplifier

  16. Improved Transmission Technologies for Wireless ― Block Time Division Multiplexing (BTDM) P/S IFFT F E Q CP insertion D/A CP Removal A/D channel FFT S/P Transmitter Receiver • BTDM System Block Diagram • Nice Properties • no performance flooring in the presence of spectrall nulls • very small peak-to-average power ratio • shown to be the optimal channel independent transmitter that minimizes the bit error rate (BER)

  17. Improved Transmission Technologies for Wireless ― Block Time Division Multiplexing (BTDM) OFDM OFDM BTDM BTDM • Example Results • Linear Time Invariant Channel with a Spectral Null • Rayleigh fading Channel • 3-10 db Gain over OFDM Observed

  18. Wireless Multi-media over IP- Year 2001 • Multi-media • virtual reality • video • audio • speech • text • Basic Signal • Processing • analysis • segmentation • detection Robustness Error Concealment Scalability • Most Link/Transport Level Technologies have been Well Standardized and Widely Used • impact due to enhancement less easy • Most Efforts Concentrated on Application Level

  19. Video Streaming (I) – Reversible Variable Length Code (RVLC) • Variable Length Code • different code length for different symbols with different probabilities • average bit rate minimized for compression purposes • encoded/decoded in forward direction only • errors may propagate • Reversible Variable Length Code (RVLC) • can be decoded in both the forward and backward directions • to eliminate error propagation and enhance error-resilience capabilities

  20. Video Streaming (I) – Reversible Variable Length Code (RVLC) • Two types of RVLC • Symmetric RVLC: only one code table required for both forward and backward decoding • Asymmetric RVLC: two code tables required, one for forward decoding and the other backward

  21. Video Streaming (I) – Reversible Variable Length Code (RVLC) • Better Compression Achieved by Proposed Techniques

  22. Experimental Results • Error rate: Random error 10 -4 Without RVLC With RVLC Video Streaming (I) – Reversible Variable Length Code (RVLC)

  23. Video Streaming (II) – An Integrated Error Handling System • Different Error Handling Technologies at Different Stages Integrated • Based on H.263 Video Codec • Technologies Integrated • Resynchronization • Periodically Refreshing • Data Partitioning • Reversible Variable Length Code • Forward Error Correction Code • Error Concealment

  24. Video Streaming (II) – An Integrated Error Handling System • System Interface

  25. Video Streaming (II) – An Integrated Error Handling System • Typical Experimental Result Example

  26. proxy Uplink data/ARQ Terminal data Bitstream & Frame Buffer Packet loss proxy Uplink data/ARQ Terminal (truncated Bitstream Decoding) data Important Data Resent Bitstream & Frame Buffer Video Streaming (III) – Error Handling with Video Proxy • No Error • bitstream stored at Proxy and deleted after ARQ confirmed • Errors Occur • important information (e.g. motion vectors) resent, while others depending on available bandwidth • packet loss message sent to source requesting rate control

  27. Typical Experimental Result Example • Error rate =10 -3 Error Free Errors plus Video Proxy with Errors Video Streaming (III) – Error Handling with Video proxy

  28. Video Streaming (III) – Error Handling with Video proxy Results Error Free Random Error 1e-3 Random Error 1e-3 w/ FEC Random Error 1e-3 w/ Video Proxy Forman 64kbit/sec QCIF 10fps

  29. Video Streaming (IV) – Scalable Video Compression • Data Rate Scalability • bandwidth/data rate much smaller and time-varying in wireless networks • Spatial Scalability • different user terminals requiring different video sizes • Temporal Scalability • trading frame rate with picture resolution • SNR Scalability • different user terminals requiring different SNR conditions • Core Technologies • discrete Wavelet Transform for spatial compression • motion estimation/compensation for temporal compression

  30. SP3 SP1 SP2 SP4 bitstream Video Streaming (IV) – Scalable Video Compression • Spatial Scalability Examples

  31. (a) (b) Video Streaming (IV) – Scalable Video Compression • Temporal Scalability Example – Trading Frame Rate with Picture Resolution 30fps 15fps

  32. Wireless Multi-media over IP- Year 2001 • Multi-media • virtual reality • video • audio • speech • text • Basic Signal • Processing • analysis • segmentation • detection Robustness Error Concealment Scalability • Most Link/Transport Level Technologies have been Well Standardized and Widely Used • impact due to enhancement less easy • Most Efforts Concentrated on Application Level

  33. Public Information and Services multi-media presentation Future Networks user-network interaction multi-media information retrieval user instructions Private and Personal Services network response Multi-media Information Retrieval • To Find the Desired Multi-media Information out of the Huge Quantities of Data over the Networks • Multi-media Information is Difficult for Pattern Matching, Browsing and Retrieving • Errors and Bandwidth Limitations for Wireless • Content-based Multi-media Information Retrieval

  34. Multi-media Information Retrieval (I) –Image Retrieval • Shape Indexing Features • MPEG-7 Shape Descriptor • Color IndexingFeatures • Color Histogram • Regional Color • Texture Indexing Features • Angular Second Moment • Inverse Difference Moment • Contrast • Entropy

  35. Multi-media Information Retrieval (I) –Image Retrieval • Typical Experimental Result Examples • Color Histogram-based Image Query

  36. Multi-media Information Retrieval (I) –Image Retrieval • Typical Experimental Result Examples • Regional Color-based Image Query with/without Logic Scheme

  37. Multi-media Information Retrieval (I) –Image Retrieval • Typical Experimental Result Examples • Texture based Image Query • Both Color and Texture Image Query

  38. Multi-media Information Retrieval (II) – 3D Object Retrieval • Primarily Based on the Skeletal and Topological Structures of 3D Objects • Calculating the sum of the Geodesic distance from each vertex to all others • A Reeb Graph generated with each node representing a region according to these values • Similarity between two objects calculated from the nodes on the Reeb Graphs • Robust against Translation, Rotation, Scaling, Simplification, Subdivision, Noise, Deformation

  39. Multi-media Information Retrieval (II) – 3D Object Retrieval Desired Object Similarity Retrieved Objects • An Example • http://3dsite.dhs.org/~dynamic • 445 objects in the database • 0.08 sec for comparing two objects on the average

  40. Multi-media Information Retrieval (III) – Music Retrieval Original Detected Results • Initial Progress – Melody Extraction including Octave Dection • A Typical Example

  41. Multi-media Information Retrieval (IV) –Voice-based Information Retrieval Voice Instructions Text Instructions 我想找有關紐約受到恐怖攻擊的新聞? Text Information d1 Voice Information d2 d1 d3 d2 美國總統布希今天早上… d3 • Speech may become a New Data Type, if the Difficulties in Browsing and Retrieval can be Overcome • Application Examples: Personal Memo、Meeting Minutes、Personal Phone Records、Voice Mail Databases、Course Lectures Databases、Broadcast Programs …

  42. Multi-media Information Retrieval (IV) –Voice-based Information Retrieval Typical Experimental Results for Mandarin Chinese in Non-interpolated Average Precision Syllable-based is the best when S(N), N=1~3, P(M), M=1~3 included, except character-based is the best for TQ/TD S(N), N=1~3, P(M), M=1~3 S(N), N=1~2 S(N), N=1 TQ: Text Queries TD: Text Document SQ: Spoken Queries SD: Spoken Document

  43. User Voice-based Multi-media Information Retrieval — Demo • Chinese Text/Speech/Video News Collected from Internet and Broadcast Radio and Stored in a Server • User Retrieving the Desired News via a Notebook PC • News Corpora Easy to Obtain, with Plenty of New Words and a Variety of Subjects • Relatively High Degree of Difficulties Suitable for Academic Research • “請幫我找”or“請幫我查”as the Keywords for Initializing the Retrieval Radio Station TV Server Radio Internet NTU Campus Backbone

  44. Voice-based Multi-media Information Retrieval — Demo • An Example

  45. 一、國際期刊論文已發表及接受32篇 Y.P.Lin and S.M.Phoong,〝ISI free FIR filter bank transceivers for frequency selective channels, 〞IEEE Trans. Signal Processing, pp. 2648-2658,Nov.2001. Y.P.Lin and S.M.Phoong, 〝Optimal ISI free DMT transceivers for distorted channels with colored noise,〞IEEE Trans. Signal Processing,pp.2702-2712, Nov.2001. Y.P.Lin and S.M.Phoong, 〝Minimum Redundancy for ISI free FIR DMT Transceivers, 〞accepted for publication in IEEE Trans. Signal Processing,2002. S.M.Phoong and Y.P.Lin, 〝Optimal Biorthogonal Transform for Colored Noise Suppression with Subband Wiener Filtering, 〞accepted for publication in IEEE Signal Processing Letters,2002. Lin-Shan Lee and Yumin Lee, 〝Voice Access of Global Information for Broadband Wireless: Technologies of Today and Challenges of Tomorrow, 〞Proceedings of the IEEE,Vol.89, No.1, pp.1-17, Jan. 2001. 90年度計畫產出列表

  46. S.C.Pei and P.H.Wang,〝Analytical design of maximally flat FIR fractional Hilbert transformers, 〞Signal Processing, Vol.81, pp.643-661,2001. S.C.Pei and C.C.Tseng,〝A new eigenfilter based on total least squares error criterion, 〞IEEE Trans. on Circuits and Systems, part I: Fundamental Theory and Applications, Vol.48, No.6, pp.699-709, June 2001. S.C.Pei and P.H.Wang, 〝Closed-form design of maximally flat FIR Hilbert transformers, Differentiators, and Fractional Delayers by Power Series Expansion, 〞to appear in IEEE Trans. on Circuits and Systems, Part I: Fundamental Theory and Applications, Vol.48, No.4, pp.389-398, April 2001. S.C.Pei and J.J.Ding, 〝Two-dimensional generalized affine fractional Fourier transform, 〞IEEE Trans. on Signal Processing, Vol.49, No.4, p.878-897, April 2001. S.C.Pei and J.J.Ding,〝Relations between fractional operations and time-frequency distributions and their applications, 〞IEEE Transon Signal Processing, Vol.49, No.8, pp.1638-1655, Aug.2001. 90年度計畫產出列表

  47. C.C.Tseng and S.C.Pei, 〝Stable IIRnotch filter design with optimal pole placement, 〞IEEE Trans. on Signal Processing, Vol.49, No.11, pp.2673-2681, Nov.2001. S.C.Pei and M.H.Yeh,〝The discrete fractional cosine and sine transforms,〞IEEE Trans on Signal Processing,Vol.49,No.6,pp.1198-1207,June2001. C.K.Chen, P.C.Tseng, Y.C.Chang and L.G.Chen,〝A Digital Signal Processor with Programmable Correlator Array Architecture for 3rd Generation Wireless Communication System, 〞IEEE Trans. on Circuits and Systems II: Analog and Digital Signal Processing, 2001(accepted). T.H.Tsai, R.J.Wu and L.G.Chen, 〝A Cost-Effective Design for MPEG-2 Audio Decoder with Embedded RISC Core, 〞Journal of VLSI Signal Processing, Vol.29, pp.255-265, Nov. 2001(SCI&EI). 90年度計畫產出列表

  48. P.C.Tseng, C.K.Chen and L.G.Chen, 〝CDSP: An Application-Specific Digital Signal Processor for Third Generation Wireless Communications, 〞IEEE Trans. on Consumer Electronics, Vol.47, pp.672-677, No.3, Aug.2001. J.F.Shen, T.C.Wang and L.G.Chen, 〝A Novel Low-Power Full Search Block-Matching Motion Estimation Design for H.263+, 〞IEEE Trans. on Circuits and Systems for Video Technology, Vol.11, No.7, pp.890-897, July 2001(SCI&EI). C.Y.Chen, Z.L.Yang, T.C.Wang and L.G.Chen, 〝A Programmable Parallel VLSI Architecture for 2D Discrete Wavelet Transform, 〞Journal of VLSI Signal Processing, Vol.28, pp.151-163, July 2001(SCI&EI). P.C.Wu and L.G.Chen,〝An efficient architecture for two-dimensional discrete wavelet transform, 〞IEEE Trans. on Circuits and Systems for Video Technology, Vol.11, No.4, pp.536-545, April 2001(SCI&EI). 90年度計畫產出列表

  49. Yi-Shin Tung, Ja-Ling Wu, His-Kang Tsao and Chi-Hui Huang, 〝A Highly Rate-Scalable System Based on MPEG-4 Spatial-Temporal-FGS Video Coding, 〞Advances in Automation, Multimedia and Video Systems, and Modern Computer Science, ISBN 960-8052-44-0, WSES Press, pp. 320-325. Chien-Wu Tsai and Ja-Ling Wu, 〝Modified Symmetrical Reversible Variable Length Code And Its Theoretical Bounds, 〞IEEE Transactions on Information Theory, Vol. 47, No. 6, pp. 2543-2548, Sept. 2001. Chien-Wu Tsai and Ja-Ling Wu,〝On Constructing the Huffman-Code Based Reversible Variable Length Codes, 〞IEEE Transactions on Communications, Vol. 49, No. 9, pp. 1506-1509, Sept. 2001. P.C.Cheng, J.H.Chen, S.C.Hwang and C.K.Sun,〝Three-dimensional reconstructions of Zea mays stem by magnetic resonance imaging (MRI) technology, 〞 Scanning Vol. 23,2 pp. 98-99, (2001). 90年度計畫產出列表

  50. Kai-Hsiang Chuang and Jyh-Horng Chen,〝IMPACT: Image-based Physiological Artifacts Estimation and Correction Technique for fMRI, 〞 Magnetic Resonance in Medicine, 46:344-353, 2001. Ching-Po Lin, Wen-Yih Isaac Tseng, Hui-Cheng Cheng and Jyh-Horng Chen,〝Validation of Diffusion Tensor Magnetic Resonance Axonal Fiber Imaging with Registered Manganese-enhanced Tractography, 〞 NeuroImage, 14:1035-1047, 2001. M.J.Chiu, C.C.Lin, K.H.Chuang, J.H.Chen and K.M.Huang,〝Tissue Segmentation-Assisted Analysis of fMRI for Human Motor Response: An Approach Combining Artificial Neural Network and Fuzzy C Means, 〞 J. Digital Imaging, Vol. 14, No.1, March 2001, pp38-47. H.S.Chen, F.R.Guo, C.Y.Chen, J.H.Chen and T.S.Kuo, 〝Review of Telemedicine Projects in Taiwan, 〞 Int. J. Medical Informatics, May 2001; 61(2-3): 17-29. 90年度計畫產出列表

More Related