建構於 IP 之無線多媒體技術及應用 Wireless Multi-media over IP ： Technologies and Applications 主持人：李琳山共同主持人：貝蘇章，吳家麟參與研究教授：鄭士康，陳良基，陳志

建構於IP之無線多媒體技術及應用 • Wireless Multi-media over IP：Technologies and Applications • 主持人：李琳山 • 共同主持人：貝蘇章，吳家麟 • 參與研究教授：鄭士康，陳良基，陳志宏，馮世邁 • 李宇旼，歐陽明，陳文進，黃肇雄

計劃主題及目標 本年度主要工作項目本年度重要成績舉例及展示計劃產出列表下年度工作計劃總檢討及結論大綱

90年度教育部考評委員會評語： “在原核定項目下進行良好，唯有若干核定項目與電信關連性較低，應考慮調整計劃重心” 90年7月決定調整方向： “智慧型訊號處理與多媒體資訊環境” (Intelligent Signal Processing and Multi-media Information Environment) “建構於IP之無線多媒體技術及應用” (Wireless Multi-media over IP: Technologies and Applications) 以無線通訊及IP環境下之多媒體技術為主軸方向調整

Future Integrated Networks • Real–time • Information • weather, traffic • flight schedule • stock price • sports scores • Private Services • personal notebook • business databases • home appliances • network • entertainments • Intelligent Working • Environment • e–mail processors • intelligent agents • teleconferencing • distant learning • Knowledge • Archieves • digital libraries • virtual museums • Electronic • Commerce • virtual banking • on–line transactions • on–line investments Future Network Era • Multi–media, Multi–lingual, Multi–functionalities • • Cross–cultures, Cross–domains, Cross–regions • • Integrating All Knowledge/Information/Services Globally • Most Attractive Representation for Knowledge/Information/Services will be in form of Multi-media Signals

Public Information and Services Future Networks Private and Personal Services Wireless Communications are Creating a Whole Variety of User Terminals and a Completely New User Environment • Network Access at Any Time, from Anywhere • Personal Notebooks, Handsets, PDA’s , Hand-held Devices … • Small in Size, Light in Weight, Ubiquitous… • Keyboard/Mouse Most Convenient for PC’s not Convenient any longer • human fingers never shrink, and application environment is changed • Multi-media Signals will be the Most Convenient Interface for ALL User Terminals at Any Time, from Anywhere

Information Flow from Servers to Users via Both Wired and Wireless Networks Many Different Network Technologies for Wired and Wireless Co-existing Wireless Communications are Creating a Completely New Transmission/Networking Environment

Wireless Links examples: Personal Area Networks (Bluetooth, etc.), Wireless LAN (IEEE 802.11), Cellular (GSM, GPRS, 3G), etc. time-varying fading and noise characteristics time-varying signal level and signal-to-noise ratios bursty errors with much higher error rates much smaller and dynamic bandwidth, much lower and changing bit rates highly unreliable supporting mobility : at any time, from anywhere Link Level Characteristics for Wireless vs Wired

UDP/IP- Real-time, No Retransmission/Delay as in TCP/IP Errors Packet Loss Real-time Transmission with Lost Information Packets out of Sequence Problems of Wireless Multi-media multi-media applications require real-time continuous transmission of huge quantities of data smaller and dynamic bandwidth offers only limited and time-varying data rates bursty errors with higher error rates are difficult to correct errors translated to packet loss, which are not acceptable packets out of sequence also creating problems time-varying transmission quality even varies across different wireless technologies Transport Level Issues in Wireless Multi-media

Improved Link Level Technologies better link quality Enhanced Transport Level Protocols to accommodate multi-media traffic and link unreliability Developing “Convergence Functionalities” in the Application Level wireless link level problems and transport level variations become transparent to “core technologies” “core technologies” shielded from and equally applicable to all different link/transport environments examples: scalability, error concealment,etc. Robust “Core Technologies” including Intelligent User Interfaces Combatting with residual errors Examples: signal verification,error resilience, etc. Intelligent user interface for different terminals Proposed Approach for Wireless Multi-media

Wireless Multi-media over IP

Wireless Multi-media over IP- Year 2001 • Multi-media • virtual reality • video • audio • speech • text • Basic Signal • Processing • analysis • segmentation • detection Robustness Error Concealment Scalability • Most Link/Transport Level Technologies have been Well Standardized and Widely Used • impact due to enhancement less easy • Most Efforts Concentrated on Application Level

Mission Statement • Basic Research • developing fundamental technologies for multi-media considering issues created by wireless communications • link unreliability and user interface requirements • link level, transport level, convergence functionalities and core technologies • Application Tasks • multi-media information streaming and retrieval • Academic Prototype Systems • technology demonstration Academic Prototype Systems Application Tasks Basic Research

satellites servers Integration with Other Subprojects • Microwave and Millimeter-wave Technologies • Wireless Multi-media over IP: Technologies and Applications Networks fiber C radio Global Knowledge, Information and Services cable ... 0110... ...1101... • Communications and Networking Technologies

Improved Transmission Technologies for Wireless ― Block Time Division Multiplexing (BTDM) IFFT P/S F E Q CP insertion D/A CP Removal A/D channel FFT S/P Receiver Transmitter • Orthogonal Frequency Division Multiplexing (OFDM)- A Mainstream Transmission Technology for Wireless • Multi-carrier with Many Attractive Properties • Difficulties • suffering from performance flooring in the presence of spectral nulls • high peak-to-average power ratio (PAPR) requiring special considerations for transmitter amplifier

Improved Transmission Technologies for Wireless ― Block Time Division Multiplexing (BTDM) P/S IFFT F E Q CP insertion D/A CP Removal A/D channel FFT S/P Transmitter Receiver • BTDM System Block Diagram • Nice Properties • no performance flooring in the presence of spectrall nulls • very small peak-to-average power ratio • shown to be the optimal channel independent transmitter that minimizes the bit error rate (BER)

Improved Transmission Technologies for Wireless ― Block Time Division Multiplexing (BTDM) OFDM OFDM BTDM BTDM • Example Results • Linear Time Invariant Channel with a Spectral Null • Rayleigh fading Channel • 3-10 db Gain over OFDM Observed

Video Streaming (I) – Reversible Variable Length Code (RVLC) • Variable Length Code • different code length for different symbols with different probabilities • average bit rate minimized for compression purposes • encoded/decoded in forward direction only • errors may propagate • Reversible Variable Length Code (RVLC) • can be decoded in both the forward and backward directions • to eliminate error propagation and enhance error-resilience capabilities

Video Streaming (I) – Reversible Variable Length Code (RVLC) • Two types of RVLC • Symmetric RVLC: only one code table required for both forward and backward decoding • Asymmetric RVLC: two code tables required, one for forward decoding and the other backward

Video Streaming (I) – Reversible Variable Length Code (RVLC) • Better Compression Achieved by Proposed Techniques

Experimental Results • Error rate: Random error 10 -4 Without RVLC With RVLC Video Streaming (I) – Reversible Variable Length Code (RVLC)

Video Streaming (II) – An Integrated Error Handling System • Different Error Handling Technologies at Different Stages Integrated • Based on H.263 Video Codec • Technologies Integrated • Resynchronization • Periodically Refreshing • Data Partitioning • Reversible Variable Length Code • Forward Error Correction Code • Error Concealment

Video Streaming (II) – An Integrated Error Handling System • System Interface

Video Streaming (II) – An Integrated Error Handling System • Typical Experimental Result Example

proxy Uplink data/ARQ Terminal data Bitstream & Frame Buffer Packet loss proxy Uplink data/ARQ Terminal (truncated Bitstream Decoding) data Important Data Resent Bitstream & Frame Buffer Video Streaming (III) – Error Handling with Video Proxy • No Error • bitstream stored at Proxy and deleted after ARQ confirmed • Errors Occur • important information (e.g. motion vectors) resent, while others depending on available bandwidth • packet loss message sent to source requesting rate control

Typical Experimental Result Example • Error rate =10 -3 Error Free Errors plus Video Proxy with Errors Video Streaming (III) – Error Handling with Video proxy

Video Streaming (III) – Error Handling with Video proxy Results Error Free Random Error 1e-3 Random Error 1e-3 w/ FEC Random Error 1e-3 w/ Video Proxy Forman 64kbit/sec QCIF 10fps

Video Streaming (IV) – Scalable Video Compression • Data Rate Scalability • bandwidth/data rate much smaller and time-varying in wireless networks • Spatial Scalability • different user terminals requiring different video sizes • Temporal Scalability • trading frame rate with picture resolution • SNR Scalability • different user terminals requiring different SNR conditions • Core Technologies • discrete Wavelet Transform for spatial compression • motion estimation/compensation for temporal compression

SP3 SP1 SP2 SP4 bitstream Video Streaming (IV) – Scalable Video Compression • Spatial Scalability Examples

(a) (b) Video Streaming (IV) – Scalable Video Compression • Temporal Scalability Example – Trading Frame Rate with Picture Resolution 30fps 15fps

Public Information and Services multi-media presentation Future Networks user-network interaction multi-media information retrieval user instructions Private and Personal Services network response Multi-media Information Retrieval • To Find the Desired Multi-media Information out of the Huge Quantities of Data over the Networks • Multi-media Information is Difficult for Pattern Matching, Browsing and Retrieving • Errors and Bandwidth Limitations for Wireless • Content-based Multi-media Information Retrieval

Multi-media Information Retrieval (I) –Image Retrieval • Shape Indexing Features • MPEG-7 Shape Descriptor • Color IndexingFeatures • Color Histogram • Regional Color • Texture Indexing Features • Angular Second Moment • Inverse Difference Moment • Contrast • Entropy

Multi-media Information Retrieval (I) –Image Retrieval • Typical Experimental Result Examples • Color Histogram-based Image Query

Multi-media Information Retrieval (I) –Image Retrieval • Typical Experimental Result Examples • Regional Color-based Image Query with/without Logic Scheme

Multi-media Information Retrieval (I) –Image Retrieval • Typical Experimental Result Examples • Texture based Image Query • Both Color and Texture Image Query

Multi-media Information Retrieval (II) – 3D Object Retrieval • Primarily Based on the Skeletal and Topological Structures of 3D Objects • Calculating the sum of the Geodesic distance from each vertex to all others • A Reeb Graph generated with each node representing a region according to these values • Similarity between two objects calculated from the nodes on the Reeb Graphs • Robust against Translation, Rotation, Scaling, Simplification, Subdivision, Noise, Deformation

Multi-media Information Retrieval (II) – 3D Object Retrieval Desired Object Similarity Retrieved Objects • An Example • http://3dsite.dhs.org/~dynamic • 445 objects in the database • 0.08 sec for comparing two objects on the average

Multi-media Information Retrieval (III) – Music Retrieval Original Detected Results • Initial Progress – Melody Extraction including Octave Dection • A Typical Example

Multi-media Information Retrieval (IV) –Voice-based Information Retrieval Voice Instructions Text Instructions 我想找有關紐約受到恐怖攻擊的新聞？ Text Information d1 Voice Information d2 d1 d3 d2 美國總統布希今天早上… d3 • Speech may become a New Data Type, if the Difficulties in Browsing and Retrieval can be Overcome • Application Examples: Personal Memo、Meeting Minutes、Personal Phone Records、Voice Mail Databases、Course Lectures Databases、Broadcast Programs …

Multi-media Information Retrieval (IV) –Voice-based Information Retrieval Typical Experimental Results for Mandarin Chinese in Non-interpolated Average Precision Syllable-based is the best when S(N), N=1~3, P(M), M=1~3 included, except character-based is the best for TQ/TD S(N), N=1~3, P(M), M=1~3 S(N), N=1~2 S(N), N=1 TQ: Text Queries TD: Text Document SQ: Spoken Queries SD: Spoken Document

User Voice-based Multi-media Information Retrieval — Demo • Chinese Text/Speech/Video News Collected from Internet and Broadcast Radio and Stored in a Server • User Retrieving the Desired News via a Notebook PC • News Corpora Easy to Obtain, with Plenty of New Words and a Variety of Subjects • Relatively High Degree of Difficulties Suitable for Academic Research • “請幫我找”or“請幫我查”as the Keywords for Initializing the Retrieval Radio Station TV Server Radio Internet NTU Campus Backbone

Voice-based Multi-media Information Retrieval — Demo • An Example

一、國際期刊論文已發表及接受32篇 Y.P.Lin and S.M.Phoong,〝ISI free FIR filter bank transceivers for frequency selective channels, 〞IEEE Trans. Signal Processing, pp. 2648-2658,Nov.2001. Y.P.Lin and S.M.Phoong, 〝Optimal ISI free DMT transceivers for distorted channels with colored noise,〞IEEE Trans. Signal Processing,pp.2702-2712, Nov.2001. Y.P.Lin and S.M.Phoong, 〝Minimum Redundancy for ISI free FIR DMT Transceivers, 〞accepted for publication in IEEE Trans. Signal Processing,2002. S.M.Phoong and Y.P.Lin, 〝Optimal Biorthogonal Transform for Colored Noise Suppression with Subband Wiener Filtering, 〞accepted for publication in IEEE Signal Processing Letters,2002. Lin-Shan Lee and Yumin Lee, 〝Voice Access of Global Information for Broadband Wireless: Technologies of Today and Challenges of Tomorrow, 〞Proceedings of the IEEE,Vol.89, No.1, pp.1-17, Jan. 2001. 90年度計畫產出列表

S.C.Pei and P.H.Wang,〝Analytical design of maximally flat FIR fractional Hilbert transformers, 〞Signal Processing, Vol.81, pp.643-661,2001. S.C.Pei and C.C.Tseng,〝A new eigenfilter based on total least squares error criterion, 〞IEEE Trans. on Circuits and Systems, part I: Fundamental Theory and Applications, Vol.48, No.6, pp.699-709, June 2001. S.C.Pei and P.H.Wang, 〝Closed-form design of maximally flat FIR Hilbert transformers, Differentiators, and Fractional Delayers by Power Series Expansion, 〞to appear in IEEE Trans. on Circuits and Systems, Part I: Fundamental Theory and Applications, Vol.48, No.4, pp.389-398, April 2001. S.C.Pei and J.J.Ding, 〝Two-dimensional generalized affine fractional Fourier transform, 〞IEEE Trans. on Signal Processing, Vol.49, No.4, p.878-897, April 2001. S.C.Pei and J.J.Ding,〝Relations between fractional operations and time-frequency distributions and their applications, 〞IEEE Transon Signal Processing, Vol.49, No.8, pp.1638-1655, Aug.2001. 90年度計畫產出列表

C.C.Tseng and S.C.Pei, 〝Stable IIRnotch filter design with optimal pole placement, 〞IEEE Trans. on Signal Processing, Vol.49, No.11, pp.2673-2681, Nov.2001. S.C.Pei and M.H.Yeh,〝The discrete fractional cosine and sine transforms,〞IEEE Trans on Signal Processing,Vol.49,No.6,pp.1198-1207,June2001. C.K.Chen, P.C.Tseng, Y.C.Chang and L.G.Chen,〝A Digital Signal Processor with Programmable Correlator Array Architecture for 3rd Generation Wireless Communication System, 〞IEEE Trans. on Circuits and Systems II: Analog and Digital Signal Processing, 2001(accepted). T.H.Tsai, R.J.Wu and L.G.Chen, 〝A Cost-Effective Design for MPEG-2 Audio Decoder with Embedded RISC Core, 〞Journal of VLSI Signal Processing, Vol.29, pp.255-265, Nov. 2001(SCI&EI). 90年度計畫產出列表

P.C.Tseng, C.K.Chen and L.G.Chen, 〝CDSP: An Application-Specific Digital Signal Processor for Third Generation Wireless Communications, 〞IEEE Trans. on Consumer Electronics, Vol.47, pp.672-677, No.3, Aug.2001. J.F.Shen, T.C.Wang and L.G.Chen, 〝A Novel Low-Power Full Search Block-Matching Motion Estimation Design for H.263+, 〞IEEE Trans. on Circuits and Systems for Video Technology, Vol.11, No.7, pp.890-897, July 2001(SCI&EI). C.Y.Chen, Z.L.Yang, T.C.Wang and L.G.Chen, 〝A Programmable Parallel VLSI Architecture for 2D Discrete Wavelet Transform, 〞Journal of VLSI Signal Processing, Vol.28, pp.151-163, July 2001(SCI&EI). P.C.Wu and L.G.Chen,〝An efficient architecture for two-dimensional discrete wavelet transform, 〞IEEE Trans. on Circuits and Systems for Video Technology, Vol.11, No.4, pp.536-545, April 2001(SCI&EI). 90年度計畫產出列表

Yi-Shin Tung, Ja-Ling Wu, His-Kang Tsao and Chi-Hui Huang, 〝A Highly Rate-Scalable System Based on MPEG-4 Spatial-Temporal-FGS Video Coding, 〞Advances in Automation, Multimedia and Video Systems, and Modern Computer Science, ISBN 960-8052-44-0, WSES Press, pp. 320-325. Chien-Wu Tsai and Ja-Ling Wu, 〝Modified Symmetrical Reversible Variable Length Code And Its Theoretical Bounds, 〞IEEE Transactions on Information Theory, Vol. 47, No. 6, pp. 2543-2548, Sept. 2001. Chien-Wu Tsai and Ja-Ling Wu,〝On Constructing the Huffman-Code Based Reversible Variable Length Codes, 〞IEEE Transactions on Communications, Vol. 49, No. 9, pp. 1506-1509, Sept. 2001. P.C.Cheng, J.H.Chen, S.C.Hwang and C.K.Sun,〝Three-dimensional reconstructions of Zea mays stem by magnetic resonance imaging (MRI) technology, 〞 Scanning Vol. 23,2 pp. 98-99, (2001). 90年度計畫產出列表

Kai-Hsiang Chuang and Jyh-Horng Chen,〝IMPACT: Image-based Physiological Artifacts Estimation and Correction Technique for fMRI, 〞 Magnetic Resonance in Medicine, 46:344-353, 2001. Ching-Po Lin, Wen-Yih Isaac Tseng, Hui-Cheng Cheng and Jyh-Horng Chen,〝Validation of Diffusion Tensor Magnetic Resonance Axonal Fiber Imaging with Registered Manganese-enhanced Tractography, 〞 NeuroImage, 14:1035-1047, 2001. M.J.Chiu, C.C.Lin, K.H.Chuang, J.H.Chen and K.M.Huang,〝Tissue Segmentation-Assisted Analysis of fMRI for Human Motor Response: An Approach Combining Artificial Neural Network and Fuzzy C Means, 〞 J. Digital Imaging, Vol. 14, No.1, March 2001, pp38-47. H.S.Chen, F.R.Guo, C.Y.Chen, J.H.Chen and T.S.Kuo, 〝Review of Telemedicine Projects in Taiwan, 〞 Int. J. Medical Informatics, May 2001; 61(2-3): 17-29. 90年度計畫產出列表

建構於 IP 之無線多媒體技術及應用 Wireless Multi-media over IP ： Technologies and Applications 主持人：李琳山共同主持人：貝蘇章，吳家麟參與研究教授：鄭士康，陳良基，陳志