智慧型訊號處理與多媒體資訊環境 Intelligent Signal Processing and Multi-media Information Environment 主持人：李琳山 共同主持人： 貝蘇章，吳家麟 參與研究教授： 鄭士

1. 智慧型訊號處理與多媒體資訊環境 Intelligent Signal Processing and Multi-media Information Environment 主持人：李琳山 共同主持人：貝蘇章，吳家麟 參與研究教授：鄭士康，陳良基，陳志宏，曹建和，馮世邁 李宇旼，歐陽明，陳文進，黃肇雄

2. 計劃主題 主要工作項目 重要成績舉例 計劃產出列表 下年度工作計劃 錄影帶及系統展示 總檢討及結論 大綱

3. Utilizing External Knowledge example: speech recognition using lexicon and language model Adaptive Algorithms based on Signal Characteristics/Conditions example: video processing considering local signal characteristics Learning Capabilities learning new knowledge and developing new adaptation mechanisms Intelligent Signal Processing Signal Processing Signals Output Networks External Knowledge Adaptive Algorithm Learning Capabilities

4. Information-related Activities, Applications and Services in Future Network Era Future Integrated Networks Real-time Information • weather, traffic • flight schedule • stock price • sports scores Private Services • personal notebook • business databases • home appliances • network entertainments Intelligent Working Environment • e-mail processors • intelligent agents • teleconferencing • distant learning Knowledge Archieves • digital libraries • virtual museums Electronic Commerce • virtual banking • on-line transactions • on-line investments

5. Information-related Activities, Applications and Services in Future Network Era Future Integrated Networks Real-time Information Private Services Intelligent Working Environment Knowledge Archieves Electronic Commerce • Multi-media, Multi-lingual, Multi-functionalities • Cross-cultures, Cross-domains, Cross-regions • Integrating All Knowledge Systems and Information-related Activities and Services Globally • All Knowledge and Information/Services Represented in Form ofMulti-media, Multi-lingual Signals • Multi-media, Multi-lingual Signals will be the Core for Future Human Knowledge and Information/Services

6. Vision - Intelligent Multi-media Information Environment Terminal Equipments: • Personal Computers • Telephone Sets • PDA’s • Handsets • Vehicular Electronics • Home Appliances Future Integrated Networks Application Tasks for Intelligent Signal Processing: • Voice Conversational Interfaces • Video/Audio Compression and Manipulation • Graphics/Virtual Reality • Multi-media Information Retrieval Users Information-related Activities and Services: • Knowledge Archieves • Real-time Information • Private Services . . . Basic Signal Processing Technologies : • Intelligent Signal Processing • Speech Signal Processing • Video/Audio Signal Processing • Biomedical Signal Processing • Multi-media Signal Processing . . . . . . . . .

7. Information Environment terminal equipments, computers, software, networks, knowledge/information/services Content Engineering – Processing of Network Knowledge/Information/Services in Form of Multi-media Signals User Interface in Form of Multi-media Signals Intelligent Multi-media Signal Processing Vision - Intelligent Multi-media Information Environment Future Integrated Networks Terminal Equipments: Application Tasks for Intelligent Signal Processing: Users . . . . . . Information-related Activities and Services: Basic Signal Processing Technologies : . . . . . .

8. Intelligent Signal Processing and Multi-media Information Environment User Interface in Form of Multi-media Signals Content Engineering - Processing of Global Knowledge/Information in Form of Multi-media Signals satellites userterminals Networks fiber C radio servers Global Information cable ... 0110... ...1101...

9. Integration with Other Projects Microwave and Millimeter-wave Technologies Multi-media Information Environment satellites userterminals Networks fiber C radio Global Information cable servers ... 0110... ...1101... Intelligent Signal Processing Communications and Networking Technologies

10. At Any Time, from Anywhere As Handset Size Shrinks While Required Functionalities Grows Continuously, Voice Interface will be the Key Vision - Voice Access of Global Multi-media Information under Broadband Wireless Environment

11. 智慧型高等訊號處理 中文自然語言及語音處理 視訊及音訊處理 生理訊號處理 多媒體訊號處理 處理器及晶片設計 系統整合 主要工作項目

12. 一、智慧型高等訊號處理 著重深入的基礎研究，建立學理基礎 二度空間離散傅立葉分數轉換之研究 投影式離散傅立葉分數轉換之研究 離散希爾伯特轉換設計： Analytic Design Type and Closed-form Maximally Flat Type 高速無線通訊之訊號處理研究： 調變解調模擬、無線通道模型建立、無線通道模擬 最小雜訊之小波原理研究： 梯形小波原理、最小雜訊之架構、非正交轉換 小波多載波原理之研究： 小波與多載波系統之關係、小波多載波系統之最佳化、最佳小波多載波系統之效益評估 卓 越 計 畫 National Taiwan University 89年度所完成之研究項目

13. Derived a New Algorithm for Optimizing FIR Decision Feedback Equalizer Giving Low Complexity and Better Performance Fast Optimization of FIR Decision Feedback Equalizer

14. 二、中文自然語言及語音處理 運用語言知識、考慮中文結構 PAT-tree為基礎之語言模型技術研究 改良型搜尋及語言解碼技術研究 口語對話技術之學理分析 卓 越 計 畫 National Taiwan University 89年度所完成之研究項目

15. Syllable-based One-pass Search Finding the Optimal Sentence from an Unknown Utterance Using 3 Knowledge Sources:Acoustic Models, Lexicon and Language Model Based on a Lattice of Syllable Candidates t Acoustic Models Syllable Lattice P(w1)P(w2 |w1)...... P(w1)P(w2 |w1)...... w1 w2 Word Graph w1 w2 Language Models Lexicon

16. unknown known verified u/k/v: unknown, known, or verified c/e/x: correct, error, or don’t care (v,c) (k,c) (u,x) (k,e) (v,e) slot lost Random Tests Rl, Rm slot i slot i User System slot j (error) Random Select j no misunderstanding Statistical Formulation and Analysis for Spoken Dialogue Systems • A Spoken Dialogue System Formulated as a Process of Transmitting a set of Semantic Slots from the User to the System • Finite State Machine Representation • Channel Model for Slot Transmission • slot lost rate Rl : desired slots lost • slot misunderstanding rate Rm : wrong slots received causing misunderstanding

17. 三、視訊及音訊處理 使所有的知識及資訊有聲有色、多采多姿 轉換壓縮域視訊處理分析研究(MPEG Compressed Video Analysis)： 換景自動偵測(Abrupt Scene Change detection and Gradual Scene Changes in dissolve sequence)、閃光燈特效偵測(Flash light detection)、字幕偵測(Caption detection) 彩色影像量化研究： Dependent Color Scalar Quantitzation、Quaternion moment preserving threshold technique、Self-organization Kohnuen Map Neural Network clustering 音訊之浮水印(Audio Watermarking)技術 音樂訊號音色辨識基礎研究： 聲音特徵值研究及萃取、類神經網路研究及程式撰寫、訓練辨識樣本、系統測試及參數調整 傳送樂音代碼之網路音樂會實作： 樂音產生器、網路連結、網路對話、系統整合及系統測試 卓 越 計 畫 National Taiwan University 89年度所完成之研究項目

18. Flashlight homogeneous white and bright Low variance unusual intra-coded MB in B frame unusual consecutive two abrupt scene changes Flashlight Detection

19. 「網路音樂會」架構 合成音效 虛擬電子琴 CSocket網路通訊（Server-Client） 網路與音樂控制的結合 網路音樂會雛型

20. 四、生理訊號處理 讓使用者接觸資訊世界更為方便自然 眼動滑鼠之眼球運動追蹤系統建立 眼動滑鼠之硬體系統 89年度所完成之研究項目

21. A New Human-computer Interface Controlled by Eye Movements Position of Pupil can be Detected and Used for Cursor Control with an Infrared Optical Sensor Array Optical Mouse by Eye Movements

22. Cursor Control 視動滑鼠不反應區之電腦畫面︰中間灰色框框為不反應區，眼球可在中間注視休息而不影響電腦游標之位置。正常操作時，此框框會消失隱藏於於幕後，以免妨礙使用者選項。

23. 五、多媒體訊號處理 不同訊號間之整合互動 即時環場實境及浮水印 可變速率及抗拒錯誤之視訊壓縮 臉部表情合成技術之開發 跨平台使用者共享瀏覽器系統規劃 建構半球型虛擬實境顯示裝置（Spherical display equipment） 卓 越 計 畫 National Taiwan University 89年度所完成之研究項目

24. Scalable decoding visual information at varying rates from a single compressed bitstream. Football: 1.5M bps, 16 fps S=3 (352x288) S=1 (88x72) S=0 (44x36) S=2 (176x144) Scalable Video Codec

25. Error – Resilience Video Without/With Error-Resilience Techniques

26. Constructing Panoramic Images Automatically in Real-time from Capturing a Scene over 360 Degrees VideoVR

27. Protecting the Intellectual Properties of Images and Associated Software Resist Various Attacks: Compression, Pixel-shifting, Cropping, etc. VideoVR with Watermark

28. 六、處理器及晶片設計 使訊號處理走出電腦，跨出視窗，進入手機等各種終端設備，融入日常生活 數位訊號處理器核心指令集之開發設計 C語言模擬分析 智慧型視訊處理系統應用模擬程式開發 卓 越 計 畫 National Taiwan University 89年度所完成之研究項目

29. Computational ALU MAC SFT CMP Data Movement Program Flow Special Instructions Total 92 Instructions DSP Core Instruction Set Design • CMAC/CMUL, SQS • Complex MAC/MUL • I*I+Q*Q • ACS • Dual Add-Compare-Select operation • TRCBK • Traceback • FIR2 • Two FIR operations in one cycle

30. DSP Simulator Interface • Step/Free Running • Run # clocks • Run to PC=# • Monitor Register • Monitor Memory • Interrupt Generation

31. An Example Layout/Specification for A 16-bit DSP for 3G Wireless

32. Block diagram for the Example

33. 七、系統整合 系統應用層次的整合 第一階段整合實驗室之初步建設 卓 越 計 畫 National Taiwan University 89年度所完成之研究項目

34. A Good Environment for Integration good results of different areas can be naturally integrated when available An Intelligent, Multi-media, Paperless Information Environment virtual electronic classroom, multi-media teleconferencing room, intelligent office, intelligent lab or any other working environment conference room setting as an example Network Architecture Campus Backbone Intranet Integration Plan

35. (2) (3) (4)為三種不同性質形態的訊號處理，共同圍繞著(1)構成核心技術 (6)提供所需關鍵性零組件 (5) (7)為整合性技術 各工作項目之整合性及關連性 核心技術 (2)中文自然語言及語音處理 整合性技術 (3) 視訊 及 音訊 處理 (4) 生理訊號處理 (5)多媒體訊號處理 (7)系統整合 (1)智慧型高等訊號處理 (6)處理器及晶片設計 關鍵性零組件

36. 訊號處理與網路通訊結合 訊號處理與知識及資訊處理結合 整合諸多訊號處理技術 前瞻性的開闊視野，集中明確的方向 整合台大在訊號處理各相關領域的基礎向前邁進 目標

37. At Any Time, from Anywhere As Handset Size Shrinks While Required Functionalities Grows Continuously, Voice Interface will be the Key Vision - Voice Access of Global Multi-media Information under Broadband Wireless Environment

38. Information Environment terminal equipments, computers, software, networks, knowledge/information/services Representation/Extraction of Network Knowledge/Information/Services in Form of Multi-media Signals User Interface in Form of Multi-media Signals Intelligent Multi-media Signal Processing Vision - Intelligent Multi-media Information Environment Terminal Equipments: • Personal Computers • Telephone Sets • PDA’s • Handsets • Vehicular Electronics • Home Appliances Application Tasks for Intelligent Signal Processing: • Voice Command and Dictation • Voice Conversational Interfaces • Video/Audio Compression and Manipulation • Graphics/Virtual Reality • Multi-media Information Retrieval Future Integrated Networks Users Information-related Activities and Services: • Knowledge Archieves • Real-time Information • Electronic Commerce • Private Services . . . Basic Signal Processing Technologies : • Intelligent Signal Processing • Speech Signal Processing • Video/Audio Signal Processing • Biomedical Signal Processing • Multi-media Signal Processing . . . . . . . . .

39. Integration with Other Projects Microwave and Millimeter-wave Technologies Multi-media Information Environment satellites userterminals Networks fiber C radio Global Information cable servers ...1101... ... 0110... Intelligent Signal Processing Communications and Networking Technologies

40. Intelligent Signal Processing and Multi-media Information Environment Content Engineering production, retrieval, presentation, protection User Interface 3-dim mouse, optical mouse, speech interface Initial Integration An intelligent, multi-media, paperless information environment Demo’s and Video

41. 一、國際期刊論文已發表及接受24篇，已投稿審查中至少4篇一、國際期刊論文已發表及接受24篇，已投稿審查中至少4篇 “MINLAB: Minimum noise structure for ladder-based biorthogonal filter banks”, IEEE Trans. Signal Processing, pp.465-77, Feb. 2000. “Prediction based lower triangular transform, ”IEEE Trans. Signal Processing, pp. 1947-56, July 2000. “Prefect discrete multitone modulation with optimal transceivers, ” IEEE Trans. Signal Processing, pp. 1702-12, June 2000. C. C. Tseng, S. C. Pei, and S. C. Hsia,“Computation of fractional derivatives using Fourier transform and digital FIR differentiator,” Signal Processing, Vol.80, No.1, pp.151-159 Jan. 2000. S. C. Pei, and J. J. Ding,“Closed form discrete fractional and affine Fourier transform,” IEEE Trans. on Signal Processing, Vol.48, No.5, pp.1338-1353, May 2000. 89年度計畫產出列表

42. S. C. Pei, and P.H. Wang,“ Closed-form design and efficient implementation of generalized maximally flat half-band FIR filters,” IEEE Signal Processing Letters, Vol.7, No.6, pp.149-151, June 2000. S. C. Pei, B.R. Chiou and P.H. Wang,“Programmable fractional sample delay filters with flatness compromise between magnitude reponse and group delay, ” IEEE Trans. On Circuits and Systems, Part II : Analog and Digital Signal Processing, Vol.47, No.8, pp.783-787, Aug 2000 S. C. Pei and PH. Wang,“Design of arbitrary cut-off 2-D diamond-shaped FIR filters using the Bernstein Polynomial, ”IEEE Signal Processing Letters, Vol.7, No.11, pp.310-313, Nov. 2000. S. C. Pei and M.H.Yeh,“Discrete fractional Hilbert transform,”IEEE Trans. On Circuits and Sytems, Part II : Analog and Digital Signal Processing, Vol.47, No.11, Nov. 2000. 89年度計畫產出列表

43. S. C. Pei and J. J. Ding,“Simplified fractional Fourior transform,”to appear in J. Opt. Soc. Am. A, Dec, 2000. S. C. Pei and J. J. Ding,“The integer transforms analogous to discrete trigometric transforms,”IEEE Trans. on Signal Processing, Vol.48, No.12, Dec 2000. Lin-shan Lee, Yumin Lee, “Voice Access of Global Information for Broadband Wireless: Technologies of Today and Challenges of Tomorrow” (invited paper), to appear on Proceedings of the IEEE, Feb. 2001. Jeih-weih Hung, Jia-lin Shen and Lin-shan Lee, “New Approaches for Domain Transformation and Parameter Combination for Improved Accuracy in Parallel Model Combination (PMC) Techniques,” paper accepted by IEEE Transactions on Speech and Audio Processing. 89年度計畫產出列表

44. Bor-shen Lin, Lin-shan Lee, “Computer-aided Analysis and Design for Spoken Dialogue Systems Based on Quantitative Simulations,” paper accepted by IEEE Transactions on Speech and Audio Processing. S. C. Pei and Y. Z. Chou,“ Efficient MPEG compressed video analysis using macroblock type information,” IEEE Trans. on Multimedia, Vol.1, No.4, pp.321-331, Dec. 1999. S. C. Pei, C. M. Cheng and L.F. Ho,“Limited color display for compressed image and video,”IEEE Trans. on Circuits and Systems for Video Technology, Vol.10, No.6, pp.913-922, Sept. 2000. Jiann-Rong Wu and Ming Ouhyoung,“On Latency Compensation and Its Effects for Head Motion Trajectories in Virtual Environments,”pp. 79-90, Vol. 16, No. 2, The Visual Computer, 2000. 89年度計畫產出列表

45. Ding-Yun Chen, Ming Ouhyoung, and Ja-Ling Wu,“A Shift-Resisting Public Watermark System for Protecting Image Processing Software,” IEEE Trans. on Comsumer Electronics, Vol. 46, No. 3, pp. 404-414, August 2000. Chien-Feng Huang, I-Chen Lin, Ming Ouhyoung,“High Resolution Calibration of Motion Capture Data for Realistic Facial animation,”Vol. 11, No. 9, pp.1141-1150, Journal of Software, ISSN 1000-9825, September 2000, China Computer Federation. Yuh-Ming Huang and Ja-Ling Wu, “Polynomial Transform Based Algorithms for Computing 2-D Generalized DFT, Generalized DHT and skew circular convolution ”to appear in Signal Processing. Yuh-Ming Huang, Ja-Ling Wu, and Chi-Lun Chang, “A Generalized Output Pruning Algorithm for Matrix-Vector Multiplication and Its Application to Compute Discrete Cosine Transform,” vol.48, No.2, pp.561-563, IEEE Trans.on Signal Processing Feb.2000. 89年度計畫產出列表

46. H. C. Chang, J. Y. Jiu, L. L. Chen, and, L. G. Chen, “Design and Implementation of Low Power DCT Chip for Portable Multimedia Terminal”, Journal of VLSI Signal Processing, Vol. 26, pp. 319-332, November 2000 R. X. Chen, L. G. Chen, and L. Chen, “System Design Consideration for Digital Wheelchair Controller”, IEEE Trans. on Industrial Electronics, Vol. 47, No. 4, pp. 898-907, August 2000. T. H. Tsai and L. G. Chen, “A Novel Architecture of Inverse Quantization and Multichannel Processing for MPEG-2 Audio Decoding”, IEEE Trans. on Circuits and Systems II: Analog and Digital Signal Processing, Vol. 47, No. 1, pp. 75-78, January 2000. “Minimum redundancy for ISI free FIR DMT transceiver,” submitted to IEEE Trans. Signal Processing. 89年度計畫產出列表

47. “Discrete multitone modulation with principle component filter banks,” submitted to IEEE Trans. Information Theory. “Optimality of orthogonal DMT Transceivers for distorted channels with colored noise”, paper submitted to IEEE Trans. Signal Processing. “ISI free FIR filterbank Transceivers for frequency selective channels”, paper submitted to IEEE Trans. Signal Processing. 89年度計畫產出列表

48. 二、國際會議論文已發表及接受44篇 “Optimality of principle component filter banks for discrete multitone communication systems,” in Proc. IEEE Int. Symp. Circ. Syst., Geneva, 2000. “Minimal factorization of lapped unimodular transforms,” in Proc. IEEE Int. Conf. Acoust. Speech, Signal Processing, Turkey, 2000. “On the Duality of Optimal DMT Systems and Biorthogonal Subband Coders,” in Proc. IEEE Int. Conf. Acoust. Speech, Signal Processing, Turkey, June 2000. “Design of Causal Stable IIR Filter Bank with Powers-of-two Coefficients,” in Proc. Eusipco, Finland, sep. 2000. “Design of FIR filter bank transceivers with effective band separation,” in Proc. Eusipco, Finland, Sep. 2000. “Minimum redundancy ISI free FIR filter bank transceiver,” in SPIE, San Diego, CA, July 2000. 89年度計畫產出列表

49. Yumin Lee, Vip Desai, “Fast Optimization of FIR DFE for Wireless Data Communications,” GLOBECOM 2000. S. C. Pei and J. J. Ding,”Integer discrete Fourier transform and its extension to integer trigometric transforms,”Proc. of IEEE Int'l Symp. on Circuits and Systems, Geneva, Switzerland, May 2000. S. C. Pei and P. H. Wang,”Closed-form design of maximally flat R-regular Mth-band FIR filter,” Proc. of IEEE Int'l Symp. on Circuits and Systems, Geneva, Switzerland, May 2000. C. C. Tseng and S. C. Pei,”Discrete-time Hilbert transformer,”Proc. of IEEE Int'l Symp. on Circuits and Systems, Geneva, Switzerland, May 2000. S. C. Pei and J. J. Ding,”Eigenfunctions of the canonical transform and the self-imaging problems in optical system,”Proc. of IEEE Int'l Conf. on Acoustics, Speech and Signal Processing, Istanbul, Turkey, June 2000. 89年度計畫產出列表

50. S. C. Pei and P.H. Wang,”Closed-form design of generalized maximally flat low-pass FIR filters using generating functions,”Proc. of IEEE Int'l Conf. On. Lin-shan Lee, Lee-Feng Chien, “Live Lexicons and Dynamic Corpora Adapted to the Network Resources for Chinese Spoken Language Processing Applications in an Internet Era”, 2nd International Conference on Language Resources and Evaluation, Athens, Greece, MayJune 2000, pp. 931-936. Berlin Chen, Hsin-min Wang, Lin-shan Lee, “Retrieval of Broadcast News Speech in Mandarin Chinese Collected in Taiwan Using Syllable-level Statistical Characteristics”, IEEE International Conference on Acoustics, Speech and Signal Processing, Istanbul, Turkey, June 2000, SPP9.14, pp. III-1771-1774 Bor-shen Lin, Lin-shan Lee, “Fundamental Performance Analysis for Spoken Dialogue Systems Based on A Quantitative Simulation Approach”, IEEE International Conference on Acoustics, Speech and Signal Processing, Istanbul, Turkey, June 2000, SPL9.2, pp. II-1221-1224. 89年度計畫產出列表