Introduction to Fourier Transform and Time-Frequency Analysis

1. Introduction to Fourier Transform and Time-Frequency Analysis Speaker: Li-Ming Chen Advisor: Meng-Chang Chen, Yeali S. Sun

2. Outline • Periodic Phenomena and Fourier Series • Non-periodic Phenomena and Fourier Transform • Why Needs Time-Frequency Analysis? • Wavelet Transform and its Applications Speaker: Li-Ming Chen

3. History of Fourier Series • Fourier series • Named in honor of Joseph Fourier (1768-1830) • Originally used to solve “heat equation” • Initially, the paper submitted in 1807 • However, the theory published in 1822 • A Fourier series decomposes a periodic function (or periodic signal) into a sum of simple oscillating functions, namely sines and cosines (or complex exponentials) (Wikipedia) Speaker: Li-Ming Chen

4. Fourier series model a general periodic phenomena by using basic building blocks; Example sin(2*pi*t) + sin(2*pi*2*t) + sin(2*pi*5*t) 3 0 Amplitude -3 1000 2000 3000 4000 5000 Time (ms) 1 sin(2*pi*t) 0 1 sin(2*pi*2*t) 1 sin(2*pi*5*t) Speaker: Li-Ming Chen

5. Fourier Transform & Fourier Series • We can consider: • Fourier transform as a limiting case of Fourier series in concerned with analysis of non-periodic phenomena • Applications of Fourier analysis: • Physics, partial differential equations, signal processing, imaging, acoustics, cryptography… • Why so popular, so applicable?  due to some properties: • Transforms are linear operators and “usually” invertible • Using complex exponential (computational convenience) • Convenient to compute convolution operation • Has fast Fourier transform (FFT) algorithm Speaker: Li-Ming Chen

6. What is a Periodic Phenomenon? • Periodic: • Some pattern that repeated and repeat regularly Called “Frequency” (e.g., number of repetitions of patterns in a second) Periodic in Time Periodic Periodic in Space Called “Period” (e.g., Heat) The temperature on a ring is periodic (depend on position) ※ Periodic in Time and Space!? e.g., wave motion Speaker: Li-Ming Chen

7. sin and cos are periodic functions Can configure their frequency, amplitude, and phase “One period, many frequencies” Let’s consider with period 1 also has period 1 !! We can modify and combine these building blocks to model very general periodic phenomena Using Periodic Function to Model Periodic Phenomena 1 Amplitude 1 1 0 1 Speaker: Li-Ming Chen Time (ms)

8. Fourier Series • A periodic function f(t) can be represented as • Using 和角公式 • Such that, “partial sums” of the Fourier series for ƒ these are constants an and bn called Fourier coefficients of f. (include the info. of amplitude and phase) Speaker: Li-Ming Chen

9. Using Complex Exponential • Euler’s formula • Fourier Series (General Form) • But, how to get ck ? (how to compute Fourier coefficients?) Ck is also a complex number Speaker: Li-Ming Chen

10. How to Compute ck ? • Say, isolate cn and then solve cn • …(utilizing the property of complex exponential), we can get and (Fourier coefficient) [Analysis] f(t) is the function we observed (Fourier series) [Synthesis] Speaker: Li-Ming Chen

11. Outline • Periodic Phenomena and Fourier Series • Non-periodic Phenomena and Fourier Transform • Why Needs Time-Frequency Analysis? • Wavelet Transform and its Applications Speaker: Li-Ming Chen

12. Fourier Transform (FT) • Not all phenomena are periodic, and periodic phenomena will die out eventually… • Let’s view non-periodic function as limiting case of period function as period  ∞ • Fourier transform is invertible • FT is the generalization of Fourier coefficient • Inverse FT is the generalization of Fourier series Speaker: Li-Ming Chen

13. Fourier Transform (FT) (cont’d) • Definition • FT: • Inverse FT: Speaker: Li-Ming Chen

14. Example (periodic function) all the amplitudes are 1 Magnitude/amplitude of the Freq. is half of the original amplitude f(t) = cos(2pi*5t) + cos(2pi*10t) + cos(2pi*20t) + cos(2pi*50t) FT Time (ms) Frequency (Hz) 5 10 20 50 Speaker: Li-Ming Chen

15. Example (non-periodic function) Integrals of the function in GREEN FT Integrals of the function in Red 3 5 Time (s) Freq. (Hz) Speaker: Li-Ming Chen Compute c3 Compute c5

16. Outline • Periodic Phenomena and Fourier Series • Non-periodic Phenomena and Fourier Transform • Why Needs Time-Frequency Analysis? • Wavelet Transform and its Applications Speaker: Li-Ming Chen

17. Example • What if the periodic components occur at different time? • Non-stationary ※ FT can still find the frequencies (4 peaks) cos(2pi*5t) | cos(2pi*10t) | cos(2pi*20t) | cos(2pi*50t) The noise is due to the sudden change between the freq. FT Time (ms) Frequency (Hz) Speaker: Li-Ming Chen 5 10 20 50

18. Why Needs Time-Frequency Analysis? • Drawback of Fourier Transform • Time information is lost !! •  unable to tell when in time a particular event (Freq.) took place • Not a problem for signals which are stationary • But when we have a signal which changes with time (non-stationary), we need more information about the signal behavior • (Idea) can we assume that, some portion of a non-stationary signal is stationary? • If so, we can do time-frequency analysis Speaker: Li-Ming Chen

19. Short Term Fourier Transform (STFT) • In STFT, the signal is divided into small enough segments, where these segments (portions) of the signal can be assumed to be stationary. • Assume the signal is NOT changed for that particular period • Using window function(mask function) to cover those periods time frequency Speaker: Li-Ming Chen

20. STFT Window function could be: Rectangular function, Gaussian function, … The length of this window function is pre-assigned. (the length will affect the results) τ= t1’ τ=t3’ τ= t2’ Speaker: Li-Ming Chen

21. Example (STFT) 300Hz 200Hz 100Hz 50Hz 0~300ms: 300Hz, 300~600ms: 200Hz, 600~800ms: 100Hz, 800~1000ms: 50Hz. if we use FFT to analyze this signal, we might have good frequency resolution but poor time resolution. Speaker: Li-Ming Chen

22. Poor freq. resolution, Good time resolution f τ 4 Gaussian window Func. with different length Good freq. resolution, Poor time resolution Speaker: Li-Ming Chen

23. Drawback of STFT • Heisenberg uncertainty principle (海森堡測不準原理) • One can not know the exact time-frequency representation of a signal • i.e., One can not know what spectral components exist at what instances of times. • What one can know are the time intervals in which certain band of frequencies exist, which is a resolution problem. • Drawback of STFT is due to the constant length windows • STFT: single-resolution for complete signal !! • Need multi-resolution Speaker: Li-Ming Chen

24. Outline • Periodic Phenomena and Fourier Series • Non-periodic Phenomena and Fourier Transform • Why Needs Time-Frequency Analysis? • Wavelet Transform and its Applications Speaker: Li-Ming Chen

25. Multi-resolution Analysis (MRA) • MRA is designed to give • “good time resolution and poor frequency resolution” at high frequencies • and “good frequency resolution and poor time resolution” at low frequencies. • This approach makes sense especially when the signal at hand has high frequency components for short durations and low frequency components for long durations. • Common practical applications are often of this type Speaker: Li-Ming Chen

26. Continuous Wavelet Transform (CWT) • 2 main differences between STFT and CWT: • 1.) The Fourier transforms of the windowed signals are not taken, and therefore single peak will be seen corresponding to a sinusoid, i.e., negative frequencies are not computed. • 2.) The width of the window is changed as the transform is computed for every single spectral component, which is probably the most significant characteristic of the wavelet transform. Speaker: Li-Ming Chen

27. Continuous Wavelet Transform • Forward wavelet transform: • Inverse wavelet transform: Location (time, translation) (t): mother wavelet energy normalization 能量守恆 Scale (~ 1/freq) a,b(t) is dual orthogonal to (t) Speaker: Li-Ming Chen

28. Mother Wavelet, (t) • (t) is a prototype for generating the other window functions •  (t)(t/b) is the function with different scale derived from the mother wavelet • b < 1, compresses the signal, (low scale  detailed view) • b > 1, dilates the signal, (high scale  non-detailed global view) (a: 調整位置) 能量守恆， 故面積一樣 (b: 調整寬度) Speaker: Li-Ming Chen

29. Example 30Hz 20Hz 10Hz 5Hz CWT Unlike the STFT which has a constant resolution at all times and frequencies, the WT has a good time and poor frequency resolution at high frequencies, and good frequency and poor time resolution at low frequencies Speaker: Li-Ming Chen

30. Discrete Wavelet Transform The discrete wavelet transform is very different from the continuous wavelet transform. It is simpler and more useful than the continuous one. (較為實用) down sampling lowpass filter xL[n] x[n] 的低頻成份 g[n]  2 x1,L[n] x[n] highpass filter down sampling xH[n] x1,H[n]  2 x[n] 的高頻成份 h[n] Speaker: Li-Ming Chen

31. Example:ImageCompression 原影像 x1,H2[m, n] x1,L[m, n] (n軸高頻) (低頻部分) 2-D DWT 的結果 x1,H1[m, n] x1,H3[m, n] (m軸高頻) Speaker: Li-Ming Chen

32. Example: Image Compression 和原圖類似，但資料量僅1/64 重複對低頻部分 進行 DWT Speaker: Li-Ming Chen

33. Example: Traffic Volume Anomaly Detection • Jianning Mai, Chen-Nee Chuah Ashwin Sridharan, Tao Ye, Hui Zang, “Is Sampled Data Sufficient for Anomaly Detection?,” IMC 2006 • Discrete wavelet transform (DWT) based detection • An off-line algorithm • 3 steps: • Decomposition • Re-synthesis • Detection Speaker: Li-Ming Chen

34. (Volume Anomaly Detection)DWT-based Detection • Decomposition • Input signal X will be separate into detail information D1 or approximation information A1 (level 1) • Repeated using Ai as an input to generate Di+1 and Ai+1 at the next level • , each level j represents the strength of a particular frequency in the signal • Higher value of j indicating a lower frequency • Re-synthesis • Aggregate the various frequency levels into low, mid and high bands • , Speaker: Li-Ming Chen

35. (Volume Anomaly Detection)DWT-based Detection (cont’d) • Detection • Compute local variability of the high and mid bands • Compute the variance of data within a sliding window • Compute deviation score • The ratio between the local variance within the window and the global variance • Windows with deviation scores higher than a threshold are marked as volume anomalies Speaker: Li-Ming Chen

36. Reference • Wikipedia (keyword search :p) • The Wavelet Tutorial (by Robi Polikar): • http://users.rowan.edu/~polikar/WAVELETS/WTtutorial.html • Standford Video Course (The Fourier Transform and its Applications) (by Brad G. Osgood): • http://academicearth.org/courses/the-fourier-transform-and-its-applications • Wavelets and Time-Frequency Analysis (by Mudasir) • http://hubpages.com/hub/wavelets1 Speaker: Li-Ming Chen