Digital Image Processing

1. بسمه‌تعالي Digital Image Processing H.R. Pourreza Wavelet and Multiresolution Processing(Chapter 7) H.R. Pourreza

2. Introduction (Robi Polikar, Rowan University)What is a Transform and Why do We Need One? • Transform: A mathematical operation that takes a function or sequence and maps it into another one • Transforms are good things because… • The transform of a function may give additional /hidden information about the original function, which may not be available /obvious otherwise • The transform of an equation may be easier to solve than the original equation • The transform of a function/sequence may require less storage, hence provide data compression / reduction • An operation may be easier to apply on the transformed function, rather than the original function (recall convolution) H.R. Pourreza

3. f F f T T-1 IntroductionProperties of Transforms • Most useful transforms are: • Linear: we can pull out constants, and apply superposition • One-to-one: different functions have different transforms • Invertible:for each transform T, there is an inverse transform T-1 using which the original function f can be recovered (kind of – sort of the undo button…) • Continuous transform: map functions to functions • Discrete transform: map sequences to sequences H.R. Pourreza

4. IntroductionWhat Does a Transform Look Like? • Complex function representation through simple building blocks • Compressed representation through using only a few blocks (called basis functions / kernels) • Sinusoids as building blocks: Fourier transform • Frequency domain representation of the function H.R. Pourreza

5. IntroductionWhat Transforms are Available? • Fourier series • Continuous Fourier transform • Laplace transform • Discrete Fourier transform • Z-transform H.R. Pourreza

6. Jean B. Joseph Fourier (1768-1830) IntroductionFourier Who …? “An arbitrary function, continuous or with discontinuities, defined in a finite interval by an arbitrarily capricious graph can always be expressed as a sum of sinusoids” J.B.J. Fourier December, 21, 1807 H.R. Pourreza

7. IntroductionHow Does FT Work Anyway? • Recall that FT uses complex exponentials (sinusoids) as building blocks. • For each frequency of complex exponential, the sinusoid at that frequency is compared to the signal. • If the signal consists of that frequency, the correlation is high  large FT coefficients. • If the signal does not have any spectral component at a frequency, the correlation at that frequency is low / zero,  small / zero FT coefficient. H.R. Pourreza

8. IntroductionFT at Work H.R. Pourreza

9. F F F IntroductionFT at Work H.R. Pourreza

10. F IntroductionFT at Work H.R. Pourreza

11. IntroductionFT at Work Complex exponentials (sinusoids) as basis functions: F H.R. Pourreza An ultrasonic A-scan using 1.5 MHz transducer, sampled at 10 MHz

12. IntroductionStationary and Non-stationary Signals • FT identifies all spectral components present in the signal, however it does not provide any information regarding the temporal (time) localization of these components. Why? • Stationary signals consist of spectral components that do not change in time • all spectral components exist at all times • no need to know any time information • FT works well for stationary signals • However, non-stationary signals consists of time varying spectral components • How do we find out which spectral component appears when? • FT only provides what spectral components exist, not where in time they are located. • Need some other ways to determine time localization of spectral components H.R. Pourreza

13. Concatenation IntroductionStationary and Non-stationary Signals • Stationary signals’ spectral characteristics do not change with time • Non-stationary signals have time varying spectra H.R. Pourreza

14. 50 Hz 25 Hz 5 Hz IntroductionNon-stationary Signals Perfect knowledge of what frequencies exist, but no information about where these frequencies are located in time H.R. Pourreza

15. IntroductionFT Shortcomings • Complex exponentials stretch out to infinity in time • They analyze the signal globally, not locally • Hence, FT can only tell what frequencies exist in the entire signal, but cannot tell, at what time instances these frequencies occur • In order to obtain time localization of the spectral components, the signal need to be analyzed locally, BUT • HOW ? H.R. Pourreza

16. IntroductionShort Time Fourier Transform (STFT) • Choose a window function of finite length • Put the window on top of the signal at t=0 • Truncate the signal using this window • Compute the FT of the truncated signal, save. • Slide the window to the right by a small amount • Go to step 3, until window reaches the end of the signal • For each time location where the window is centered, we obtain a different FT • Hence, each FT provides the spectral information of a separate time-slice of the signal, providing simultaneous time and frequency information H.R. Pourreza

17. IntroductionShort Time Fourier Transform (STFT) H.R. Pourreza

18. IntroductionShort Time Fourier Transform (STFT) Frequency parameter Time parameter Signal to be analyzed FT Kernel (basis function) STFT of signal x(t): Computed for each window centered at t=t’ Windowing function Windowing function centered at t=t’ H.R. Pourreza

19. 1 1 0.5 0.5 0 0 -0.5 -0.5 -1 -1 -1.5 -1.5 0 100 200 300 0 100 200 300 1 1 0.5 0.5 0 0 -0.5 -0.5 -1 -1 -1.5 -1.5 0 100 200 300 0 100 200 300 IntroductionSTFT at Work Windowed sinusoid allows FT to be computed only through the support of the windowing function H.R. Pourreza

20. IntroductionSTFT 300Hz 200Hz 100Hz 50Hz H.R. Pourreza

21. IntroductionSTFT • STFT provides the time information by computing a different FTs for consecutive time intervals, and then putting them together • Time-Frequency Representation (TFR) • Maps 1-D time domain signals to 2-D time-frequency signals • Consecutive time intervals of the signal are obtained by truncating the signal using a sliding windowing function • How to choose the windowing function? • What shape? Rectangular, Gaussian, Elliptic…? • How wide? H.R. Pourreza

22. IntroductionSelection of STFT Window Two extreme cases: • W(t) infinitely long:  STFT turns into FT, providing excellent frequency information (good frequency resolution), but no time information • W(t) infinitely short:  STFT then gives the time signal back, with a phase factor. Excellent time information (good time resolution), but no frequency information Wide analysis window  poor time resolution, good frequency resolution Narrow analysis window  good time resolution, poor frequency resolution Once the window is chosen, the resolution is set for both time and frequency. H.R. Pourreza

23. IntroductionHeisenberg Uncertainty Principle Frequency resolution: How well two spectral components can be separated from each other in the transform domain Time resolution: How well two spikes in time can be separated from each other in the transform domain Both time and frequency resolutions cannot be arbitrarily high!!! We cannot precisely know at what time instance a frequency component is located. We can only know what interval of frequencies are present in which time intervals H.R. Pourreza

24. a=0.01 Gaussian window function: a=0.0001 a=0.00001 IntroductionSTFT H.R. Pourreza

25. IntroductionWavelet Transform • Overcomes the preset resolution problem of the STFT by using a variable length window • Analysis windows of different lengths are used for different frequencies: • Analysis of high frequencies Use narrower windows for better time resolution • Analysis of low frequencies  Use wider windows for better frequency resolution • This works well, if the signal to be analyzed mainly consists of slowly varying characteristics with occasional short high frequency bursts. • Heisenberg principle still holds!!! • The function used to window the signal is called the wavelet H.R. Pourreza

26. IntroductionWavelet Transform A normalization constant Translation parameter, measure of time Scale parameter, measure of frequency Signal to be analyzed Continuous wavelet transform of the signal x(t) using the analysis wavelet (.) The mother wavelet. All kernels are obtained by translating (shifting) and/or scaling the mother wavelet Scale = 1/frequency H.R. Pourreza

27. IntroductionWT at Work High frequency (small scale) Low frequency (large scale) H.R. Pourreza

28. IntroductionWT at Work H.R. Pourreza

29. IntroductionWT at Work H.R. Pourreza

30. IntroductionWT at Work H.R. Pourreza

31. IntroductionTime and Frequency Resolution H.R. Pourreza

32. Background H.R. Pourreza

33. Image Pyramids H.R. Pourreza

34. Image Pyramids • Compute a reduced-resolution approximation of the input image • Filtering (Averaging, Gaussian) • Down-sampling • Up-sample the output of the previous by a factor 2 • Compute the difference between the prediction of step 2 and the input toStep 1. H.R. Pourreza

35. Image Pyramids H.R. Pourreza

36. Multi-Resolution Expansion • In Multi-resolution Analysis (MRA), a Scaling Function is used to create a series of approximations of a function or image, each differing by a factor 2 from its nearest neighboring approximations. Additional functions, called Wavelet, are used to encode the difference in information between adjacent approximation H.R. Pourreza

37. If the expansion is UNIQUE- that is, there is only one set of for any given - the are called basis functions, and the expansion set, , is called a BASIS for the class of functions that can be so expressed. Multi-Resolution ExpansionSeries Expansion Real-valued expansion functions Real-valued expansion coefficients The expressible functions form a function space that is referred to as the close span of the expansion set H.R. Pourreza

38. Multi-Resolution ExpansionSeries Expansion Dual Functions H.R. Pourreza

39. Multi-Resolution ExpansionSeries Expansion CASE 1: Expansion functions form an orthonormal basis CASE 2: Expansion functions are not orthonormal, but are an orthogonal basis (biorthogonal basis) CASE 3: Expansion set is not a basis H.R. Pourreza

40. Multi-Resolution ExpansionScaling Functions H.R. Pourreza

41. Multi-Resolution ExpansionScaling Functions H.R. Pourreza

42. Multi-Resolution ExpansionMRA Requirements • The scaling functions is ORTHOGONAL to its integer translations. • The subspace spanned by the scaling function at low scales are nested within those spanned at higher scales. • The only function that is common to all Vj isf(x)=0 • Any function can be represented with arbitrary precision H.R. Pourreza

43. Multi-Resolution ExpansionMRA Requirements Scaling Vector H.R. Pourreza

44. Multi-Resolution ExpansionWavelet Functions Union of Spaces H.R. Pourreza

45. Multi-Resolution ExpansionWavelet Functions The all members of Vj are orthogonal to the Wj We can now express the space of all square integrable functions as: or or H.R. Pourreza

46. Multi-Resolution ExpansionWavelet Functions Approximation of f(x) H.R. Pourreza

47. Multi-Resolution ExpansionWavelet Functions H.R. Pourreza

48. Wavelet TransformWavelet Series Expansion Detail (Wavelet) Coefficients Approximation (Scaling) Coefficients If the expansion functions form an orthonormal basis: H.R. Pourreza

49. Wavelet TransformWavelet Series Expansion H.R. Pourreza

50. Wavelet TransformDiscrete Wavelet Transform (DWT) H.R. Pourreza