Extensions of wavelets

1. Extensions of wavelets ECE 802

2. M-Band Wavelet Systems • Generalization of dyadic wavelets • Scale factor of M • More flexible tiling of the time-frequency plane

3. Properties • Scaling Equation: • Existence and Orthogonality: • M-1 wavelets:

4. MRA • At each scale j, there are M-1 wavelet functions and one scaling function • If the wavelets are orthogonal to the scaling function at the same scale

5. Analysis and Synthesis • The expansion is • The filter bank structure will now have M branches • Gives a mixture of a logarithmic and linear frequency resolution. • Easier to design for M=2k

6. Wavelet Packets • M=2 results in a logarithmic frequency resolution. The low frequencies have narrow bandwidths and the high frequencies have wide bandwidths. • Wavelet packet system proposed by Coifman • Adjustable resolution of frequencies at high frequencies • Computational complexity O(NlogN)

7. Wavelet Packet Decomposition • In order to have higher resolution decomposition at high frequencies, iterate the highpass wavelet branch • Split both the lowpass and highpass bands at all stages • Evenly spaced frequency resolution • In DWT we consider the outputs of each channel. • In WPD, we have more outputs than inputs redundant system • Choose an independent set as basis (not one unique basis)

8. Optimization Criteria • Search based on minimizing a cost function on the transform coefficients. • Binary search algorithm for additive cost function • How do we choose the ‘best’ basis? • Shannon entropy • Thresholding the coefficients • Log Energy • Norm of the coefficients • Two approaches: • Choose a particular decomposition based on the signal class • Adapt the decomposition to each signal

9. Complexity • P(J): The number of J-scale orthonormal wavelet packet transforms • P(1)=1 • P(J)=P(J-1)2+1 • Application: FBI standard for fingerprint image compression

10. Haar Wavelet Packets

11. Wavelet Packet Tree

12. Optimization Functions • Shannon Entropy: • Norm: • Log-Energy: • Threshold Entropy: Number of times the coefficient is larger than a threshold

13. Example: Minimum Entropy Decomposition • Start with a constant original signal. w00 = ones(1,16)*0.25; • Compute entropy of original signal. • e00 = wentropy(w00,'shannon') e00 = 2.7726 • Then split w00 using the haar wavelet. • [w10,w11] = dwt(w00,'db1'); • Compute entropy of approximation at level 1. • e10 = wentropy(w10,'shannon') e10 = 2.0794 • The detail of level 1, w11, is zero; the entropy e11 is zero. Due to the additivity property the entropy of decomposition is given by e10+e11=2.0794. This has to be compared to the initial entropy e00=2.7726. We have e10 + e11 < e00, so the splitting is interesting. • Now split w10 (not w11 because the splitting of a null vector is without interest since the entropy is zero). • [w20,w21] = dwt(w10,'db1'); • We have w20=0.5*ones(1,4) and w21 is zero. The entropy of the approximation level 2 is • e20 = wentropy(w20,'shannon') e20 = 1.3863 • Again we have e20 + 0 < e10, so splitting makes the entropy decrease. • Then • [w30,w31] = dwt(w20,'db1'); e30 = wentropy(w30,'shannon')     e30 = 0.6931 [w40,w41] = dwt(w30,'db1')     w40 = 1.0000     w41 = 0 e40 = wentropy(w40,'shannon')     e40 = 0 • Perform wavelet packets decomposition of the signal s. • t = wpdec(s,4,'haar','shannon');

14. Best Tree

15. Overcomplete Representations, Frames, Redundant Transforms • There are many cases where a single basis is not effective for signal representation. • Example: Fourier basis is good for sinusoids, but bad for transients • Efficiency of the transform can be improved by combining several basis systems. • Combination of basis systemsOvercomplete • Collection of basis systems is called a dictionary.

16. Desired Criteria • Sparsity: Efficient representation • Separation: Better ability to separate a mixture of signals • Superresolution: Higher resolution or detail compared to a single basis • Stability: Robust under noise, the selected atoms do not change • Speed

17. Definitions • Frame: Generalization of a basis, a collection of functions that span the vector space, but are not linearly independent • The frame condition: • 0<A<B<∞ • If A=B, tight frame • If A=B=1, orthonormal basis

18. Frame Examples • Tight Frame: 4 basis functions in 3-dimensional space

19. Matching Pursuit • Matching pursuit (MP) algorithm finds a sub-optimal solution to the problem of an adaptive approximation of a signal in a redundant set (dictionary) of functions. • Look for a linear expansion of a signal in terms of elements (atoms) of a dictionary.

20. Algorithm [Mallat, Zhang 1993] • At each step, try to find the element of the dictionary that ‘best’ fits the signal. • Energy Conservation: • For a complete dictionary as M∞, the residue should go to zero. • Stopping Criteria: Threshold the residue or pre-determine M

21. Dictionary • Commonly used dictionary: Gabor functions, dictionary of time-frequency atoms • General and compact model for oscillations • Compact time-frequency localization • Restrict the search to a range of time, frequency, and scale values

22. Applications • EEG Spike Parametrization:

23. Extensions • Multichannel MP: Jointly represent a class of signals using the same elements of the dictionary • Orthogonal Matching Pursuit (OMP): • Efficient greedy algorithm • Applies Gram-Schmidt orthogonalization to the selected atoms before computing the residue • The selected atoms are orthogonalized with respect to the residue • Faster convergence than MP

24. Basis Pursuit[Chen, Donoho] • Convex optimization: Find the representation that minimizes the l1 norm of the coefficients • Solved using linear programming • Nearly linear time • l1 norm guarantees sparsity, l2 norm does not (Method of Frames)

25. Examples • FM-Cosine signal:

26. Comparisons • MP and OMP are iterative algorithms. • MP starts from an ‘empty’ signal model and builds it up one atom at a time • BP starts from a ‘full’ model and iteratively improves the full model. • Wavelet Packet Decomposition (Best Orthogonal Basis) focuses only on the orthogonal bases. • MOF l2 solution, not sparse, can be noisy.