Basic Concepts and Definitions

1. Basic Concepts and Definitions Vector and Function Space. A finite or an infinite dimensional linear vector/function space described with set of non-unique spanning vectors/functions and an inner product defined in the space.

2. Linear Independence • Linear independence is necessary for orthogonality but not sufficient • Overcomplete Representation of a Function/Signal. A linear expansion of a function with a bases set which are linearly dependent

3. Inner product space • . A function space with a functional ( linear or bilinear) defined in the space. Inner product is an assignment of a scalar to a functional described between every pair of elements in the function space. It is used to describe relation between pair of elements such as distance between two elements in a given space, length (norm) of a function, angle between two elements in the given space, etc..

4. Basis Set • Basis set are set of functions with an inner product between every pair of the basis that is equal to zero. An example is set of orthogonal spanning vectors in a given space. Basis functions are used to describe all the elements in a given function space.

5. Dimension of a function space • . Dimension of a given function space is the number of basis set sufficient to describe all the elements of the function space. Dimension of a function space is independent of the particular basis set chosen for the space.

6. Non-uniqueness of Bases Set. Bases sets are not unique since a linear combination of the set, can also be used as a bases set.

7. A Complete Space • . A space in which every Cauchy sequence converges to a vector in the space, is a complete space. It can also be interpreted as a set by which every element of the space including the elements of space boundary can be described by the basis set

8. Hilbert space • . An infinite or finite dimensional inner product space which is also a complete space. Hilbert space is an L2 norm space as defined below.

9. An L2 norm space. Space with a norm defined as the integral of squared signal values i.e. to be a finite value. In finite dimensional space, it corresponds to square summable functions ( finite energy of the signal ). L2 norm is described in terms of the inner product between pair of elements defined in a given space. • <f(t),f(t)>= =

10. Inner product rules( please refer to class discussion and reference material), • Definition of length of a function: <f(t),f(t)>1/2, • Definition of a distance between two functions: • <f(t)-g(t),f(t)-g(t)>1/2=[ ]1/2

11. Riesz basis and concept of frame • Consider an inner product space with a norm induced by the inner product defined in the space. Then a set {vi, i=1,..n } not necessarily linearly independent, is a Reisz basis if every element in the space f(t) can be expressed as a linear combination of the set {v} having non-unique coefficients of the expansion, and. • Frames may be discussed in a more detail at a later stage in the course

12. Frames b- Such that transform of the signal and coefficients {ck} defined as orthogonal projection of f(t) on each element of the basis, are bounded, i.e. there exists positive scalors A and B such that norm of signal and coefficients of the orthogonal projection {ck} can be expressed in an inequality relation given as follows: • A ≤ ∑jk|cjk|2 /||f(t)||2 ≤ B (1) • A||f(t)||2 ≤ ∑jk|cjk|2 ≤ B||f(t)||2

13. Frames • cjkis the projection of f(t) on the frame set φjk(t) i.e. cjk=<f(t),φjk(t)>. Accordingly, coefficient norm when normalized to signal energy, is constrained within lower and upper bounds A and B respectively. A and B are scalars independent of f(t) and A>0 and B<∞.

14. Comments • Comments: • In representation of a signal using Riesz basis, there exists a redundancy the degree of which is determined by constants A and B. When A=B=1, we have orthogonal set ( i.e. with no redundancy since we have a unique representation). Redundant representation as stated above is obtained by decomposition of f(t) by set of functions that may be linearly dependent and referred to as frame. • Commonly used degree of redundancy is given by (A+B)/2.

15. Examples • Examples of the spanning vectors which is a basis but not a Riesz basis is polynomial set 1. t. t2,…( an infinite dimensional basis set). • Frame structure of a basis is useful in function analysis as it allows redundancy in function expansion. An expansion of a function with a bases set which is constructed from an arbitrary translation of a basis with a finite support length, results in a redundant set.

16. Continuous wavelet transform • Continuous wavelet transform having an arbitrary translation and scale parameters, is a redundant transform that can be described in terms of frame structure. Decomposition generates redundant coefficients in a signal representation in transform domain.

17. Function Expansion, Signal Decomposition Goals of Function Expansion: • Representation of signal in a domain other than the given signal domain that is considered to be suitable for a given application, e.g. frequency domain for spectral analysis, transformed domain for approximation, coding, transmission, etc.

18. Optimal or near optimal signal description • In engineering applications we search for bases that are considered suitable to meet the following objectives. • Efficient function approximation, • Reduction in computational complexity, • Effective information extraction. • We search for bases that are suitable for compression in the sense that signal information can be extracted and represented effectively using a few number of coefficients of expansion in the transformed domain. (refer to discussion in the class).

19. Wavelets in Function Expansion • Wavelets are considered to be near optimal bases for a wide class of signals found in engineering applications • Examples of non-wavelet bases used in signal analysis: Karhunen-Loeve Transform (KLT) or Principal Component Analysis, Laguerre bases, Short-time Fourier transform, Vigner-Wille Transform.

20. Requirements • Requirements for wavelet bases to be considered as suitable in signal processing applications are as follows: • Well localized both in time and freq. • Oscillatory behavior to capture the transient signal features • Higher order of moment cancellation necessary for compression • Regularity for desirable smoothness property of wavelets and corresponding filter. • Computational efficiency for signal processing (such as Fast Wavelet Transform used in quadratic filter bank (QMF) of DWT)

21. Biorthogonal and Orthogonal Analysis • Biorthogonal vs. Orthogonal Analysis • In biorthogonal expansion, two sets of basis functions are utilized. Basis functions of each set are linearly independent of each other but are not orthogonal while the elements within the two sets are pair-wise orthogonal to each other.

22. Biorthogonal and Orthogonal Analysis • A given signal can be represented by either one of the basis sets ( please refer to class discussion for further analysis). The selection of one wavelet for decomposition and other for reconstruction or vice versa, is determined by requirement for decomposition and reconstruction. For example, if we are looking for compression i.e. a few number of coefficients during decomposition stage, we will select wavelet which has higher moment cancellation

23. Decomposition of a Signal • Decomposition of a signal is achieved by the projecting the signal onto one set where the coefficient of the projection in one set are used to describe the signal in terms of the dual bases set ( refer to class notes for derivations and discussions).

24. Biorthogonal vs. Orthogonal Analysis • Advantages and Disadvantages: • a) Orthogonal basis • Orthogonal expansion yields a unique expansion coefficients. It results in an efficient and non-redundant signal representation and is computationally more attractive. • Non-robustness of the signal expansion to errors where the occurrence of an error in derivation of the coefficient or loss of single coefficient value, can result in inaccuracies and degradation of overall signal representation

25. Biorthogonal basis • In biorthogonal expansion there exists an added degree of freedom for function expansion i.e., one can choose either set for a suitable signal representation • In wavelet analysis, dual basis provides possibility of choosing two different bases: one for analysis and another for synthesis stage and thus an additional degree of freedom is available to design wavelet filters.

26. Frame and Redundancy • In frames redundancies are accepted, • Signal representation is not unique, • Robustness property. Loss of a single coefficients may be recovered from other coefficients and have minor effect in the overall results • We can have different order of redundancy in a given expansion determined by the requirement of a signal expansion and computational efficiency requirements. Higher redundancy has lower computational efficiency but higher robustness against errors

27. Parseval Relation • In an orthogonal expansion, Parseval relation plays an important and useful role in providing information about the distribution of signal energy along each basis ( refer to class discussion for further discussion). However, this is not true if biorthogonal expansion is utilized. • In wavelet analysis, absolute value of wavelet coefficients also contain information about the signal. As such they are used for function approximation and information extraction. Wavelet bases are referred as unconditional bases set ( refer to class discussion and chap.2 of course text book)