Fundamentals from Real Analysis

1. Fundamentals from Real Analysis Ali Sekmen, Ph.D.2 Professor and Department Chair Department of Computer Science College of Engineering Tennessee State University 1st Annual Workshop on Data Sciences

2. Outline • Spaces • Normed Vector Space • Banach Space • Inner Product Space • Hilbert Space • Metric Space • Topological Space • Subspaces and their Properties • Subspace Angles and Distances

3. Big Picture Vector Spaces Normed Vector Spaces Inner-Product Vector Spaces Hilbert Spaces Banach Spaces

4. Big Picture Vector Spaces Topological Spaces Normed Space Inner Product Space Metric Spaces

5. What is a Vector Space? • A vector space is a set of objects that may be added together or multiplied by numbers (called scalars) • Scalars are typically real numbers • But can be complex numbers, rational numbers, or generally any field • Vector addition and scalar multiplication must satisfy certain requirements (called axioms)

6. What is a Vector Space? • A vector space may have additional structures such as a norm or inner product • This is typical for infinite dimensional function spaces whose vectors are functions • Many practical problems require ability to decide whether a sequence of vectors converges to a given vector • In order to allow proximity and continuity considerations, most vector spaces are endowed with a suitable topology • A topology is a structure that allows to define “being close to each other” • Such topological vector spaces have richer theory • Banach space topology is given by a norm • Hilbert space topology is given by an inner product

7. What is a Vector Space? Associativity Commutativity Identity Element Inverse Element Compatibility Distributivity Identity Element

8. Vector Spaces - Applications • The Fourier transform is widely used in many areas of engineering and science • We can analyze a signal in the time domain or in the frequency domain We can show that is a measure for the amount of the frequency s What does this have to do with vector spaces?

9. Vector Spaces - Example • When we define the Fourier transform, we need to also define when the transform is well-defined • The Fourier transform is defined on a vector space For the sake of simplicity, we are not considering equivalence classes of functions that are the same almost everywhere

10. Vector Spaces - Example • What kind of functions have a Fourier series • Periodic functions • Let us say periodic functions • We can have Fourier series of functions that belongs to the vector space • If the function does not belong to this space, then the Fourier coefficients may not be well-defined

11. Basis • Every vector space has a basis • Every vector in a vector space can be written in a unique way as a finite linear combination of the elements in this basis • # of elements in a basis is dimension of vector space

12. Basis

13. Vector Spaces - Example • All of us know a very well-known vector space: • For general vector spaces, we need a concept that corresponds to length in • We use “norm” instead of “length”

14. Norm This concept is defined by mimicking what we know about

15. Euclidean Vector Norm

16. p-Norms Surface of the sphere of radius c includes all the vectors whose 2-norm is c Surface of the diamond of includes all the vectors whose 1-norm is c

17. Question • Are there any vector spaces for which we cannot define any norms? • Every finite dimensional real or complex topological vector space has a norm • There are infinite dimensional topological vector spaces that do not have a norm that induces the topology

18. Subspaces • Before we introduce some interesting vector spaces, we will now introduce “subspaces”

19. Subspaces A line through origin in is a 1-dimensional subspace of A plane through origin in is a 2-dimensional subspace of

20. Subspaces • Consider trigonometric polynomials, i.e., a finite linear combination of exponential functions • We can show that trigonometric polynomials form a subspace of

21. Subspaces

22. Closed Subset

23. A Vector Space

24. A Vector Space

25. A Vector Space

26. A Set of Polynomials

27. Set of Polynomials

28. Set of Polynomials

29. Set of Polynomials

30. Banach Space • An important group of normed vector spaces in which a Cauchy sequence of vectors converges to an element of the space • Banach spaces play an important role in functional analysis • In many areas of analysis, the spaces are often Banach spaces

31. Convergence

32. Convergence

33. Convergence

34. Cauchy Sequences

35. Convergence and Cauchy

36. Banach Space

37. Banach Space

40. Inner Product for Inner product is a very important tool for analysis in . It is a measure of angle between vectors

41. General Inner Product

42. Example Inner Product Spaces

43. Cauchy-Schwarz Inequality

44. Important • In any inner product vector space, regardless of the inner product we can always define a norm • But opposite is not true. We may not always define an inner product from a given norm

45. Important • We may define an inner product from a given norm if the parallelogram law holds for the norm • In this case, the induced inner product from the norm is defined as

46. Hilbert Space • A Hilbert space is a vector space equipped with an inner product such that when we consider the space with the corresponding induced norm, then that space is a Banach space

47. Hilbert Space

48. Big Picture Vector Spaces Normed Vector Spaces Inner-Product Vector Spaces Hilbert Spaces Banach Spaces

49. Metric Space

50. Topological Space • Inner Product Spaces • Angles • Normed Vector Spaces • Length • Not as strong as angles • Metric Spaces • Distance • Not as strong as length • Topological Spaces • What do we do if we do not have a notion of distance between elements? • Nearness • Not as strong as distance • Via neighborhoods