Reproducing Kernel Hilbert spaces Dr. M. Asaduzzaman Professor Department of Mathematics

1. Reproducing Kernel Hilbert spaces Dr. M. Asaduzzaman Professor Department  of  Mathematics University of Rajshahi Rajshahi -6205, Bangladesh E-mail: md_asaduzzaman@hotmail.com

2. Definition Let H be a Hilbert space comprising of complex valued functions on a set E. Then, H is called a reproducing kernel Hilbert space if there exists a function defined on EE such that (a) for every y E, K(,y)  H; (b) (reproducing property): for any fH, In this case, K(x,y) is called a reproducing kernel of H.

3. Theorem A reproducing kernel Hilbert space admits a unique reproducing kernel. Proof. Let H be a reproducing kernel Hilbert space comprising complex valued functions on a set E and let and be reproducing kernels of H. Then for any , we have Therefore, for all .

4. Theorem Let H be a Hilbert space comprising complex valued functions defined on a set E. Then, H is a reproducing kernel Hilbert space if and only if for every yE, the point evaluation at y is continuous on H; that is Ty:HC defined by Ty(f)=f(y) is a bounded linear functional on H, for every yE. Proof. Let H be a reproducing kernel Hilbert space and let K(x,y) be its reproducing kernel, then we have from the reproducing property and the Schwarz inequality, Hence, TyH*, for all yE. Conversely, let TyH*, for all yE. Then, from the Riesz representation theorem there exists a unique KyH such that f, Ky =Ty(f), for all f H. If we define K(x,y)= Ky, KxH for x,yE, then it is obvious that K(x,y) is a reproducing kernel of H.

5. Example Let be a finite dimensional complex vector space of functions. A finite dimensional space endowed with any inner product is always complete and therefore it is a Hilbert space. Let be an orthonormal basis in and define . Then for any in , and for any function in , we have . Any finite dimensional Hilbert space of functions has a reproducing kernel.

6. Let and let be the set of complex sequences such that . is a Hilbert space with the inner product if and . Consider . and

7. (3) For any n-dimensional space Hn comprising complex valued functions on a set E, let {w1,w2,…,wn} be any basis on Hn. Then any member f of Hn is uniquely expressible in the form Let A=(jk) be an n-square positive definite Hermitian matrix. Introduce norm in Hn by the formula where C* is the conjugate transpose of the coordinate vector of f. Then, we have that is, A is the Gramm matrix the of the system {w1,w2,…,wn}. Let B=(ij) be the conjugate inverse of A and define Then K(p,q) is the reproducing kernel for the space Hnequipped with the norm given by (1).

8. In particular, for any positive definite nnHermitian matrix A=(a) , Is the reproducing kernel for the for the vector space Cn equipped with The inner product Where * denotes complex conjugate transpose and is the complex conjugate inverse of A. (4) For any abstract set E and for any complex valued function f(p) (|f(p)|<) on E, the function is the reproducing kernel for the one dimensional space generated by f(p) and equipped with the inner product

9. is a Hilbert space with inner product . Definition of reproducing kernel does not strictly apply in that case. (6) Let and is defined almost everywhere as the derivative of . is a Hilbert space with the inner product . belongs to the class of Sobolev spaces.

10. Consider . The weak derivative of is the function and Then is a reproducing kernel. All infinite dimensional separable Hilbert spaces are isomorphic to which has a reproducing kernel. This property gives no guarantee that a given separable Hilbert space of functions has a reproducing kernel.

11. Definition Let E be a set and let be a complex valued function defined on EE. Then K(x,y) is called a positive matrix or a positive type function on E if, for any finite set of points in E and for any 1, 2,,n, in C

12. Theorem A positive matrix on a set E satisfies the following relations: is a positive matrix. Proof. From the definition of a positive matrix (a) is clear. Again, from the definition of a positive matrix,

13. for any x1, x2E and 1, 2 C. If we substitute x1 and x2 by x and y respectively, and if we put 1=1 and 2=z , then from (1) we have is a nonnegative real number for arbitrary x,yE and zC. Hence, and so is real for an arbitrary z C. Thus, we have (b). Next, let x, yEand let  be a complex number such that | |=1 and K(y,x)=|K(x,y)|. Let r be any real number. Then by setting x1=x, x2=y, 1=- and 2= r in (1), we have That is If K(y,y)=0, we must have K(x,y)=0; otherwise (2) is false for a large positive number r. If K(y,y)>0, it is necessary that and we have (c).

14. Since K(x,y) is a positive matrix on E, for any finite set of points in E and for any 1, 2,,n, in C Hence, we have (d).

15. Example (1) Any constant non negative function onis of positive type function. For every and for every

16. (2) Let be any non empty set. The delta function is of positive type. For every and for every

17. Lemma : Let be some Hilbert space with inner product and let . Then , the function is of positive type. Proof: For every and for every . Lemma tells us that writing in some is sufficient to prove definiteness of

18. Examples of positive matrices : (1) Let then any complex valued function on E can be considered as an element of ℂm . Let e1, e2, e3, …,en be any vectors from ℂm Let A be the mnmatrix whose i-th column vector is ei. Then the matrix K=AA* is a positive matrix on E, where A* denotes the conjugate transpose of A. (2) For x,yℝ and let Then K(x,y) is a positive type function on ℝ. For which is the inner product of and in (3) For and let Then is a positive type function on ℝ. For

19. Reproducing Kernels and Positive type functions Lemma: Any reproducing Kernel is Positive type function. Proof: Suppose is the reproducing kernel of For every and for every Hence is positive type function.

20. Lemma A real function defined on is a positive type function if and only if i.e. is symmetric and for every

21. Lemma In a reproducing kernel Hilbert space a sequence converging in the norm sense converges point wise to the same limit. Proof: Let converges to in the norm sense . Then Hence converges point wise to .

22. Theorem Let be any subspaces of ,the space of complex functions on , on which an inner product , with associate norm . In order that there exists a Hilbert space such that a) and the topology defined on by the inner product coincides with the topology induced on by b) has a reproducing kernel . It is necessary and sufficient that c) the evaluation functional are continuous on . • any Cauchy sequence in converging pointwise to 0 converges also to 0 in norm sense. Assumption d) is equivalent to d’)for any function in and any Cauchy sequence in converging pointwise to , converges also to in the norm sense.

23. Proof Let is exist with conditions a) and b) are satisfied. Then the evaluation functionals are continuous on and therefore on . Let be a cauchy sequencein converging pointwise to 0. As is complete, converges in the norm sense to some . Thus we have and Hence converges to 0 in norm sense.

24. Conversely , suppose c) and d) hold. Define as being the set of functions in for which there exists a Cauchy sequence in converging pointwise to . Obviously . Lemma: Let and belong to . Let and be two Cauchy sequences in converging pointwise to and . Then the sequence is convergent and its limit only depends on and . Proof We know that any Cauchy sequence is bounded. , by Cauchy –Schwarz inequality. This shows that is a Cauchy sequence in and therefore convergent.

25. In the same way, if and are two other Cauchy sequences in converging pointwise respectively to and , we have , and are Cauchy sequences in converging pointwise to From assumption d) they also converges to 0 in the norm sense . It follows that and have the same limit.

27. Lemma Let and be a Cauchy sequence in converging pointwise to . Then converges to in the norm sense. Proof Let and let be such that and . Fix . Then sequence is a Cauchy sequence in converging pointwise to . Therefore . Thus converges to in the norm sense. Corollary : is dense in . Proof By definition, for any there exists a Cauchy sequence in converging pointwise to . Again converges to in the norm sense. Hence is dense in .

28. Lemma: The evaluation functionals are continuous on . Proof: As the evaluation functionals are linear it suffices to show that they are continuous at 0. Let . The evaluation functionals are continuous on . Fix and let such that and . For any function in with there exists a function in such that and . This entails Hence and . Thus is continuous on .

29. Lemma: is a reproducing kernel Hilbert space. Proof It remains to prove that is complete. Let be a Cauchy sequence in and let . As evaluation functional is linear and continuous , is a Cauchy sequence in and thus converges to some . One has to prove that such defined f belongs to . Let be any sequence of positive numbers tending to zero as n tends to . As is dense in such that . From the inequalities and from the properties of it follows that tends to as tends to .

30. We have . Thus is a Cauchy sequence in converging pointwise to and so. Again converges to in the norm sense. Now . Therefore converges to in the norm sense and is complete.

31. Remark: As is dense in is isomorphic to the completion of. It is the smallest Hilbert space of functions on satisfying (a) .is called the functional completion of . Theorem: (Moore- Aronszajn) Let be a positive type function on .There exists only one Hilbert space of functions on with as a reproducing kernel. The subspace of spanned by the functions is dense in and is the set of functions on which are pointwise limits of Cauchy sequences in with the inner product where and

32. Proof The complex number defined does not depend on the representations not necessarily unique of and : ,this shows that depends onand only through their values. Then, taking and we get Thus the inner product with ‘ reproduces ’ the values of functions in . In particular .

33. As is a positive type function, is a semi-positive Hermitian from on . Now, suppose that . From the Cauchy-Schwarz inequality we have and . Let us consider endowed with the topology associated with the inner product and check conditions : The evaluation functional are continuous on . Any Cauchy sequence in converging pointwise to 0 converges also to 0 in norm sense.

34. Let and in . Therefore the evaluation functionals are continuous on . Let be a Cauchy sequencein converging pointwise to 0 and let be an upper bound for . Let and such that . Fix , and such that .

35. As we have, for , hence . As is arbitrary This implies that converges to 0 in norm sense. Then there exist a Hilbert space of functions on satisfying and the topology defined on by the inner product coincides with the topology induced on by and has a reproducing kernel .

36. is the set of functions in for which there exists a Cauchy sequence in converging pointwise to . Then such a sequence converges to in norm sense: is dense in . Therefore is unique and Thus is the reproducing kernel of . Theorem claims that a RKHS of functions on a set is characterized by its kernel on and that the property for of being a reproducing kernel is equivalent to the property of being a positive type function.

37. : The space is the set of complex sequences satisfying endowed with the inner product Theorem: A complex function defined on is a reproducing kernel or a positive type function if and only if there exists a mapping from to some space such that .

38. Proof Let be a RKHS of functions on a set with kernel . Consider the mapping Since any Hilbert space, is isometric to some space . If denotes any isometry from to , the mapping meets the requirements.

39. Conversely, Suppose that there is a mapping such that Since is a Hilbert space with inner product . This implies that is a positive type function and hence is Reproducing Kernel of a Hilbert Space. This theorem provides a characterization of ALL reproducing kernels on an abstract set . It turns out that the definition of a positive type function or of a reproducing kernel on or of a RKHS of functions on is equivalent to the definition of a mapping on with values in some .

40. Particularizing to a set , a pre-Hilbert space which can be considered through a suitable isomorphism as a part of a space and mapping from to , one can construct as many reproducing kernels as desired. Example: Let and By previous Theorem we get that defined on by is a reproducing kernel. where if if 0 and

41. Integral transforms and isometrical identities: Let dm be a positive measure on a set D. For a set E, let h(t,x) be a complex valued function defined on DE such that, for every xE, h(t,x) belongs to as a function of t. Then it is obvious that the function is a positive matrix on E, and so it is interesting to determine the reproducing kernel Hilbert space admitting K(x,y) as a reproducing kernel. We consider the problem under more general situation. Let H be a Hilbert space and let h: EH be a function from an abstract set E. Let F(E) be the complex vector space comprising all complex valued functions on E. Then the null space N(L) of the linear map L: HF(E) defined by is a closed subset of H. For, the point evaluation map defined by Lp(f)=(Lf)(p) is bounded. Hence, N(Lp) is a closed subset of H. Since N(L) is a closed subset of H. Thus the quotient space is a Hilbert space.

42. Therefore, by the fundamental theorem of linear maps, there is a linear bijection map defined by If we define a norm in R(L) by Then  becomes an isometry and hence, becomes a Hilbert space. In fact we have Theorem The space is a reproducing kernel Hilbert space with reproducing kernel Furthermore, the mapping L is an isometry from H into R(L) if and only if is complete in H.