USING A PRIORI INFORMATION FOR CONSTRUCTING REGULARIZING ALGORITHMS

1. USING A PRIORI INFORMATION FOR CONSTRUCTING REGULARIZING ALGORITHMS Anatoly Yagola Department of Mathematics, Faculty of Physics, Moscow State University, Moscow 119899 Russia E-mail: yagola@inverse.phys.msu.ru

2. Main publications: 1.   Tikhonov, A.N., Goncharsky, A.V., Stepanov, V.V. and Yagola, A.G. (1995). Numerical methods for the solution of ill-posed problems. Kluwer Academic Publishers, Dordrecht. 2.  Tikhonov, A.N., Leonov, A.S. and Yagola, A.G. (1998). Nonlinear ill-posed problems. Chapman and Hall, London. 3.   Kochikov, I.V., Kuramshina, G.M., Pentin, Yu.A. and Yagola, A.G. (1999). Inverse problems of vibrational spectroscopy. VSP, Utrecht, Tokyo.

3. Introduction (1) is a linear operator, are linear normed spaces. The problem (1) is called well-posed on the class of its “admissible” data if for any pair from the set of “admissible” data the solution of (1): • exists, • is unique, • continuously depends on errors in and (is stable).

4. Stability means that if instead of we are given “admissible” such that , , the approximate solution converges to the exact one as . The numbers and are error estimates for the approximate data of (1) with the exact data . Denote . If at least one of the mentioned requirements is not met, then the problem (1) is called ill-posed.

5. As a generalized solution, it is often taken the so-called normal pseudosolution . It exists and is unique for any exact data of the problem (1) if , , . Here and denote the ranges of the operator and its orthogonal complement in , and stands for the operator pseudoinverse to . Below we find as a normal pseudosolution, i.e., .

6. What is to solve an ill-posed problem? Tikhonov answered: to solve an ill-posed problem means to produce a map (regularizing algorithm) . such that • brings an element into correspondence with any data , , . of the problem (1); • has the convergence property as , .

7. All inverse problems may be divided into three groups: well-posed problems, ill-posed regularizable problems, ill-posed nonregularizable problems.

8. Is it possible to construct a regularizing algorithm that does not depend on , ? Theorem 1: Let be a map of the set into . If is a regularizing algorithm (not depending explicitly on ), then the map . is continuous on its domain . . Proof The second condition in the definition of RA implies in valid for each . and the convergence as valid for . . The map is continuous on . .

9. It is clear from Theorem 1 that a regularizing algorithm not using and explicitly can only exist for problems (1) well-posed on the set of the data . The theorem generalized the assertion proved by Bakushinskii. Tikhonov proved the similar theorem when was studying ill-posed SLAE. As result, L-curve and GCV methods cannot be applied for the solution of ill-posed problems.

10. It is very curious that the most popular error free methods cannot solve well-posed problems also! As the first example we consider so-called the “L-curve method” (P.C. Hansen). In this method the regularization parameter in Tikhonov functional  is selected as a point maximum curvature of the L-curve {(ln||Ahz - u||, ln||z||):   0}. But this method cannot be used for the solution of ill-posed problems because the L-curve doesn’t depend on h and  (see the theorem). Everybody can easily prove that this method is inapplicable to solving the simplest finite-dimensional well-posed problems.

11. Let us consider the equation:z = 1. Here Z = U = R1, A = I (unit operator), u = 1. Let approximate data Ah= I and u = 1 for any h and . Independently on h and , the regularization parameter selected by the L-curve method L(Ah, u) = 1. Therefore, the approximate solution zL = 0.5, and it doesn’t converge to ze = 1 as h,  0. Using L-curve method we’ve received 0.5 instead of 1 independently on errors!!!

12. For another popular form of L-curve {(||Ahz - u||2, ||z||2):  0} it is possible to prove that such method has systematic error for all well-posed systems of linear algebraic equations (A. Leonov, A. Yagola). Another very popular “error free” method is GCV – the generalized cross-validation method (G. Wahba), where (Ah, u) is found as the point of the global minimum of the function G() = ||(AhAh* + I)-1u|| [tr(AhAh* + I)-1]-1,  0. This method is not applicable for the solution of ill-posed problems including ill-posed systems of linear algebraic equations (see the theorem above). It is possible construct well-posed systems of linear algebraic equations the GCV method failed for their solution.

13. Is it possible to estimate an error of an approximate solution of an ill-posed problem? The answer is negative. The main and very important result was obtained by Bakushinskii. Assume . Let be a RA. Denote by . the error of a solution of (1) at the point using the algorithm . If (1) is regularizable by a continuous map and there is an error estimate, which is uniform on then the restriction of to is continuous on . .

14. The accuracy of the approximate solution . of the problem (1) could be estimated as , where does not depend on and the function defines the convergence rate of to . Pointwise and uniform error estimations should be distinguished.

15. Consider the results obtained by Vinokurov. Let be a linear continuous injective operator acting in Banach space and the inverse operator . be unbounded on . Suppose that . is an arbitrary positive function such that . as , and is an arbitrary method to solve the problem. The following equality holds for elements except maybe for a first category set in : A uniform error estimate can only exist on a first category subset in .

16. A compact set is a typical example of the first category set in a normed space . For this set special regularizing algorithms may be used and a uniform error estimation may be constructed. Clearly, a uniform error estimate exists only for well-posed problems.

17. A posteriori error estimation For some ill-posed problems it is possible to find a so-called a posteriori error estimation. Let be an exact injective operator with closed graph and be a -compact space. Introduce a function such that . , , , : The function is an a posteriori error estimation for the problem (1), if as .

18. The generalized discrepancy method Let be Hilbert spaces, be a closed convex set of a priori constraints such that , ., be linearoperators. On a set introduce the Tikhonov's functional: where is a regularization parameter. (2) For any , and bounded linear operator.the problem (2) is solvable and has a uniquesolution .

19. A priori choice of . A regularizing algorithm using theextreme problem (2) for : to construct such thatas . If is an injectiveoperator, and , .as , then as , i.e., there is the a priori choice of .

20. A posteriori choice of . The incompatibility measure of (1) on : Let it can be computed with an error , i.e., instead of there is such that The generalized discrepancy: The generalized discrepancy is continuous and monotonically non-decreasing for .

21. The generalized discrepancy principle to choose the regularization parameter: If the condition is not just, then is an approximate solution of (1); If the condition is just, then the generalized discrepancy has a positive zero and . If is an injective operator, then . Otherwise, , where is the normal solution of (1), i.e., .

22. If are bounded linear operators, is a closed convex set, , , the generalized discrepancy principle are equivalent to the generalized discrepancy method: find

23. Inverse problem for the heat conduction equation. There is a function , we want to find such that as . We may write that

24. The problem may be written in the form of integral equation where is the Green function: The problem is solved for the parameters . , the function is taken such that .

25. The exact solution ( ) and the approximate solution ( ).

26. The Euler equation The Tikhonov's functional is a strongly convex functional in a Hilbert space. The necessary and sufficient condition for to be a minimum point of on a set of a priori constraints is If is an interior point of , then , or We obtain the Euler equation.

27. Sourcewise represented sets (1) is a linear injective operator. Assume the next a priori information: is sourcewise represented with a linear compact operator : (3) Here is a reflexive Banach space. Suppose is injective, is known exactly, .

28. Set and define the set Minimize the discrepancy on . If , then the solution is found. Denote . Otherwise, we change to and reiterate the process. If is found, then we define the approximate solution of (1) as an arbitrary solution of the inequality

29. Theorem 2: The process described above converges: . . There exists (generally speaking, depending on ) such that for . Approximate solutions strongly converge to . as . Proof The ball is a bounded closed set in . The set is a compact in for any , since is a compact operator. Due to Weierstrass theorem the continuous functional attains its exact lower bound on . Clearly, , where . is the integer part of a number.

30. Therefore is a finite number and there is such that for any . The inequality for any is evident. Thus, for all the approximate solutions . belong to the compact set , and the method coincides with the quasisolutions method for all sufficiently small positive . The convergence follows from the general theory of ill-posed problems. Remark: The method is a variant of the method of extending compacts.

31. Theorem 3: For the method described above there exists an a posteriori error estimate. It means that a functional exists such that as and at least for all sufficiently small positive . Remark 2: The existence of the a posteriori error estimation follows from the following. If by . we denote the space of sourcewise represented with the operator solutions of (1), then . Since is a compact set, then . is a -compact space.

32. An a posteriori error estimate is not an error estimate in general meaning that is impossible in principle for ill-posed problems. But it becomes an upper error estimate of the approximate solution for “small” errors , where depends on the exact solution .

33. The operators and are known with errors. Let there be linear operators , such that . , . Denote the vector of errors by . For any integer define a compact set . Find a minimal positive integer number such that the inequality has a nonempty set of solutions. Then the a posteriori error estimation is

34. Inverse problem for the heat conduction equation For any moment of time there is where . Suppose . We solve the problem using the method of extending compacts. Let , , , , .

35. The approximate solution and its a posteriori error estimation. We obtain .

36. Compact sets There is the additional a priori information: the exact solution of (1) belongs to a compactset and is a linear continuous injective operator. Asa set of approximate solutions of (1) it is possible to accept Then as in for any .

37. After finite dimensional approximation we obtain that , where is a convex polyhedron for convex or monotonic functions and . is a matrix, and are vectors. To find it is possible to use the method of conditional gradient or the method of projection conjugated gradients.

38. Error estimation • Find the minimum and the maximum values for each coordinate of . Denote them by , , . . • Secondly, using the found we construct functions and close to such that . for each . Therefore, we should minimize a linear function on a convex set. We may approximate the set by a convex polyhedron and solve a linear programming problem. The simplex-method or the method to cut convex polyhedrons may be used.

39. Inverse problem for the heat conduction equation. Let be a set of convex upward functions such that . Assume that , , . , , the number of nodes 20.

40. The exact solution ( ), the functions , .

41. We shall formulate now general conditions for constructing of regularizing algorithms for the solution of nonlinear ill-posed problems in finite-dimensional spaces. These conditions could be easily checked for an inverse vibrational problem which we consider as a problem to find the normal pseudosolution of nonlinear ill-posed problem on a given set of constraints. We shall discuss typical a priori constraints.

42. In this section the main problem for us is an operator equation (1) where D is a nonempty set of constraints, Z and U are finite-dimensional normed spaces, is a class of operators from D into U. Let us give a general formulation of Tikhonov's scheme of constructing a regularizing algorithm for solving the main problem: for the operator Eq. (1) on D find an element z* for which (2)

43. We assume that to some element there corresponds the nonempty set Z* in D of quasisolutions and that Z* may consist of more than one element. Furthermore, we suppose that a functional is defined on D and bounded below: The -optimal quasisolution problem for Eq. (1) is formulated as follows: find a such that

44. We suppose that instead of the unknown exact data (A, u), we are given approximate data which satisfy the following conditions: Here the function represents the known measure of approximation of precise operator A by approximate operator .We are given also numerical characterizations of the “closeness” of to (A, u). The main problem is to construct from the approximate data an element  which converges to the set -optimal pseudosolutions as

45. Let us formulate our basic assumptions. 1) The class consists of the operators A continuous from D to U. 2) The functional is lower semicontinuous on D. 3) If K is an arbitrary number such that then the set is compact in Z. 4) The measure of approximation is assumed to be defined for , to depend continuously on all its arguments, to be monotonically increasing with respect to for any h  0, and satisfy the equality

46. Conditions 1)-3) guarantee that Tikhonov’s scheme for constructing regularizing algorithms is based on using the smoothing functional in the conditional extreme problem: for fixed 0, find an element such that

47. Here f[x] is an auxiliary function. A common choice is We denote the set of extremals of (5) which correspond to a given  0 by Conditions 1)-3) imply that

48. The scheme of construction of an approximation to the set includes: (i) the choice of the regularization parameter ; (ii) the fixation of the corresponding to , and a special selection of an element in this set: as  0.

49. It is in this way that the generalized analogs of a posteriori parameter choice strategies are used. They were introduced by A.S. Leonov. We define for their formulation some auxiliary functions and functionals: Here is a generalized measure of incompatibility for nonlinear problems having the properties [2]: .

50. All these functions are generally many-valued. They have the following properties. Lemma. The functions are single-valued and continuous everywhere for  >0 except perhaps not more a countable set of their common points of discontinuity of the first kind, which are points of multiple-valuedness, then there exists at least two elements in the set such that . The functions , are monotonically nondecreasing and , are nonincreasing. The generalized discrepancy principle (GDP) for nonlinear problems consists of the following steps.