Total Variation and related Methods Numerical Schemes

1. Total Variation and related MethodsNumerical Schemes

2. Total Variation Numerical schemes • Usual starting point are variational formulation or optimality condition • Which formulation to be used: • Primal ? • Dual ? • Primal-Dual ? • We aim to give a unified perspective on all such methods

3. Total Variation General setup • Consider Variational Problem

4. Total Variation General Setup • Particular form of K

5. Total Variation General Setup • Introduce „gradient“ variable and constraint

6. Total Variation General Setup • Minimization problem

7. Total Variation General Setup • Optimality

8. Total Variation General Setup • Form of F

9. Total Variation Linear Analogue • F quadratic

10. Total Variation Iterative Schemes for Solution of TV Problems • Mainly three classes of iterative schemes: • Fixed point methods • Thresholding methods • Newton Methods

11. Total Variation 1. Fixed point methods • Solve first and third equation exactly in each step • (possibly with preconditioning for A*A ) • Do fixed-point iteration for w instead of second equation

12. Total Variation 1. Fixed point methods • Rewrite subgradient relation in some form • Eliminating v

13. Total Variation 1. Fixed point methods • Matrix form

14. Total Variation 2. Thesholding methods • Solve first equation exactly in each step • (possibly with preconditioning for A*A ) • Do fixed-point iteration for v instead of second equation • Possibly add a damping term in w for the last equation

15. Total Variation 2. Thesholding methods • C is damping matrix, possible perturbation • T is thresholding operator

16. Total Variation 2. Newton type methods • Approximate F in a reasonably smooth way and perform (inexact) • Newton iteration • Linearized coupled system solved in each iteration step

17. Total Variation 2. Newton type methods • For consistency (superlinear convergence)

18. Total Variation 2. Newton type methods • Matrix form

19. Total Variation Singular case • Our case has the same structure except the nonlinearity and multi-valuedness in the second equation • Several numerical approaches and distinctions can be understood by approximating the pointwise relation

20. Total Variation Primal Approximation • Simple approach: approximate F by smooth functionEquation q equals derivative of this smooth function • Example for Euclidean norm:

21. Total Variation Primal Approximation • General approach to obtain a differentiable approximation with Lipschitz gradient is Moreau-Yosida regularization • Example for Euclidean norm: Huber norm

22. Total Variation Fixed point form • Alternative without approximation

23. Total Variation Fixed point form • Leads to shrinkage

24. Total Variation Fixed point form • Equivalent fixed point relation

25. Total Variation SOCP / LP formulations • Roughly introduce new variable f and inequalitySubgradients characterized by minimization ofNote: we want to minimize f + something, hence optimal solution will always satisfies

26. Total Variation SOCP / LP formulations • Example: usual total variation, F equals Euclidean normConstraint is a quadratic (second-order cone) condition

27. Total Variation SOCP / LP formulations • Example: anisotropic TVAnalogous introduction, now many f yields even LP

28. Total Variation SOCP / LP formulations: Interior Points • Interior point methods further approximate the constrained problemvia an unconstrained problem with additional barrier term • ( - log of the constraint) enforcing that the solution is in the interior of the constraint set • This can be rewritten as using a smoothed version of F

29. Total Variation Dual Approaches • Alternative interpretation of subgradient relation

30. Total Variation Dual approaches • Primal and bidual are the same under suitable conditions

31. Total Variation Dual Approximation • Penalty Methods

32. Total Variation Dual Approximation • Barrier methods

33. Total Variation Dual Fixed Point • Construct fixed-point form

34. Total Variation Dual Fixed Point • Adding constants we see that q minimizes

35. Total Variation Dual Fixed Point for Primal Relation • Consider primal relation in special case

36. Total Variation Dual Fixed Point for Primal Relation • Fixed point equation