1 / 51

Chebyshev Estimator

Chebyshev Estimator. Presented by: Orr Srour. References. Yonina Eldar, Amir Beck and Marc Teboulle, "A Minimax Chebyshev Estimator for Bounded Error Estimation" (2007), to appear in IEEE Trans. Signal Proc.

fitzgerald-gardner
Download Presentation

Chebyshev Estimator

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Chebyshev Estimator Presented by: Orr Srour

  2. References • Yonina Eldar, Amir Beck and Marc Teboulle, "A Minimax Chebyshev Estimator for Bounded Error Estimation" (2007), to appear in IEEE Trans. Signal Proc. • Amir Beck and Yonina C. Eldar, Regularization in Regression with Bounded Noise: A Chebyshev Center Approach, SIAM J. Matrix Anal. Appl. 29 (2), 606-625 (2007). • Jacob (Slava) Chernoi and Yonina C. Eldar, Extending the Chebyshev Center estimation Technique, TBA

  3. Chebyshev Center - Agenda • Introduction • CC - Basic Formulation • CC - Geometric Interpretation • CC - So why not..? • Relaxed Chebyshev Center (RCC) • Formulation of the problem • Relation with the original CC • Feasibility of the original CC • Feasibility of the RCC • CLS as a CC relaxation • CLS vs. RCC • Constraints formulation • Extended Chebyshev Center

  4. Notations • y– boldface lowercase = vector • yi - i’th component of the vector y • A - boldface uppercase = matrix • - hat = the estimated vector of x • = A – B is PD, PSD

  5. The Problem • Estimate the deterministic parameter vector from observations with: • A– n x m model matrix • w– perturbation vector.

  6. LS Solution • When nothing else is known, a common approach is to find the vector that minimizes the data error: • Known as “least squares”, this solution can be written explicitly: (Assuming A has a full column rank)

  7. But… • In practical situations A is often ill-conditioned -> poor LS results

  8. Regularized LS • Assume we have some simple prior information regarding the parameter vector x. • Then we can use the regularized least squares (RLS):

  9. But… • But what if we have some prior information regarding the noise vector as well…? • What if we have some more complicated information regarding the parameter vector x?

  10. Assumptions • From now on we assume that the noise is norm-bounded : • And that x lies in a set defined by: (hence C is the intersection of k ellipsoids)

  11. Assumptions • The feasible parameter set of x is then given by: • (hence Q is compact) • Q is assumed to have non-empty interior

  12. Constrained Least Squares (CLS) • Given the prior knowledge , a popular estimation strategy is: • Minimization of the data error over C • But: the noise constraint is unused… • More importantly, it doesn’t necessarily lead to small estimation error:

  13. Chebyshev Center • The goal: estimator with small estimationerror • Suggested method: minimize the worst-case error over all feasible vectors

  14. Chebyshev Center – Geometric Interpretation • Alternative representation: -> find the smallest ball (hence its center and its radius r ) which encloses the set Q.

  15. Chebyshev Center – Geometric Interpretation

  16. Chebyshev Center • This problem is more commonly known as finding “Chebyshev’s Center”. Pafnuty Lvovich Chebyshev 16.5.1821 – 08.12.1894

  17. Chebyshev Center –The problem • The inner maximization is non-convex • Computing CC is a hard optimization problem • Can be solved efficiently over the complex domain for intersection of 2 ellipsoids

  18. Relaxed Chebyshev Center (RCC) • Let us consider the inner maximization first: and:

  19. Relaxed Chebyshev Center (RCC) • Denoting , we can write the optimization problem as: • with: Concave Not Convex

  20. Relaxed Chebyshev Center (RCC) • Let us replace G with: • And write the RCC as the solution of: Convex Convex

  21. Relaxed Chebyshev Center (RCC) • T is bounded • The objective is concave (linear) in • The objective is convex in • We can replace the order: • min-max to max-min

  22. Relaxed Chebyshev Center (RCC) • The inner minimization is a simple quadratic problem resulting with • Thus the RCC problem can be written as: Note: this is a convex optimization problem.

  23. RCC as an upper bound for CC • RCC is not generally equal to the CC (except for k = 1 over the the complex domain) • Since we have: • Hence the RCC provides an upper bound on the optimal minimax value.

  24. CC – The problem

  25. RCC Solution • Theorem: • The RCC estimator is given by:

  26. RCC Solution Where are the optimal solution of: subject to:

  27. RCC Solution – as SDP • Or as a semidefinite program (SDP): s.t.:

  28. Feasibility of the CC • Proposition: is feasible. • Proof: • Let us write the opt. problem as: • with: 3. has a UNIQUE solution 2. strictly convex: 1. Convex in

  29. Feasibility of the CC • Let us assume that is infeasible, and denote by y its projection onto Q. • By the projection theorem: and therefore:

  30. Feasibility of the CC • So: • Which using the compactness of Q implies: • But this contradicts the optimality of . Hence: is unique and feasible.

  31. Feasibility of the RCC • Proposition: is feasible. • Proof: • Uniqueness follows from the approach used earlier. • Let us prove feasibility by showing that any solution of the RCC is also a solution of the CC.

  32. Feasibility of the RCC • Let be a solution for the RCC problem. Then: • Since: • We get:

  33. CLS as CC relaxation • We now show that CLS is also a (looser) relaxation of the Chebyshev center. • Reminder:

  34. CLS as CC relaxation • Note that is equivalent to • Define the following CC relaxation: unharmed relaxed

  35. CLS as CC relaxation • Theorem: The CLS estimate is the same as the relaxed CC over V (here CCV). • Proof: • Les us assume is the CCV solution, and the RCC solution, . • The RCC is a strictly convex problem, so its solution is unique:

  36. CLS as CC relaxation • Define • It is easy to show that (hence it is a valid solution for the CCV)

  37. CLS as CC relaxation • Denote by the objective of the CCV. • By definition: • contradicting the optimality of . > 0

  38. CLS vs. RCC • Now, as in the proof of the feasibility of the RCC, we know that: • And so: • Which means that the CLS estimate is the solution of a looser relaxation than that of the RCC.

  39. Modeling Constraints • The RCC optimization method is based upon a relaxation of the set Q • Different characterizations of Q may lead to different relaxed sets. • Indeed, the RCC depends on the specific chosen form of Q. (unlike CC and CLS)

  40. Linear Box Constraints • Suppose we want to append box constraints upon x: • These can also be written as: • Which of the two is preferable…?

  41. Linear Box Constraints • Define:

  42. Linear Box Constraints • Suppose , then: • Since , it follows that: • Which can be written as:

  43. Linear Box Constraints • Hence: • T1 is a looser relaxation -> T2 is preferable.

  44. Linear Box Constraints • An example in R2: • The constraints have been chosen as the intersection of: • A randomly generated ellipsoid • [-1, 1] x [-1, 1]

  45. Linear Box Constraints

  46. Linear Box Constraints

  47. An example –Image Deblurring • x is a raw vector of a 16 x 16 image. • A is a 256 x 256 matrix representing atmospheric turbulence blur (4 HBW, 0.8 STD). • w is a WGN vector with std 0.05 . • The observations are Ax+w • We want x back…

  48. An example –Image Deblurring • LS: • RLS: with • CLS: • RCC:

  49. Chebyshev Center - Agenda • Introduction • CC - Basic Formulation • CC - Geometric Interpretation • CC - So why not..? • Relaxed Chebyshev Center (RCC) • Formulation of the problem • Relation with the original CC • Feasibility of the original CC • Feasibility of the RCC • CLS as a CC relaxation • CLS vs. RCC • Constraints formulation • Extended Chebyshev Center

More Related