1 / 70

Minimal Bounds on the Mean Square Error: A Tutorial

This tutorial provides an overview of minimal bounds on the mean square error (MSE) in statistical signal processing. It covers motivation, framework, and applications of MSE bounds. The tutorial also explores unification and perspectives in MSE bounds.

jameskthompson
Download Presentation

Minimal Bounds on the Mean Square Error: A Tutorial

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Minimal bounds on the Mean Square Error: A Tutorial Alexandre Renaux Washington University in St. Louis

  2. Outline Minimal bounds on the Mean Square Error Framework and motivations Minimal bounds on the MSE: unification Minimal bounds on the MSE: application Conclusion and perspectives 1 2 3 4

  3. Outline Framework and motivations Minimal bounds on the MSE: unification Minimal bounds on the MSE: application Conclusion and perspectives 1 2 3 4

  4. Framework and motivations 1 Statistical signal processing Extract informations (estimation) Applications: Radar/Sonar Digital communications Medical imaging Astrophysic …

  5. Framework and motivations 1 Statistical framework Parameters space Observations space Physical process Observations model performances Estimation rule

  6. Performances Framework and motivations 1 ’’Distance’’ between and r.v. Mean Square Error Bias Estimates distibution Variance

  7. Framework and motivations 1 Performances: Cramér-Rao inequality For unbiased with Estimates distribution Fisher Information Matrix Cramér-Rao If equality, then efficient estimator

  8. Framework and motivations 1 Context Maximum Likelihood estimators Direction of Arrivals estimation Frequency estimation The parameters support is finite

  9. Framework and motivations 1 Rife and Boorstyn 1974 Van Trees 1968 MSE behavior of ML estimator: 3 areas Mean Square Error (dB) Non-Information Threshold Asymptotic SNRT Signal to Noise Ratio (dB)

  10. Framework and motivations 1 Single frequency estimation (100 observations) SNR = 10 dB Run 1 MSE SNR Normalized frequency

  11. Framework and motivations 1 Single frequency estimation (100 observations) SNR = 10 dB Run 2 MSE SNR Normalized frequency

  12. Framework and motivations 1 Single frequency estimation (100 observations) SNR = 10 dB Run 20 MSE SNR Normalized frequency

  13. Framework and motivations 1 Single frequency estimation (100 observations) SNR = -6 dB Run 1 MSE SNR Normalized frequency

  14. Framework and motivations 1 Single frequency estimation (100 observations) SNR = -6 dB Run 2 MSE SNR Normalized frequency

  15. Framework and motivations 1 Single frequency estimation (100 observations) SNR = -6 dB Run 7 Outlier MSE SNR Normalized frequency

  16. Framework and motivations 1 Single frequency estimation (100 observations) SNR = -20 dB Outlier Run 1 MSE SNR Normalized frequency

  17. Framework and motivations 1 Single frequency estimation (100 observations) SNR = -20 dB Outlier Run 2 MSE SNR Normalized frequency

  18. Framework and motivations 1 Single frequency estimation (100 observations) SNR = -20 dB Run 20 Outlier MSE SNR Normalized frequency

  19. Framework and motivations 1 - Asymptotic MSE - Asymptotic efficiency - Threshold prediction - Global MSE - Ultimate performances Mean Square Error (dB) Non-Information Threshold Asymptotic SNRT Signal to Noise Ratio (dB)

  20. Outline Framework and motivations Minimal bounds on the MSE: unification Minimal bounds on the MSE: application Conclusion and prospect 1 2 3 4

  21. Minimal bounds on the MSE: unification 2 Mean Square Error (dB) Insuffisancy of the Cramér- Rao bound Non-Information • Optimistic • Bias • Threshold Threshold Other minimal Bounds (tightest) Cramér-Rao bound Asymptotic SNRT Signal to Noise Ratio (dB)

  22. Minimal bounds on the MSE: unification 2 Two categories Deterministic parameters Random parameters Determininstic Bounds Bayesian Bounds Bound the local MSE Bound the global MSE

  23. Minimal bounds on the MSE: unification 2 Two categories Deterministic parameters Random parameters Determininstic Bounds Bayesian Bounds Bound the local MSE Bound the global MSE

  24. Minimal bounds on the MSE: unification 2 Deterministic bounds unification Glave IEEE IT 1973 Barankin Approach In a class of unbiased estimator , we want to find the particular estimator for which the variance is minimal at the true value of the parameter Constrained optimization problem Class of unbiased estimator ????

  25. Minimal bounds on the MSE: unification 2 Deterministic bounds unification Bias Barankin (1949)

  26. Minimal bounds on the MSE: unification 2 Deterministic bounds unification Barankin Needs the resolution of an integral equation Sometimes, doesn’t exist

  27. Minimal bounds on the MSE: unification 2 Deterministic bounds unification Bias Cramér-Rao

  28. Minimal bounds on the MSE: unification 2 Deterministic bounds unification Cramer Rao Fisher Frechet Darmois

  29. Minimal bounds on the MSE: unification 2 Deterministic bounds unification Bias Bhattacharyya (1946)

  30. Minimal bounds on the MSE: unification 2 Deterministic bounds unification Bias Bhattacharyya Barankin

  31. Minimal bounds on the MSE: unification 2 Deterministic bounds unification ? Bhattacharyya Guttman Fraser

  32. Minimal bounds on the MSE: unification 2 Deterministic bounds unification Bias McAulay-Seidman (1969) (Barankin) Test points

  33. Minimal bounds on the MSE: unification 2 Deterministic bounds unification Bias McAulay-Seidman Barankin Test points

  34. Minimal bounds on the MSE: unification 2 Deterministic bounds unification How to choose test points ?

  35. Minimal bounds on the MSE: unification 2 Deterministic bounds unification Bias Chapman-Robbins (1951) 1 test point

  36. Minimal bounds on the MSE: unification 2 Deterministic bounds unification Chapman Robbins Hammersley Kiefer

  37. Minimal bounds on the MSE: unification 2 Deterministic bounds unification Bias Abel (1993) Test points

  38. Minimal bounds on the MSE: unification 2 Deterministic bounds unification

  39. Minimal bounds on the MSE: unification 2 Deterministic bounds unification Bias Quinlan-Chaumette-Larzabal (2006) Test points

  40. Minimal bounds on the MSE: unification 2 Deterministic bounds f0=0, K=32 observtions Don’t take into accout the support of the parameter

  41. Minimal bounds on the MSE: unification 2 Deterministic bounds Already used in Signal Processing CRB for wide range of topics ChRB and Barankin (McAulay-Seidman version) Time delay estimation DOA estimation Digital communications (synchronization parameters) Abel bound Digital communications (synchronization parameters)

  42. Minimal bounds on the MSE: unification 2 Two categories Deterministic parameters Random parameters Determininstic Bounds Bayesian Bounds Bound the local MSE Bound the global MSE

  43. Minimal bounds on the MSE: unification 2 Bayesian bounds unification Best Bayesian bound: MSE of the conditional mean estimator (MMSEE) is the solution of

  44. Minimal bounds on the MSE: unification 2 Bayesian bounds unification For your information

  45. Minimal bounds on the MSE: unification 2 Bayesian bounds unification Best Bayesian bound Minimal bound

  46. Minimal bounds on the MSE: unification 2 Bayesian bounds unification Constrained optimization problem Degres of freedom

  47. Minimal bounds on the MSE: unification 2 Bayesian bounds unification s 1 h Best Bayesian bound

  48. Minimal bounds on the MSE: unification 2 Bayesian bounds unification s 1 h Bayesian Cramér-Rao bound (Van Trees 1968)

  49. Minimal bounds on the MSE: unification 2 Bayesian bounds unification Van Trees

  50. Minimal bounds on the MSE: unification 2 Bayesian bounds unification s 1 … Test points h Reuven-Messer bound (1997) (Bayesian Barankin bound) Bobrovsky-Zakaï bound (1976) (1 test point)

More Related