COMM 1004: Detection & Estimation

1. Prof. Ahmed El-Mahdy Dean of the faculty of IET The German University in Cairo COMM 1004: Detection & Estimation

2. H.L. Van Trees, Detection, Estimation, and Linear Modulation Theory, vol. I. John Wiley& sons, New York, 2001. • Don. H. Johnson, Statistical Signal Processing: Detection Theory, Houston, TX, 2013. • S. Kay, Fundamentals of Statistical Signal Processing: Estimation Theory, Prentice Hall, 1993. • S. Kay, Fundamentals of Statistical Signal Processing: Detection Theory, Prentice Hall, 1993. Text Books

3. Grading • Quizzes (2Quizzes) 15% (No Compensation Quizzes) • Assignments 15% • Project 30% • Final Exam 40%

4. Course Contents 1-Estimation Theory: Parameter Estimation random Applications: Communication channel estimation, Range Estimation, Sinusoidal Parameter Estimation, communication receivers, Noise Canceller 2-Detection: Simple binary hypothesis testing, likelihood ratio, Bayes criterion, Neyman-Pearson Criterion, Min-Max Performance

5. COMM 1004: Detection & Estimation Lecture 1- Introduction - Estimation Theory

6. Introduction to Detection & Estimation What is the detection and estimation?? Goal: Extract useful information from noisy signals Detection: Decision between two (or a small number of) possible hypothesis to choose the best of the two hypothesis. Parameter Estimation: Given a set of observations and given an assumed probabilistic model, we get the best estimate of the parameters of the model.

7. Detection: example 1: digital Communications

9. Detection example 3:In a speaker classification problem we know the speaker is German, British, or American. There are three possible hypotheses Ho, H1, H2. Decision: After observing the outcome in the observation space, we guess which hypothesis is true.

10. Examples for Estimation Estimation of a DC level of a signal: Estimation of the phase of the signal: Useful in coherent modulation:

11. Estimate h[m]??? Estimation of fading Channel: Parameter estimation of a signal:

12. Difference between Detection & Estimation? Detection: Estimation: Try to extract a parameter from them

13. Estimation theory

14. Definitions

15. Parameter Estimation random

16. Performance of Estimators • 1- Unbiased Estimators: • For an estimator to be unbiased we mean that on the averagethe estimator will yield the true value of the unknown parameter. • Since the parameter value may in general be anywhere in the • interval , unbiasedness asserts that no matter what • the true value of θ, our estimator will yield it on the average. • ]= • Otherwise, the estimate is said to be biased: ]

17. The bias is usually considered to be additive, so that: ]=. When we have a biased estimate, the bias usually depends on the number of observations N. An estimate is said to be asymptotically unbiased if the bias tends to zero for large N: =0 Variance of Estimator: The variance of an estimator is defined as: )=] Expectations are taken over x (meaning is random but not ). An estimate’s variance equals the mean-squared estimation error only if the estimate is unbiased.

18. Performance of Estimators

19. Example:

20. Unbiased Estimators • An estimator is unbiased does not necessarily mean that it is a good estimator. We need to Check some other performance measure. • It only guarantees that on the average it will attain the true value. • A continuousbias will always result in a poor estimator.

22. 2-Efficiency: • An unbiased estimator is said to be efficient if it has lower variance than all other estimators. • Example: If we compare two unbiased estimators . • Cramer-Rao bound is a lower bound of the variance of any unbiased estimators. Then: • An estimator is said to be efficient if: • It is unbiased • It satisfies Cramer-Rao bound. • If an efficient estimate exists, it is optimum in the mean-squared sense: No other estimate has a smaller mean-squared error. • Efficiency states that the estimator is “best”

23. 3- Consistency: • An unbiased estimator is consistent if its variance decreases as sample size increases. • In consistent unbiased estimator, • the distribution of the estimator • converges to the true value as the • sample size increases. Thus, a consistent estimate must be at least asymptotically unbiased. • Consistency is a relatively weak property in contrast to optimal properties such as efficiency. Unbiased and Consistent Estimator

24. Appendix A :Revision of Matrices

25. Revision of Matrices

27. Determinant of matrices

29. Inverse of matrices For the matrix A: There exist an inverse of the matrix A when det (A) does not equal to zero.

31. Eigen values and Eigen vectors of a matrix :

32. Example to find the Eigen values and vectors of a matrix :

33. Appendix B :Revision of Random Variables

34. Revision of Random Variables

35. Mean of a Random Variable

36. Covariance of a Random Variable

37. Independence and Uncorrelation

38. Remember: Two Statistically Independent Random Variables If X and Y are statistically independent, then

40. LMMSE