COMPE 467 - Pattern Recognition

1. Bayesian Decision Theory COMPE 467 - Pattern Recognition

2. Covariance matrices for all of the classes are identical, • But covariance matrices are arbitrary.

3. and are independent of i. So,

7. Covariance matrices are different for each category.

8. only is independent of i. So,

12. EXAMPLE:

13. EXAMPLE (cont.):

14. EXAMPLE (cont.): Find the discriminant function for the first class.

15. EXAMPLE (cont.): Find the discriminant function for the first class.

16. EXAMPLE (cont.): Similarly, find the discriminant function for the second class.

17. EXAMPLE (cont.): The decision boundary:

18. EXAMPLE (cont.): The decision boundary:

19. EXAMPLE (cont.): Using MATLAB we can draw the decision boundary: (to draw the decision boundary in MATLAB) >> s = 'x^2-10*x-4*x*y+8*y-1+2*log(2)'; >> ezplot(s)

20. EXAMPLE (cont.): Using MATLAB we can draw the decision boundary:

21. EXAMPLE (cont.):

22. Error Probabilities and Integrals

23. Error Probabilities and Integrals

24. Error Probabilities and Integrals

25. Receiver Operating Characteristics • Another measure of distance between two Gaussian distributions. • found a great use in medicine, radar detection and other fields.

26. Receiver Operating Characteristics

27. Receiver Operating Characteristics

28. Receiver Operating Characteristics

29. Receiver Operating Characteristics • If both diagnosis and test are positive,it is called a true positive. The probability of a TP to occur is estimated by counting thetrue positives in the sample and divide by the sample size. • If the diagnosis is positive andthe test is negative it is called a false negative (FN). • False positive (FP) and true negative(TN) are defined similarly.

30. Receiver Operating Characteristics • The values described are used to calculate different measurements of the quality of thetest. • The first one is sensitivity, SE, which is the probability of having a positive test among the patients who have a positive diagnosis.

31. Receiver Operating Characteristics • Specificity, SP, is the probability of having a negative test among the patients who have anegative diagnosis.

32. Receiver Operating Characteristics • Example:

33. Receiver Operating Characteristics • Example (cont.):

34. Receiver Operating Characteristics • Overlap in distributions:

35. Bayes Decision Theory – Discrete Features • Assume x = (x1.......xd) takes only m discrete values • Components of x are binary or integer valued, x can take only one of m discrete values v1, v2, …, vm

36. Bayes Decision Theory – Discrete Features (Decision Strategy) • Maximize class posterior using bayes decision rule P(1 | x) = P(x | j) . P (j) S P(x | j) . P (j) Decide i if P(i | x) > P(i | x)for all i ≠ j or • Minimize the overall risk by selecting the action i = *: deciding 2 * = arg min R(i | x)

37. Bayes Decision Theory – Discrete Features (Independent binary features in 2-category problem)

38. Bayes Decision Theory – Discrete Features (Independent binary features in 2-category problem)

39. Bayes Decision Theory – Discrete Features (Independent binary features in 2-category problem)

40. Bayes Decision Theory – Discrete Features (Independent binary features in 2-category problem)

41. Bayes Decision Theory – Discrete Features (Independent binary features in 2-category problem)

42. Bayes Decision Theory – Discrete Features (Independent binary features in 2-category problem)

43. Bayes Decision Theory – Discrete Features (Independent binary features in 2-category problem) EXAMPLE

44. Bayes Decision Theory – Discrete Features (Independent binary features in 2-category problem) EXAMPLE

45. Bayes Decision Theory – Discrete Features (Independent binary features in 2-category problem) EXAMPLE

46. Bayes Decision Theory – Discrete Features (Independent binary features in 2-category problem) EXAMPLE

47. Bayes Decision Theory – Discrete Features (Independent binary features in 2-category problem) EXAMPLE

48. APPLICATION EXAMPLE Bit – matrix for machine – printed characters a pixel Here, each pixel may be taken as a feature For above problem, we have is the probabilty that for letter A,B,… B A

49. Summary • To minimize the overall risk, choose the action thatminimizes the conditional risk R (α |x). • To minimize the probability of error, choose the class thatmaximizes the posterior probability P (wj |x). • If there are different penalties for misclassifying patternsfrom different classes, the posteriors must be weightedaccording to such penalties before taking action. • Do not forget that these decisions are the optimal onesunder the assumption that the “true” values of theprobabilities are known.

50. References • R.O. Duda, P.E. Hart, and D.G. Stork, Pattern Classification, New York: John Wiley, 2001. • Selim Aksoy, Pattern Recognition Course Materials, 2011. • M. Narasimha Murty, V. Susheela Devi, Pattern Recognition an Algorithmic Approach, Springer, 2011. • “Bayes Decision Rule Discrete Features”, http://www.cim.mcgill.ca/~friggi/bayes/decision/binaryfeatures.html, access: 2011.