MEDICAL IMAGING INFORMATICS: Lecture # 1 Basics of Medical Imaging Informatics: Estimation Theory

1. MEDICAL IMAGING INFORMATICS:Lecture # 1Basics of Medical Imaging Informatics:Estimation Theory Norbert Schuff Professor of Radiology VA Medical Center and UCSF Norbert.schuff@ucsf.edu Department of Radiology & Biomedical Imaging

2. What Is Medical Imaging Informatics? • Picture Archiving and Communication System (PACS) • Imaging Informatics for the Enterprise • Image-Enabled Electronic Medical Records • Radiology Information Systems (RIS) and Hospital Information Systems (HIS) • Digital Image Acquisition • Image Processing and Enhancement • Image Data Compression • 3D, Visualization and Multi-media • Speech Recognition • Computer-Aided Detection and Diagnosis (CAD). • Imaging Facilities Design • Imaging Vocabularies and Ontologies • Data-mining from medical image databases • Transforming the Radiological Interpretation Process (TRIP)[2] • DICOM, HL7 and other Standards • Workflow and Process Modeling and Simulation • Quality Assurance • Archive Integrity and Security • Teleradiology • Radiology Informatics Education • Etc. Department of Radiology & Biomedical Imaging

3. Pro-active Improve Data collection Refine Model knowledge Measurements Imaging Model Extract information Compare with model Re-active What Is Our Focus? Learn using computation tools to maximize information and gain knowledge Department of Radiology & Biomedical Imaging

4. Challenge:Extract Maximum Information • Q: How can we estimate quantities of interest from a given set of uncertain (noise) measurements? A: Apply estimation theory (1st lecture by Norbert) • Q: How can we code the quantities? A: Apply information theory (2nd lecture by Wang) Department of Radiology & Biomedical Imaging

5. Estimation Theory: Motivation Example I Gray/White Matter Segmentation Hypothetical Histogram GM/WM overlap 50:50; Can we do better than flipping a coin? Department of Radiology & Biomedical Imaging

6. Estimation Theory: Motivation Example II Goal: Capture dynamic signal on a static background Courtesy of Dr. D. Feinberg Advanced MRI Technologies, Sebastopol, CA Department of Radiology & Biomedical Imaging

7. Suppose we have N scalar measurements : We want to determine M quantities (parameters): Basic Concepts Definition: An estimator is: Error estimator: Department of Radiology & Biomedical Imaging

8. Mean Value: Amplitude: Variance Frequency: Phase: Decay: Examples of Estimators Department of Radiology & Biomedical Imaging

9. Unbiased: Mean value of the error should be zero Consistent: Error estimator should decrease asymptotically as number of measurements increase. (Mean Square Error (MSE)) If estimator is biased bias variance Some Desirable Properties of Estimators I: Department of Radiology & Biomedical Imaging

10. Efficient: Co-variance matrix of error should decrease asymptotically to its minimal value, i.e. inverse of Fisher’s information matrix) for large N Some Desirable Properties of Estimators II: Department of Radiology & Biomedical Imaging

11. Mean: The sample mean is an unbiased estimator of the true mean Variance: The variance is a consistent estimator because It approaches zero for large number of measurements. Example:Properties Of Estimators Mean and Variance Department of Radiology & Biomedical Imaging

12. where N > M Linear Model: Generally: Least-Squares Estimation Department of Radiology & Biomedical Imaging

13. The best what we can do: Minimizing ELSEwith regard to  leads to Least-Squares Estimation • LSE is popular choice for model fitting • Useful for obtaining a descriptive measure • But • Makes no assumptions about distributions of data or parameters • Has no basis for statistics Department of Radiology & Biomedical Imaging

14. We have: Goal: Find  that gives the most likely probability distribution underlying xN. Max likelihood function ML can be found by Maximum Likelihood (ML) Estimator Xn is random sample from a pool with certain probability distribution. Department of Radiology & Biomedical Imaging

15. ML function of a normal distribution log ML function 1st log ML equation MLE of the mean 2nd log ML equation MLE of the variance Example I: ML of Normal Distribution Department of Radiology & Biomedical Imaging

16. f(y|n=10,w) y Example II: Binominal Distribution(Coin Toss) Probability density function: n= number of tosses w= probability of success Department of Radiology & Biomedical Imaging

17. Likelihood Function Of Coin Tosses Given the observed data f (y|w=0.7, n=10) (and model), find the parameter w that most likely produced the data. Department of Radiology & Biomedical Imaging

18. Compute log likelihood function ML Estimation Of Coin Toss Evaluate ML equation According to the MLE principle, the PDF f(y|w=y/n) for a given n is the distribution that is most likely to have generated the observed data of y. Department of Radiology & Biomedical Imaging

19. Assume: • are independent of vN ML and vN have the same distribution vN is zero mean and gaussian ML function of a normal distribution Connection between ML and LSE Clearly, p(x|) is maximized when LSE is minimized Department of Radiology & Biomedical Imaging

20. Properties Of The ML Estimator • is consistent: the MLE recovers asymptotically the true parameter values that generated the data for N  inf; • Is efficient: The MLE achieves asymptotically the minimum error (= max. information) Department of Radiology & Biomedical Imaging

21. We have random sample: We also have random parameters: Goal: Find the most likely  (max. posterior density of ) given xN. Maximize joint density MAO can be found by Maximum A-Posteriori (MAP) Estimator Department of Radiology & Biomedical Imaging

22. We have random sample: The sample mean of MAP is: If we do not have prior information on ,  inf or T inf MAP Of Normal Distribution Department of Radiology & Biomedical Imaging

23. Posterior Density and Estimators p(|x) MSE  MAP Department of Radiology & Biomedical Imaging

24. Summary • LSE is a descriptive method to accurately fit data to a model. • MLE is a method to seek the probability distribution that makes the observed data most likely. • MAP is a method to seek the most probably parameter value given prior information about the parameters and the observed data. • If the influence of prior information decreases, i.e. many any measurements, MAP approaches MLE Department of Radiology & Biomedical Imaging

25. Some Priors in Imaging • Smoothness of the brain • Anatomical boundaries • Intensity distributions • Anatomical shapes • Physical models • Point spread function • Bandwidth limits • Etc. Department of Radiology & Biomedical Imaging

26. Imaging Software Using MLE And MAP Department of Radiology & Biomedical Imaging

27. Segmentation Using MLE A: Raw MRI B: SPM2 C: EMS D: HBSA from Habib Zaidi, et al, NeuroImage 32 (2006) 1591 – 1607 Department of Radiology & Biomedical Imaging

28. EM-Simultaneous EM-Affine EM-NonRigid Manual Brain Parcellation Using MLE Kilian Maria Pohl, Disseration 1999 Prior information for Brain parcellation Department of Radiology & Biomedical Imaging

29. MAP Estimation In Image Reconstruction Human brain MRI. (a) The original LR data. (b) Zero-padding interpolation. (c) SR with box-PSF. (d) SR with Gaussian-PSF. From: A. Greenspan in The Computer Journal Advance Access published February 19, 2008 Department of Radiology & Biomedical Imaging

30. Literature Mathematical • H. Sorenson. Parameter Estimation – Principles and Problems. Marcel Dekker (pub)1980. Signal Processing • S. Kay. Fundamentals of Signal Processing – Estimation Theory. Prentice Hall 1993. • L. Scharf. Statistical Signal Processing: Detection, Estimation, and Time Series Analysis. Addison-Wesley 1991. Statistics: • A. Hyvarinen. Independent Component Analysis. John Wileys & Sons. 2001. • New Directions in Statistical Signal Processing. From Systems to Brain. Ed. S. Haykin. MIT Press 2007. Department of Radiology & Biomedical Imaging