Download
1 / 58
580 likes | 739 Views
THE HONG KONG UNIVERSITY OF SCIENCE & TECHNOLOGY CSIT 5220: Reasoning and Decision under Uncertainty L07: Parameter Learning. Nevin L. Zhang Room 3504, phone: 2358-7015, Email: lzhang@cs.ust.hk Home page. Page 2. Overview of Course. Next
E N D
THE HONG KONG UNIVERSITY OF SCIENCE & TECHNOLOGYCSIT 5220: Reasoning and Decision under Uncertainty L07: Parameter Learning Nevin L. ZhangRoom 3504, phone: 2358-7015, Email: lzhang@cs.ust.hkHome page
Page 2 Overview of Course • Next • L07 Parameter learning: Estimate parameters from data • L08 Structure learning: Determine both structure and parameters from data • We have done: • Concept of Bayesian networks • D-Separation • Inference • Manual model building
Page 3 L07: Parameter Learning
Page 4 Outline • The MLE Principle • Parameter Learning from Complete Data • Missing Values • Parameter Learning from Incomplete data • Bayesian Parameter Learning • Reading: Jensen & Nielsen, Chapter 6; Zhang & Guo, Chapter 7
Page 9 • So, m_h and m_t contain all the information that is necessary for computing the likelihood function. Other information about the data does not matter • Because of this, they are called sufficient statistics
Page 11 Outline • The MLE Principle • Parameter Learning from Complete Data • Missing Values • Parameter Learning from Incomplete data • Bayesian Parameter Learning
Page 14 The General Case
Page 24 Outline • The MLE Principle • Parameter Learning from Complete Data • Missing Values • Parameter Learning from Incomplete data • Bayesian Parameter Learning
Page 28 Outline • The MLE Principle • Parameter Learning from Complete Data • Missing Values • Parameter Learning from Incomplete data • Bayesian Parameter Learning
Page 35 Can we implement the idea directly? • 1 incomplete data case becomes 2 partial data cases because the data case has 1 missing value? • What is a data case has 10 missing values, 100 missing values? • Exponential number of partial data cases • Fortunately, there is no need to explicitly complete the data. • Next: • Formalize the idea • Figure out exactly what to compute
Page 45 Convergence of EM
Page 46 • Need to run multiple times to avoid local maxima.
Page 47 Outline • The MLE Principle • Parameter Learning from Complete Data • Missing Values • Parameter Learning from Incomplete data • Bayesian Parameter Learning
