1 / 60

Undirected Graphical Models

Discover the intricacies of Gaussian Graphical Models, Ising Models, and Semiparametric Gaussian Copula Models for real and discrete random variables. Learn about sparsity, precision matrices, and efficient estimation methods like Graphical LASSO and Locally Linear Approximation. Uncover insights into neighborhood selection, nonparametric approaches, and the historical context of Ising Models.

cdudley
Download Presentation

Undirected Graphical Models

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Undirected Graphical Models Yuan Yao Peking University

  2. What’s a graphical model?

  3. Markov property: Conditional Independence

  4. Hammersley-Clifford Theorem • A clique is a complete subgraph • A maximal clique is a clique where no other clique contains it • A joint probability admits the following factorization with cliques • where Z is the partition function

  5. Clique Factorization is not unique

  6. Example I: 西游记 • west.Rdata • 408-by-303 data matrix • The first column contains chapter ID (1,…,100) • 302 characters appeared {1,0} in 408 scenes (samples) • 16 main characters who appeared no less than 40 samples

  7. An Ising model Green edges：positive interactions Red edges: negative interactions

  8. Example II: 红楼梦 • dream.Rdata • 475-by-375 data matrix • 374 characters appeared {1,0} in 475 scenes (samples) • The first column is an indicator if the scene is in the first 80 chapters (by Xueqin Cao) or later (by E Gao) • 18 main characters who appeared no less than 30 scenes in the first 80 chapters

  9. Left: 曹雪芹前80回； Right：高鹗后40回

  10. Main Content • Gaussian Graphical Models for real random variables • Semiparametric Gaussian Copula Graphical Models • Ising Models (Boltzman Machine) for discrete random variables

  11. Gaussian Graphical Model

  12. Precision Matrix

  13. Sparsity in High Dimensional Statistics

  14. Gaussian Graphical Models

  15. Proof: Linear regression Y ~ Z whose coefficient:

  16. Sparse precision matrix estimation

  17. Neighborhood Selection

  18. Recall:

  19. Parallel LASSO

  20. Estimator and Symmetrization

  21. L1-penalized Maximum Likelihood Estimator (MLE)

  22. Graphical LASSO, also known as

  23. CLIME: motivation

  24. CLIME: Dantzig Selector

  25. CLIME as Linear Programming

  26. Symmetrization

  27. Nonconvex Penalized MLE

  28. SCAD Penalty

  29. Locally Linear Approximation:Adaptive LASSO

  30. Reference

  31. Normality?

  32. Semiparametric Gaussian Copula Model

  33. Nonparanormal Gaussian Model

  34. Semiparametric Gaussian Copula Model

  35. Conditional Independence

  36. Nonparametric Part: Estimate of the marginal monotone transform

  37. Rank Correlation

  38. Semiparametric Graphical LASSO R package: huge

  39. Ising Model

  40. A Brief History

  41. Ising Model

  42. Sparsity

  43. Boltzman Distribution

  44. Penalized MLE

  45. Sparsity Enforced Estimates

  46. Partition function is intractable

  47. Conditional Likelihood

  48. Neighborhood Selection: L1-regularized Logistic Regression

More Related