100 likes | 322 Views
Generative Models. for probabilistic inference. Michael Stewart. Remember the Joint Distribution?. What about very large/complex models?. "Generative" Modeling. Implement probability theory in computer science to infer a joint distribution
E N D
Generative Models for probabilistic inference Michael Stewart
"Generative" Modeling Implement probability theory in computer science to infer a joint distribution Bayesian prior -> posterior provides learning opportunity Sampling methods are their own field of study Applications in neuroscience, machine learning, biology [1] Requires a model and sampling method...
Define a Model This is a broad procedure; what is the goal? • Bio: • observe fMRI data, infer latent locality function in brain • observe genome data, infer latent gene relationships • AI: • observe words, infer latent topics or semantic information
How to sample Rejection sampling: noUsually a kind of Gibbs/MCMC
Some implementations: • [2] BLOG • [3] Church • [4] Python modules provided by Tom Haines
Recommended reading A. Daud, J. Li, L. Zhou, and F. Muhammad, "Knowledge discovery through directed probabilistic topic models: a survey," Frontiers of Computer Science in China, vol. 4, no. 2, pp. 280-301, Jun. 2010. [Online]. Available: http://dx.doi.org/10.1007/s11704-009-0062-y M. Steyvers and T. Griffiths, Probabilistic Topic Models. Lawrence Erlbaum Associates, 2007. [Online]. Available: http://www.worldcat.org/isbn/1410615340 N. Goodman, J. Tenenbaum, T. O'Donnell, and the Church Working Group. Probabilistic Models of Cognition http://projects.csail.mit.edu/church/wiki/Probabilistic_Models_of_Cognition Coin examples: Church Learning as Conditional Inference
References [1] A. Venkataraman, Y. Rathi, M. Kubicki, C.-F. Westin, and P. Golland, "Joint generative model for fMRI/DWI and its application to population studies." Medical Image Computing and Computer-Assisted Intervention, vol. 13, no. Pt 1, pp. 191-199, 2010. [Online]. Available: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=3056120&tool=pmcentrez&rendertype=abstract [2] B. Milch, B. Marthi, S. Russell, D. Sontag, D. L. Ong, and A. Kolobov, "Blog: Probabilistic models with unknown objects," in In IJCAI, 2005, pp. 1352-1359. [Online]. Available: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.116.2131 [3] N. D. Goodman, V. K. Mansinghka, D. Roy, K. Bonawitz, and J. B. Tenenbaum, "Church: a language for generative models," in Uncertainty in Artificial Intelligence, 2008. [Online]. Available: http://web.mit.edu/droy/www/papers/GooManRoyBonTenUAI2008.pdf [4] http://code.google.com/p/haines/ plate notation example: M. Steyvers and T. Griffiths, Probabilistic Topic Models. Lawrence Erlbaum Associates, 2007. [Online]. Available: http://www.worldcat.org/isbn/1410615340