1 / 43

Research Introspection “ICML does ICML”

Research Introspection “ICML does ICML”. Andrew McCallum Computer Science Department University of Massachusetts Amherst. Relational Modeling of the Research Literature & other Entities. Better understand structure of our own research area. Tools to help us learn a new sub-field.

kennan
Download Presentation

Research Introspection “ICML does ICML”

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Research Introspection“ICML does ICML” Andrew McCallum Computer Science Department University of Massachusetts Amherst

  2. Relational Modeling of theResearch Literature & other Entities • Better understand structure of our own research area. • Tools to help us learn a new sub-field. • Aid collaboration • Map how ideas travel through social networks of researchers. • Aids for hiring and finding reviewers! • Many opportunities for rich relational learning • ... in a domain we understand well.

  3. Previous Systems

  4. Previous Systems Cites Research Paper

  5. More Entities and Relations Expertise Cites Grant Research Paper Person Venue University Groups

  6. Extract metadata (title, authors, abstract, venue, citations; 14 fields in total) Convert to text (with layout & format) Topic Analysis & other Data Mining Reference resolution (of papers, authors & grants) Browsable Web Interface Spider Web for PDFs Rexa System Overview NSF grant DB WWW Discriminativelytrainedgraph partitioning (competition-winningaccuracy) Home-grownJava+MySQL (~1m PDF/day) Enhancedps2text (better word stiching,plus layout in XML) ConditionalRandom Fields (99% word accuracy)

  7. From Text to Actionable Knowledge Spider Filter Data Mining IE Segment Classify Associate Cluster Discover patterns - entity types - links / relations - events Database Documentcollection Actionableknowledge Prediction Outlier detection Decision support

  8. Joint Inference Uncertainty Info Spider Filter Data Mining IE Segment Classify Associate Cluster Discover patterns - entity types - links / relations - events Database Documentcollection Actionableknowledge Emerging Patterns Prediction Outlier detection Decision support

  9. Discriminatively-trained undirected graphical models Conditional Random Fields [Lafferty, McCallum, Pereira] Conditional PRMs [Koller…], [Jensen…], [Geetor…], [Domingos…] Complex Inference and Learning Just what we researchers like to sink our teeth into! Unified Model Spider Filter Data Mining IE Segment Classify Associate Cluster Discover patterns - entity types - links / relations - events Probabilistic Model Documentcollection Actionableknowledge Prediction Outlier detection Decision support

  10. Information Extraction Markov dependencies ...and long-range & KB dependencies?

  11. IE from Research Papers [McCallum et al ‘99] @article{ kaelbling96reinforcement, author = "Leslie Pack Kaelbling and Michael L. Littman and Andrew P. Moore", title = "Reinforcement Learning: A Survey", journal = "Journal of Artificial Intelligence Research", volume = "4", pages = "237-285", year = "1996",

  12. where Wide-spread interest, positive experimental results in many applications. Noun phrase, Named entity [HLT’03], [CoNLL’03]Protein structure prediction [ICML’04] IE from Bioinformatics text [Bioinformatics ‘04],… Asian word segmentation [COLING’04], [ACL’04]IE from Research papers [HTL’04] Object classification in images [CVPR ‘04] (Linear Chain) Conditional Random Fields [Lafferty, McCallum, Pereira 2001] Undirected graphical model, trained to maximize conditional probability of output sequence given input sequence Finite state model Graphical model OTHERPERSONOTHERORGTITLE … output seq y y y y y t+2 t+3 t - 1 t t+1 FSM states . . . observations x x x x x t t +2 +3 t - t +1 t 1 input seq said Jones a Microsoft VP …

  13. Entity Resolution Joint inference among all pairwise coref ...models of entities, attributes, first-order...

  14. Joint Co-reference Decisions,Discriminative Model [Culotta & McCallum 2005] People Stuart Russell Y/N Stuart Russell Y/N Y/N S. Russel

  15. Co-reference for Multiple Entity Types [Culotta & McCallum 2005] People Organizations Stuart Russell University of California at Berkeley Y/N Y/N Stuart Russell Y/N Berkeley Y/N Y/N Y/N S. Russel Berkeley

  16. Joint Co-reference of Multiple Entity Types [Culotta & McCallum 2005] People Organizations Stuart Russell University of California at Berkeley Y/N Y/N Stuart Russell Y/N Berkeley Y/N Y/N Y/N Reduces error by 22% S. Russel Berkeley

  17. Structured Topic Models Discovering latent structurein jointly modeling words, time, relations...

  18. Topical N-gram Model [Wang, McCallum 2005]   z1 z2 z3 z4 . . . y1 y2 y3 y4 . . . w1 w2 w3 w4 . . . D  2 1  1 2 W W T T

  19. Finding Topics with TNG Traditional unigram LDArun on 1.6 milliontitles / abstracts (200 topics) ...select ~300k papers onML, NLP, robotics, vision... Find 200 TNG topics among those papers.

  20. Topical Transfer Citation counts from one topic to another. Map “producers and consumers”

  21. Trends in 17 years of NIPS proceedings

  22. Topic Distributions Conditioned on Time topic mass (in vertical height) time

  23. Topical Transfer Through Time • Can we predict which research topicswill be “hot” at ICML next year? • ...based on • the hot topics in “neighboring” venues last year • learned “neighborhood” distances for venue pairs

  24. How do Ideas Progress Through Social Networks? Hypothetical Example: “ADA Boost” SIGIR(Info. Retrieval) COLT ICML ICCV(Vision) ACL(NLP)

  25. How do Ideas Progress Through Social Networks? Hypothetical Example: “ADA Boost” SIGIR(Info. Retrieval) COLT ICML ICCV(Vision) ACL(NLP)

  26. How do Ideas Progress Through Social Networks? Hypothetical Example: “ADA Boost” SIGIR(Info. Retrieval) COLT ICML ICCV(Vision) ACL(NLP)

  27. Preliminary Results Mean Squared Prediction Error (Smaller Is better) TransferModel # Venues used for prediction Transfer Model with Ridge Regression is a good Predictor

  28. Other Relational Opportunities • Categorizing citations. • Map transfer of ideas through science. • Rank CS departments by various criteria. • What 10 papers tell the story of ASR research? • Predicting when a student will graduate. • Help me find the right postdoc. • Suggest best collaborative opportunities. • Who should chair the next ICML?

More Related