1 / 60

Mirrors and Crystal Balls A Personal Perspective on Data Mining

Mirrors and Crystal Balls A Personal Perspective on Data Mining. Raghu Ramakrishnan. Outline. This award recognizes the work of many people, and I represent the many A warp-speed tour of some earlier work What’s a data mining talk without predictions?

medge-mckee
Download Presentation

Mirrors and Crystal Balls A Personal Perspective on Data Mining

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Mirrors and Crystal BallsA Personal Perspective on Data Mining Raghu Ramakrishnan

  2. Outline • This award recognizes the work of many people, and I represent the many • A warp-speed tour of some earlier work • What’s a data mining talk without predictions? • Some exciting directions for data mining that we’re working on at Yahoo!

  3. A Look in the Mirror …(and the faces I found there:unfortunately, couldn’t find photos for some people)(and apologies in advance for not discussing the related work that provided context and, often, tools and motivation)

  4. 1987 2007

  5. Sequences, Streams • SEQ • Sequence Data Processing. P. Seshadri, M. Livny and R. Ramakrishnan. SIGMOD 1994 • SEQ: A Model for Sequence Databases. P. Seshadri, M. Livny, and R. Ramakrishnan, ICDE 1995 • The Design and Implementation of a Sequence Database System. P. Seshadri, M. Livny and R. Ramakrishnan. VLDB 1996 • SRQL • SRQL: Sorted Relational Query Language. R. Ramakrishnan, D. Donjerkovic, A. Ranganathan, K. Beyer, and M. Krishnaprasad. SSDBM 1998

  6. Scalable Clustering • Birch • BIRCH: A Clustering Algorithm for Large Multidimensional Datasets. T. Zhang, R. Ramakrishnan and M. Livny. SIGMOD 96 • Fast Density Estimation Using CF-Kernels. T. Zhang, R. Ramakrishnan, and M. Livny. KDD 1999 • Clustering Large Databases in Arbitrary Metric Spaces. V. Ganti, R. Ramakrishnan, J. Gehrke, A. Powell, and J. French. ICDE 1999 • Clustering Categorical Data • CACTUS: A Scalable Clustering Algorithm for Categorical Data. V. Ganti, J. Gehrke, and R. Ramakrishnan. KDD 1999

  7. Scalable Decision Trees • Rain Forest • RainForest: A Framework for Fast Decision Tree Construction of Large Datasets. J. Gehrke, R. Ramakrishnan and V. Ganti. VLDB 1998 • Boat • BOAT: Optimistic Decision Tree Construction. J. Gehrke, V. Ganti, R. Ramakrishnan, and W-Y. Loh. SIGMOD 1999

  8. Streaming and Evolving Data, Incremental Mining • FOCUS • FOCUS: A Framework for Measuring Changes in Data Characteristics. V. Ganti, J. Gehrke, R. Ramakrishnan, and W-Y. Loh. PODS 1999 • DEMON • DEMON: Mining and Monitoring Evolving Data. V. Ganti, J. Gehrke, and R. Ramakrishnan. ICDE 1999

  9. QUESTION QUESTION Customer KNOWLEDGE KNOWLEDGE BASE BASE SELF SERVICE SELF SERVICE Answer added to power self service - Partner Experts Answer added to - - Customer Champions power self service - Employees ANSWER Support Agent Mass Collaboration • The QUIQ Engine: A Hybrid IR-DB System. N. Kabra, R. Ramakrishnan, and V. Ercegovac. ICDE 2003 • Mass Collaboration: A Case Study. R. Ramakrishnan, A. Baptist, V. Ercegovac, M. Hanselman, N. Kabra, A. Marathe, U. Shaft. IDEAS 2004

  10. OLAP, Hierarchies, and Exploratory Mining • Prediction Cubes. B-C. Chen, L. Chen, Y. Lin, R. Ramakrishnan. VLDB 2005 • Bellwether Analysis: Predicting Global Aggregates from Local Regions. B-C. Chen, R. Ramakrishnan, J.W. Shavlik, P. Tamma. VLDB 2006

  11. Hierarchies Redux • OLAP Over Uncertain and Imprecise Data. D. Burdick, P. Deshpande, T.S. Jayram, R. Ramakrishnan, S. Vaithyanathan. VLDB 2005 • Efficient Allocation Algorithms for OLAP Over Imprecise Data. D. Burdick, P.M. Deshpande, T. S. Jayram, R. Ramakrishnan, S. Vaithyanathan. • Learning from Aggregate Views. B-C. Chen, L. Chen, D. Musicant, and R. Ramakrishnan. ICDE 2006 • Mondrian: Multidimensional K-Anonymity. K. LeFevre, D.J. DeWitt, R. Ramakrishnan. ICDE 2006 • Workload-Aware Anonymization. K. LeFevre, D.J. DeWitt, R. Ramakrishnan. KDD 2006 • Privacy Skyline: Privacy with Multidimensional Adversarial Knowledge. B-C. Chen, R. Ramakrishnan, K. LeFevre. VLDB 2007 • Composite Subset Measures. L. Chen, R. Ramakrishnan, P. Barford, B-C. Chen, V. Yegneswaran. VLDB 2006

  12. Many Other Connections … • Scalable Inference • Optimizing MPF Queries: Decision Support and Probabilistic Inference. H. Corrada Bravo, R. Ramakrishnan. SIGMOD 2007 • Relational Learning • View Learning for Statistical Relational Learning, with an Application to Mammography. J. Davis, E.S. Burnside, I. Dutra, David Page, R. Ramakrishnan, V. Santos Costa, J.W. Shavlik.

  13. Community Information Management • Efficient Information Extraction over Evolving Text Data. F. Chen, A. Doan, J. Yang, R. Ramakrishnan. ICDE 2008 • Toward Best-Effort Information Extraction. W. Shen, P. DeRose, R. McCann, A. Doan, R. Ramakrishnan. SIGMOD 2008 • Declarative Information Extraction Using Datalog with Embedded Extraction Predicates. W. Shen, A. Doan, J.F. Naughton, R. Ramakrishnan. VLDB 2007 • Source-aware Entity Matching: A Compositional Approach. W. Shen, P. DeRose, L. Vu, A. Doan, R. Ramakrishnan. ICDE 2007

  14. … Through the Looking Glass Prediction is very hard, especially about the future. Yogi Berra

  15. Information Extraction … and the challenge of managing it

  16. DBLife • Integrated information about a (focused) real-world community • Collaboratively built and maintained by the community • CIMple software package

  17. Search Results of the Future yelp.com Gawker babycenter New York Times epicurious LinkedIn answers.com webmd (Slide courtesy Andrew Tomkins)

  18. Opening Up Yahoo! Search Phase 1 Phase 2 BOSS takes Yahoo!’s open strategy to the next level by providing Yahoo! Search infrastructure and technology to developers and companies to help them build their own search experiences. Giving site owners and developers control over the appearance of Yahoo! Search results. (Slide courtesy Prabhakar Raghavan)

  19. Custom Search Experiences Social Search Vertical Search Visual Search (Slide courtesy Prabhakar Raghavan)

  20. Economics of IE • Data $, Supervision $ • The cost of supervision, especially large, high-quality training sets, is high • By comparison, the cost of data is low • Therefore • Rapid training set construction/active learning techniques • Tolerance for low- (or low-quality) supervision • Take feedback and iterate rapidly

  21. Example: Accepted Papers • Every conference comes with a slightly different format for accepted papers • We want to extract accepted papers directly (before they make their way into DBLP etc.) • Assume • Lots of background knowledge (e.g., DBLP from last year) • No supervision on the target page • What can you do?

  22. Down the Page a Bit

  23. Record Identification • To get started, we need to identify records • Hey, we could write an XPath, no? • So, what if no supervision is allowed? • Given a crude classifier for paper records, can we recursively split up this page?

  24. First Level Splits

  25. After More Splits …

  26. Now Get the Records • Goal: To extract fields of individual records • We need training examples, right? • But these papers are new • The best we can do without supervision is noisy labels. • From having seen other such pages

  27. Partial, Noisy Labels

  28. Extracted Records

  29. Refining Results via Feedback • Now let’s shift slightly to consider extraction of publications from academic home pages • Must identify publication sections of faculty home pages, and extract paper citations from them • Underlying data model for extracted data is • A flexible graph-based model (similar to RDF or ER conceptual model) • “Confidence” scores per-attribute or relationship

  30. Extracted Publication Titles

  31. A Dubious Extracted Publication… PSOX provides declarative lineage tracking over operator executions

  32. Where’s the Problem? Use lineage to find source of problem..

  33. Source Page Hmm, not a publication page .. (but may have looked like one to a classifier)

  34. Feedback User corrects classification of that section..

  35. Faculty or Student? • NLP • Build a Classifier • Or…

  36. …Another Clue…

  37. Prof Prof …Stepping Back… • Leads to large-scale, partially-labeled relational learning • Involving different types of entities and links Prof-List Prof Student-List Student AdvisorOf Student

  38. p1 p2 p3 Maximizing the Value of What You Select to Show Users

  39. Content Optimization • PROBLEM: Match-making between content, user, context • Content: • Programmed (e.g., editors); Acquired (e.g., RSS feeds, UGC) • User • Individual (e.g., b-cookie), or user segment • Context • E.g., Y! or non-Y! property; device; time period • APPROACH: Scalable algorithms that select content to show, using editorially determined content mixes, and respecting editorially set constraints and policies.

  40. Team from Y! Research BeeChung Chen Pradheep Elango Deepak Agarwal Raghu Ramakrishnan Wei Chu Seung-Taek Park

  41. Team from Y! Engineering Nitin Motgi Joe Zachariah Scott Roy Todd Beaupre Kenneth Fox

  42. Yahoo! Home Page Featured Box • It is the top-center part of the Y! Front Page • It has four tabs: Featured, Entertainment, Sports, and Video

  43. Traditional Role of Editors • Strict quality control • Preserve “Yahoo! Voice” • E.g., typical mix of content • Community standards • Quality guidelines • E.g., Topical articles shown for limited time • Program articles periodically • New ones pushed, old ones taken out • Few tens of unique articles per day • 16 articles at any given time; editors keep up with novel articles and remove fading ones • Choose which articles appear in which tabs

  44. Content Optimization Approach • Editors continue to determine content sources, program some content, determine policies to ensure quality, and specify business constraints • But we use a statistically based machine learning algorithm to determine what articles to show where when a user visits the FP

  45. Modeling Approach • Pure feature based (did not work well): • Article: URL, keywords, categories • Build offline models to predict CTR when article shown to users • Models considered • Logistic Regression with feature selection • Decision Trees, Feature segments through clustering • Track CTR per article in user segments through online models • This worked well; the approach we took eventually

  46. Challenges • Non-stationary CTR • To ensure webpage stability, we show the same article until we find a better one • CTR decays over time; sharply at F1 • Time-of-day; day-of-week effect in CTR

  47. Modeling Approach • Track item scores through dynamic linear models (fast Kalman filter algorithms) • We model decay explicitly in our models • We have a global time-of-day curve explicitly in our online models

  48. Explore/Exploit • What is the best strategy for new articles? • If we show it and it’s bad: lose clicks • If we delay and it’s good: lose clicks • Solution: Show it while we don’t have much data if it looks promising • Classical multi-armed bandit type problem • Our setup is different than the ones studied in the literature; new ML problem

More Related