1 / 81

Easy BI

Easy BI. Data Mining for Fun and Profit. How it works and how to work it. Why do it…. “The half-life of BI is typically shorter than the life of the project needed for its implementation.” --Industry whitepaper (see references). Predicting is Hard. “Predicting is hard…

ronat
Download Presentation

Easy BI

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Easy BI

  2. Data Mining for Fun and Profit How it works and how to work it

  3. Why do it… • “The half-life of BI is typically shorter than the life of the project needed for its implementation.” --Industry whitepaper (see references)

  4. Predicting is Hard • “Predicting is hard… • …especially about the future” --Yogi Berra

  5. Data Mining to the rescue!

  6. Why are we here? A recent Gartner Group Advanced Technology Research Note listed data mining at the top of the five key technology areas that "will clearly have a major impact across a wide range of industries within the next 3 to 5 years."

  7. What it is… Data Mining finds patterns in data

  8. What it is… • Data Mining finds patterns in data • Using Machine Learning Algorithms • Don’t worry: the hard yards are done • A lot at Microsoft Research

  9. What it is… Data Mining finds patterns in data Uses these patternsto make predictions

  10. What it’s not SSAS ≠ Cube

  11. Look, Ma, No Cube! Dimensional Modelling: Build a Cube  Learn MDX   Construct Analyses …of the PAST Data Mining: Build Structure  Use Model  Make Predictions …about the Future

  12. Why No Cube? • Cubes summarize facts: • For Example: • Sums of Sales in all regions for all months • Aggregated by Gender and Age • For each Product • …

  13. Why No Cube? • Cubes summarize facts: • For Example: • Sums of Sales in all regions for all months • Aggregated by Gender and Age • For each Product • … • Data mining find patterns in data

  14. Why No Cube? • Cubes summarize facts: • For Example: • Sums of Sales in all regions for all months • Aggregated by Gender and Age • For each Product • … • Data mining find patterns in data • Cubes abstract much of the interesting information

  15. Why No Cube? • Cubes summarize facts: • For Example: • Sums of Sales in all regions for all months • Aggregated by Gender and Age • For each Product • … • Data mining find patterns in data • Cubes abstract much of the interesting information • Facts that form the patterns are lost in the Cube’s summations

  16. Demo: Excel Data Mining Add-In • Connect to Data Source • Highlight Exceptions • Forecasting • Key Influencers

  17. But is it Respectable? Is it all just smoke and mirrors???

  18. But is it Respectable? • Is it all just smoke and mirrors??? • “Excel data mining add-in was invented to make astrology look respectable!” • Donald Data, industry pundit

  19. Foundation: SSAS Data Mining

  20. Overall Process

  21. Logical Architecture

  22. Physical Architecture Jargon: ADO = ActiveX Data Objects ADO MD = ADO Multidimensional AMO = Analysis Management Objects DSO = Decision Support Objects XMLA = XML for Analytics

  23. Data Mining Tutorials Books Online  Contents or… Search For Data Mining Tutorials

  24. Data Mining Tutorials

  25. Data Mining Designer • Business Intelligence Development Studio • Demo: Key Influencers • Models and Model Viewers • Decision Tree • Cluster • Naïve Bayes • Neural Network

  26. Decision Tree Algorithm Correlation  Tree Node

  27. Decision Tree Algorithm Correlation  Tree Node

  28. Decision Tree Algorithm • Hybrid • Linear regression & association & classification

  29. Decision Tree Algorithm • Hybrid • Linear regression & association & classification • Algorithm highlights • Remove rare attributes (“Feature Selection”)

  30. Decision Tree Algorithm • Hybrid • Linear regression & association & classification • Algorithm highlights • Remove rare attributes (“Feature Selection”) • Group values into bins for performance

  31. Decision Tree Algorithm • Hybrid • Linear regression & association & classification • Algorithm highlights • Remove rare attributes (“Feature Selection”) • Group values into bins for performance • Correlate input attributes with outcomes

  32. Decision Tree Algorithm • Hybrid • Linear regression & association & classification • Algorithm highlights • Remove rare attributes (“Feature Selection”) • Group values into bins for performance • Correlate input attributes with outcomes • Find attribute separating outcomes with maximum information gain

  33. Decision Tree Algorithm • Hybrid • Linear regression & association & classification • Algorithm highlights • Remove rare attributes (“Feature Selection”) • Group values into bins for performance • Correlate input attributes with outcomes • Find attribute separating outcomes with maximum information gain • Split tree and re-apply

  34. Cluster Algorithm

  35. Cluster Algorithm • Algorithm options: • Non-scalable (all records)

  36. Cluster Algorithm • Algorithm options: • Non-scalable (all records) • Scalable (50,000 records + 50,000 more if needed) • 3 x faster than non-scalable

  37. Cluster Algorithm • Algorithm options: • Non-scalable (all records) • Scalable (50,000 records + 50,000 more if needed) • 3 x faster than non-scalable • K – means (hard)

  38. Cluster Algorithm • Algorithm options: • Non-scalable (all records) • Scalable (50,000 records + 50,000 more if needed) • 3 x faster than non-scalable • K – means (hard) • Expectation Maximization (soft) (default)

  39. Cluster Algorithm • Algorithm options: • Non-scalable (all records) • Scalable (50,000 records + 50,000 more if needed) • 3 x faster than non-scalable • K – means (hard) • Expectation Maximization (soft) (default) • Form initial cluster

  40. Cluster Algorithm • Algorithm options: • Non-scalable (all records) • Scalable (50,000 records + 50,000 more if needed) • 3 x faster than non-scalable • K – means (hard) • Expectation Maximization (soft) (default) • Form initial cluster • Assign probability each attribute-value in each cluster

  41. Cluster Algorithm • Algorithm options: • Non-scalable (all records) • Scalable (50,000 records + 50,000 more if needed) • 3 x faster than non-scalable • K – means (hard) • Expectation Maximization (soft) (default) • Form initial cluster • Assign probability each attribute-value in each cluster • Iterate until model = likelihood of data

  42. Naïve Bayes Algorithm • Simple, fast, surprisingly accurate

  43. Naïve Bayes Algorithm • Simple, fast, surprisingly accurate • “Naïve”: attributes assumed to be independent of each other

  44. Naïve Bayes Algorithm • Simple, fast, surprisingly accurate • “Naïve”: attributes assumed to be independent of each other • Pervasive use throughout Data Mining

  45. Naïve Bayes Algorithm • Simple, fast, surprisingly accurate • “Naïve”: attributes assumed to be independent of each other • Pervasive use throughout Data Mining P(Result | Data) = P(Data | Result) * P(Result) / P(Data)

  46. Naïve Bayes Algorithm P(Girl | Trousers) = ? P(Trousers | Girl) = 20/40 P(Girl) = 40/100 P(Trousers) = 80/100

  47. Naïve Bayes Algorithm P(Girl | Trousers) = ? P(Trousers | Girl) = 20/40 P(Girl) = 40/100 P(Trousers) = 80/100 P(Girl | Trousers) = P(Trousers | Girl) P(Girl) / P(Trousers)

  48. Naïve Bayes Algorithm P(Girl | Trousers) = ? P(Trousers | Girl) = 20/40 P(Girl) = 40/100 P(Trousers) = 80/100 P(Girl | Trousers) = P(Trousers | Girl) P(Girl) / P(Trousers) = (20/40)(40/100)/(80/100) = 20/80 = 0.25

  49. Neural Network Algorithm Cars W W W W Weight 2 W W W W W W Buy W Cars Weight 3 W No W Age W Weight W Output Neurons Input Neurons Hidden Neurons

  50. Neural Network Algorithm • Multilayer Perceptron Network =

More Related