1 / 39

DATA CONFUSION

DATA CONFUSION. How to confuse yourself and others with Data Analysis. AGENDA FOR TODAY’S TALK. Good Graphs – Bad Graphs The Law of Averages PTBD Analysis Enumerative & Analytical Problems PARC Analysis Wrong Methods of Analysis. “There are three kinds of lies:

december
Download Presentation

DATA CONFUSION

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. DATA CONFUSION How to confuse yourself and others with Data Analysis

  2. AGENDA FOR TODAY’S TALK • Good Graphs – Bad Graphs • The Law of Averages • PTBD Analysis • Enumerative & Analytical Problems • PARC Analysis • Wrong Methods of Analysis

  3. “There are three kinds of lies: Lies, damned lies and statistics” Attributed to Benjamin Disraeli by Mark Twain

  4. GOOD GRAPHS AND BAD GRAPHS

  5. DATA RELEVANCE • Graphs are only as good as the data they display • No amount of creativity can produce good graphs from dubious data

  6. DATA CONTENT • Don’t produce graphs from very small amounts of data • The human brain can grasp 1, 2 or 3 numbers without a graph

  7. RULES FOR PRODUCING GOOD GRAPHS • KEEP IT SIMPLE AND STUPID • Jesse Ventura • Tell the truth – don’t distort the data

  8. GOOD GRAPHS • Portray information without distortion • Contain no distracting elements • No false third dimensions, irrelevant decoration, or colour (chartjunk) • Use an appropriate scale • Label axes and tick marks properly, including measurement units • Have a descriptive title and/ or caption and legend • Have a low ink – to – information ratio

  9. GOOD GRAPH BAD GRAPH EVEN BETTER GRAPH BAD GRAPH

  10. BAD GRAPH GOOD GRAPH GOOD GRAPH

  11. GRAPHS THAT CONFUSE

  12. CHART JUNK

  13. GRAPHS THAT TELL A STORY

  14. HISTOGRAMS • No meaningless gaps • Reasonable Choice of bins • Easy to choose or adjust bins • Good aspect ratio • Meaningful labels on axes • Appropriate labels on bin tick marks

  15. TRENDING RANDOM VARIATION “Upward trend” “Setback” “Downturn” “Turnaround” “Rebound” “Downward trend”

  16. THE LAW OF AVERAGES “If I sit in a freezer and plunge my head into a pan of boiling chip fat. . . . . on average, I’m quite comfortable.”

  17. SHEWHART’S RULES FOR PRESENTATION OF DATA • Rule One • Data should always be presented in a way that preserves the evidence in the data • Rule Two • When an average, standard deviation or histogram is used to summarize data, the user should not be misled into to taking action they would not take if the data were presented in a time series

  18. USING THE WRONG METHODS Descriptive Statistics: A, B, C, D Variable N Mean StDev CoefVar Minimum Maximum A 20 11.950 0.102 0.85 11.83 12.08 B 20 11.950 0.100 0.84 11.85 12.25 C 20 11.950 0.102 0.86 11.75 12.15 D 20 11.950 0.100 0.84 11.81 12.14

  19. NO SIGNIFICANT DIFFERENCE HERE!

  20. NO DIFFERENCE?!?

  21. ALWAYS CARRY OUT PTBD ANALYSIS PLOT THE B….. DOTS!

  22. TYPES OF STATISTICAL STUDIES • Descriptive • Enumerative • Analytic

  23. DESCRIPTIVE STUDY • Count all fish in barrel • Count number of goldfish • Proportion of goldfish applies to the fish population in this barrel and no other barrels of fish

  24. ENUMERATIVE STUDY • Take a sample of fish from the barrel, and count the number of goldfish in the sample • Point estimate of the proportion of goldfish in the barrel population • Many statistical procedures do this • Cannot make any inference about any other barrels of fish

  25. ANALYTICAL STUDY Fish Packing Process over Time • Will we get the same proportion of goldfish in the future as we got in the past? • An analytical study allows prediction within limits

  26. ANALYTICAL STUDY • Proportion of goldfish is stable over time • Fish packing process is predictable within limits • We can expect, on average, 4 goldfish per barrel, but as many as 10 and as few as 0 in any single barrel

  27. ENUMERATIVE vs ANALYTICAL METHODS • Enumerative methods • seek to provide numeric summaries, confidence intervals,etc • use significance tests, ANOVA, descriptive stats, etc., assume single, stable population • Analytical methods • seek to understand the system under study • use primarily graphical tools such as run charts, control charts, histograms, box plots, etc • in the real world, most problems are analytical

  28. “Analysis of variance, t-tests, confidence intervals, and other statistical techniques taught in books,….., are inappropriate because they provide no basis for prediction and because they bury the information contained in the order of production.” W.E. Deming, Out of the Crisis Traditional statistical methods have their place, but are widely abused in the real world. When this is the case, statistics do more to cloud the issue than to enlighten.

  29. PARC ANALYSIS Practical Accumulated Records Compilation Profound Analysis Relying (on) Computers Passive Analysis (by) Regression Correlations Planning After Research Completed note inverse relationship with Continuous Recording (of) Administrative Procedures Constant Repetition (of) Anecdotal Perceptions

  30. PLANNING A PROCESS IMPROVEMENT STUDY • Why collect the data? • What statistical methods for analysis? • What data will be collected? • How much data do we need? • How will the data be measured? • How good is the measurement system? • When and where will data be collected? • Who will collect the data? • Remember:

  31. GARBAGE IN – GARBAGE OUT

  32. WHAT’S SIGNIFICANT? Two-sample T for C1 vs C2 N Mean StDev SE Mean A 5 13.652 0.487 0.22 B 5 14.369 0.646 0.29 Difference = mu (C1) - mu (C2) Estimate for difference: -0.716615 95% CI for difference: (-1.551531, 0.118301) T-Test of difference = 0 (vs not =): T-Value = -1.98 P-Value = 0.083 DF = 8 Both use Pooled StDev = 0.5725 Mean A = 13.7, Mean B = 14.4 Not significant? Two-sample T for C3 vs C4 N Mean StDev SE Mean A 200 13.510 0.501 0.035 A 200 13.667 0.492 0.035 Difference = mu (C3) - mu (C4) Estimate for difference: -0.157292 95% CI for difference: (-0.254935, -0.059649) T-Test of difference = 0 (vs not =): T-Value = -3.17 P-Value = 0.002 DF = 398 Both use Pooled StDev = 0.4967 Mean A = 13.5, Mean B = 13.7 Significant?

  33. WHAT SHOULD I DO WITH OUTLIERS? • Data point far away from the rest of the data • Don’t remove outliers to make data “look good” • Do you know why it is different? • If you do, remove it. If you don’t, leave it in • Could have a big impact on the analysis • Re – run analysis without outlier, and compare results

  34. “REGRESSION” WITH EXCEL • Usually means drawing an X-Y plot, fitting a straight line and coming up with an R2 value. • As long as R2 is high, everything’s hunky-dory. WRONG!

  35. “REGRESSION” WITH EXCEL Relationship is clearly not linear, and should not be presented as such

  36. “REGRESSION” WITH EXCEL • Regression model checking – in Excel? • Residual plots: • Normally distributed • Random pattern when plotted vs fitted values OK Variance not homogeneous Model incorrect

  37. PITFALLS OF REGRESSION ANALYSIS • Non-Linear Relationships • Influential Points • Extrapolating • Lurking Variables • Summary Data • Assuming Causation

  38. THAT’S (WITH REASONABLE PROBABILITY) THE END FOLKS! And remember, With statistics, you never have to say you’re certain!

  39. THANK YOU FOR YOUR ATTENTION ARE THERE ANY QUESTIONS? GOOD LUCK!!

More Related