1 / 25

Exploratory Data Analysis

Exploratory Data Analysis. John Tukey. Developed these procedures to help one get a first look at distributions of scores. What is the shape of the distribution? Are there any suspicious scores. Stem and Leaf Display Box and Whiskers Plot. Stem and Leaf Display.

simeon
Download Presentation

Exploratory Data Analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Exploratory Data Analysis

  2. John Tukey • Developed these procedures to help one get a first look at distributions of scores. • What is the shape of the distribution? • Are there any suspicious scores. • Stem and Leaf Display • Box and Whiskers Plot

  3. Stem and Leaf Display • See the pulse rate data at Exploratory Data Analysis (EDA). • The scores range from 48 to 104. • We probably want to group them into 5 to 15 intervals. • I’ll use two intervals for the 40’s, two for the 50’s, etc.

  4. The Stem Consists of a column of leading (aka “most significant” digits, the leftmost digits in the scores. I’ll add to the stem the leaves, the trailing (rightmost, least significant) digits of each score

  5. The Stem With Leaves Next, I’ll arrange the leaves (within each row) from lowest to highest and add a “depth” column.

  6. Leaves Arranged in Order

  7. The Depth Column • This column tells you how many scores there are in that row and all rows between it and the closer tail of the distribution. • The row that contains the median has the row frequency in parentheses.

  8. Rotated Display It looks like a histogram, but the bars made up of the scores. From this display, can you identify any scores that are odd, compared to most of the other scores?

  9. Box and Whisker Plot • Median Location = (N + 1)/2 = 97/2 =48.5. • The median will be located between the 48th and the 49th scores from either tail.

  10. Are 40 scores from 68 to 48. Count up 8 more scores, starting with the first 70. The 48th score is a 70, the 49th score is a 70, the median is 70.

  11. The Hinge Location • = (Median Location + 1)/2. • Drop any decimal on the median location • For our data, hinge location = (48 + 1)/2 = 24.5. • Now, the upper hinge is the 24.5th score from the upper end of the distribution.

  12. There are 24 scores from 80 up to 104. Go in toward the median one more score. The 25th score from the highest is a 78. The upper hinge is (78 + 79)/2 = 79.

  13. The 26th score from the lowest score is a 64. Move towards the lower tail by one score and you see the 25th score is also a 64. One more, the 24th score is also a 64. The lower hinge is 64.

  14. The H-Spread • = the difference between the upper hinge and the lower hinge. For our data, 79 - 64 = 15. • This is the range of the middle 50% of the scores. • You also know this as the interquartile range.

  15. The Inner Fences • The upper inner fence= the upper hinge plus 1.5 H‑spreads. For our data, 79 + 1.5(15) = 101.5. • The lower inner fence is the lower hinge minus 1.5 H‑spreads, 64 - 1.5(15) = 41.5. • These are invisible fences, they are not plotted.

  16. Adjacent Values • These are scores that are outside of the middle 50% of the scores but within the inner fences. • For our data, these will be scores that fall • between 79 and 101.5 or • between 41.5 and 64

  17. Outliers • These are scores that are beyond the inner fences. • For our data, these are scores that are • Less than 41.5 or • Greater than 101.5

  18. Outer Fences • These invisible fences are 3 H-spreads beyond the hinges. • For our data the lower outer fence is at79 - 3(15) = 34 • and the upper outer fence is at 79 + 3(15) = 124 • Scores that are beyond the outer fences are called way-outliers.

  19. Drawing the Plot • Prepare a numerical scale. • Draw a box that extends from the lower hinge to the upper hinge. • Draw a line through the box at the median. • May also insert a symbol at the mean. • Draw whiskers out to the most extreme adjacent values on each side

  20. Whiskers • For our data, the lowest adjacent value is the 48, so we draw the whiskers on the lower end out to 48. • We do not go all the way out the inner fence unless there is a score there. • The highest adjacent value is a 99, so we draw whiskers on the upper end out to 99.

  21. Outliers • Every outlier is plotted with a special symbol, often a O for a regular outlier and an * for a way-outliers. • Some programs will also print the identification number next to every outlier • These days, we use statistical software to make these displays and plots rather than doing them by hand.

  22. Plots Produced by SAS

  23. How tall, in inches, is your ideal mate?

  24. Eight Foot Tall Mate ! • That is a WAY-OUTLIER for sure ! • Investigation of the original data sheets revealed that the actual response was 69 inches, not 96 inches.

  25. Exploratory Data Analysis (EDA) • It is highly recommended that you read the document linked above. • It includes additional examples and a bit of silliness that might help you remember key concepts. • Do watch the video clip of the Id attempting to cross an outer fence on the Forbidden Planet.

More Related