MATH2114 Week 7, Thursday

1. MATH2114Week 7, Thursday Organizing and Visualizing Variables Slides Adopted & modified from “Business Statistics: A First Course” 7th Ed, by Levine, Szabat and Stephan, 2016 Pearson Education Inc.

2. Some feedback from Tuesday’s class • Do I have to buy the textbook? • We’re teaching from it, and the exam is open book • Not strictly required for any assessment • What does the assessment for this class look like? • Weblearn tests • 5 Tutorials (First week is not assessed, leaving 4 assessed) • Exam • Little class participation :(

3. Objective/Goals • Methods to organize variables. • Methods to visualize variables. • Methods to organize or visualize more than one variable at the same time. • Principles of proper visualizations • How not to mess it up too badly

4. Unambiguous, can be difficult to communicate and understand Tables

5. DCOVA Categorical Data Categorical Data Tallying/Counting Data One Categorical Variable Two Categorical Variables Summary Table Contingency Table

6. DCOVA Summary Table (1 variable) • Tallies frequencies or percentages of items in different categories Main Reason Young Adults Shop Online Source: Data extracted and adapted from “Main Reason Young Adults Shop Online?” USA Today, December 5, 2012, p. 1A.

7. DCOVA Contingency Tables (2 or more variables) • Cross tabulates or tallies jointly the responses of the categorical variables • Used to study patterns that may exist between the responses of two or more categorical variables • For two variables the tallies for one variable are located in the rows and the tallies for the second variable are located in the columns

8. DCOVA Raw frequencies Record of all permits issued by City of Melbourne for public events, film/photo shoots, weddings and promotions/sampling. Data sourced from City of Melbourne Open Data: Accessed 6/9/2017, last updated August 22, 2017 https://data.melbourne.vic.gov.au/People-Events/Event-permits-2014-2017-including-film-shoots-phot/sex6-6426

9. DCOVA Row % Percentages Permits issued in winter tended to be for filming Total % Column % 11.4% of permits issued were for summer weddings Very few weddings requiring permits occurred in winter

10. DCOVA Numerical Data Frequency Distributions Cumulative Distributions Ordered Array Numerical Data

11. DCOVA Ordered Array • An ordered array is a sequence of data, in rank order, from the smallest value to the largest value. • Shows range (minimum value to maximum value) • May help identify outliers (unusual observations)

12. DCOVA Frequency Distribution • The frequency distribution is a summary table in which the data are arranged into numerically ordered classes. • You must give attention to selecting the appropriate number of class groupings for the table, determining a suitable width of a class grouping, and establishing the boundaries of each class grouping to avoid overlapping. • The number of classes depends on the number of values in the data. With a larger number of values, typically there are more classes. In general, a frequency distribution should have at least 5 but no more than 15 classes. • To determine the width of a class interval, you divide the range (Highest value–Lowest value) of the data by the number of class groupings desired.

13. DCOVA Frequency Distribution Example • Example: A manufacturer of insulation randomly selects 20 winter days and records the daily high temperature • 24, 35, 17, 21, 24, 37, 26, 46, 58, 30, 32, 13, 12, 38, 41, 43, 44, 27, 53, 27

14. DCOVA Frequency Distribution Example • Sort raw data in ascending order:12, 13, 17, 21, 24, 24, 26, 27, 27, 30, 32, 35, 37, 38, 41, 43, 44, 46, 53, 58 • Find range: 58 - 12 = 46 • Select number of classes: 5 (usually between 5 and 15) • Compute class interval (width): 10 (46/5 then round up) • Determine class boundaries (limits): • Class 1: 10 but less than 20 • Class 2: 20 but less than 30 • Class 3: 30 but less than 40 • Class 4: 40 but less than 50 • Class 5: 50 but less than 60 • Compute class midpoints: 15, 25, 35, 45, 55 • Count observations & assign to classes

15. DCOVA Class Midpoints Frequency 10 but less than 20 15 3 20 but less than 30 25 6 30 but less than 40 35 5 40 but less than 50 45 4 50 but less than 60 55 2 Total 20 Frequency Distribution Example Data in ordered array: 12, 13, 17, 21, 24, 24, 26, 27, 27, 30, 32, 35, 37, 38, 41, 43, 44, 46, 53, 58

16. DCOVA Relative Frequency Class Frequency Percentage 10 but less than 20 3 .15 15% 20 but less than 30 6 .30 30% 30 but less than 40 5 .25 25% 40 but less than 50 4 .20 20% 50 but less than 60 2 .10 10% Total 20 1.00 100% Relative Frequency and Percentage Distribution Relative Frequency = Frequency / Total, e.g. 0.10 = 2 / 20

17. DCOVA Cumulative Frequency Distribution Cumulative Frequency Cumulative Percentage Class Frequency Percentage 10 but less than 20 3 15% 3 15% 20 but less than 30 6 30% 9 45% 30 but less than 40 5 25% 14 70% 40 but less than 50 4 20% 18 90% 50 but less than 60 2 10% 20 100% Total 20 100 20 100% Cumulative Percentage = Cumulative Frequency / Total * 100 e.g. 45% = 9/20 * 100

18. DCOVA Why is this a good idea? • It condenses the raw data into a more useful form; • It allows for a quick visual interpretation of the data; • It enables the determination of the major characteristics of the data set including where the data are concentrated and/or clustered.

19. DCOVA Some Tips • Different class boundaries may provide different pictures for the same data (especially for smaller data sets) • Shifts in data concentration may show up when different class boundaries are chosen • As the size of the data set increases, the impact of alterations in the selection of class boundaries is greatly reduced • When comparing two or more groups with different sample sizes, you must use either a relative frequency or a percentage distribution

20. DCOVA Easy to communicate and understand, ambiguous and simplified Graphs

21. DCOVA Categorical Data Visualizing Data Contingency Table For Two Variables Summary Table For One Variable Bar Chart Pareto Chart Side By Side Bar Chart Pie Chart Graphical Displays

22. DCOVA Categorical Data Pie Chart Bar Chart (mind the gap)

23. DCOVA Pareto Chart • Used to portray categorical data • A vertical bar chart, where categories are shown in descending order of frequency • A cumulative polygon is shown in the same graph • Used to separate the “vital few” from the “trivial many” Ordered Summary Table for Causes of Incomplete ATM Transactions

24. DCOVA Pareto Chart The “Vital Few”

25. DCOVA Side-by-Side Bar Charts Invoices with errors are much more likely to be of medium size (61.54% vs 30.77% and 7.69%)

26. DCOVA Numerical Data Frequency Distributions and Cumulative Distributions Ordered Array Stem-and-Leaf Display Histogram Polygon Ogive Visualising Univariate Numerical Data

27. DCOVA Stem and Leaf Display • A stem-and-leaf display organizes data into groups (called stems) so that the values within each group (the leaves) branch out to the right on each row. • A simple way to see how the data are distributed and where concentrations of data exist • Good for small amounts of data Morning 0|1 1|55 2|00 3|000 4|05555 5| 6|05 7| 8| 9| Evening 0| 1|0005 2|055 3|005 4|0555 5|000000 6|00000000 7|0 8|0 9|0

28. DCOVA Relative Frequency Percentage Class Frequency 10 but less than 20 3 .15 15 20 but less than 30 6 .30 30 30 but less than 40 5 .25 25 40 but less than 50 4 .20 20 50 but less than 60 2 .10 10 Total 20 1.00 100 Histogram • A vertical bar chart of the data in a frequency distribution is called a histogram. • In a histogram there are no gaps between adjacent bars. • The class boundaries (or class midpoints) are shown on the horizontal axis. • The vertical axis is either frequency, relative frequency, or percentage. • The height of the bars represent the frequency, relative frequency, or percentage. • Good for large amounts of data

29. DCOVA Polygon • Use midpoint of each class represent the data in that class • Connect the sequence of midpoints at their respective class percentages. • The cumulative percentage polygon, or ogive, displays the variable of interest along the X axis, and the cumulative percentages along the Y axis. • Useful when there are two or more groups to compare

30. DCOVA Two Numerical Variables Scatter Plot Time-Series Plot Visualising Multivariate Numeric Data

31. DCOVA Scatter Plots • Scatter plots are used for numerical data consisting of paired observations taken from two numerical variables • One variable is measured on the vertical axis and the other variable is measured on the horizontal axis • Scatter plots are used to examine possible relationships between two numerical variables

32. DCOVA Time Series Plot • Used to study patterns in a numeric variable over time • Numeric variable is measured on the vertical axis, time on the horizontal

33. DCOVA Organising many categorical variables: Multidimensional Contingency Table • Amultidimensional contingency tableis constructed by tallying the responses of three or more categorical variables. • In Excel creating a Pivot Table to yield an interactive display of this type. • Dedicated statistical platforms have specialised methods for analysing/visualising this sort of data in a more sophisticated ways

34. DCOVA Pivot Tables A pivot table: • Summarizes variables as a multidimensional summary table • Allows interactive changing of the level of summarization and formatting of the variables • Allows you to interactively “slice” your data to summarize subsets of data that meet specified criteria • Can be used to discover possible patterns and relationships in multidimensional data that simpler tables and charts would fail to make apparent

35. DCOVA Pivot Table Examples Three Dimensional Table Showing The Mean 10 Year Return % Broken Out By Type Of Fund, Market Cap, &Risk Level Two Dimensional Table Showing The Mean 10 Year Return % Broken Out By Type Of Fund & Risk Level

36. DCOVA How to work your way through the data?Data Discovery Methods • Drill-down is perhaps the simplest form of data discovery Results of drilling down to the details about small market cap value funds with low risk

37. DCOVA A quick guide in what not to do • People can only comprehend so much information at once • Presentation plays a huge role in how useful visualisation is • It’s very easy to make data summaries that are misleading • Obscure the data • Give the wrong impression Far more subtle than this Photo taken by /u/benjaminteeeee, posted on reddit.com/r/australia

38. DCOVA Information Overload leads to obscured data

39. DCOVA False Impressions are easy to make“There are three kinds of lies: Lies, damned lies, and statistics” • Selective summarization • Presenting only part of the data collected • Improperly constructed charts • Potential pie chart issues • Improperly scaled axes • A Y axis that does not begin at the origin or is a broken axis missing intermediate values • Chart junk

40. DCOVA Selective SummarisationLife is looking great

41. DCOVA Selective SummarisationLife is looking great… Until you look further

42. DCOVA How obvious is it that both of these summarise the same data? Why is it hard to tell? What would you do to improve?

43. DCOVA Graphical Errors:No relative Basis Freq.: Number of Students who participated in the survey %: Students participation rate in each year level

44. DCOVA Graphical Errors:Compressing the Vertical Axis

45. DCOVA Graphical ErrorsNo Zero Point on the Vertical Axis

46. DCOVA Chart Junk

47. DCOVA So much chart junk

48. DCOVA Please make the chart junk stop

49. DCOVA <Cries in chart junk>

50. DCOVA It’s very easy to accidentally create distortions in Excel • Excel often will create a graph where the vertical axis does not start at 0 • Excel offers the opportunity to turn simple charts into 3-D charts and in the process can create distorted images • Unusual charts offered as choices by Excel will most often create distorted images