Robert delMas, Joan Garfield, and Ann Ooms University of Minnesota

1. Using Assessment Items to Study Students’ Difficulty Reading and Interpreting Graphical Representation of Distributions Robert delMas, Joan Garfield, and Ann Ooms University of Minnesota

2. OVERVIEW • Motivation for paper-and-pencil items to assess understanding of graphs • ARTIST Project • Development of 11 topic scales and CAOS test • Spring 2005 study • 3 types of Literacy and Reasoning items • Student responses to items • Implications for Instruction

3. Assessment of Understanding About Distribution • Research to date has used observation, interview, tasks,and problems • These studies provide rich data on reasoning about Distribution • These approaches not easily administered by instructors in the classroom • Need for paper-and-pencil assessments

4. Assessment Resources Tools for Improving Statistical ThinkingARTIST Project • Past year and a half: Develop and validate assessments • Advisory group identified 11 topic areas and CAOS • 8 to 12 multiple choice items for each topic • 37 multiple choice items for CAOS • Advisors met twice to review assessment items • March 2004: Suggested changes, identified missing areas • Summer 2004: Items revised, new items generated • August, 2004: Reviewed second time, judged validity • Final versions of topic scales and CAOS produced • Piloted topic scales and CAOS fall 2004 • Started collecting data spring 2005

5. Spring 2005 PARTICIPANTSInstructor and Student Characteristics 11 Topic Scales (555 Students) • 4 High Schools, 205 AP Students • 15 Colleges and Universities, 350 Students CAOS (909 Students) • 3 High Schools, 97 AP Students • 15 Colleges and Universities, 812 Students Advanced Placement (AP) students are high ability; College more heterogeneous

6. Items on Literacy and Reasoning about Distribution • Three types • Literacy (2 tasks) • Read a graph • Interpret a graphic display • Reasoning (2 tasks) • Match graph to description • Match graphs of same data • Literacy and Reasoning

7. Literacy: Reading a graph

8. Literacy: Reading a graph ITEM 1: Here is a histogram for a set of test scores from a 10-item makeup quiz given to a group of students who were absent on the day the quiz was given. How many people received scores higher than 4?

9. Literacy: Reading a graph ITEM 1: Here is a histogram for a set of test scores from a 10-item makeup quiz given to a group of students who were absent on the day the quiz was given. How many people received scores higher than 4?

10. Literacy: Reading a graph ITEM 1: Here is a histogram for a set of test scores from a 10-item makeup quiz given to a group of students who were absent on the day the quiz was given. How many people received scores higher than 4?

11. Literacy: Reading a graph ITEM 1: Here is a histogram for a set of test scores from a 10-item makeup quiz given to a group of students who were absent on the day the quiz was given. How many people received scores higher than 4?

12. Literacy: Reading a graph ITEM 1: Here is a histogram for a set of test scores from a 10-item makeup quiz given to a group of students who were absent on the day the quiz was given. How many people received scores higher than 4? ITEM 2: How many people took the test and have scores represented in the graph?

13. Literacy: Reading a graph ITEM 1: Here is a histogram for a set of test scores from a 10-item makeup quiz given to a group of students who were absent on the day the quiz was given. How many people received scores higher than 4? ITEM 2: How many people took the test and have scores represented in the graph?

14. Literacy: Reading a graph ITEM 1: Here is a histogram for a set of test scores from a 10-item makeup quiz given to a group of students who were absent on the day the quiz was given. How many people received scores higher than 4? ITEM 2: How many people took the test and have scores represented in the graph?

15. Literacy: Reading a graph • ITEM 3: What do the numbers on the vertical axis represent?

16. Literacy: Reading a graph • ITEM 3: What do the numbers on the vertical axis represent?

17. Literacy: Reading a graph • ITEM 3: What do the numbers on the vertical axis represent?

18. Literacy: Reading a graph ITEM 4: Scores for a quiz were calculated as the number of correct responses. Below is a graphical display of the quiz scores. How many of the scores are above 15? (Note: all scores are integers and bars begin at left endpoints)

19. Literacy: Reading a graph ITEM 4: Scores for a quiz were calculated as the number of correct responses. Below is a graphical display of the quiz scores. How many of the scores are above 15? (Note: all scores are integers and bars begin at left endpoints)

20. Literacy: Reading a graph ITEM 4: Scores for a quiz were calculated as the number of correct responses. Below is a graphical display of the quiz scores. How many of the scores are above 15? (Note: all scores are integers and bars begin at left endpoints)

21. Literacy: Interpreting a Graphic Display

22. Literacy: Interpreting a Graphic Display ITEM 5: One of the items on the student survey for an introductory statistics course was "Rate your aptitude to succeed in this class on a scale of 1 to 10" where 1 = Lowest Aptitude and 10 = Highest Aptitude. The instructor examined the data for men and women separately. Below is the distribution of this variable for the 30 women in the class. How should the instructor interpret the women's perceptions regarding their success in the class?

23. Literacy: Interpreting a Graphic Display ITEM 5: One of the items on the student survey for an introductory statistics course was "Rate your aptitude to succeed in this class on a scale of 1 to 10" where 1 = Lowest Aptitude and 10 = Highest Aptitude. The instructor examined the data for men and women separately. Below is the distribution of this variable for the 30 women in the class. How should the instructor interpret the women's perceptions regarding their success in the class?

24. Literacy: Interpreting a Graphic Display ITEM 5: One of the items on the student survey for an introductory statistics course was "Rate your aptitude to succeed in this class on a scale of 1 to 10" where 1 = Lowest Aptitude and 10 = Highest Aptitude. The instructor examined the data for men and women separately. Below is the distribution of this variable for the 30 women in the class. How should the instructor interpret the women's perceptions regarding their success in the class?

25. Literacy: Interpreting a Graphic Display ITEM 6: The following graph shows a distribution of hours slept last night by a group of college students. Select the statement below that gives the most complete description of the graph in a way that demonstrates an understanding of how to statistically describe and interpret the distribution of a variable.

26. Literacy: Interpreting a Graphic Display ITEM 6: The following graph shows a distribution of hours slept last night by a group of college students. Select the statement below that gives the most complete description of the graph in a way that demonstrates an understanding of how to statistically describe and interpret the distribution of a variable.

27. Literacy: Interpreting a Graphic Display ITEM 6: The following graph shows a distribution of hours slept last night by a group of college students. Select the statement below that gives the most complete description of the graph in a way that demonstrates an understanding of how to statistically describe and interpret the distribution of a variable.

28. Literacy: Interpreting a Graphic Display ITEM 7: A college statistics class conducted a survey. They gathered data from a large random sample of students who estimated how much money they typically spent each week in different categories (e.g., food, entertainment, etc.). A distribution of the survey results is presented below. One student claims the distribution of food costs basically looks bell-shaped, with one outlier. How would you respond?

29. Literacy: Interpreting a Graphic Display ITEM 7: A college statistics class conducted a survey. They gathered data from a large random sample of students who estimated how much money they typically spent each week in different categories (e.g., food, entertainment, etc.). A distribution of the survey results is presented below. One student claims the distribution of food costs basically looks bell-shaped, with one outlier. How would you respond?

30. Literacy: Interpreting a Graphic Display ITEM 7: A college statistics class conducted a survey. They gathered data from a large random sample of students who estimated how much money they typically spent each week in different categories (e.g., food, entertainment, etc.). A distribution of the survey results is presented below. One student claims the distribution of food costs basically looks bell-shaped, with one outlier. How would you respond?

31. REASONING: Matching a Graph to a Description

32. REASONING: Matching a Graph to a Description Match each description to the appropriate histogram below. III I II IV

33. REASONING: Matching a Graph to a Description Match each description to the appropriate histogram below. III I II IV ITEM 8: A set of quiz scores where the quiz was very easy.

34. REASONING: Matching a Graph to a Description Match each description to the appropriate histogram below. III I II IV ITEM 8: A set of quiz scores where the quiz was very easy.

35. REASONING: Matching a Graph to a Description Match each description to the appropriate histogram below. III I II IV ITEM 8: A set of quiz scores where the quiz was very easy.

36. REASONING: Matching a Graph to a Description Match each description to the appropriate histogram below. III I II IV ITEM 9: The last digits of phone numbers sampled from a phone book. Match this description to the appropriate histogram below.

37. REASONING: Matching a Graph to a Description Match each description to the appropriate histogram below. III I II IV ITEM 9: The last digits of phone numbers sampled from a phone book. Match this description to the appropriate histogram below.

38. REASONING: Matching a Graph to a Description Match each description to the appropriate histogram below. III I II IV ITEM 9: The last digits of phone numbers sampled from a phone book. Match this description to the appropriate histogram below.

39. REASONING: Matching a Graph to a Description Match each description to the appropriate histogram below. III I II IV ITEM 10: A set of average weights (measured in pounds) compiled monthly over the course of two years, of one healthy adult. Match this description to the appropriate histogram below.

40. REASONING: Matching a Graph to a Description Match each description to the appropriate histogram below. III I II IV ITEM 10: A set of average weights (measured in pounds) compiled monthly over the course of two years, of one healthy adult. Match this description to the appropriate histogram below.

41. REASONING: Matching a Graph to a Description Match each description to the appropriate histogram below. III I II IV ITEM 10: A set of average weights (measured in pounds) compiled monthly over the course of two years, of one healthy adult. Match this description to the appropriate histogram below.

42. REASONING: Matching Graphs of the Same Data

43. REASONING: Matching Graphs of the Same Data ITEM 11: One of the items on the student survey for an introductory statistics course was "Rate your aptitude to succeed in this class on a scale of 1 to 10" where 1 = Lowest Aptitude and 10 = Highest Aptitude. The instructor examined the data for men and women separately. On the right is the distribution of this variable for the 30 women in the class. Which of the following boxplots represents the same data set as the histogram shown above?

44. REASONING: Matching Graphs of the Same Data ITEM 11: One of the items on the student survey for an introductory statistics course was "Rate your aptitude to succeed in this class on a scale of 1 to 10" where 1 = Lowest Aptitude and 10 = Highest Aptitude. The instructor examined the data for men and women separately. On the right is the distribution of this variable for the 30 women in the class. Which of the following boxplots represents the same data set as the histogram shown above?

45. REASONING: Matching Graphs of the Same Data ITEM 12: The following graph shows a distribution of hours slept last night by a group of college students. Which box plot seems to be graphing the same data as this histogram?

46. REASONING: Matching Graphs of the Same Data ITEM 12: The following graph shows a distribution of hours slept last night by a group of college students. Which box plot seems to be graphing the same data as this histogram?

47. LITERACY and REASONING

48. LITERACY and REASONING ITEM 13: A local running club has its own track and keeps accurate records of each member's individual best lap time around the track, so members can make comparisons with their peers. Here are graphs of these data. Which of the above graphs allows you to most easily see the shape of the distribution of running times?

49. LITERACY and REASONING ITEM 13: A local running club has its own track and keeps accurate records of each member's individual best lap time around the track, so members can make comparisons with their peers. Here are graphs of these data. Which of the above graphs allows you to most easily see the shape of the distribution of running times?

50. LITERACY and REASONING ITEM 14: A baseball coach wanted to get an idea of how well his team did during the past baseball season. He recorded the proportion of hits obtained by each player based on their number of times at bat as shown in the table below.