Dr. Craig McBride , PhD University of Washington Tacoma mcbridec@uw

1. Flipping an Introductory Statistics Class:Students’ Attitudes About and Success with the use of Online Tools Dr. Craig McBride, PhD University of Washington Tacoma mcbridec@uw.edu

2. Purpose of the Study • Determine the effectiveness of flipping a college level Introductory Statistics course • Assess the effectiveness of using online homework and quizzes • Assess students’ attitudes towards both “flipping” and online tools • Analyze retention of knowledge and long-term attitudes

3. Sample The sample consisted of (N=40) mostly Freshman (n=21) and Sophomore (n=9) students in a highly selective private liberal arts college located in the Pacific Northwest. 21 Female, 19 Male.

4. College • 2,600 students, 43% men and 57% women from 44 states, 16 countries; 76% from out of state • 99% of tenure-line faculty hold a doctorate or equivalent • 93% of faculty are full-time • Student-faculty ratio of 12 to 1 • 13 Math faculty • All classrooms equipped with lectern computer with overhead projector NEXT TO chalkboards.

5. Design The class was “flipped” from the beginning of the term utilizing Powerpoints as the major mode of delivery for lecture material and supplemented with applets, online resources, and video lessons and tutorials. Students were expected to read through the Powerpoints on their own or in groups outside of class. Classes met 4 days per week (Mon, Tue, Thu, Fri) for 50 minutes each day.

6. Design Class time was devoted to answering some guided questions and working through examples of the techniques discussed in the Powerpoints in groups using real world data. Additionally, I would use class time to present “mini-lectures” (10-20 minutes) whenever the subject matter required it.

7. Design Midway through the term, I introduced students to the publisher’s online learning tool MyStatsLab. Students were asked to try doing the rest of their homework and quizzes for the remainder of the term online. Use of the online site was voluntary, and nine of the students opted to continue working out of the book for homework and quizzes. The last five chapters (8-12) were done online.

8. Participants 23 participated fully in the online version 9 opted out of the online version completely 2 opted out of just the online quizzes • 1 student did the last quiz online* 2 opted out of just the online homework 2 quit online HW & switched back to the book* 2 quit online quizzes (both came back for last quiz) 1 quit both homework and quizzes *1 student did both

9. Technology • Microsoft PowerPoints • Applets • Videos • Wacom Bamboo Create Tablet & Smoothdraw or PPT etc. with Camtasia (JING if you can keep it short) • MyStatLab • TI-83 calc/emulator • StatCrunch • The Island

10. Typical Class • 5 minutes asking probing questions about the PPT material to guide if they “got it” or not • Always be prepared to lecture on every topic! • 5-10 minutes doing a few examples (10-20 if it seems the group didn’t “get it” from slides etc.) • 30 minutes working in groups on more examples – slightly less if a “mini-lecture” was needed

11. Results – Course Grade Total course grade Percentage

12. Results – Course Grade Total course grade Percentage No Difference

13. Results – Final Exam Score Total course grade Percentage

14. Results – Final Exam Score Total course grade Percentage No Difference

15. Results – Exam 3 Total course grade Percentage No Difference – but closer!

16. Results – Quiz Average Total course grade Percentage No Difference

17. Results – Quiz Difference No Difference

18. Summary Results - Statistical There seems to be no real statistical difference in student performance when using online tools for quizzes and homework versus using regular paper and pencil quizzes and homework. Is this a good or a bad thing or neither?

19. What about Attitudes?

20. Please rate the pace of the class from 1 to 5 (1 = way too slow, 2 = kind of too slow, 3 = just right, 4 = kind of too fast, 5 = way too fast) - You can use decimals etc.

21. Please rate the effectiveness of the Homework on a scale from 1 to 5 (1 = Not at all helpful, 5 = Extremely helpful) Please COMPARE the effectiveness of the ONLINE Homework to the Paper Homework on a scale from 1 to 5 with the options: 1 = Book was Much Better 2 = Book was Slightly Better 3 = Book and Online were the same 4 = Online was Slightly Better 5 = Online was Much Better

22. Online vs Book Homework 1 = Book was Much Better 5 = Online was Much Better 46% Book 54% Online MEAN = 3.04 1 = Not at all helpful 5 = Extremely helpful MEANS: 3.38(online) vs 3.58 N = 13

23. Online vs Book Quizzes 1 = Book was Much Better 5 = Online was Much Better 53% Book 33% Online MEAN = 2.67 1 = Not at all helpful 5 = Extremely helpful MEANS: 3.14(online) vs 3.60 N = 15

24. Time Spent on Task How much time do you spend each week outside of class working on… ~2.5 HOURS ON HW

25. How did you like the format of the "flipped" class? Instead of "lecturing" each day, we tried to have you learn the concepts on your own outside of class with the PPTS and then we used the class time to discuss the major ideas, do some examples, and work on sample problems etc. Tell me how you like that concept compared to the "standard" version of a math class with lectures etc.

26. I liked it a lot I did not like it. Standard was much better. Eh it was alright. Yes, very much Much better learning tool. It worked pretty well.

27. I firmly believe that class time is poorly spent learning new material. To do so would be to degrade a highly skilled professor to nothing more than an interactive text book and- even more importantly- to ignore the contribution that fellow students can have on an individual's learning. Rather, the "flipped" class allowed students who struggled with the material to come to class ready to advance their understanding, allowed students with some grasp of the material to reinforce their understanding by participating with their classmates, and gave an opportunity for students who had mastered the material to investigate beyond the narrower constrains of the text book itself.

28. Retention Survey Have you thought about stats since you finished the course? IE: Have you come across something online or in your everyday life that made you think about something you learned in stats? 10 = YES (91%) 1 = NO

29. Retention Survey Now that the stats course is over, how much more do you feel you know about stats? Do you think you know more about stats now than you did BEFORE you took the course • I don't know anything more than I did before I took the course • I think I know a little more about stats • I think I know a lot more 3 (27%) = Little More 8 (73%) = Lot More

30. Retention Survey Do you like math more? After your Excellent (or not so excellent) experience in the stats course, how has your attitude about math and stats changed? • I hate math and stats even more 0 • I dislike math and stats a little more 1 • My level of like or dislike did not change 5 • I like math and stats a little more 3 • I like math and stats a lot more 2

31. Knowledge Retention I asked them to take a knowledge retention test to assess how much they remember (It was administered online during the second week of fall 2013 classes, they took the stats class in the spring of 2013). It covered some of the more important CONCEPTUAL parts of the course including correlation, confidence intervals, hypotheses, and ANOVA.

32. Knowledge Retention - Results

33. Suggestions or Questions? • Are there any analyses you would like to see or can suggest? • Suggestions for what to try next? • Any other suggestions or questions?

34. THANK YOU Dr. Craig McBride, PhD University of Washington Tacoma mcbridec@uw.edu