End of Lecture? Active Learning Works Across STEM Disciplines

1. End of Lecture?Active Learning Works Across STEM Disciplines Sarah L. Eddy

2. My Context:

3. Outline • What is the evidence for active learning? • Can active learning also reduce achievement gaps between student groups? • How does active learning produce these outcomes?

4. What is the Evidence for Active Learning? • Calls from national agencies to incorporate active learning into all college STEM classrooms. • But what is the evidence for these calls? • Needed a synthesis of experiments on STEM college classrooms

5. What is the Evidence for Active Learning? • Turn to your neighbor: • Define Active Learning • 1 minute

6. What is the Evidence for Active Learning? • Active Learning (a working definition): • Engages students in the process of learning through activities and/or discussions in class • Emphasizes higher order thinking • Often involves groupwork • Traditional Lecture (control condition): • Students passively listen to expert • Methods: Meta-analysis • Hand searched 55 STEM education journals from 6/1/1998 to 1/1/2010 • 642 initial studies identified • 225 studies met criteria for rigor Freeman, Eddy et al. 2014 PNAS

7. What is the Evidence For active Learning? Highly Collaborative Effort DozieOkoroafor Michelle Smith Scott Freeman Mary Pat Wenderoth Miles McDonough Hannah Jordt

8. What is the EvidenceFor Active Learning? 95% confidence interval • Question:Does Active Learning increase student performance on assessments across STEM disciplines relative to Traditional Lecture? Estimate of Effect Size Effect Size: Hedges’s g = Mean Performance in Trt – Mean in Control Pooled SD Number of Studies Two groups not statistically different from each other Freeman, Eddy et al. 2014 PNAS

9. What is the Evidence For Active learning? • Question: Does Active Learning increase student performance on assessments across STEM disciplines relative to Traditional Lecture? • Yes! Student in Active Learning classes perform almost ½ a standard deviation better on identical assessments than students in Traditional Lecture classes. Freeman, Eddy et al. 2014 PNAS

10. What is the Evidence for Active Learning? • Question: Does Active Learning increase student performance on assessments across STEM disciplines relative to Traditional Lecture? • Putting results in context: • Collected SDs for exam performance from multiple years of three introductory STEM courses at UW. • In intro STEM, 6% increase in exam scores; 0.3 increase in average course grade (or B to B+, C+ to B, etc). Effect Size in Standard Deviations

11. What is the evidence for active learning? • Question: Does active learning decrease Ds, Fs earned in and/or withdrawals from courses across STEM disciplines relative to traditional lecture? • Yes: Risk difference = 12.0% • Avg. Traditional: 33.8% DFW • Avg. Active Learning: 21.8% DFW • Students 1.5 times more likely to fail in traditional course than an active learning course. Risk Difference: % DFW in Control – % DFW in Trt

12. What is the Evidence for active learning? • Question: Does active learning decrease Ds, Fs earned in and/or withdrawals from courses across STEM disciplines relative to traditional lecture? • Putting results in context: • Meta-analysis of biomedical trials showed trials stopped early for benefit when: • P-value < 0.001 • Our p-value: p<< 0.001 • relative risk: 0.53 (0.22-0.66) • Our relative risk: 0.64 • In our sample: 3,516 fewer students would fail Pocock 2006

13. Outline Active Learning • What is the evidence for active learning? • Increases student performance by ½ SD (~6%) • 12% fewer students would fail • Can active learning also reduce achievement gaps between student groups? • How does active learning produce these outcomes? Increased Student Performance Decreased DFW rates

14. What is the Evidence for Active Learning? • Question: Does Active Learning increase student performance on assessments across STEM disciplines relative to Traditional Lecture? • Putting results in context: • Collected SDs for exam performance from multiple years of three introductory STEM courses at UW. • In intro STEM, 6% increase in exam scores; 0.3 increase in average course grade (or B to B+, C+ to B, etc). Effect Size in Standard Deviations

15. What is the Evidence for Active Learning? Given this new information, what more do you want to know before beginning to conclude that Active Learning improves learning for all students? What questions remain for you? 1 minute • Studies in Meta-analysis: • Most done at one type of institution: selective R1s. • Majority of students at these institutions: White Americans. • Looked at instruction as a binary – really it’s a range. • Pooled across all types of active learning. Freeman, Eddy et al. 2014 PNAS

16. Outline Active Learning • What is the evidence for active learning? • Increases student performance by ½ SD (~6%) • Decreases failure rates by 12% • Can active learning also reduce achievement gaps between student groups? • How does active learning produce these outcomes? Increased Student Performance Decreased DFW rates

17. Can Active learningClose achievement gaps? • Why might active learning work differently for different students? • Students differ in their preparedness for college • High schools differ in how well they prepare students for college (ETS 2015) • Families differ in how much they can help students prepare and adjust to college • Students come from different cultural contexts • Interdependence vs. Independence: Latina/o, Asian, and Working Class cultures > White and Middle Class cultures (Stephens et al. 2014; Kim and Markus 2002; Crisp 2014) • Talking While Learning: Black, Working Class cultures > White or Middle Class > Asian (Kim and Markus; Hitchcock 2011) • Prediction: Active learning will help students from historically underrepresented groups in science more than those from majority groups.

18. Can active learning Reduce achievement gaps? • Context: • Large Southeastern US university • Course: all of biology in one semester • Student population: • Non-majors and underprepared majors • Race/Ethnicity: • Native American: 1.1% • Asian: 7.4% • Latin@: 10.3% • Black: 13.9% • White: 59% • Undeclared, Mixed or International: 8% • Generation Status: 24% First-Generation • Gender: 66.3% Female Kelly Hogan, Senior Lecturer Assistant Dean for the Office of Instructional Innovation, College of Arts & Sciences

19. Can Active learningReduce achievement gaps? Kelly Hogan facilitating her class

20. Can active learning Reduce achievement gaps? • Study Design: • Low Structure: 3 terms • Increased Structure: 3 terms • Statistical methods: • Outcome: Total exam points earned • Checked for exam equivalency between terms: Cognitive task exam questions ask them to do • Model: Linear Regression • Controlled for: Student SAT score Kelly Hogan facilitating her class

21. Can active learning Reduce achievement gaps? Predicted Performance based on regression model. Students matched by SAT score, gender, Term.

22. Can Active learning Reduce Achievement gaps? • Yes! Everyone sees an increase in exam performance with ‘increased structure’, but Black students and First-Generation experience an additional increase. 3.2% 6.3% 3.7% 6.1% Halves the achievement gap with White students Closes the achievement gap with Continuing-Generation students Performance Predicted based on regression model. Students matched by SAT score, gender, Term.

23. Outline Active Learning • What is Discipline Based Education Research? • What is the evidence for active learning? • Increases student performance by ½ SD (~6%) • Decreases failure rates by 12% • Can active learning also reduce achievement gaps between student groups? • Decreases gap between Black and White students • Closes gap between first- and continuing generation students • How does active learning produce these outcomes? • Structured Learning Environment that meets best practices for how people learn • Classroom climate where students feel safe to focus on learning Increased Student Performance Decreased DFW rates

24. Outline Active Learning • What is the evidence for active learning? • Increases student performance by ½ SD (~6%) • Decreases failure rates by 12% • Can active learning also reduce achievement gaps between student groups? • Decreases gap between Black and White students • Closes gap between first- and continuing generation students • How does active learning produce these outcomes? • Structured Learning Environment that meets best practices for how people learn • Classroom climate where students feel safe to focus on learning Reduces Achievement Gaps Decreased DFW rates Increased Student Performance

25. How does active learning produce outcomes? • Why should we care about the ‘how’? • We want people to be able to adapt intervention to meet their needs, but these adaptions need to be wise • Can flexibility around surface features • Need to have fidelity of implementation of critical features • Understanding how something works helps us figure out what is a critical and what is a surface feature

26. How does active learning produce outcomes?

27. How does active learning produce outcomes? Why do you think the changes in these courses led to greater student achievement? Come up with a hypothesis for at least 2 that is 10 words or less. 2 minutes

28. How does Active learning Produce outcomes? • Why did ‘increased structure’ increase student performance? • We see hints from a classroom survey: • Administered survey in one semester of low structure and three semesters of increased structure. • Survey focused on course related behaviors and attitudes: • How students allocate time outside of class • Perception of classroom climate • Value students saw in class Kelly Hogan facilitating her class

29. How does Active Learning Produce outcomes? • How did ‘increased structure’ increase student performance? • All students: • 2.6 times more likely to spend more hours studying a week for course. • 2 times more likely to read textbook before coming to class. • 2 times more likely to see class as a community. • Specifically Black students: • More likely to participate in class (White students did not change). • See more value in out of class work than White Students. Kelly Hogan facilitating her class

30. How does active learning produce outcomes? • Hypothesis 1: Structures course so students employ best practices for learning. • Required Preparatory and Review assignments: • Distributed learning vs. cramming • Testing effect • In class practice: • Elaborative interrogation and Self-explaining • Students explain to themselves/their group in their own words why something is true when answering questions

31. How does active learning produce outcomes? • Other increased structure replications fail to reduce performance gaps: • Comprehensive University in Midwest Replication: • High Structure (modified) • 4% gains for all students • No impact on Black/White exam performance gap • Caveat: Did reduce failure rate gap • Community College in PNW Replication: • Moderate structure • 3.7% gains for all students • No impact on achievement gap Casper, Eddy et al., in prep Pape-Lindstrom, Eddy et al., in prep

32. How does active learning produce outcomes? • Hypothesis 2: Creates a climate more conducive to risk taking which is important for learning. • Norms: implementations of active learning can establish a norm of ‘sense making’ rather than ‘answer-making’ (Turpen and Finkelstein 2009) • Relationships: Student perception of their relationship to the instructor, and their classmates can impact outcomes: • Feeling the instructor cares about students increases how much student enjoys subject and how much they learn (Ellis, 2000; 2004; Goodboy& Myers, 2008) • When students believe the instructor believes they can improve they are more motivated and their performance increases ( Good et al. 2012; Cohen 1999) • Climate may be particularly important for students from historically underserved populations.

33. Outline Active Learning • What is Discipline Based Education Research? • What is the evidence for active learning? • Increases student performance by ½ SD (~6%) • Decreases failure rates by 12% • Can active learning also reduce achievement gaps between student groups? • Decreases gap between Black and White students • Closes gap between first- and continuing generation students • How does active learning produce these outcomes? • Structured Learning Environment that matches best practices for how people learn • Classroom climate where students feel safe to focus on learning Structured Learning Environment Perceived Classroom Climate Increased Student Performance Decreased DFW rates Increased Student Performance

34. Thank You and Questions • Thank you to: • The students and instructors who agreed to participate in these studies. • Collaborators on Gender Projects: • Sara Brownell, Bio Educ, Arizona State Univ • Mary Pat Wenderoth, Bio Educ, Univ of Washington (UW) • Dan Grunspan, Anthropology, UW • Steve Goodreau, Anthropology, UW • Collaborators on Values Affirmation project: • Hannah Jordt, Biology, UW • Scott Freeman, Bio Educ, UW • Collaborator on Persistence Patterns Across STEM disciplines: • Erin Dolan, Bio Educ, Univ Georgia • Collaborators on student motivation projects: • Lisa Corwin, Bio Educ, UC Boulder • Mark Graham, Educ Psych, Yale • Melissa Aikens, Bio Educ, Univ New Hampshire • Chris Runyon, Bio Educ, UT Austin • Collaborators on Active Learning in introductory Biology projects: • Pam Pape-Lindstrom, Biology, Everrett Community College • Anne Casper, Biology, Eastern Michigan University • Bill Hoese, Biology, Cal State Fullerton • Carol Pollock, Bio Educ, University of British Columbia • Jerry Timbrook, Quantitative Educ Psych, University of Nebraska-Lincoln • Alison Crowe, Bio Educ, UW • Ben Wiggins, Educ, UW

36. Methods: A quantitative review of the literature • 7 criteria for inclusion of a study: • Contrast any active learning intervention with traditional lecturing (same class and institution). • Occurred in a regularly scheduled course for undergrads. • Involve changes in the conduct of class sessions (or recitation/discussion). • Involved a course in Astronomy, Bio, Chem, CompSci, Engineering, Geo, Math, Physics, Psych, Stats. • Included data on some aspect of academic performance—exam/concept inventory scores or failure rates (DFW). • If measure achievement: assessment equivalent in both treatments. • Presents data necessary to compute test statistic for meta-analysis. • Meta-analysis: a quantitative statistical analysis of separate but similar studies in order to test the pooled data for statistical significance.