First-Year STEM Student Cohorts: Assessment and Best Practices

1. First-Year STEM Student Cohorts:Assessment and Best Practices • Gary Hunt • Boise State University; • Edmund Tsang • Western Michigan University

2. Session Structure • Introduction – Tsang/Hunt • Part I: Building Large Cohorts – Tsang • Part II: Assessing Cohorts – Hunt • Wrap-up – Hunt

3. Cohorts – Introduction • Why do we create cohorts? • Improved retention • Tinto: social and academic engagement • Social: organized social activities; informal • Academic: performance; interactions with faculty & staff • Campus: resource connections • Enrollment logistics • Learning communities can help campuses with managing course offerings

4. Cohorts, examples • Living Learning Communities • Bridge programs • Learning communities • UROPs (undergraduate research opportunities programs)

5. Social Learning Theory: COPs • Communities of practice are formed by people who engage in a process of collective learning in a shared domain of human endeavor: a tribe learning to survive, a band of artists seeking new forms of expression, a group of engineers working on similar problems, a clique of pupils defining their identity in the school, a network of surgeons exploring novel techniques, a gathering of first-time managers helping each other cope. In a nutshell: • Communities of practice are groups of people who share a concern or a passion for something they do and learn how to do it better as they interact regularly. http://www.ewenger.com/theory/

6. Communities of Practice • COP are groups of people (community) • Who share a concern or a passion for something they do (domain) • And learn how to do it better (practice). We are trying to create COPs with most freshmen cohorts

7. Example:Engineering Residential Dormitory

8. Questions this session addresses • Logistics associated with large university creation of cohorts; • Assessment: how does an institution or program know if their cohort program is cost effective?

9. Building and Assessing Learning Community/Cohorts Since evidence shows learning community/cohort is a powerful strategy to support student success and retention • Ideally should include as many students as possible • Logistics of building LC involving several hundred students (particularly if they have diverse academic preparation background) • WMU/CEAS Objectives – students in a cohort take the same 3-5 courses together in fall semester and 2-4 courses in spring semester of their first year • Constrained to small number of students • Selection criteria are important – how to get the biggest impact • Avoid “false positive” results Highly Motivated  Moderately Motivated  Not Motivated (Well/Moderately/ (Well/Moderately/ (Well/Moderately/ Poorly Prepared) Poorly Prepared) Poorly Prepared)

10. Build Cohorts Involving ~400 Students at WMU Constraints at WMU/CEAS • Entering 1st-year CEAS students place into Algebra II (~25%), Precalculus (~30%), and Calculus I (~45%) • 13 engineering and applied sciences program organized into 6 departments • No common first-year curriculum in CEAS • CCE, ECE, CHEG, PAPR have their own 1st-semester course • Mathematics, chemistry, physics courses outside CEAS and they do not want to “zero-out” any courses/labs What are the constraints on your campus?

11. Build Cohorts Involving ~400 Students at WMU Factors to consider in building cohorts • Chemistry lab sections have capacities of 24 students/lab • Math sections have capacity of 40 students/section • Physics lab sections have capacities of 20 students/lab • Engineering Graphics lab sections have capacity of 24 students/lab • Technical Communication sections have capacity of 24 students/section What are the factors on your campus to consider?

12. Build Cohorts Involving ~400 Students at WMU Strategies to build cohorts at WMU • # students/cohort = 24 based on section size of technical communication/engineering graphics lab, chemistry lab (take 12 seats from 2 labs) and other labs, and mathematics • Importance of an anchor class • Use admit and historic yield data, as well as MATH ACT sub-score to project # seats required • Submit seat requests in January for summer orientation/fall registration; early October for spring registration; complete iterations in February/late October • Keep departments informed of seat usage; release unused seats immediately What strategies would work best on your campus?

13. Build Cohorts Involving ~400 Students at WMU Examples of cohort course templates

14. Build Cohorts Involving ~400 Students at WMU Examples of cohort course templates

15. Build Cohorts Involving ~400 Students at WMU Examples of cohort course templates

16. Build Cohorts Involving ~400 Students at WMU Lessons learned • Placing students in cohorts has become standard practice in CEAS summer orientation/registration since 2005, shifting focus of summer orientation from course registration to learning more about their programs and how to be successful • Captures 85-90% of students (not placed in a cohort are students with many AP credits; athletes; students who opt out) • Math, chemistry, and physics are very supportive of the practice  reduces uncertainty because they know which sections will make

17. Part II – Assessing CohortsGary Hunt, Boise State University • Case Study 1: Stem Summer Adventure • Any questions? • Here is a summary of your responses… • Which would you implement? • What are the two most significant barriers to assessment? • Discussion

18. Part II – Assessing CohortsGary Hunt, Boise State University • Case Study 2: Research • Any questions? • Here is a summary of your responses… • Which would you implement? • What are the two most significant barriers to assessment? • Discussion

19. Wrap-up:A specific cohort, andhow it is being assessed.

20. What we did --Stem Summer Adventure August 2012 5 day extended field program

21. Why? • Goal: to increase first year retention, leading to increased STEM graduation rates • The lever we are pulling: Build community, increase student engagement into campus • Hypothesis: Incoming freshmen STEM students who participate in the summer adventure and the associated seminar class will be more integrated into the university than the average student.

22. Treatment: • 5 days/4 nights of rafting, camping, social and curricular activities • Team building and social activities led by the outdoor program • Science curricular activities -- faculty lead with volunteers • Weekly seminar class in fall • 15 one hour meetings include – • Student success information • Tours of labs and departments • Social activities

23. How to measure success? • NSSE-like question survey (national survey of student engagement) • What is NSSE? • Nsse.lub.edu

24. How to measure success? • Qualitative: student feedback in form of student videos and question responses • Quantitative: NSSE question survey (national survey of student engagement) • BSU had administered NSSE • We had aggregate scores for the campus to compare with • Students in the summer adventure program took survey at the end of their first semester

25. Qualitative Feedback • Typical student responses to: • What did you like the best about this trip? • What needs improvement? • Would you recommend this trip to a friend? http://www.youtube.com/watch?v=PAq6Cpzxqag&feature=plcp

26. Quantitative results • Selected NSSE questions were used to measure student engagement • Statistical results showed that Cohort participants showed greater engagement. The result was significant. • χ2(3, N = 12) = 8.9, p <.01

27. Discussion: • What would you do to assess this program? • What do you like about this program? • What are you concerned about? • What is your best take away from this session?