First-Year STEM Student Cohorts: Assessment and Best Practices

1. First-Year STEM Student Cohorts:Assessment and Best Practices • Janet Callahan, Doug Bullock, Gary Hunt • Boise State University; • Edmund Tsang • Western Michigan University

2. Session Structure • Introduction – Callahan • Part I: Building Large Cohorts – Tsang • Part II: Assessing Cohorts – Bullock • Wrap-up – Hunt

3. Cohorts – Introduction • Why do we create cohorts? • Improved retention • Tinto: social and academic engagement • Social: organized social activities; informal • Academic: performance; interactions with faculty & staff • Campus: resource connections • Enrollment logistics • Learning communities can help campuses with managing course offerings

4. Cohorts, examples • Living Learning Communities • Bridge programs • Learning communities • UROPs (undergraduate research opportunities programs)

5. Social Learning Theory: COPs • Communities of practice are formed by people who engage in a process of collective learning in a shared domain of human endeavor: a tribe learning to survive, a band of artists seeking new forms of expression, a group of engineers working on similar problems, a clique of pupils defining their identity in the school, a network of surgeons exploring novel techniques, a gathering of first-time managers helping each other cope. In a nutshell: • Communities of practice are groups of people who share a concern or a passion for something they do and learn how to do it better as they interact regularly. http://www.ewenger.com/theory/

6. Communities of Practice(as Tania Siemens expressed last night) • COP are groups of people (community) • Who share a concern or a passion for something they do (domain) • And learn how to do it better (practice). It seems to me that we’re trying to create COPs with most freshmen cohorts

7. Engineering Residential College

8. Questions this session addresses • Logistics associated with large university creation of cohorts; • Assessment: how does an institution or program know if their cohort program is cost effective?

9. Building and Assessing Learning Community/Cohorts Since evidence shows learning community/cohort is a powerful strategy to support student success and retention • Ideally should include as many students as possible • Logistics of building LC involving several hundred students (particularly if they have diverse academic preparation background) • WMU/CEAS Objectives – students in a cohort take the same 3-5 courses together in fall semester and 2-4 courses in spring semester of their first year • Constrained to small number of students • Selection criteria are important – how to get the biggest impact • Avoid “false positive” results Highly Motivated  Moderately Motivated  Not Motivated (Well/Moderately/ (Well/Moderately/ (Well/Moderately/ Poorly Prepared) Poorly Prepared) Poorly Prepared)

10. Build Cohorts Involving ~400 Students at WMU Constraints at WMU/CEAS • Entering 1st-year CEAS students place into Algebra II (~25%), Precalculus (~30%), and Calculus I (~45%) • 13 engineering and applied sciences program organized into 6 departments • No common first-year curriculum in CEAS • CCE, ECE, CHEG, PAPR have their own 1st-semester course • Mathematics, chemistry, physics courses outside CEAS and they do not want to “zero-out” any courses/labs What are the constraints on your campus?

11. Build Cohorts Involving ~400 Students at WMU Factors to consider in building cohorts • Chemistry lab sections have capacities of 24 students/lab • Math sections have capacity of 40 students/section • Physics lab sections have capacities of 20 students/lab • Engineering Graphics lab sections have capacity of 24 students/lab • Technical Communication sections have capacity of 24 students/section What are the factors on your campus to consider?

12. Build Cohorts Involving ~400 Students at WMU Strategies to build cohorts at WMU • # students/cohort = 24 based on section size of technical communication/engineering graphics lab, chemistry lab (take 12 seats from 2 labs) and other labs, and mathematics • Importance of an anchor class • Use admit and historic yield data, as well as MATH ACT sub-score to project # seats required • Submit seat requests in January for summer orientation/fall registration; early October for spring registration; complete iterations in February/late October • Keep departments informed of seat usage; release unused seats immediately What strategies would work best on your campus?

13. Build Cohorts Involving ~400 Students at WMU Examples of cohort course templates

14. Build Cohorts Involving ~400 Students at WMU Examples of cohort course templates

15. Build Cohorts Involving ~400 Students at WMU Examples of cohort course templates

16. Build Cohorts Involving ~400 Students at WMU Lessons learned • Placing students in cohorts has become standard practice in CEAS summer orientation/registration since 2005, shifting focus of summer orientation from course registration to learning more about their programs and how to be successful • Captures 85-90% of students (not placed in a cohort are students with many AP credits; athletes; students who opt out) • Math, chemistry, and physics are very supportive of the practice  reduces uncertainty because they know which sections will make

17. Part II – Assessing CohortsDoug Bullock, Boise State University • read & brainstorm assessment strategies; • rotate papers; discuss; select one or two that your group would implement • report out

18. Wrap-upGary Hunt, Boise State UniversityA specific cohort;how it is being assessed.

19. What we did --Stem Summer Adventure August 2012 5 day extended field program

20. Why? • Goal: to increase first year retention, leading to increased STEM graduation rates • The lever we are pulling: Build community, increase student engagement into campus • Hypothesis: Incoming freshmen STEM students who participate in the summer adventure and the associated seminar class will be more integrated into the university than the average student.

21. Treatment: • 5 days/4 nights of rafting, camping, social and curricular activities • Team building and social activities led by the outdoor program • Science curricular activities -- faculty lead with volunteers • Weekly seminar class in fall

22. How to measure success? • NSSE-like question survey (national survey of student engagement) • What is NSSE? • Nsse.lub.edu

23. How to measure success? • Qualitative: student feedback in form of student videos and question responses • Quantitative: NSSE question survey (national survey of student engagement) • BSU had administered NSSE • We had aggregate scores for the campus to compare with • Students in the summer adventure program took survey at the end of their first semester

24. Qualitative Feedback • Typical student responses to: • What did you like the best about this trip? • What needs improvement? • Would you recommend this trip to a friend? http://www.youtube.com/watch?v=PAq6Cpzxqag&feature=plcp

25. Quantitative results • Selected NSSE questions were used to measure student engagement • Statistical results showed that Cohort participants showed greater engagement. The result was significant. • χ2(3, N = 12) = 8.9, p <.01

26. Discussion: • What would you do to assess this program? • What do you like about this program? • What are you concerned about? • What is your best take away from this session?