RP Regional Research Groups – Spring Convening

1. RP Regional Research Groups – Spring Convening Andrew LaManque, Foothill-De Anza CCD Alice VanOmmeren, CCC Chancellor's Office Ryan Fuller, CCC Chancellor's Office Daylene Meuschke, College of the Canyons Barbara McNeice-Stallard, Mt. San Antonio College Terrence Willett, RP Group

2. - RP Report on Prerequisite Validation - Student Success Scorecard - Wage Tracker - Professional Development Opportunities - IRB Issues - RP Group Update A Brief Overview Measuring Our Success, RP Conference, April 2013

3. Update on the Scorecard (formerly know as ARCC) and WageMart Alice van Ommeren Ryan Fuller

4. Developments • In 2012, the Scorecard Advisory Group and TRIS developed metrics and reporting format • In January 2013, researchers had 45-days to review report format and Scorecard metrics • April 9, the public release of the Scorecard • Last week, the Scorecard on the DataMart • System of the State this summer and the advisory group reconvenes in the fall

5. College Responsibilities • ARCC- Alias List for Updates and Revisions • Two changes already, ESL and Math Remedial Rate • Board of Trustee Interactions (due March 2014) • Use the 45-day review period in January for questions and data resubmission (data on demand) • Check other Scorecard resources on webpage • New peer groups and % of missing SSN report • Use scorecard@cccco.edu or arcc@cccco.edu for questions and feedback

6. Most FAQs/Issues of Concern • Why do Prepared students have lower rates than Unprepared (persistence)? • What are the N’s behind the metrics? • To disclose, or not? When is the cell size too small? • Revise Data on Demand files to include flags • Why are some of my remedial courses not qualifying for the cohort? • TOP code and CB04

7. Remedial ESL Rate (cohort) Cohort is first attempt of an ESL course in any levels below transfer: • CB03 COURSE-TOP-CODE = 4930.84, 4930.85, 4930.86, 4930.87 • SB01 STUDENT IDENTIFIER STATUS = S (only students with SSN) • CB21 COURSE-PRIOR-TO-COLLEGE-LEVEL = A, B, C, D, E, or F • CB04 COURSE-CREDIT-STATUS = C

8. Remedial Math Rate (cohort) Cohort is first attempt of a Math course in two to four levels below transfer: • CB03 COURSE-TOP-CODE = 1701* • SB01 STUDENT IDENTIFIER STATUS = S (only students with SSN) • CB21 COURSE-PRIOR-TO-COLLEGE-LEVEL = B, C, or D • CB04 COURSE-CREDIT-STATUS = C (not D)

9. We are working on… • Revisions for next years’ Scorecard • Advisory Group in the fall, student-counselor ratio • Guidelines for Measuring the Achievement Gap using the Scorecard metrics • ISP and OOS Report (modifying methodology) • Exploring uses of the Scorecard metrics for other reporting requirements • WageTracker or WageMart (release May 29th)

10. New Wage Tracker Tools on Datamart

11. Methodology • Two different methodologies • Systemwide wages for award completers • By campus wages for award completers • Award completer data joined with EDDUI wage data. • Both wage methodologies break out by award discipline (6 digit TOP code) and award type.

12. Systemwide Award Completer Wages • Award completers in a given academic year, exclude: • Enrolled anywhere in higher education after award • 21 and under at time of award • Join with EDDUI (Unemployment Insurance) wage file • Sum all quarters of wages to student annual wage • All wages inflation adjusted using California CPI-U • Calculate median wages by discipline and award type • Median calculated for students with wages 2 years before award, 2 years after and 5 years after award

13. By College, Award Completer Wages • Same methodology as statewide wages, except: • Series of 8 years of award completers (to increase the number of students in the cohorts) • Median calculated for students in cohort with wages 3 years after award • Calculate median wages by “college,” discipline and award type

14. Systemwide vs. By College Methodology Systemwide 05-06 Awards Wages 04-05 07-08 10-11 03-04 05-06 08-09 06-07 09-10 By College 05-06 02-03 04-05 06-07 00-01 01-02 03-04 07-08 Awards Wages 07-08 10-11 04-05 06-07 08-09 03-04 05-06 09-10

15. Data Caveats • EDDUI Data contains wages of occupations covered by CA Unemployment Insurance • Excludes military, federal government, self-employed, out of state and unemployed • No way of determining part time or full time wages. Hours worked not in the EDDUI data. • Only students with SSNs • Wages are adjusted for inflation to current dollars using the California CPI-U

16. Analysis Caveats • Data reflects only the wage outcomes of award earners who remained in California • Region/locale significantly impacts wages (i.e. cost of living), caution in comparing earnings across the state. • Wages are not necessarily from employment associated with a particular award discipline, no information on job • Short term wage outcomes should not be a sole measure of institutional effectiveness or program quality • Other factors besides earnings motivate students to earn an award in a specific discipline

19. Accessing the new tools • Query modules using both the Systemwide and ‘By College’ methods will be available in datamart (System Wage Tracker & College Wage Tracker) • Release on May 29th • Median wages by program code will be available for colleges in Data on Demand.

20. Proposed RP Professional Development Plan 2013-14 Daylene Meuschke, College of the Canyons Barbara McNeice-Stallard, Mt. San Antonio College

21. The following slides present the proposed professional development plan for the RP Group in 2013-14. • The first slide presents the an overview of the plan. • Slides 3-5 provide detail for each segment of the plan. • Slides 6-8 identify possible webinar topics for each of the three big training areas. A Brief Overview Measuring Our Success, RP Conference, April 2013

22. RP GroupProfessional Development

29. RP Group Prerequisite Guide Overview Andrew LaManque, Ph.D. President, RP Group Regional Research Meetings RP Regional Meeting, May 3, 2013

30. Thanks! This project was made possible through the efforts of many dedicated people within the California Community College system! Special thanks goes to: • Anne Danenberg, Sacramento City College • LanHao, Citrus College • Andrew LaManque, Foothill De Anza Community College District • Katherine McLain, Cosumnes River College • DayleneMeuschke, College of the Canyons • Ken Meehan, formerly Fullerton College • Steven Spurling, San Francisco City College • Keith Wurtz, Crafton Hills • LijuanZhai, Fresno City College

31. Using Research Questions to Frame the Discussion Pre-implementation • Are students highly unlikely to succeed in the target course without the knowledge, skills, and abilities learned in the prerequisite course? • Do student success rates in the target course vary for those students with or without the pre-requisite? • Is the projected impact on course availability, enrollment and success different depending on student demographics? Post-implementation • Did the student success rates of a target course increase after the implementation of a pre-requisite? • Is the actual impact on course enrollment and success different depending on student demographics?

32. The Prerequisite Research Framework (pre)

33. The Prerequisite Research Framework (post)

34. Next Steps • Will be seeking input in the coming weeks: • Institutional Researchers • Faculty • Chancellor’s Office • Instructional and Student Service Officers • Will be reviewing the Disproportionate Impact section

35. Questions? Please submit any additional questions to your webinar moderator and they will get back to you with answers. Find out more: Andrew LaManque, Foothill-De Anza CCD lamanqueandrew@fhda.edu

36. RP Group Update Terrence Willett, The RP Group For the latest on our projects visit: http://www.rpgroup.org/projects/current