Holistic Course Placement and Whole-Student Model of Early Intervention

1. Holistic Course Placement and Whole-Student Model of Early Intervention • Kevin Li, Dean of Instruction • Sara Schupack, Director of Developmental Education • Lawrence Buonaguidi, Quality Assurance CoordinatorSteve Robbins, Director of Research Innovation, Academic and Workforce Success • The Innovations 2013 Conference • March 12, 2013; Dallas, Texas

2. Part 1: Holistic Course Placement

3. ETS/CCC Collaboration Objectives • Combine COMPASS and Psychosocial Measure to accelerate student progress and college completion • Students placed within the upper-band and combined with high psychosocial scores can enroll in the next level course • Allow more individualized, actionable plans for students tied to curricular goals • Non-cognitive survey and in-class behavioral ratings will be used to trigger Early Alert and intervention strategies • Create supplementary services tied to Early Alert and to behavioral monitoring

4. Background • Two new studies from the Community College Research Center (CCRC) at Teachers College, Columbia University suggest that a significant number of students may be accelerated if provided proper supports • Placement tests alone do not yield strong predictions of how students will perform in college • Success requires intervening with at-risk students to ensure completion and success

5. An Alternative Measure Based on Grounded Research: Non-Cognitive Assessment • Ameta-analysis was conducted as an attempt to integrate the psychological and educational literatures regarding the postsecondary outcomes of achievement and retention (Robbins, S., Lauver, K., Le, H., Davis, D., Langley, R., & Carlstrom, A., 2004) • Examined the relationship between psychosocial and study skill factors (PSFs) across 109 studies • Results indicated relationships between retention and academic goals, academic self-efficacy, and academic-related skills

6. An Alternative Measure Based on Grounded Research: Non-Cognitive Assessment • Respect the whole student (examine cognitive, non-cognitive, behavioral factors) • Focus on the unique individuality of students • Focus on factors educators can control and empower students to modify: such as motivation, social connectedness, as opposed to socio-economic, situational factors • CCC found that behavioral monitoring and intervention was key to student success [HAND-OUT for group exercise 1]

7. Group interaction 1: 5 min. [HAND-OUT for group exercise 1] Form a 3-4 person group Look over the hand-outs Discuss what “non-cognitive” or “psycho-social skills” mean to you and how they are manifested. How would you assess, monitor, and address these components at your campus? Report out

9. Risk Indices: Separate indices for both classroom and enrollment success. Based on background, cognitive , and psycho-social information and supported by statistical relationships with success. Background Information: Communicate key student information from both SuccessNavigator and SIS to faculty/advisor. Domain Scores: Four general areas of student strengths and weaknesses. Scores are presented normatively. Feedback: Determine by more specific “facet” scores (see next page). Action Plans: Suggested interaction with programs and services on campus.

11. Risk Indices: Separate indices for both classroom and enrollment success. Based on background, cognitive , and psycho-social information and supported by statistical relationships with success. Background Information: Communicate key student information from both SuccessNavigator and SIS to faculty/advisor. Domain Scores: Four general areas of student strengths and weaknesses. Scores are presented normatively. Feedback: Determine by more specific “facet” scores (see next page). Action Plans: Suggested interaction with programs and services on campus.

13. Risk Indices: Separate indices for both classroom and enrollment success. Based on background, cognitive , and psycho-social information and supported by statistical relationships with success. Background Information: Communicate key student information from both SuccessNavigator and SIS to faculty/advisor. Domain Scores: Four general areas of student strengths and weaknesses. Scores are presented normatively. Feedback: Determine by more specific “facet” scores (see next page). Action Plans: Suggested interaction with programs and services on campus.

15. The “Success Navigator” - Purpose • Low Stakes designed to assess the dispositions and noncognitive skills of incoming college students • Can be used with or without academic markers (SAT, HS GPA, course placement score) • Three primary intended uses: • Identifying likelihood of persistence to degree and academic failure • Informing course placement decisions • Developmental Feedback for advising, FYE, etc

16. The “Success Navigator” - Logistics • Length: 20-25 minute on-line assessment • Items • Initially, ~120 self-report items • Incorporating anchoring vignettes and forced-choice items • Demographic and life event information • Administration: Varies per institution: could be at home, during new student orientation, or in first-year experience course

17. Sample of Holistic Course Placement Chart

18. Expected Outcomes of Holistic Placement More accurate placement Acceleration: Less time spent in developmental education Help us get to know our incoming students early and efficiently More nuanced understanding of the whole student in a meaningful way Provide data to help develop placement and intervention programs

19. Part 2: Whole-Person Model of Early Alert

20. Whole-person model: 3 domains of assessment Whole-student/Whole-person Early Alert Model

21. Research shows that behavioral monitoring and intervention are key to student success

22. Group interaction 2: 10 minutes Form a group of 3-4 Look over the hand-outs. Think about your students and your programs: What are ways that you could best support the whole student on your campus? What collaborations across departments and units could you encourage? What are different ways that you can reach out to at-risk students? What are different ways that you can monitor students’ behaviors in order to intervene with a robust support program? Report out

23. Use the Whole Student Model to Drive Intervention Strategies [HAND-OUT for group exercise 2] Map support services to individual students’ needs based on non-cognitive scores Feedback loop to share non-cognitive report with students to arrive at action plans for students Consider other strategies to further student success, such as supplementary instruction, early alert and support other new and existing initiatives

24. Comprehensive Early Interventions • Model and coach at-risk students toward a set of desirable behaviors. • Based on previous research (Li et al., 2012), low-level dev ed math students can increase the likelihood of passing the course by 30% if they exhibit desirable in-class behaviors: • Active participation in groupwork (Student is actively engaged during group work; helps other students with assignments; does his/her fair share of the work, etc) • Active participation in lecture (Student's alert, attentive during class, asks/answers questions, etc) • Attendance (Attends class, stays for whole period, etc) • Completion of homework assignments (Student completes assignments thoroughly and completely, turns assignments in on time) [HAND-OUT for group exercise 2]

25. Interventions: Yes we can! [HAND-OUT for group exercise 2] • Three basic implications on early interventions: • As educators, we can motivate our students, shape and reinforce their intentionality and academic determination. (Surrounding non-cognitive assessment) • Outward behaviors are indicators of inwards states such as motivation and engagement • We have the opportunity and capability to coach our students towards a set of desirable classroom/academic behaviors.

26. Comprehensive Early Interventions Map support services to non-cognitive psychological constructs: Academic Skills, Self Management, Motivation, Social Support

27. Academic Readiness Instruction and Institutional Resources Behavioral Factors Recommendations: The 3 Pillars of Success • Place students more accurately using multiple measures, including non-cognitive skills for possible acceleration • Smarter, strategic interventions: coordinate efforts to support the whole student • Institutional commitment: Faculty, advisors, and staff collaborate in early alert program • Generate fine-grained reporting to empower frontline workers and drive further research and program development

28. References ACT, Inc. (2002). Creating seamless educational transitions for urban African American and Hispanic students. Iowa City, IA: Author. Scott-Clayton, J. (2012). Do High-Stakes Placement Exams Predict College Success? Predicting Success in College: The Importance of Placement Tests and High School Transcripts. CCRC Working Paper No. 41. New York: Community College Research Center, Teachers College, Columbia University. ACT, Inc. (2004). Schools involving parents in early postsecondary planning. Iowa City, IA: Author. ACT, Inc. (2008). Student Readiness Inventory user’s guide. Retrieved from ACT, Inc. website: http://www.act.org/sri/pdf/UserGuide.pdf ACT, Inc. (2008). What we know about college success: Using ACT data to inform educational issues. Iowa City, IA: Author. ACT, Inc. (2008). Student Readiness Inventory Toolshop. Retrieved from ACT, Inc. website: http://www.act.org/sri/studentguide/toolshop.html Allen, J. & Robbins, S. (2010). Effects of interest-major congruence, motivation, and academic performance on timely degree attainment. Journal of Counseling Psychology, 57, 23-35. Allen, J., Robbins, S., Casillas, A., & Oh, I.-S. (2008). Third-year college retention and transfer: effects of academic performance, motivation, and social connectedness. Research in Higher Education, 49(7), 647-664. American Association of Community Colleges (AACC). (2009). Community college fast facts. Retrieved July 10, 2009 from http://www2.aacc.nche.edu/research/index.htm Belfield, C.R., Crosta, P.M. (2012). Predicting Success in College: The Importance of Placement Tests and High School Transcripts. CCRC Working Paper No. 42. New York: Community College Research Center, Teachers College, Columbia University.

29. References (cont.) Berkner, L., & Choy, S. (2008).Descriptive summary of 2003-04 beginning postsecondary students: Three years later. Washington, DC: U.S. Department of Education, Institute of Education Sciences. Bragg, D. D. (2001). Community college access, mission and outcomes: Considering intriguing intersections and challenges. Peabody Journal of Education. 76(1), 93-116. Community College Research Center. (2002a). Accountability and learning outcomes in community colleges. Retrieved July 7, 2008, from http://ccrc.tc.columbia.edu/Seminar.asp?uid=8 Community College Research Center. (2002b). Accountability and learning outcomes in community colleges. Retrieved July 7, 2008, 2008, from http://ccrc.tc.columbia.edu/Seminar.asp?uid=8 Habley, W. & McClanahan, R. (2004).What Works in student retention – All survey colleges. Iowa City, IA: ACT, Inc. Harris, S. (1991) Can’t you guys read? Cartoons on academia. New Brunswick, NJ: Rutgers University Press. Herzog, K. & Miller, M. P. (Eds.). (1985).Scholarship: More great cartoons from the Kappan. Bloomington, IN: Phi Beta Kappa International Inc. Le, H., Casillas, A., Robbins, S., & Langley, R. (2005). Motivational and skills, social, and self-management predictors of college outcomes: Constructing the Student Readiness Inventory. Educational and Psychological Measurement, 65, 482-508. Li, K, & Cordes, C. (2010).Individualizing the Delivery of Math Developmental Education. Talk presented at the 34th Annual National Association of Developmental Education Conference, Columbus, OH. Lotkowski, V., Robbins, S., & Noeth, R. (2004).The role of academic and non-academic factors in improving college retention. Retrieved from ACT, Inc. website: www.act.org/research/policy/index.html. McCabe, R.H. (2000).No one to waste. Washington, DC: Community College Press. NCES (2004).Remedial education at degree-granting postsecondary institutions in fall 2000 (NCES 2004-010). NISOD (2012). Assessing College-readiness assessments. Webinar delivered on 3/28/2012.

30. References (cont.) Pascarella, E. T., & Terenzini, P. T. (2005).How college affects students: A third decade of research. San Francisco: Jossey-Bass. Peterson, C. H., Casillas, A., & Robbins, S. B. (2006). The Student Readiness Inventory and the Big Five: Examining social desirability and college academic performance. Personality and Individual Difference, 41, 663-673. Porchea, S., Allen, J., Robbins, S., Phelps, R. (in press). Predictors of long-term enrollment and degree outcomes for community college students: Integrating academic, psychosocial, socio-demographic, and situational factors. Journal of Higher Education. President’s Commission on Higher Education. (1947). Organizing higher education. Washington, DC: Author. Robbins, S. B., Allen, J. Casillas, A., Akamigbo, A., Saltonstall, M., Campbell, R., Mahoney, E. & Gore, P.A. (2009). Associations of resource and service utilization, risk level, and college outcomes. Research in Higher Education, 50(1), 101-118. Robbins, S., Allen, J., Casillas, A., Peterson, C., & Le, H. (2006). Unraveling the differential effects of motivational and skills, social, and self-management measures from traditional predictors of college outcomes. Journal of Educational Psychology, 98, 598-616. Robbins, S. B., Lauver, K., Le, H., David, D., Langley, R., & Carlstrom, A. (2004). Do psychosocial and study skill factors predict college outcomes? A meta-analysis. Psychological Bulletin, 130, 261-288. Robbins, S., Oh, I., Le, H., & Button, C. (2009). Intervention effects on college performance and retention as mediated by motivational, emotional, and social control factors: Integrated meta-analytic path analyses. Journal of Applied Psychology, 94, 1163-1184. Swail, W. S. (2004, January 23). Legislation to improve graduation rates could have the opposite effect. The Chronicle of Higher Education, B16.

31. Refining Course Placement Practices and Driving toward Student Success using a Holistic Model Correspondence regarding this presentationshould be addressed to: Steve Robbins, Director, Research Innovation, ETS srobbins@ets.org Kevin Li, Dean of Instruction Wilbur Wright College,One of the City Colleges of Chicago kli@ccc.edu