1 / 27

Why a measure of noncognitive abilities is essential to drive student success

Innovations 2013 Dallas, Texas March 2013. Why a measure of noncognitive abilities is essential to drive student success. Victoria Monaghan and Steve Robbins Educational Testing Service. More than 1.7 million students begin community college each year. But many are unprepared.

tracey
Download Presentation

Why a measure of noncognitive abilities is essential to drive student success

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Innovations 2013 Dallas, Texas March 2013 Why a measure of noncognitive abilities is essential to drive student success Victoria Monaghan and Steve Robbins Educational Testing Service

  2. More than 1.7 million students begin community college each year. But many are unprepared We don’t need to remind you: • 50 percent of community college students need remedial – or below college level – courses • 20 percent of 4-year college students also start in remediation • 40 percent of these students – frustrated by their low placement –don’t even bother to enroll in classes

  3. And the path to college-level work has detours 4 our of every 10 students in remedial classes never complete their remedial sequence Placement tests alone do not yield strong predictions of how students will perform in college We believe there is a critical need to use a holistic, multiple measures approach to college course placements

  4. What are Noncognitive Skills? Conscientiousness Metacognition Social Support Anything outside of “academic ability” or “academic intelligence” that contributes to or is part of student learning Study Skills Teamwork Motivation Self Efficacy Test Taking Strategies Worry Goal Setting Institutional Commitment

  5. The appeal of these skills “Genius is 1% inspiration and 99% perspiration.” ~Thomas Edison • We like them • We (and others) value them • They do indeed work • We can change them

  6. Our mission statements agree, identifying noncognitive skills frequently Taxonomy of Valued Academic Skills: Mission Statements from 35 schools(College Board/Michigan State University)1,2 • Social responsibility, citizenship and involve • Physical and psychological health • Career orientation • Adaptability and life skills • Perseverance • Interpersonal skills • Ethics and integrity • Knowledge, learning, mastery of general principles • Continuous learning, intellectual interest and curiosity • Artistic cultural appreciation and curiosity • Multicultural tolerance and appreciation • Leadership 1 Schmitt, Oswald, Kim, Imus, Drzakowski, Friede, & Shivpuri (2007). 2 Oswald, Schmitt, Kim, Ramsay, & Gillespie (2004).

  7. And employers care as well Conference Board/SHRM survey of > 400 employers from across the United States (Casner-Lotto & Barrington, 2006)

  8. Adding a noncognitive assessment allows for a holistic view of students • Respects the whole student (examines cognitive, non-cognitive, behavioral factors) • Focuses on the unique individuality of students • Focuses on factors educators can control and empower students to modify: such as motivation, social connectedness, as opposed to socio-economic, situational factors • A common language for early warning, advising, and acceleration decisions

  9. From the “so what” to the “now what”…….. How do we... Data Integration & Use Across Silos • Identify these skills in an operational setting? • Best disseminate that information to students, faculty, staff, and institutions? • Develop and target sound and practical interventions? • Consider individual differences in background, skills, abilities, and experiences? Holistic Assessment Student Commitment & Classroom Success

  10. Early Intervention is key Three basic implications: • Educators have a profound influence on students • Faculty can motivate and inspire students • Educators can also shape and foster their intentionality and academic determination • Observable behaviors are indicative of internal states • E.g., Motivation, engagement • Educators have the opportunity and capability to guide students towards desirable classroom/academic behaviors

  11. Noncognitive Assessment Experience at Wright College semester Note. Results from Wright College (Spring 2009). n = 453; All paths are statistically significant.

  12. Research shows that behavioral monitoring and intervention are key to student success

  13. ETS/CCC Collaboration Objectives • Combine COMPASS and Psychosocial Measure to accelerate student progress and college completion • Students placed within the upper-band with high psychosocial scores can enroll in the next level course • Allow more individualized, actionable plans for students tied to curricular goals • Non-cognitive survey and in-class behavioral ratings will be used to trigger Early Alert and intervention strategies • Create supplementary services tied to Early Alert and to behavioral monitoring

  14. Comprehensive Early Interventions Map support services to non-cognitive psychological constructs: Academic Skills, Self Management, Motivation, Social Support

  15. Student Name: Juan Rodriguez • Student ID: 12-34567 • Score Date: Sep 12, 2013 Advisor Report NOT RECOMMENDED LOW LOW COURSE ACCELERATION ACADEMIC SUCCESS SCORE* RETENTION SUCCESS SCORE* SKILL REPORT MODERATE MODERATE *Course Placement: Recommended academic placement action for student. * Academic Success Score: Weighted composite of student’s SuccessNavigator™ profile and other academic indicators of student preparedness, such as self or institution reported high school GPA, SAT/ACT, etc. * Retention Success Score: Projected likelihood that student will return for a second year at the institution. MODERATE LOW

  16. Juan Background: Latino Male Major: Business Status: Part-time (10 credits) Current GPA: 2.29 Age:18 Parental Education: Unknown (Mother), Unknown (Father) Misc: Undocumented ESL student who is in the National Guard to obtain citizenship. He works 40 hours/week at UPS to cover his tuition and help his family financially. • Academic Preparation: • 2.3 HS grade point average • Scored 900 on SAT • Placed in bottom third of placement tests • Taking remedial Algebra and English Success Navigator profile indicates: He has moderate Academic Skills, Commitment, and Self- Management, but low Social Support Notes: Overall Juan is struggling both academically and socially and is at-risk for dropping out. His financial / work responsibilities and full-time student status have limited his ability to focus on his academics and interact with other students. Since he has little support for attending college he would benefit from engaging with formal (e.g. tutoring) and informal resources on campus. Also, because he remains dedicated to earning his degree and is confident in his ability to succeed, he might benefit from reducing his course load even further which will help him balance all his responsibilities. Student Withdrew

  17. Student Name: Sarah Smith • Student ID: 13-75680 • Score Date: Sep 2, 2013 Advisor Report RECOMMENDED MODERATE HIGH COURSE ACCELERATION ACADEMIC SUCCESS SCORE* RETENTION SUCCESS SCORE* SKILL REPORT MODERATE MODERATE *Course Placement: Recommended academic placement action for student. * Academic Success Score: Weighted composite of student’s SuccessNavigator™ profile and other academic indicators of student preparedness, such as self or institution reported high school GPA, SAT/ACT, etc. * Retention Success Score: Projected likelihood that student will return for a second year at the institution. HIGH MODERATE

  18. Sarah Background: Caucasian Female Major: Liberal Arts Status: Full-time Current GPA: 3.0 Age:19 Parental Education: Below High School (Mother), High School (Father) Misc: She is the first member of her family to attend college. She was very involved in high school extra-curriculars. She was class secretary, earned a varsity letter in soccer, and was a member of the chorus. • Academic Preparation: • 2.8 HS grade point average • Scored below 900 on SAT • Placed in remedial Algebra Noncognitive profile indicates: She has strong Commitment and above average Academic Skills, Self Management, and Social Support Prime candidate for acceleration Notes: Although Sarah struggles academically, especially in math, she is a conscientious student who is in regular contact with her advisor and has used study groups to help her academically. Since she has very high commitment both to the institution and to the goal of earning a degree, she would benefit from continual encouragement to make use of the academic resources on campus such as the tutoring center. She should also be encouraged to maintain regular contact with her professors.

  19. Student Name: Janet Jerzyk • Student ID: 18-34287 • Score Date: Sep 25, 2013 Advisor Report RECOMMENDED HIGH MODERATE COURSE ACCELERATION ACADEMIC SUCCESS SCORE* RETENTION SUCCESS SCORE* SKILL REPORT HIGH HIGH *Course Placement: Recommended academic placement action for student. * Academic Success Score: Weighted composite of student’s SuccessNavigator™ profile and other academic indicators of student preparedness, such as self or institution reported high school GPA, SAT/ACT, etc. * Retention Success Score: Projected likelihood that student will return for a second year at the institution. MODERATE MODERATE

  20. Janet Background: Multiracial/Multi-ethnic Female Major: Education Status: Full-time Current GPA: 3.25 Age:19 Parental Education: Unknown (Mother), Unknown (Father) Misc: Janet has significant financial hardship and has received considerable financial aid. She also works 25 hours/week as a waitress. She picks up extra shifts whenever possible. • Academic Preparation: • 3.3 HS grade point average (top 20% of class) • Scored 1200 on SAT • Placed into college level courses Noncognitive profile indicates: She has strong Academic Skills, and Self-Management, and average Commitment and Social Support. Notes: Janet is a capable student with strong organizational skills with promising academic potential. She is confident in her ability to succeed and does not easily get distressed by her school work. Janet is not inclined to seek out help and should be encouraged to make use of campus resources. In particular, Janet has financial constraints and should be encouraged to reach out to her financial aid counselor to discuss the availability of scholarships Also, although Janet is committed to earning a degree, she has expressed interest in transferring soon and may not feel connected to institution. She may benefit from seeing a transfer counselor and engaging in extra-curricular activities.

  21. Student Name: Julia Fuentes • Student ID: 13-34588 • Score Date: Sep 15, 2013 Advisor Report NOT RECOMMENDED HIGH LOW COURSE ACCELERATION ACADEMIC SUCCESS SCORE* RETENTION SUCCESS SCORE* SKILL REPORT LOW MODERATE *Course Placement: Recommended academic placement action for student. * Academic Success Score: Weighted composite of student’s SuccessNavigator™ profile and other academic indicators of student preparedness, such as self or institution reported high school GPA, SAT/ACT, etc. * Retention Success Score: Projected likelihood that student will return for a second year at the institution. HIGH LOW

  22. Julia Background: Latina Female Major: Liberal Arts, transfer to Psychology Status: Full-time Current GPA: 4.0 Age:19 Parental Education: Some College (Mother), Some College (Father) Misc: Is the first in her family to attend college. She has five younger siblings whom she is responsible for while her parents work. She was a top student in high school and a member of the National Honor Society. • Academic Preparation: • 3.9 HS grade point average (top 5% of class) • Scored 27 on ACT (over 1200 on SAT) • Placed into college level courses Noncognitive profile indicates: She has significantly above average Commitment, moderate Academic Skills and Self-Management, and low Social Support Notes: Julia is an academically able student who made the Dean’s List in her first semester despite having taken 17 credits. She is very committed to earning a degree and her long term educational aspirations of earning a Ph.D. Due to her desire to do well academically, she is easily stressed and gets very nervous prior to tests. In addition, because of her heavy course load and family responsibilities she has not made many friends on campus. Julia would benefit from seeing a counselor to help her overcome her anxiety and should be encouraged to engage in extra-curricular activities.

  23. Questions? Or ask us later ……. Steve Robbins, Director, Research Innovations, Academic and Workforce Readiness and Success, ETS. srobbins@ets.org Victoria Monaghan, Director of Strategic Initiatives in Higher Education, ETS. vmonaghan@ets.org

  24. References Allen, J., & Robbins, S. (2010). Effects of interest-major congruence, motivation, and academic performance on timely degree attainment. Journal of Counseling Psychology, 57, 23-35. Allen, J., Robbins, S., Casillas, A., & Oh, I.-S. (2008). Third-year college retention and transfer: effects of academic performance, motivation, and social connectedness. Research in Higher Education, 49(7), 647-664. American Association of Community Colleges (AACC). (2009). Community college fast facts. Retrieved July 10, 2009 from http://www2.aacc.nche.edu/research/index.htm Belfield, C. R., & Crosta, P. M. (2012). Predicting success in college: The importance of placement tests and High school transcripts. CCRC Working Paper No. 42. New York: Community College Research Center, Teachers College, Columbia University. Berkner, L., & Choy, S. (2008).Descriptive summary of 2003-04 beginning postsecondary students: Three years later. Washington, DC: U.S. Department of Education, Institute of Education Sciences. Bragg, D. D. (2001). Community college access, mission and outcomes: Considering intriguing intersections and challenges. Peabody Journal of Education. 76(1), 93-116. Community College Research Center. (2002a). Accountability and learning outcomes in community colleges. Retrieved July 7, 2008, from http://ccrc.tc.columbia.edu/Seminar.asp?uid=8 Community College Research Center. (2002b). Accountability and learning outcomes in community colleges. Retrieved July 7, 2008, 2008, from http://ccrc.tc.columbia.edu/Seminar.asp?uid=8 Habley, W., Bloom, J, & Robbins, S. (2012). Increasing persistence: Research-based strategies for college student success. San Francisco: Jossey-Bass

  25. References (cont.) Habley, W., & McClanahan, R. (2004).What works in student retention – All survey colleges. Iowa City, IA: ACT, Inc. Harris, S. (1991) Can’t you guys read? Cartoons on academia. New Brunswick, NJ: Rutgers University Press. Herzog, K., & Miller, M. P. (Eds.). (1985).Scholarship: More great cartoons from the Kappan. Bloomington, IN: Phi Beta Kappa International Inc. Le, H., Casillas, A., Robbins, S., & Langley, R. (2005). Motivational and skills, social, and self-management predictors of college outcomes: Constructing the Student Readiness Inventory. Educational and Psychological Measurement, 65, 482-508. Li, K., & Cordes, C. (2010).Individualizing the Delivery of Math Developmental Education. Talk presented at the 34th Annual National Association of Developmental Education Conference, Columbus, OH. Lotkowski, V., Robbins, S., & Noeth, R. (2004).The role of academic and non-academic factors in improving college retention. Retrieved from ACT, Inc. website: www.act.org/research/policy/index.html. Marke, R., & Robbins, S., (2013). Breaking the developmental education logjam: A holistic approach to acceleration. Princeton, NJ: Educational Testing Service. Marke, R., & Robbins, S., (2013). A holistic view of course placement fecisions – Avoiding the HS GPA trap. Princeton, NJ: Educational Testing Service. McCabe, R. H. (2000).No one to waste. Washington, DC: Community College Press. NCES (2004).Remedial education at degree-granting postsecondary institutions in fall 2000 (NCES 2004-010). NISOD (2012, March). Assessing college-readiness assessments. Webinar delivered at the annual NISOD Conference. Pascarella, E. T., & Terenzini, P. T. (2005).How college affects students: A third decade of research. San Francisco: Jossey-Bass.

  26. References (cont.) Peterson, C. H., Casillas, A., & Robbins, S. B. (2006). The Student Readiness Inventory and the Big Five: Examining social desirability and college academic performance. Personality and Individual Difference, 41, 663-673. Porchea, S., Allen, J., Robbins, S., & Phelps, R. (in press). Predictors of long-term enrollment and degree outcomes for community college students: Integrating academic, psychosocial, socio-demographic, and situational factors. Journal of Higher Education. President’s Commission on Higher Education. (1947). Organizing higher education. Washington, DC: Author. Robbins, S. B., Allen, J. Casillas, A., Akamigbo, A., Saltonstall, M., Campbell, R., Mahoney, E., & Gore, P.A. (2009). Associations of resource and service utilization, risk level, and college outcomes. Research in Higher Education, 50(1), 101-118. Robbins, S., Allen, J., Casillas, A., Peterson, C., & Le, H. (2006). Unraveling the differential effects of motivational and skills, social, and self-management measures from traditional predictors of college outcomes. Journal of Educational Psychology, 98, 598-616. Robbins, S. B., Lauver, K., Le, H., David, D., Langley, R., & Carlstrom, A. (2004). Do psychosocial and study skill factors predict college outcomes? A meta-analysis. Psychological Bulletin, 130, 261-288. Robbins, S., Oh, I., Le, H., & Button, C. (2009). Intervention effects on college performance and retention as mediated by motivational, emotional, and social control factors: Integrated meta-analytic path analyses. Journal of Applied Psychology, 94, 1163-1184. Scott-Clayton, J. (2012). Do High-Stakes Placement Exams Predict College Success? Predicting Success in College: The Importance of Placement Tests and High School Transcripts. CCRC Working Paper No. 41. New York: Community College Research Center, Teachers College, Columbia University. Swail, W. S. (2004, January 23). Legislation to improve graduation rates could have the opposite effect. The Chronicle of Higher Education, B16.

More Related