Facilitating Probationary Students' Success: Design, Evaluation, and Impact of Probationary Student Re-Orientation

1. Facilitating Probationary Students' Success: Design, Evaluation, and Impact of Probationary Student Re-Orientation Esau Tovar  Merril A. Simon Presentation for the 2004 NACADA Pacific Region Conference Pasadena, CA  April 21-23, 2004

2. Contact Information Esau Tovar, M.S. Faculty Leader/Counselor, Assessment Center Project Director, Student Enhancement & Educational Research Project Santa Monica College 1900 Pico Blvd., Santa Monica, CA 90405 (310) 434-4012 tovar_esau@smc.edu Merril A. Simon, Ph.D. Assistant Professor, Educational Psychology and Counseling California State University Northridge 18111 Nordhoff St. Northridge, CA 91330-8265 merril.simon@csun.edu Note: A full report addressing the design, evaluation, and outcomes of this project can be found at: http://homepage.smc.edu/tovar_esau/esauprof/SEER%20Counseling%20Based%20Interventions.doc

3. Presentation Abstract Budgetary constraints continually force colleges to design more effective interventions to serve the most students possible. With up to 35% of first-time students at this urban, diverse, public California community college ending up on probation after their first semester, a probationary student “re-orientation” was developed to address their specific needs. Participants attended a two-hour orientation (based on collaborative/problem-based learning) by engaging them in small group counseling discussions on factors leading to their poor academic performance; readiness, motivation, and commitment for college; understanding institutional expectations; balancing personal, academic, and social commitments; and connecting with students and faculty. Compared to controls, participants were retained, persisted, made greater academic gains, and overcame probation to a significantly higher degree. Orientation content and methods will be shared.

4. Presentation Objectives • Participants will leave session with a better understanding on how to work with at-risk college students. • Participants will leave session with concrete ideas on how to implement a successful probationary student “re-orientation.” • Participants will be provided with an overview on how to design a comprehensive assessment/evaluation plan to measure the effectiveness of the orientation program and its impact on student success, course completion, and retention. • Participants will be given the opportunity to brainstorm and consider possible counseling/advising strategies during a question-and-answer period.

5. Intervention Strategies Rationale

6. Pre-Program Decisions • Follow-up of three year study on the effects of extended orientation; intentional instructor and counselor involvement to create greater social and academic integration; increased retention, persistence, and GPA resulted. • Successful, so undertook this study of first-time, first-semester probationary students to assess effect of select intervention strategies on same outcomes.

7. Rationale for Program Development • 35% of all first-time students are on probation at the end of their first term. • 54 - 73% success rate for first semester students based on ethnic breakdown) • Persistence rate of 54% from semester one to semester two. • Commitment by institution to successfully serve students.

8. Factors Underlying High Probationary Rates • Initial orientation to college does not meet students’ needs • Lack of social and/or academic integration • Delayed or flawed educational planning and undefined career goals. • Need for intrusive advisement and mandatory assessment • Delayed completion of math and English courses but needed for success in other courses

9. Program Goals • The aim of the Student Enhancement & Educational Research Project (SEER)—mainly through the re-orientation program was to increase the percentage of probationary (i.e., academic probation, progress probation, disqualified) students who: • Completed courses successfully (i.e., retention); • Persisted in higher proportions to the subsequent semester; • Attained higher grades that would allow them to overcome their probationary status.

10. Strategies Designed • The intervention strategies designed and implemented included the following: • Development of an innovative probationary student re-orientation; • Use of intrusive and developmental advising; • Assessment of students’ readiness and motivation to change existing patterns of unsuccessful behaviors; and • Increase rate of assessment of students’ writing, reading, and mathematical skills.

11. Funding Support • Provided by: • Santa Monica College • Fund for Instructional Improvement of the State of California

12. Counseling Faculty Training • Developmental, intrusive advisement strategies. • Using ‘flashpoints’ (Hirsch, 2001) • Student involvement and I-E-O Model (Astin, 1993) • Retention strategies (Basham & Lunenburg, 1998) • Social & academic integration (Tinto, 1993) • Training on the administration and interpretation of the College Student Inventory (Stratil, 1988)

13. Characteristics of Programs Addressing Academic Performance • Hirsch (2001) indicates that such programs must: • Use a holistic approach to diagnosing causes for academic difficulties; • Be cost-effective; • Account for students’ motivation and readiness for change; • Individualized interventions given the type of difficulty; • Interventions based on study & learning skills development (cognitive and affective).

14. Probationary Student “Re-Orientation” • Researched existing programs for probationary student – in community colleges and four-year institutions. • Intended to address the specific needs of continuing students. • Discussed topics including commitment to college, motivation to succeed, understanding of institutional expectations, balancing, school, work and personal commitments; and connecting with faculty members and peers (social and academic integration).

15. Re-Orientations Conducted • Summer 2002 (small pilot—focus group based; 150 students) • Winter 2003 (350 first semester students) • Summer 2003 (850 students) • Winter 2004 (fully institutionalized; 700 students)

16. Participant Invitations • Invitations for re-orientation sent to all students who had enrolled for the first time and were placed on academic or progress probation after their first semester; follow-up phone call reminders to those who didn’t respond. • Re-Orientation based on small group format focusing on problem-based-learning and collaborative learning strategies. • 10-15 students per group/20-30 group sessions • Led by a professional counselor

17. PBL: Defined Problem-based learning (PBL) is an instructional method that challenges students to "learn to learn," working cooperatively in groups to seek solutions to real world problems. These problems are used to engage students' curiosity and initiate learning the subject matter. PBL prepares students to think critically and analytically, and to find and use appropriate learning resources. -- Barbara Duch (Editor of The Power of Problem-Based Learning)

18. Re-Orientation Beginning • Distribution of demographic questionnaire focusing on: • Time/Distance to college; Parents’/Guardians’ educational level of attainment • Hours employed per week • Hours studied per week • High school GPA (approximate) • Student’s reason for probation and strategies for success • Introductions: • Brief introduction of counselor and student worker • Overview of Orientation/Purpose of Program • Counselor briefly explained the purpose and history • Counselor presented an overview of orientation • Discussed ‘What is Probation’ handout

19. Icebreaker Exercises • Gave students five minutes to write down a response to one of the following two questions: • What was your biggest adjustment in starting college? • What is something important thing you have learned about yourself since starting college? • Volunteers from the group shared their responses.

20. Re-Orientation Discussion Questions(1 of 2) • After icebreaker activities, the following questions/topics were discussed: • How many people have jobs? For those of you who do work, how does working affect your academic and social experience at SMC? • If you found yourself having trouble keeping up in class this coming semester, what would be the most effective method of improving your situation? Why would you choose this method? What has worked in the past? What has not worked?      Distributed Math and English Tutoring Schedule Reminder of English/Math placement tests completion • Each semester, about 3,500 new students are placed on probation. What factors will those students need to address to become successful in college? Distribute ‘Calculating Your GPA’ handout

21. Re-Orientation Discussion Questions(2 of 2) • What does time management mean to you? How do you manage your time so that you can study enough hours each week?(If relevant, distribute: Time Management handouts) • Have your instructors or counselors talked with you about effective study skills? What study skills would you recommend to others in your group?(If relevant, distribute Study Skills handouts) • Research has shown that college students face many personal obstacles while working to achieve their academic  goals. How do you deal with personal obstacles so that they  do not impede your progress towards your goals?

22. College Student Inventory • All students were asked to complete the assessment—and all but two did. • Version B—100 items. • Assesses motivation, coping skills, and receptivity to support services. Provides a general overview of student’s likelihood to dropout and experience academic difficulty. • Takes approximately 30 minutes to complete. • Student and counselor will discuss results in a subsequent appointment.

23. CSI cont. • Norms for the CSI are available for four-year, and two-year schools. • Results include profiles for advisors and profiles for students with suggested interventions. • Also available is a full-length version (A—164 items). May be completed online or sent in for processing.

24. Completion of Re-Orientation Process • Orientation Evaluation • Students completed evaluation and answered final demographic question when they finished their assessment: • Based on what you learned here today, what do you plan to do differently next term? • Encouraged students to make an appointment to speak with a counselor during the following semester to solidify goals.

25. Financial Aid Information • Answered student questions regarding financial aid. • Distributed and discussed the green paper titled, ‘Warning: Being on Academic Probation Does Affect Financial Aid’ • Distributed ‘’Financial Aid Myths’’ pamphlet

26. Campus Resources • Distributed and discussed the ‘Contact Information’ Handout • Distributed and discussed the ‘Campus Resources’ Handout • Including program-provided math & English tutoring. • Showed dates and deadlines in SMC catalog (encouraged the students to buy one) and the Schedule of Classes. • Showed Student Planning Guide and where to get it on-line. • Showed Student Planner—buy in the bookstore.

27. Evaluation Results • Reasons given for being on probation were assessed in terms of attribution theory with four areas (and a general-non-categorized) identified: • Course Specific • Internal-Stable-Specific Attributions • Internal-Unstable-Specific Attributions • External-Stable-Specific Attributions • External-Unstable-Specific Attributions

28. Course Specific Attributions • Student states “obvious” reasons for lack of success (e.g., probationary status) in the following ways: • Withdrew from too many courses • Low Grade Point Average

29. Internal-Stable-Specific Attributions • Student identifies an aspect of him/herself which contributed to poor performance, and seems to impact only a given course or only college. • Lack of Academic Preparation for Course • Adjustment to college • Too much fun • Lateness or attendance problems • Enrolling in too many classes

30. Internal-Unstable-Specific Attributions • Student identifies an aspect of him/herself which contributed to poor performance in course, but not other aspects of school. • Lack of Enthusiasm or Interest in the Course

31. External-Stable-Specific Attributions • Student states specific conflicts that contributed to poor performance and attributes lack of success to these entities. • Persistent Work Conflicts • “Another person made me do it” • Unreliable transportation/Distance

32. External-Unstable-Specific Attributions • Student states multiple conflicts that contributed to poor performance, but did not affect other aspects of life. • Poor performance (changed throughout) in class • Lack of studying • Poor time management • Family, personal, financial problems

33. Program Assessment Outcomes Based on Winter 2003 Re-Orientation Attendees

34. Study & Control Groups • Study Group: • Students attending college for the first time in fall 2002 and subsequently placed on academic and/or progress probation at the conclusion of the semester AND participated in the probationary student re-orientation conducted in winter 2003. • Control Group 1: • Students attending college for the first time in fall 2002 and subsequently placed on academic and/or progress probation at the conclusion of the semester AND DID NOT participate in the probationary student re-orientation conducted in winter 2003. • Control Group 2: • The cohort of first-time college students placed on academic and/or progress probation after completing the fall 1999 semester.

35. Orientation Participation Demographics • 29% of invited probationary students attended orientation. • 80% were strictly on academic probation (< 2.0 GPA) • 315 students completed the College Adjustment Inventory

36. Gender & Ethnicity • No Gender X Ethnicity differences found for participation. • Attendees were predominantly Latino (39%). However, they constitute 26% of SMC students. • Age: 93% were 22 or younger (M = 19.5, SD = 3.5).

37. Demographics (cont.) • Male and females differed on self-reported HS GPA (Female: 2.8 vs. 2.7, p < .05). • Ethnic differences in Distance traveled, travel time, and self-reported HS GPA (p < .05). • Mode of Transportation: 27% use public transportation to get to the college; 55% drive; 13% are driven by another person; and 5% walk or ride a bike.

38. Course-Taking Characteristics

39. Outcomes Attained Effect of Re-Orientation on Select Student Outcomes

40. Re-Orientation Participation by Probationary Outcomes • Although study and control group 1 students completed their first semester with a similar standing, those participating in the re-orientation were more likely to decrease their probationary rates: • Academic Probation status • Participants: 40 percentage point decrease in just one semester; • Non-Participants: 25% percentage points decrease

41. Re-Orientation Participation by Probationary Outcomes

42. Influence of Re-Orientation Participation AND Counseling on Probationary Outcomes • Probationary students attending the re-orientation & meeting with a counselor: • Accounted for fewer cases of academic probation in subsequent semester • 34% for participants vs. 42% for non-participants; • Accounted for a greater proportion of “good standing” cases: • 24% for participants vs. 18% for non-participants

43. Influence of Re-Orientation on Course Completion • Participants completed more coursework successfully—“C” or better—after the re-orientation intervention: • 57% for P vs. 47% for NP in spring 2003; and • 73% for P vs. 65% for NP in summer 2003 • Increased significantly higher when also accounting for counseling intervention: • 60-63% for P vs. 53 % for NP in spring 2003; and • 75-87% for P vs. 64% for NP in summer 2003

44. Influence of Re-Orientation on Semester GPA • GPA increased from a mean first-semester GPA of 1.09 to: • 1.53 mean GPA in one semester • Higher when also accounting for counseling intervention: • 1.65-1.66 for those meeting with counselor in second semester vs. 1.45 for those not meeting with one; • 2.29-2.21 for those meeting with counselor in summer session vs. 1.61 for those not meeting with one; • Significant because of mathematical difficulty in raising GPA.

45. Influence of Re-Orientation on Persistence to Subsequent Semesters • Participants persisted to a significantly higher degree than non participants: • Fall to Spring: • 72% for P vs. 23% for NP--a near 50 percentage point difference; • Fall to Fall: • 43% for P vs. 14% for NP--a 30 percentage point difference. • Accounting for counseling Intervention, participating students meeting with counselor: • Fall to Spring: • 71% for those not meeting and 74% for those meeting 1+ times • Fall to Fall: • 39% for those not meeting; 44% for those meeting 1; 51% for 2+ times

46. Re-Orientation Participation by Success Outcomes

47. Adjustment Findings & Student Receptivity Outcomes for Gender & Ethnicity

48. CSI Composite Scales • Ethnic differences in dropout proneness, predicted academic difficulty, and educational receptivity. Educational stress, not significant. • Latino students are most likely to dropout and experience academic difficulties; however, are also more willing to accept assistance. • Asian students are under somewhat more educational stress than other students.

49. CSI Academic Motivation Scales • Consistent with theory, students with poor academic achievement, these students generally express a low tolerance toward instructors. Latino’s had a slightly more favorable impressions than other students. African American students had the least impression. • White students expressed a higher degree of verbal confidence compared to African American and Latino students. This is particularly crucial as it impacts students’ attitudes and subsequent success in courses where extensive reading, writing, and public speaking is expected. • Regardless of ethnicity, females expressed a higher degree of intellectual interests (i.e., enjoys the learning process). Men, on the other hand, expressed higher degrees on confidence in their perceived capacity to do well in math and science, and where communication skills are highly emphasized.

50. CSI Coping Scales • No statistically significant differences were found for gender or ethnicity in the General Coping scales. This indicates that all students have developed similar coping mechanisms, albeit not particularly favorable. • As such, counselors must work all the harder with these students through personal work as we strive to find the “flashpoint” (Hirsch, 2001) to effect change and ensure student success.