1 / 42

Examining Retention of Sophomores from a Consumer Satisfaction Perspective

Examining Retention of Sophomores from a Consumer Satisfaction Perspective. Authors: M . Rita Caso, Xiaohong Li Rebecca Bowyer & John J. Scariano. Office of Institutional Research & Assessment Sam Houston State University A Member of The Texas State University System.

amanda
Download Presentation

Examining Retention of Sophomores from a Consumer Satisfaction Perspective

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Examining Retention of Sophomores from a Consumer Satisfaction Perspective Authors: M. Rita Caso, Xiaohong Li Rebecca Bowyer & John J. Scariano Office of Institutional Research & Assessment Sam Houston State University A Member of The Texas State University System

  2. Absent Author Rebecca Bowyer

  3. Evolution of Research Objective Why Sophomores? • Sophomores often represent the university’s second largest attrition group. • Much research devoted to predicting freshman retention but less to sophomores

  4. Evolution of Research Objective (cont.) Much research has predicted retention from key variables student attrition theories • Authors: Tinto, Bean, Dey, Terenzini & Pascarella • Variables: Major Certainty, GPA, Institutional Fit, Social Integration, Finances/Support

  5. Evolution of Research Objective (cont.) Variables that are historically associated with persistence/ attrition, which were available : • Entry At-Risk Individual Attributes, Tinto 1975 • 1st Sophomore Semester GPA Academic Integration, Tinto 1975 • Campus Residence (FTF) Academic Integration, Tinto 1975 • Minority Individual Attributes, Tinto 1975 • Financial Aid Family Background and Individual Attributes , Tinto 1975

  6. Evolution of Research Objective (cont.) At our university, research on retention of Sophomores to Junior year using traditional attrition-theory variables had not produced useful prediction or explanation

  7. Evolution of Research Objective (cont.) Other Possible predictors of persistence or attrition: • Langbein & Snider (1999) and Ronco & Cahill (2004) • Classes taught by faculty who are not tenured • Favorable rating of classes in the university’s class evaluation survey system as related to tenure/tenure-track level of instruction • Suddarth (1975) • Classes that are required as part of a General Education/ Core Curriculum program ** In our current study we add Required Remedial Courses to General Education Courses

  8. Evolution of Research Objective (cont.) Could student satisfaction with class experiences significantly influence the decision to persist? • Student as Consumer  Consumer Satisfaction • Brand Loyalty Student-Consumer Satisfaction  Decision to Stay • Abandonment of Brand Student-Consumer Dissatisfaction  Decision to Leave

  9. Variables of Interest Hypothesized predictors of Sophomore-to-Junior persistence representing the customer satisfaction perspective : • High sophomore year exposure to classes positively rated overall* • High sophomore year exposure to classes positively rated for instruction* • High sophomore year exposure to classes taught by tenured or tenure-track faculty in sophomore year • Low sophomore year exposure to required General Education (Core Curriculum) or Remedial courses (* hypothesized to be a direct indicator of customer satisfaction)

  10. Variables of Interest from IDEAOperational Definitions • Percentage of a student’s sophomore year class schedule spent in classes positively rated* overall - IDEA class evaluation Question 42 • “Overall, I rate this course as excellent.” • 1 - 5 Scale (1 = Definitely False, 5 = Definitely True) • Percentage of a student’s sophomore year class schedule spent in classes positively rated* for instruction - IDEA class evaluation Question 41 • “Overall, I rate this instructor as an excellent teacher.” • 1 - 5 Scale (1 = Definitely False, 5 = Definitely True) *Positively Rated classes have mean class ratings >=4

  11. IDEA Class / Instructor Evaluation System • IDEA Class/Instructor Evaluation– the standardized, national, classroom evaluation system used at SHSU by students near the end of each semester to rate their perceptions about classes and instructors.

  12. Variables of Interest: Operational Definitions in the Study Data • Percentage of student’s sophomore year class schedule spent in classes that were taught by Tenured/Tenure-Track faculty • Percentage of student’s sophomore year class schedule spent in classes that were required General Education (Core Curriculum) or Remedial courses

  13. Persistence-Related Covariates: Operational Definitions • Entry At-Risk - Whether a student was originally admitted with one or more College Readiness Deficiencies • GPA in 1st Sophomore semester - End of semester GPA in first semester of Sophomore year • Campus Residence (Off/On) - Residence off vs. on campus in 1st semester of entry year. • Financial Aid - Whether or not the student received financial aid during Sophomore year • Minority – Hispanic, African-American, Native-American Vs. White, Asian-American, International • Entry Type - Entry as First Time Freshmen (FTF) vs. First Time Transfer (FTT)

  14. Sophomore PopulationOperational Definition • The first Fall semester by which a student has completed between 30 and 59 semester credit hours defines her/his Sophomore Cohort. E.g., • If a student is first classified as a sophomore in Spring 2004 and is still classified as a sophomore in Fall 2004, that student is a member of the Fall 2004 Sophomore Cohort. • If a student is first classified as a sophomore in Fall 2004, but is still classified as a sophomore in Fall 2005, that student is still a member of the Fall 2004 Sophomore Cohort

  15. Dependent Variable Operational Definition • One-Year retention is defined as continued enrollment in the fall term following the first fall term in which the student meets the criteria for sophomore classification • The student is considered retained even if he/she has not earned enough credits to progress to junior classification.

  16. SPSS Procedure Backward, Stepwise,Binary Logistic Regression • Under Tool Bar Analyze, select Regression - Binary Logistic • Select variable ‘Retention1Yr’ as the dependent variable. • Select the variables of interest as Covariates • Method - Select Backward Stepwise Likelihood Ratio • Click Option in the logistic Regression Dialog box • Select Hosmer-Lemeshow goodness-of-fit to test the model fit • Enter .2 as the classification cutoff point • Keep the default Probability for stepwise setting- Entry Probability .05 and Removal Probability .10. Example of The syntax: LOGISTIC REGRESSION VARIABLES Retention1Yr /METHOD = BSTEP(LR) PercentQ42Coded PercentQ41coded PercentRemiCorePercentTenureCoursesEnter_RiskEnter_TYPEUGACoded FAID Minority /CONTRAST (Enter_Risk)=Indicator /CONTRAST (Enter_TYP)=Indicator /CONTRAST (FAID)=Indicator /CONTRAST (Minority)=Indicator /CONTRAST (PecentQ42Coded)=Indicator /CONTRAST (PercentQ41coded)=Indicator /CONTRAST (EUGACoded)=Indicator /PRINT = GOODFIT /CRITERIA = PIN(.05) POUT(.10) ITERATE(20) CUT(.2) .

  17. Methodological Definitions • Binary (or binomial) logistic regression is a non-linear form of regression which is used when the dependent is a dichotomy (binary) and the independents are of any type (continuous, categorical, binary). • Backwards stepwise logistic regression methods allow one to enter all variables at once. Backwards stepwise logistic regression methods determine automatically which variables to drop from the model. • Hosmer-Lemeshow goodness-of-fit test is a recommended test for the overall fit of a binary logistic regression model • A finding of non-significance allows the researcher to conclude that the model adequately fits the data. Well-fitting models show non-significance, indicating that model prediction is not significantly different from observed values.

  18. Methodological Definitions • Coefficient (B ) - The coefficient (B)’s estimations tell the amount of increase (or decrease, if sign is negative) in the predicted log odds of retention by an increase or decrease in the variable’s values, holding all others constant. The proportional change in the odds of being retained, for a one unit change in the independent variable • Odds Ratio = Exp(B) - General speaking, the odds ratio is a measure of the strength of association between a predictor and the response of interest. Can be used to compare whether the probability of a certain event is the same for two groups. The value of odds ratio is from 0 to infinity. If the odds ratio is one, there is no association, which implies that the event is equally likely in both groups, and an odds ratio greater than one implies that the event is more likely in the first group. An odds ratio less than one implies that the event is less likely in the first group.

  19. Populations • “Combined” - Sophomore Cohort population (n=10983) includes any enrolled student who has completed between 30 and 59 semester credit hours at the start of Fall 2005, Fall 2006, Fall 2007, or Fall 2008 regardless of whether he/she originally entered the university as a First Time Freshman (n=5568) or a First Time Transfer (n=5415). • “FTF” - F 05-08 Sophomore Cohort members who entered as First Time Freshman (FTF) (n=5568) • “FTT” - F 05-08 Sophomore Cohort members who entered as First Time Transfer Cohort Population (FTT) (n=5415)

  20. Continuous Variables Recoded as Categorical

  21. Combined Population Analysis Model FitBackward Stepwise Hosmer and Lemeshow Test

  22. Combined Population Analysis Variables in the Equation at Final Step

  23. Combined PopulationInterpretation of Coefficients, Odds Ratios • End of 1st Sophomore Semester GPA,EUGAcoded, is the most important factor is the persistence-related covariate. • EUGAcoded is statistically significant overall, and for each of its dummy-coded values. • Sophomores with End-of-1st-Semester GPAs between 3.00-3.49 were the group most likely to be retained into next year. • Interesting that students in the highest GPA group (3.5-4.0)-EUGACoded(1)- with an odds ratio of 8.24, had slightly lower odds of being retained than students in the lower GPA group (2.5-3.0-EUGACoded(3)- with odds ratio of 8.33

  24. Combined PopulationInterpretation of Coefficients, Odds Ratios • The effect of Percent of required General Education or Remedial Classes in student’s schedule, %RemiCore, is smaller than that of 1st semester sophomore GPA • The relationship to retention is negative • For each one point increase in the % of required General Education or Remedial courses in a student’s sophomore schedule, the log odds of the student being retained decreased by 1.51.

  25. Combined PopulationInterpretation of Coefficients, Odds Ratios • Both of the variables representing our customer satisfaction hypothesis (%Q41Coded and %Q42Coded) are statistically significant overall • However, some of their dummy-coded value levels are not statistically significant. • There is a NEGATIVE relationship between retention and the groups with the highest percent of positively rated classes, e.g.,: • %Q41Coded(1) = >66% has Coefficient=-.12 and Odds Ratio =0.89 • %Q42Coded(1) = >50% has Coefficient=-.28 and Odds Ratio = 0.76 • Students with 50% or more classes with favorable overall ratings have odds of NOT being retained that are 1.24 times higher than for those with the lowest % of favorably rated classes (%Q42Coded(5)) • All other dummy-coded levels have positive relationships with retention.

  26. Are FTF and FTT Populations Different?

  27. FTF Population Model Fit Hosmer and Lemeshow Test

  28. FTF Population Variables in the Equation at Final Step

  29. FTF Population Interpretation of Coefficients, Odds Ratios FIVE variables influence retention of sophomore students who entered the university as FTFs: 1. EUGAcoded 2. %RemiCore 3. Q42coded 4. ONOFF 5. Enter_Risk • First Semester Sophomore GPA isthe most important variable in predicting retention. The overall EUGAcoded variable is statistically significant as are each of its dummy-coded values. • Students with end of semester sophomore GPAs between 3.00-3.49 have 30.77 times higher odds of being retained than students in the EUGACoded(6) group who have GPAs <1.5 • The Percent of GenEd/Remedial Courses effect is smaller, and NEGATIVE in relation to retention. • For each one point increase in a student’s %RemiCore, the odds of NOT being retained increase by 1.91

  30. FTF Population Interpretation of Coefficients, Odds Ratios 1. Percent of Favorably Rated Courses Overall, is statistically significant, although two of its dummy-coded values are not. • There is a NEGATIVE relationship between retention and membership in the %Q42Coded(1) group, despite the fact that this group has the highest % of classes that were rated favorably overall(>= 50%) • In contrast, there is a POSITIVE relationship between retention and membership in the %Q42Coded(3) groupwhose members have much lower exposure to favorably rated classes (30% -39%) 2. The ONOFFodds ratio of 0.724, indicates that sophomores who entered the university as FTFs and did NOT live on campus in their entering semester were 1.27 times more likely to NOT be retained than those who lived on campus 3. The Enter_Riskeffect is the smallest, of all the variables. a. There is a negative relationship between having entered the university at risk and being retained. b. With an odds ratio of 0.773, students who entered at risk have 0.22 greater odds of NOT being retained than students who do not enter at risk.

  31. FTT Population Variables in the Equation at Final Step

  32. FTT Population Interpretation of Coefficients, Odds Ratios Same 4 variables influence retention of sophomore students who entered the university as FTTs, as for Combined Population: 1. EUGAcoded; 2. %RemiCore; 3. %Q41coded ; and 4. %Q42coded • The most important factor is EUGAcoded, which is statistically significant overall and for each dummy-coded value. • Students in the EUGAcoded(2) group have GPAs between 3.00-3.5 and the highest coefficient and odds ratio of all EUGAcodedvalue groups. With an odds ratio = 7.91, the odds of students in this group being retained is 7.91 times greater than for students in the EUGACoded(6) referencegroup who have GPAs <1.5. • The %RemiCore effect is smaller, and NEGATIVE in relation to retention. For each one point increase in a student’s %RemiCore, the odds of NOT being retained increase by 1.21

  33. FTT Population Interpretation of Coefficients, Odds Ratios Both %Q41Coded and %Q42Coded are statistically significant overall, however, some of their dummy-coded value levels are not statistically significant. • %Q41Coded (3) , the group with 38-49% exposure to courses positively rated for instruction is 1.38 times more likely to be retained than the group with <38% exposure However, %Q41Coded (2), the group with >50% exposure to courses positively rated for instruction has slightly lower odds of retention (1.27) • Curiously, for %Q42Coded the only value level with a significant relationship to retention is %Q42Coded (1), whichis a negative one. Students who have a >50% exposure to courses positively rated overall have 1.29 times greater odds of NOT being retained than students in the reference group, who had only <20% exposure to courses positively rated overall

  34. Comparing Combined, FTF and FTT Population Variables in the Equation at Final Step

  35. Comparing Combined, FTF and FTT Populations Interpretation • For all 3 populations ,GPA at the end of 1st sophomore semester is extremely important for retention into next year, and this variable is relatively more important in relation to other variables in the model for sophomores who entered as FTF students • The % of General Education or Remedial Courses is the next most important variable common to all three populations. This variable is a consistent NEGATIVE predictor to retention. • Less clear in its predictive importance for all three populations is the % of classes favorably rated overall. However, students in the group with >50% exposure to favorably rated classes are consistently less likely to be retained than students with much lower (<20%) exposure to favorably rated classes. .

  36. Discussion • EUGACoded - Common Sense and 30 years of literature on Retention by Tinto, Dey, Bean, Astin, Pascarella, Terenzini support & explains the association of GPA with retention. • %RemiCore - The experimental research findings of Betty Suddarth, at Purdue University (1975), indicated a direct relationship between retention and removing General Education course requirements, as well as a direct relationship with students’ expressed satisfaction with the university experience and removing General Education course requirements • Enter_Risk & ONOFF - The specific indicators of Risk and Off/On Campus Residence had been associated with retention of freshmen and retention of Sophomores in previous studies conducted at our university ( i.e., Li,X & Caso,R, Spring 2009 & Fall 2009 and Yi,J, Caso,R, Li,X, Kokatla,L, Li,Q, Spring 2009)

  37. Comparing Combined, FTF and FTT Populations Interpretation • Less clear and also less consistent in its predictive importance is the % of classes positively rated for instructor, which is significant for the retention of Combined and FTT populations: • There seems to be a significant relationship between retention and being a member of either of the two groups who have 38-49% and 50-65% exposure to classes that were favorably rated for instructor. This is not so for members of groups with higher or lower percentages of exposure. • For Sophomores who entered as FTFs their initial Entry-at-Risk and On-Campus Residence in first semester at the university seem to continue to play a role in persistence from sophomore into junior year. Neither of these variables impacted retention of students who entered as FTTs and neither contributed to retention in the combined population.

  38. Comparing Combined, FTF and FTT Populations Interpretation • Variables rejected in models for all populations: • % Exposure to Tenured/Tenure-track Faculty • Financial aid • Minority vs. Non-Minority status • Commitment to major had been ruled out in preliminary examinations and was never entered into any of the models.

  39. Discussion • %Q42Coded & %Q41Coded – Represent the Customer Satisfaction hypothesis in this study, and while both of these variables are statistically significant contributors to the prediction of sophomore retention, the observed nature of their relationship to retention is not logically explained. • We conclude that these particular measures may not have been the most appropriate indicators of the type of customer satisfaction / dissatisfaction that drives brand loyalty or brand abandonment for students at this university • A survey study of the sophomores conducted at our university in winter-spring of 2010 ( Rogers,K, May 4,2010) suggests that non-retention at this university is not driven by educational dissatisfaction.

  40. Questions & Comments? THANK YOU!

  41. References • Langbein, Laura I. &Kevin Snider. “The Impact of Teaching on Retention: Some Quantitative Evidence”. Social Science Quarterly 80.3, Sept 1999. • Li,Xiaohong ; Li,Qiyu &Caso, Rita. “Looking Backwards for Profiles of Success”. Paper presented at the 2009 TAIR Conference, Lubbock, TX, March 2009. • Li,Xiaohong ; Li, Jia; Li, Qiyu; Kokatla, Lakshmi & Caso, Rita. “Exploration, Examination, Excavation and Explanation”. Paper presented at the 2009 TAIR Conference, Lubbock, TX, March 2009. • Rogers, Keri. “Success Advisory Board 2010 Sophomore Survey Study”. Study results presented at the May 4, 2010 meeting of Sam Houston State University Success Advisory Board, Huntsville, TX. • Ronco, Sharron,& Cahill, John.   "Does it Matter Who’s in the Classroom? Effect of Instructor Type on Student Retention, Achievement & Satisfaction".  Paper presented at the 44th Annual Forum of the Association for Institutional Research, Boston, Massachusetts, May 2004. ) • Suddarth, Betty, ”An Investigation of General Education Requirements in College Curricula”. Research in Higher Education, Vol. 3, No. 3 (1975), pp. 197-204. Springer . Stable URL: http://www.jstor.org/stable/40194970 • Tinto, Vincent. “Dropout from Higher Education: A Theoretical Synthesis of Recent Research.” Review of Educational Research, Vol. 45, No. 1 (Winter, 1975), pp. 89-125 . American Educational Research Association . Stable URL: http://www.jstor.org/stable/1170024

  42. Contact Information • M. Rita Caso • Email: mrc007@shsu.edu • Phone: 936-294-3618 • Xiaohong Li • Email: xxl001@shsu.edu • Phone: 936-294-3989 • Rebecca Bowyer • Email: rlb013@shsu.edu • Phone: 936-294-4433

More Related