A Dozen Ideas about First-Year Assessment

1. A Dozen Ideas about First-Year Assessment FYE Annual Conference Atlanta, GA February 26, 2006 Randy L. Swing Co-Director & Senior Scholar, Policy Center on the First Year of College Fellow, National Resource Center for The First-Year Experience and Students in Transition Bradley E. Cox Coordinator of Research and Public Information, National Resource Center for The First-Year Experience and Students in Transition

2. Attendance & Early Intervention • Does it matter if students miss class? • Are all class cuts the same? • Do colleges “set up” students to miss class?

3. A Study of Attendance Patterns University of Mississippi Freshman Absence-Based Intervention (pilot) Experimental design (treatment & control groups) Students who missed 2 classes/8 weeks were reportedGraduate student made personal contact Cum GPA = C or better(end of term) Anderson, C. (2004).

4. Institutionalizing an Intervention (year 2) University of Mississippi Pilot taken campus-wideStudents who missed 2 classes/8 weeks were reportedResidence Hall Advisor made personal contact Cum GPA = C or better(end of term) Anderson, C. (2004).

5. Attendance Patterns - Semester weeks 1-4 weeks 5-8 weeks 9-12 weeks 13-16 2 4 8 6 Average number of class cuts per 2 week period Anderson, C. (2004).

6. Grade Point Average by Perfect or Not Perfect Attendance NS On average, students with perfect attendance had higher grade point averages – statistically significant in periods 1, 2, and 4. Anderson, C. (2004).

7. GPA & Absence = Oversleeping NS Difference in GPA is statistically significant in periods 1, 2, & 4. Not in period 3. Anderson, C. (2004).

8. GPA & Absence = Too Tired to Go NS NS Significant in periods 1, & 2.Not significant for period 3 & 4. Anderson, C. (2004).

9. GPA & Absence = Minor Illness NS NS NS Significant only for first period of term. Anderson, C. (2004).

10. We can’t get there! Porter and Umbach studied the time it takes for students to move between classrooms. Findings.... First-year students attend classes in more buildings and travel the greatest distances Some class changes were “impossible” Advisors did not consider distance in making schedules. Porter, S. R., & Umbach, P. D.

11. Retention • Put it into context • Prediction formulas • Peer institutions • Calculate the savings • Financial Implications • Find out what happens next • Student persistence at other institutions

12. Predictors of College Graduation • Biggest predictors of graduation • HS GPA (30.84% of variability) • SAT (additional 4.02%) • Gender (additional 0.73%) • Race (additional 0.42%) • Additional input & environment variables improve predictive value (R2) to 51.5%. Source: Astin & Oseguera, 2005

13. Retention Rate Comparison Detailed information from all member schools. Allows comparison of selected peers. Breakdown on race & gender. Includes 1yr & 2yr retention. ?Includes graduation? National Average Consortium for Student Retention Data Exchange & ACT

14. Calculate the Savings Noel-Levitz Retention Revenue Calculator

15. Find out where they go Where do your transfer-outs go? Where do your recruits go? Where have your transfers come from? www.studentclearinghouse.com

16. Before & AfterHow do students change in the first year? Pre/post surveys pretest at orientation posttest at end of the first year match on student id numbers

17. Pre/Post Survey Pairings

18. Expectations vs. Reality Go to Art Gallery or Exhibit on Campus Reality = 36% fewer students than expectation. 73.7% 38% 38.2% Expected (orientation) End of 1st Year End of 2nd Year Source: CSEQ National Data

19. Change in First-Year Students’Activities and Feelings Self-Report: “Above Average” or “Highest 10%” in Emotional Health • 12.3% 55.7% 43.4% At Entry End of First Year Anticipated behavior: “Some” or “good chance” I will seek personal counseling: 30.1% Actual first-year behavior: Sought personal counseling: 17.4% Example from CIRP and YFCY data

20. Religion and Spirituality Life goal: Integrating spirituality into my life is “very important” or “essential” “Frequently” or “Occasionally” attend a religious service At Entry81% End of First Year 49.6% End of First Year 55.7% At Entry40.5% -25.3% + 9.1 Example from CIRP and YFCY data

21. Student Voices • Ask them questions • KISS & QOTD/W • Prompt their thinking • Consider institutional “promises” • Let their pictures tell the story • Reflexive photography

22. KISS & QOTD/W • Keep it… Simple, Short, Straight, Strong • Asking questions & getting answers • Students talk to students • Administrators like short & straight to point • Question of the Day/Week • Easy enough for one person office • Directly addressing timely issues Cox, B.E. & Stirton, E. R (FYA-list essay)

23. Prompts Project • Quick, simple way to get lots of data • 10 minute response to open-ended prompts during required English 101 or FY seminar • Write for 10 minutes about your hopes, dreams fears, and expectations for this semester. • What is partying like here at VCU? How do you feel about the role of partying at VCU? • What types of serious crises have you and your friends experienced this semester, and how have you coped? Claffey & Laff (2004)

24. Promise Audit • Consider the various “promise-makers” • Staff, websites, publications, etc. • Friends, media, the “grapevine”? • Find out which promises are being broken. • Can use existing collection processes • What are the effects of broken promises? Claffey & Laff (2004)

25. Reflexive Photography • Pictures worth 1,000 words • Pictures plus words  Priceless • Students love to take pictures • Inexpensive and very powerful • Costs: Less than $100 plus staff time • Benefit: Images verify student statements Harrington, C.E. & Schibik, T.J. (2003)

26. Student Voices: Example • “The campus housing at USI is supposed to be “dry” (alcohol free), but every weekend after all the partying is over this is all that remains. Sure, we have a “dry” campus!” - Mary Harrington, C.E. & Schibik, T.J. (2003)

27. Student Voices: Example • “New construction project. This is a very good sign that the campus is growing and that more space is needed.” - Kirk Harrington, C.E. & Schibik, T.J. (2003)

28. Student Voices: Example • “My roommates took this picture of me in the morning one day. This picture means feeling real tired in the morning after staying up late studying. I was thinking how upset I was because they were making noise outside doing construction and it woke me up.” - Mercedes Harrington, C.E. & Schibik, T.J. (2003)

29. Student Voices: Example • “This is the mail truck. It is amazing how personal people can be when they write. I have become closer to my friends through letters. I also feel like my mom has become one of my best friends rather than just a parent. The mail has taught me a lot about myself and the important people in my life.” - Abby Harrington, C.E. & Schibik, T.J. (2003)

30. (Baker’s) Dozen Ideas

31. Additional (free, web) Resources • FYA-List • Monthly essays by assessment experts • http://nrc.fye.sc.edu/resources/FYAlist • Typology of Assessment Instruments • http://www.sc.edu/fye/resources/assessment/typology.html • Overview of Instruments, Pricing, Contacts • http://nrc.fye.sc.edu/resources/survey/search/index.php

32. Summer Institute on First-Year Assessment • Intentionally structured to create a cumulative learning experience for: • Beginners and experts • Institutions of all types • June 25-27, 2006 • Asheville, NC www.sc.edu/fye/events/upcoming.html

33. References: Attendance Anderson, C. (2004). Freshman absence-based intervention at the University of Mississippi. In R. Swing (Ed.), Proving and improving, Volume II: Tools and techniques for assessing the first college year (monograph no. 37) (pp. 19 -21). Columbia, SC: University of South Carolina, National Resource Center for the First-Year Experience and Students in Transition. Porter, S. R., & Umbach, P. D. We can’t get there in time: Assessing the time between classes and classroom disruptions. Planning in Higher Education, 32(2), 35-40.

34. References: Retention • Astin, & Oseguera (2005) Degree Attainment Rates at American Colleges and Universities, revised edition. • Consortium for Student Retention Data Exchange • http://www.ou.edu/csrde/index.html • ACT National Retention Statistics • http://www.act.org/path/policy/reports/retain.html • Noel-Levitz Retention Revenue Calculator • http://www.noellevitz.com/Papers+and+Research/Retention+Calculator/ • National Student Clearinghouse • Ward, J.P. (2003). Using EnrollmentSearch to Track First-Year Success. In Swing, R.L. (Ed.) Proving and Improving, Volume II. • www.studentclearinghouse.com

35. References: Before & After HERI – UCLA http://www.gseis.ucla.edu/heri/heri.html NSSE http://www.indiana.edu/~nsse/ CSEQ & CSXQ http://www.indiana.edu/~cseq/

36. References: Student Voices • Cox & Stirton (2005) Start Simple: The Value of Simple Assessment Techniques • http://www.sc.edu/fye/resources/assessment/essays/Cox&Stirton-5.18.05.html • Hodges & Yerian (2004) The First-Year Prompts Project: A Qualitative Research Study Revisited • Claffey & Laff (2004) The Promise Audit: Who’s Promising What to Students? Both of the above articles are in:Swing, R. (Ed.) (2004) Proving and Improving, Volume II: Tools and Techniques for Assessing the First College Year

37. References: Student Voices • Harrington, C.E. & Schibik, T.J. (2003) Reflexive Photography as an Alternative Method for the Study of the Freshman Year Experience. NASPA Journal, 41(1) • University of Southern Indiana • Complete PPT presentation available at https://www.usi.edu/depart/instires/reflexphoto.asp

38. Contact Info Randy L. Swing Co-Director & Senior Scholar, Policy Center on the First Year of College 400 North Broad Street Brevard, NC 28712 828. 966.5312 swing@fyfoundations.org Bradley E. Cox Coordinator, Research and Public Information National Resource Center on The First-Year Experience & Students in Transition, USC 1728 College Street Columbia, SC 29208 803-777-6225 bradcox@sc.edu