1 / 46

May 2010

Association of Institutional Research 50 th Annual Forum. FAIR Best Paper: Using Technology to Efficiently and Effectively Gather Information from Recent Alumni and Employers. May 2010. Alumni and Employer Surveys. Presenters

oscary
Download Presentation

May 2010

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Association of Institutional Research 50th Annual Forum FAIR Best Paper: Using Technology to Efficiently and Effectively Gather Information from Recent Alumni and Employers May 2010

  2. Alumni and Employer Surveys Presenters • Dr. James Coraggio, Director of Academic Effectiveness and Assessment Other Authors Not Present • Maggie Tymms, Assessment Coordinator for Academic Programs • Nicole Carr, Former Assessment Coordinator for Academic Programs Association of Institutional Research

  3. SPC Background • SPC, established in 1927, is the oldest 2-year college in Florida • First Community College in Florida to offer 4 year degrees (2002) • 9 Campuses throughout the county • 2008-09 FTE: 16,966.8 (LD), 1,676.7 (UD) • Opening Fall 2009 credit enrollment: 29,560 • Annual 2008-09 headcount: 61,414 Association of Institutional Research

  4. Institutional Effectiveness “Closing the Loop” “Institutional Effectiveness is the integrated, systematic, explicit, and documented processes of measuringperformanceagainst the SPC mission for purposes of continuousimprovement of academic programs, administrative services, and educational services offered by the college.” Association of Institutional Research

  5. Performance Improvement • From Compliance to Performance Improvement Association of Institutional Research

  6. Agenda • Student Assessment Points • Purpose of Follow-up Surveys • Alumni/Employer Survey Process • Dissemination of Survey Results and Reports Association of Institutional Research

  7. Student Assessment Points Students surveyed multiple times Association of Institutional Research

  8. Purpose of Follow-up Surveys Recent Alumni Survey was designed to: • Gain insight into alumni’s perceptions of the quality of general education and career preparation • Determine the current activities of former students such as employment information and educational status Association of Institutional Research

  9. Purpose of Follow-up Surveys Employer Survey was designed to: • Gain insight into an employer’s perceptions of hired SPC graduates in the areas of key general education competencies and foundational skills • Provide the employee’s major job responsibilities and wage information, as well as employer’s willingness to hire another SPC graduate • Identify employer’s willingness to support various college activities Association of Institutional Research

  10. Purpose of Follow-up Surveys State Requirements • Survey data are also collected for state required annual Placement & Follow-up Report (FETPIP: Florida Education Training Placement Information Program) and supplemental information to meet workforce criteria Association of Institutional Research

  11. Alumni/Employer Survey Process Previous model (Before 2005) • Labor intensive • Manually tabulated paper surveys • Telephone interviews conducted using overtime hours Association of Institutional Research

  12. Alumni/Employer Survey Process Issues with previous model (Before 2005) • Costly survey process • Declining college budget • Increasing demand for accountability • Outdated contact information • Students over-surveyed • Declining response rates • Conducted in isolation Association of Institutional Research

  13. Alumni/Employer Survey Process Goals of new survey process • Improve data collection process • Enhance quality of survey forms • Employ industry best practices • Include discipline specific information • Increase response rates Association of Institutional Research

  14. A. Improve Data Collection Process Data Collection Process • Eliminated telephone calls • Automated the process (Remark Office OMR and email merge) • Employed online survey submission (ZoomerangTM) • Obtained accurate email addresses • Used multiple points of contact Association of Institutional Research

  15. A. Improve Data Collection Process Survey Schedule • Recent Alumni Survey • Graduates surveyed six months after the end of their last term (Three times per year) • Employer Survey • Employer of graduates surveyed at the end of academic year (Once per year) Association of Institutional Research

  16. A. Improve Data Collection Process First email Second email Third email 2 weeks 2 weeks 2 weeks Recent Alumni Survey Multiple Points of Contact 2 weeks First mailing Second mailing Association of Institutional Research

  17. B. Enhance Quality Of Survey Forms Improved Survey Forms • Better ‘look and feel’ of survey format • Developed scannable paper forms • Used online branching models to reduce survey length • Designed items to gather only pertinent data Association of Institutional Research

  18. C. Employ Industry Best Practices Don Dillman’s Five Elements • Respondent-friendly questionnaire • Up to five contacts with the respondent • Inclusion of stamped envelope • Personalized correspondence • A token financial incentive that is sent with the survey request Association of Institutional Research

  19. Online Survey: Email and Form Association of Institutional Research

  20. Paper Survey: Letter and Form Association of Institutional Research

  21. C. Include Program Specific Information Program specific information • Met with Deans/Program Directors • Discussed requirements for program accreditation and timelines • Included their necessary program specific items Association of Institutional Research

  22. D. Increase Response Rates Create awareness and convey importance • Involved program faculty in the process • Requested faculty assistance to inform students • Included a flyer with graduation materials • Coordinated with programs that hand-carry surveys to local employers Association of Institutional Research

  23. Alumni Survey Response Rates • Summer 2007 Association of Institutional Research

  24. Alumni Survey Response Rates • Fall 2007 Association of Institutional Research

  25. Alumni Survey Response Rates • Spring 2008 Association of Institutional Research

  26. Alumni Survey Response Rates • Summer 2008 Association of Institutional Research

  27. Alumni Survey Response Rates • Fall 2008 Association of Institutional Research

  28. Alumni Survey Response Rates • Spring 2009 Association of Institutional Research

  29. Alumni Survey Response Rates • Summer 2009* *Summer 2009 in progress Association of Institutional Research

  30. Alumni Survey Response Rates *Summer 2009 in progress Association of Institutional Research

  31. Alumni Survey Response Rates Source of Responses 2008-09 Association of Institutional Research

  32. Alumni Survey Response Rates *Summer 2009 in progress Association of Institutional Research

  33. Alumni Survey Response Rates Overall Response Rate *Summer 2009 in progress Association of Institutional Research

  34. Alumni Survey Response Rates Note: For 2009-10, only partial Summer 2009 results have been received. Association of Institutional Research

  35. Response Rates - Employer Recent Alumni Surveys Sent Is a 44% Employer Response Rate Good? Employer Surveys Sent COE Responses Permission Not Required Other Responses Responses Permission Required Association of Institutional Research

  36. Disseminating Information • College wide access to reports • http://www.spcollege.edu/Central/Surveys/SurveyList.htm • Program Specific Reports • Published on the Educational Outcomes website • http://it.spcollege.edu/edoutcomes Association of Institutional Research

  37. Disseminating Information Association of Institutional Research

  38. Disseminating Information Association of Institutional Research

  39. Disseminating Information Association of Institutional Research

  40. Program Specific Reports Association of Institutional Research

  41. Recent Improvements • Survey invitations from program administrators (signatures) • Additional program specific reports (i.e., baccalaureate programs and PSAV) • Improved oversight (and QC) of printing functions Association of Institutional Research

  42. Lessons Learned • Do not expect that students will remember their ID#s • Administer the survey soon after the end of the session (six months) • Use multiple points of contact (two weeks) • Find the most recent contact information • Partner with programs to acquire up-to-date information Association of Institutional Research

  43. Lessons Learned • Make students aware of the importance of the survey process (survey announcements) • Provide contact updates to college data systems (PeopleSoft, Alumni Foundation) • Monitor results and make changes as necessary • Use automated processes for a quicker turn around of results and more accurate data Association of Institutional Research

  44. Future Direction • Our ultimate goal is to provide Program Administrators ‘timely’, ‘relevant’ ‘accurate’, and ‘interpretable’ information regarding their programs and their performance outcomes • Multi-measure ‘Dashboard’ style reports (Business Intelligence/Performance Measurement) Association of Institutional Research

  45. Questions/Next Steps Association of Institutional Research

  46. Association of Institutional Research 50th Annual Forum FAIR Best Paper: Using Technology to Efficiently and Effectively Gather Information from Recent Alumni and Employers May 2010

More Related