Enhancing Undergraduate Career Services and Skills with Alumni Survey Results

1. Enhancing Undergraduate Career Services and Skills with Alumni Survey Results American College Personnel Association March 31, 2014 Angie L. Miller, Ph.D. Amber D. Lambert, Ph.D. Center for Postsecondary Research, Indiana University

2. Presentation Outline • Literature Review: the need for alumni assessment • Specific connections to career services • Examples from the Strategic National Arts Alumni Project (SNAAP) • Career Outcomes: generating materials for current & prospective students • Program Assessment: the importance of opportunities for internships • Curricular Assessment: aligning with career skills

3. Literature Review • There is an increasing trend for various assessments at colleges and universities (Kuh & Ewell, 2010) • Survey use is popular in higher education (Kuh & Ikenberry, 2009) • Students, faculty, administrators, and alumni can provide information for internal purposes (curricular/program revisions, strategic planning) and external purposes (accreditation, benchmarking)

4. Literature Review • Alumni surveys provide knowledge of student satisfaction as well as acquired skills, strengths and weaknesses of the institution, and current career attainment • Institutions often encounter difficulty in transforming survey results into actionable plans for improvement

5. Literature Review • Changes to curriculum can be difficult to make, but institutions must be reactive to market needs • If institutions do not train students well, the employability of their graduates will decrease (Evers, Rush, & Berdrow, 1998) • In addition to content knowledge from a student's major, institutions also provide other skills such as communication, analytical thinking, and creative thinking (Tait & Godfrey, 1999) • Students nearing completion of their degrees are concerned with potential employment, especially in the current economic climate

6. Literature Review • Most, if not all, institutions have some type of career services office: • Aimed at providing students with much-needed information on potential careers • Focused on pre-graduation work experiences such as internships and other high impact practices (Kuh, 2008) • Certain fields, in particular, have recently been under scrutiny for the career outcomes of their graduates (Carnevale, Cheah, & Strohl, 2012) • Career services are under increasing pressure to supply students with assistance in multiple areas of career development

7. So how can alumni surveys help career servicesprograms expand their functionality and effectiveness?

8. Examples from the Strategic National Arts Alumni Project (SNAAP)

9. SNAAP • As an example, we will present some survey questions, results, and institutional uses from the Strategic National Arts Alumni Project (SNAAP) • What is SNAAP? • Online survey designed to assess and improve various aspects of arts school education • Investigates the educational experiences and career paths of arts graduates nationally • Findings are provided to educators, policymakers, and philanthropic organizations to improve arts training, inform cultural policy, and support artists

10. Who does SNAAP survey? • Participants drawn from: • Arts high schools • Independent arts colleges • Arts schools, departments, or programs in comprehensive colleges/universities • Cohort Year Sampling • 2008 and 2009 Field Tests: 5, 10, 15, & 20 years out • 2010 Field Test: 1-5, 10, 15, & 20 years out • 2011 and forward: all years to generate the most comprehensive data possible

11. Increasing Numbers… • 2011 Administration • More than 36,000 respondents • 66 institutions • 2012 Administration • More than 33,000 respondents • 70 institutions • 2013 Administration • More than 27,000 respondents • 48 institutions • Now able to combine 2011, 2012, and 2013 respondents to create a “SNAAP Database” with over 92,000 respondents

12. Questionnaire Topics • Formal education and degrees • Institutional experience and satisfaction • Postgraduate resources for artists • Career • Arts engagement • Income and debt • Demographics

13. Career Outcomes Information *List continues, 45 categories total

14. Career Outcomes Information *List continues, 45 categories total

15. Career Outcomes Information In 2011 & 2012 data, 80% of alumni reported they were “very satisfied” or “somewhat satisfied” as their overall job satisfaction

20. Program Assessment: Services

21. Program Assessment: Services • Programs and services with low satisfaction may need to be revised • Career advising had 59% report either “very dissatisfied” or “somewhat dissatisfied” • Additional resources could be devoted to developing new components of career advising such as: • Alumni career panel presentations • Résumé or portfolio building sessions • Networking opportunities for graduating students

22. Program Assessment: Internships • Data suggests that pre-graduation internship experiences are important for future success

23. Program Assessment: Internships • Those who completed internships were more satisfied with their overall institutional experience

24. Program Assessment: Internships Those who completed an internship also found a job faster, found jobs that were more closely related to their training, and were more likely to currently be working in an arts-related field

25. Program Assessment: Internships • Equity concerns: it is important to make sure opportunities for internships are promoted to all students • First generation respondents were less likely to have completed an internship

27. Curricular Assessment: Skills

28. Curricular Assessment: Skills

29. Curricular Assessment: Skills • Identify strengths: • What skills have the highest % of alumni reporting the institution helped them develop “very much” or “quite a bit”? • Identify areas for improvement: • What skills have the highest % of alumni reporting the institution helped them develop “very little” or “not at all”? • Peer group information provides context: • Do other institutions have similar strengths and weaknesses?

30. Examples From Aggregate Findings

31. Curricular Assessment: Skills • Alumni receive strong training in learning artistic techniques • Discrepancies between those who say a skill is important for their work and those who say the institution helped them develop that skill suggest some improvements that could be made, such as: • Requiring business and financial classes, or incorporating these elements into existing courses • Include classes looking at the “nontraditional” career paths of arts graduates

32. Curricular Assessment: Skills • Triangulate data with qualitative responses • Institutions can use this information to stress the importance of certain skills to current students, or address curricular areas for improvement if the training is not relevant

35. Conclusions • Assessing alumni can provide important information for career service offices, including: • Career outcomes information, which can help generate materials for students seeking advice on a major to align with career options • Specific program assessment, which can provide data needed to enhance opportunities for internship experiences that promote future success • Curricular improvements, which can better synchronize classroom experiences with necessary post-graduation skills

36. Questions or Comments? • Contact Information: • Angie L. Miller anglmill@indiana.edu • Amber D. Lambert adlamber@indiana.edu Strategic National Arts Alumni Project (SNAAP) www.snaap.indiana.edu (812) 856-5824 snaap@indiana.edu

37. References Carnevale, A.P., Cheah, B., & Strohl, J. (2012). College majors, unemployment, and earnings: Not all college degrees are created equal. Washington, DC: Center of Education and the Workforce, Georgetown University. Evers, F.T., Rush, J.C., & Berdrow, I. (1998). The bases of competence: Skills for lifelong learning and employability. San Francisco, CA: Jossey-Bass. Kuh, G.D. (2008). High-impact educational practices: What they are, who has access to them, and why they matter. Washington, DC: Association of American Colleges and Universities. Kuh, G. D. & Ewell, P. T. (2010). The state of learning outcomes assessment in the United States. Higher Education Management and Policy, 22(1), 1-20. Kuh, G. D. & Ikenberry, S. O. (2009). More than you think, less than we need: Learning outcomes assessment in American higher education, Urbana, IL: University of Illinois and Indiana University, National Institute of Learning Outcomes Assessment. Tait, H., & Godfrey, H. (1999). Defining and assessing competence in generic skills. Quality in Higher Education, 5(3), 245-253.