New SERU Graduate Student Survey October 9, 2014

1. New SERU Graduate Student SurveyOctober 9, 2014 Gregg Thomson (SERU PR) Daniel White Olena Horner Ron Huesman Tiffany Thayer (University of Minnesota) SERU Consortium

2. 2014 SERU GSS Workgroup 2014 SERU GSS Workgroup Members Chair Ron Huesman – Managing Director, SERU-AAU Consortium, UMN* Melissa Anderson – UMN Graduate School* Igor Chirikov – Higher School of Economics, Moscow Ken Doxsee – Oregon Louis Myers – Virginia Gregg Thomson – SERU PR/CSHE* Daniel White – UMN* Staff Support Olena Horner – UMN Research Assistant* Shelva Hurley – CSHE SERU Research Associate* Workgroup Consultants/Advisors: Tom Dohm – UMN Anne Maclachlan – CSHE MaresiNerad – University of Washington * Denotes Planning Team responsible for developing a preliminary draft for the Workgroup

3. Graduate SERU Workgroup Goals • Primary: • Done: Developed the core of the survey • Done: Populations, survey design (e.g., census, across academic levels), • Identified central data elements for exchange • Seed file (Done) • Supplemental file (tbd) • Fall 2014 SERU pilot: launches late October, 2014 - 6 weeks • University of Minnesota: PhD and MA students • University of Virginia: PhD only • Winter 2015 International pilot: • International: Higher School of Economics (Moscow) • Unicamp(Brazil) Winter

4. Rationale & NEED

5. Challenges Faced by Graduate Education in the U.S. • Ph.D. Attrition • 6.6% leave in their first year • 30.6% leave without a degree by year ten • Ph.D. Time to Degree • 56.6% complete their degree within ten years • 7.7 years – the national median time to degree in 2008 • International Competition • 33,000 - S&E doctoral degrees awarded in the U.S. • 28,000 - S&E doctoral degrees awarded in China • 15,000 - S&E doctoral degrees awarded in Russia • 11,000 – S&E doctoral degrees awarded in Germany Note: data on doctoral degrees are presented for 2007-2008 Source: Council of Graduate Schools (2008), National Science Foundation (2009, 2012)

6. Survey Rationale A survey can provide vital information about graduate/professional student experiences while enrolled that may help improve graduate education outcomes (i.e., increased graduation rates, faster time to degree, and help students find successful careers).

7. The Need for the New Graduate Student Survey • Existing Surveys • External exit surveys: Survey of Earned Doctorates & Survey of Doctorate Recipients (NSF) • Institutional exit surveys: encouraged to use Graduate Education Exit Survey (AAUDE) core as a component • Locally developed institutional student experience surveys: (UC-Berkeley, the University of Texas at Austin, the University of Michigan, Rutgers) • Benefits of the New Survey • a joint effort between AAU and SER-I • the survey will focus on graduate student experiences while enrolled in school • the survey will provide data for cross-institutional and international comparisons

8. The process

9. An Opportunity

10. Consensus Building: Identifying Stakeholders and Expertise

11. Carousel Brainstorming Session • What opportunitieswould an enterprise-wide graduate and professional student survey create for the University and individual units? • What challenges would an enterprise-wide graduate and professional student survey create for the University and individual units? • What benefits would result from an enterprise-wide graduate and professional student survey for the University and individual units? • What concerns or fearsdo you have in regard to an enterprise-wide graduate and professional student survey for the University and individual units? • What impactwould an enterprise-wide graduate and professional student survey have on the University and individual units?

12. Consensus Workshop If we were to create an effective enterprise-wide graduate and professional student survey, what information would need to be collected about students and their experiences?

13. Getting to Started 1. What is out there? 2. What is missing? Survey of Earned Doctorates (NSF). Survey of Doctorate Recipients (NSF/NIH). PhD Completion Project (CGS) Pew Trusts Survey on Doctorate Educ. Ivy League Graduate Student Survey. Assessment of Research Doct. Prog. (NRC). Graduate Education Survey (AAUDE).

14. Developing the Survey: A Parallel Process UMN Workgroup collaboration on initial draft. Open comment period for stakeholders to raise questions/issues. Dozen one-hour interviews with stakeholders. Revisions. SERU/CSHE collaboration on initial draft. Open comment period for stakeholders to raise questions/issues. Teleconference review of the draft. Revisions.

15. The instrument

16. Review of Current Research Literature on Student Experience

17. Graduate SERU Survey Conceptual Framework Conceptual framework: provides a basis/blue-print for survey development; mapping of final items to each domain will provide an item rationale document and a useful guide for conducting research with the survey in the future.

18. Attributes Outcomes EntryOrientations Student Experience • Student Background/ Current Status • Demographics • Parents’ educational attainment • Current program • Type of degree • Stage in the program • Socialization into the Profession & Professional Development • Professional values and ethical issues • Opportunities offered by the program/ • college/school/ • Graduate School • Program Climate • Satisfaction with the program climate • Teaching Experience • Training • Teaching experience • Impact on degree progress • Graduate/ • Professional • Degree Completion • Commitment to complete a degree in the current program • Admissions & Entrance • Goals • Reasons for selecting a program • Program communica-tions • Advising & Mentoring • Quality of advising • Areas of support from an advisor • Areas of support from a mentor • Career Plans & Changes • Changes in career plans during grad/prof studies • Careers toward which a degree program is oriented • Factors influencing career choices • Proficiency Levels • Development of academic, research, and professional skills • Previous Higher Education Experience • Degree, field, institution • Time elapsed since completing a previous degree • Financial Support • Sources • Degree of support • Other Institutional Support • Quality of amenities at the institution • Dissertation/Thesis Stage • Dissertation topic • Dissertation research process • Overall Satisfaction • Satisfaction with various aspects of a program • Fit between students’ values, expectations and the program • Choose the same field, program, university • Research Experience • Experience with research-related activities • Sources of financial support • Interdisciplinary research and its challenges • Financial Support • Satisfaction with financial support • Impact of employment outside of the university on degree progress • Anticipated consequences of debt burden • External Commitments • Work • Family • Financial Resources • Ability to pay • Undergraduate/ graduate loans • Obstacles to Degree Progress • Factors that hinder degree progress Note. Derived from Tinto (1993) and Astin (1970).

19. Graduate SERUSurveyStructure The survey covers 16 areas of student experience: • A. Your Graduate/Professional Program • B. Previous Higher Education Experience • C. Admissions and Entrance • D. Socialization into the Profession and Professional Development • E. Financial Support • F. Other Institutional Support • G. Program Climate • H. Proficiency Levels • I. Advising and Mentoring • J. Dissertation/Thesis Stage (for doctoral students only) • K. Research Experience • L. Teaching Experience • M. Career Plans & Changes • N. Obstacles to Degree Progress • O. Overall Satisfaction • P. For International Students Only

20. Graduate SERU SurveyStructure Core Survey • Factual questions: 42 • Opinion questions: 37 • Demographics: 12 International Students only • Factual questions: 5 • Opinion questions: 4

21. Examples of Survey Questions E. FINANCIAL SUPPORT To what extent do you agree or disagree with: “Having a job outside the university while going to school • helped me with career preparation • helped me secure a job • helped me advance my career • restricted my choice of classes • limited the number of classes I could take • slowed my degree progress • limited my access to campus facilities or services”? Scale: Strongly disagree, Disagree, Agree, Strongly agree, Not applicable When you complete your degree, how much do you think your debt burden will affect: • the type of job you will seek • the part of the country/world in which you will live • your family planning • your life style? Scale: Not at all, Little, Some, A great deal

22. Current Version • Graduate SERU • CORE within academic level (Ph.D. and not) and across all post-baccalaureate students • Items tailored to the academic level (similar items with term-specific references to the level or completely different items aimed at all students at a specific academic level)

23. Future • Pilot test of the survey • Reviews (staff and students) • Additional “think-aloud” interviews or verbal probing • Post collection: Item Response Theory analysis of items; examples of competency items Revise and focus survey: Winter/spring of 2015

24. SERU consortium: Vision • Graduate SERU and Undergraduate SERU, (and their associated modules) become part of a suite of surveys available to members for administration. • Costs? TBD information from pilot will be useful in estimating actual costs • SERU institutional reps, same?

25. Examples of Survey Questions PROFICIENCY LEVELS (v1) To what extent have your skills in the following areas developed during your graduate/professional program? • Analytical and critical thinking skills • Understanding your specific field of study • Ability to speak effectively in your field • Ability to write effectively in your filed • Ability to design and conduct original research • Ability to write grant proposals • Ability to work collaboratively on a project • Ability to work internationally • Ability to collaborate across disciplines • Ability to critically analyze the research literature (or performance products) from your field • Having transferable skills for various career opportunities • Adherence to high standards of ethics and professional responsibility • Ability to teach in your field • Ability to innovate, be entrepreneurial • Other (please specify): ___________________ Scale: Not at all, Not very much, To some extent, To a great extent, Not applicable

26. Examples of Survey Questions PROFICIENCY LEVELS (v2) Please rate your level of proficiency in the following areas when you started your graduate/professional program and now. Currently When you started • Analytical and critical thinking skills • Understanding your specific field of study • Ability to speak effectively in your field • Ability to write effectively in your filed • Ability to design and conduct original research • Ability to write grant proposals • Ability to work collaboratively on a project • Ability to work internationally • Ability to collaborate across disciplines • Ability to critically analyze the research literature (or performance products) from your field • Having transferable skills for various career opportunities • Adherence to high standards of ethics and professional responsibility • Ability to teach in your field • Ability to innovate, be entrepreneurial • Other (please specify): ___________________ Scale: Poor, Fair, Good Excellent, Not applicable

27. Examples of Survey Questions ADVISING AND MENTORING To what extent do you agree or disagree with the following statements concerning your current, primary advisor? • My advisor has expertise in the area I’m studying. • My advisor knows how to effectively help me. • My advisor provides me with information I need to help me think about my career. • My advisor has time for me when I need help or advice. • My advisor helps me get financial support. • My advisor assists me in writing for presentations/publications. • My advisor advises me about teaching. • My advisor teaches me the details of good research practice. • My advisor respects me as an individual. • My advisor considers my personal abilities, talents, and interests when advising me. • My advisor is interested in having students help with his/her research. • My advisor prompts me to seek out opportunities that I would not have otherwise considered. Scale: Strongly disagree, Disagree, Agree, Strongly agree, Not applicable

28. Examples of Survey Questions OBSTACLES TO DEGREE PROGRESS Rate the extent to which the following factors have been an obstacle to your degree progress? • Difficult coursework and academic requirements • Diminished interest in the field of study • Family responsibilities or obligations • Inadequate advising • Inadequate financial support • Need to work • Nonsupportive or unfriendly environment for students like me • Physical or emotional health problems • Poor or uncertain employment prospects after degree completion • Immigration issues • Other (please specify): ______________________________ Scale: Not an obstacle, A minor obstacle, A major obstacle, Not applicable

29. Questions Contact • Ron Huesmanhuesm003@umn.edu • Daniel Jones-White djwhite@umn.edu • Olena Horner glus0005@umn.edu

30. References Astin, A. W. (1970). The methodology of research on college impact, part one. Sociology of Education, 43(3), 223-254. Retrieved from http://www.jstor.org/stable/2112065 Abedi, J., & Benkin, E. (1987). The effects of students’ academic, financial, and demographic variables on time to the doctorate. Research in Higher Education, 27(1), 3-14. Ampaw, F. D., & Jaeger, A. J. (2012). Completing the three stages of doctoral education: An event history analysis. Research in Higher Education, 53, 640-660. doi: 10.1007/s11162-011-9250-3 Andrieu, S. C., & St. John, E. P. (1993). The influence of prices on graduate student persistence. Research in Higher Education, 34(4), 399-425. Baird, L. L. (1990). Disciplines and doctorates: The relationships between program characteristics and the duration of doctoral study. Research in Higher Education, 31(4), 369-385. Baird, L. L. (1993). Using research and theoretical models of graduate student progress. New Directions for Institutional Research, 80, 3-12. Bowen, W. G., & Rudenstine, N. L. (1992). In pursuit of the Ph.D. Princeton, NJ: Princeton University Press. Bowen, W. G., Lord, G., & Sosa, J. A. (1991). Measuring time to the doctorate: Reinterpretation of the evidence. Proceedings of the National Academy of Sciences, 88(3), 713–717. Council of Graduate Schools. (2008). Ph.D. Completion and attrition: Analysis of baseline program data from the Ph.D. Completion Project. Washington, D.C.: Council of Graduate Schools. de Valero, J. F. (2001). Departmental factors affecting time-to-degree and completion rates at one land-grant research institution. The Journal of Higher Education, 72(3), 341-367.

31. Ehrenberg, R. G., & Mavros, P. G. (1995). Do doctoral students’ financial support patterns affect their times-to-degree and completion rates? The Journal of Human Resources, 30(3), 581-609. Ethington, C., & Pisani, A. (1993). The RA and TA experience: Impediments and benefits to graduate study. Research in Higher Education, 34(3), 343-354. Gardner, S. K. (2009). The development of doctoral students: Phases of challenge and support. ASHE Higher Education Report, 34(6), 1-127. doi: 10.1002/aehe.3406 Gillingham, L., Seneca, J. J., & Taussig, M. K. (1991). The determinants of progress to the doctoral degree. Research in Higher Education, 32(4), 449-468. Girves, J. E., & Wemmerus, V. (1988). Developing models of graduate student degree progress. The Journal of Higher Education, 59(2), 163-189. Golde, C. M. (2000). Should I stay or should I go? Student descriptions of the doctoral attrition process. The Review of Higher Education, 23(2), 199-227. Golde, C. M. (2005). The role of the department and discipline in doctoral student attrition: Lessons from four departments. The Journal of Higher Education, 76(6), 669-700. Groen, J. A. (2012). Time to the doctorate and labor demand for new PhD recipients. U.S. Bureau of Labor Statistics. Retrieved from http://www.ilr.cornell.edu/cheri/upload/cheri_wp146.pdf Groen, J. A., Jakubson, G. H., Ehrenberg, R. G., Condie, S., & Liu, A. Y. (2008). Program design and student outcomes in graduate education. Economics of Education Review, 27(2), 111–124. Haldaway, E., Deblois, C., & Winchester, I. (1995). Supervision of graduate students. The Canadian Journal of Higher Education, XXV(3), 1-29. Jiranek, V. (2010). Potential predictors of timely completion among dissertation research students at an Australian faculty of sciences. International Journal of Doctoral Studies, 5, 1-13.

32. Nerad, M., & Miller, D. S. (1996). Increasing student retention in graduate and professional programs. New Directions for Institutional Research, 92, 61-76. Ott, M. D., Markewich, T. S., & Ochsner, N. L. (1984). Logit analysis of graduate student retention. Research in Higher Education, 21(4), 439-460. Park, C. (2005). War of attrition: patterns of non-completion amongst postgraduate research students. Higher Education Review, 38(1), 48-53. Pascarella, E .T., & Terenzini, P. T. (2005). How College Affects Students: A Third Decade of Research. San Francisco: Jossey-Bass. Picciano, J., Rudd, E., Morrison, E., & Nerad, M. (2008). Does time-to-degree matter? CIRGE Spotlight on Doctoral Education #3. CIRGE: University of Washington, Seattle, WA. Retrieved from www.cirge.washington.edu Seagram, B. C., Gould, J., & Pyke, S. W. (1998). An investigation of gender and other variables on time to completion of doctoral degrees. Research in Higher Education, 39(3), 319-335. Smith, R. L., Maroney, K., Nelson, K. W., Abel, A. L., & Abel, H. S. (Spring, 2006). Doctoral programs: Changing high rates of attrition. Journal of Humanistic Counseling, Education and Development, 45, 17-31. Stricker, L. J. (1994). Institutional factors in time to the doctorate. Research in Higher Education, 35(5), 569-587. Tinto, V. (1993). Leaving college: Rethinking the causes and cures of student attrition. Chicago and London: The University of Chicago Press. Wendler, C., Bridgeman, B., Cline, F., Millett, C., Rock, J., & McAllister, P. (2010). The Path forward: The future of graduate education in the United States. Princeton, NJ: Educational Testing Service. Wilson, S. B., Mason, T. W., & Ewing, M. J. M. (1997). Evaluating the impact of receiving university-based counseling services on student retention. Journal of Counseling Psychology, 44(3), 316-320.