1 / 19

Survey Design Workshop: Enhancing Student Success

Learn to develop effective surveys for assessing student learning and success. Gain practical feedback on survey instruments to improve data collection and analysis. Plan, design, and interpret survey findings for impactful decision-making.

jward
Download Presentation

Survey Design Workshop: Enhancing Student Success

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Division of student development Developing Surveys (Magis@Work) Divisional Assessment Committee in partnership with the Professional Development Committee February 1st, 2018

  2. Goals for the Workshop • Discuss general tips for survey design • Consider the assessment of student learning and student success • Contribute to the development of a bank of questions to use within DSD • Receive practical feedback on survey instruments

  3. WHY: Why Administer a Survey? • What you are assessing should be determined by your goals • and learning outcomes for the program / service. • Think ahead: once you get the data, what will you do with it? • Seek out information on which you can act. • What do you hope to be able to communicate with the data? • Don’t ask questions if you already have the information. • Determine whether a survey is the best format for the data you • are gathering.

  4. WHO: Asking the Right People • Consider the difference between population and sample. • Consider different sampling strategies (such as random, stratified, or convenience sampling). • Consider how representative the sample may be. • Beyond thinking about the people who will receive your survey, consider: • whether there are people with whom you may want to collaborate when administering the survey. • who can provide assistance (the Campus Labs • team can provide feedback on content and • structure)

  5. WHAT TO ASK, and HOW TO ASK IT: Steps in Survey Design • Outline topics and draft items • Choose response formats • Write and edit items • Determine the order • Plan the administration • Pilot and revise as needed • Use and share the findings

  6. Steps in Survey Design: Choosing Response Formats • Quantitative or Qualitative • Broad picture of what is happening vs. why / how it is happening • Open-ended or Closed • Single Response or Multiple Choice (“Select All that Apply”… or “Select Top 3 Options”) • Ranking • Likert Scale • Define the options • Neutral point or forced choice? • Number of levels

  7. Steps in Survey Design: Write & Edit Items • Use clear language, and ask one thing at a time. • Bloom’s Taxonomy • Avoid loaded or leading questions (no assumptions). • Provide options for all (Never, N/A). • Ask for a useful level of detail: • Categories versus specifics (example: majors, or numbers versus a range) • Yes / No & Please Explain • Consider the number of questions • Attrition increases at 22 questions or 13 minutes.

  8. Steps in Survey Design: Determine the Order • Objective before subjective (Have you participated in PROGRAM? How would you rate PROGRAM)? • Begin with less threatening questions. • Important questions near the start. • Save open-ended questions for the end. • Place questions about demographics at the end.

  9. Steps in Survey Design: Plan and Pilot • Outline topics and draft items • Choose response formats • Write and edit items • Determine the order • Plan the administration • Consider time of year, • communication, incentives • Pilot and reviseas needed • Use and share the findings

  10. Steps in Survey Design: Interpret, Use, & Share Findings • Qualitative: • Themes (coding) • Nominal / named categories (in-state or out-of-state). Present frequencies or percentages (tables, charts, graphs). • Quantitative: • Ordinal: order/ranking, but not meaningful distance between points (class year). Present frequencies or percentages. • Interval: order with equal fixed distances between data points, but no absolute zero (SAT or ACT scores). • Ratio/scale: meaningful zero point (family contribution in $)

  11. Interpret, Use, & Share Findings • Presenting qualitative findings • Simple descriptive statistics • Central tendency (most common, middle, average value) • Dispersion (variation or spread in relation to central tendency) • Measures of relationship (correlation, regression) • Measures of comparison (t-tests, ANOVA, MANOVA) • Share findings with staff when making decisions, and with students so they know their voice matters.

  12. Plan 2020 • Building a more just, humane, and sustainable world • Plan 2020 charges us to analyze and support these student success outcomes: • Retention • Timely graduation • Progression • Career readiness • Post-graduation success

  13. Career Readiness & Post-Graduation Success • Employers look for candidates who excel in areas such as: teamwork, problem-solving, communication, work ethic, and leadership (NACE, 2016). • The likelihood that alumni will be engaged at work is: (a) 2.2x higher if mentors encouraged them to pursue their goals, and (b) 1.8x higher if they were active in co-curricular activities while in university (Gallup, 2014). • The likelihood that alumni are emotionally attached to their alma mater is: (a) 4.1x higher if mentors encouraged them to pursue their goals, and (b) 2.7x higher if they were active in co-curricular activities while in university (Gallup, 2014).

  14. Student Success is Associated with: • Building formal and informal relationships with staff and faculty • High-impact educational opportunities that foster global learning, exploration of social identities, and social-perspective-taking (Kuh, 2008)* • Academic self-efficacy (belief in their academic abilities) • Campus engagement (this may be the most important condition for retention) • On-campus employment / Federal Work Study positions (financial assistance, resources, and relationships built) • A sense of belonging on campus (campus climate affects this) • *All other points are from Tinto, 2012

  15. Small Group Activity • Get into groups of 2 to 3 people and introduce • yourselves. • In small groups, please exchange copies of • your surveys so each person is reviewing a • different survey and providing feedback. You • may want to use the handout to help. • with main points from this presentation. • Then, please share feedback about each • survey with small group members.

  16. Resources and Support • Creation of bank of questions for DSD assessment • Resources • Links on DSD Assessment Website • Campus Labs • Divisional Assessment Committee • Free online course on assessment in student affairs • Day-long Assessment Conference offered by Campus Labs at LSC the week of May 21st (more details to come soon)

  17. Multi-Institutional Study of Leadership

  18. References Assessment Certificate Program Campus Labs: https://baselinesupport.campuslabs.com/hc/en-us/articles/204840519-Assessment- 101-The-ABCs-of-Survey-Design Gallup (2014). Great jobs, great lives: The 2014 Gallup-Purdue Index Report. Kuh, George D. (2008). High-impact educational practices: What they are, who has access to them, and why they matter. Washington, D.C.: AAC&U. NACE: http://www.naceweb.org/talent-acquisition/candidate-selection/the-attributes-employers-seek- on-a-candidates-resume/ Schuh, J. H. & Associates (2009). Assessment methods for student affairs. San Francisco: Jossey- Bass. Tinto, V. (2012). Completing college: Rethinking institutional Action. Chicago: The University of Chicago Press.

  19. Questions?

More Related