1 / 41

Simple Surveys: From Program Objectives to Impact Statements in 90 Minutes

Simple Surveys: From Program Objectives to Impact Statements in 90 Minutes. 2016 Extension Conference Jeff Buckley & Jennifer Cantwell January 13, 2016. Topics we will cover…. General Evaluation Overview: Why and what do we evaluate? Outcomes → Constructs → Questions Survey Templates

gaurav
Download Presentation

Simple Surveys: From Program Objectives to Impact Statements in 90 Minutes

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Simple Surveys: From Program Objectives to Impact Statements in 90 Minutes 2016 Extension Conference Jeff Buckley & Jennifer Cantwell January 13, 2016

  2. Topics we will cover….. • General Evaluation Overview: • Why and what do we evaluate? • Outcomes → Constructs → Questions • Survey Templates • Data Tabulation Templates • Impact Statement Templates • Create you own!

  3. You will leave with…. Free, adaptable resources! www.georgia4h.org/evaluationresources/

  4. The overly-busy 4-H Agent: Scene 1: Why evaluate? Topics we will cover….. • Why evaluate? • Proper Usage of data (IRB) • Survey Templates • Data Tabulation Templates • Impact Statements • Tools for creating Surveys

  5. Why do you need to evaluate? • Program Impact: Did kids learn? • Program Improvement • Professional Development & Accountability • Improve your impact statements and other reports. • Gather data for posters, workshop proposals, and awards

  6. The overly-busy 4-H Agent: Scene 2: Don’t I need permission from IRB? Topics we will cover….. • Why evaluate? • Proper Usage of data (IRB) • Survey Templates • Data Tabulation Templates • Impact Statements • Tools for creating Surveys

  7. Institutional Review Board (IRB) Research oversight committee charged with ensuring that human subjects research is conducted in compliance with the applicable federal, state, and institutional policies and procedures.

  8. When is IRB-approval needed? Program Evaluation vs. Research

  9. The overly-busy 4-H Agent: Scene 3: Okay, well that’s great and all, but I don’t know anything about evaluation! Topics we will cover….. • Why evaluate? • Proper Usage of data (IRB) • Survey Templates • Data Tabulation Templates • Impact Statements • Tools for creating Surveys

  10. The Process – Keep it simple!

  11. Types of Outcomes • Short-term • Change in Knowledge, Skills, Attitude • Behavioral Intent • Intermediate • Change in Behavior • Long-term • Improved Condition

  12. Today we will focus on: • Short-term Outcomes • Increase in Knowledge or Skill • Demonstration of Knowledge or Skill • Change in Behavioral Intention • As a result of intervention, I plan to….

  13. Outcomes What are some of the outcomes you hope youth or adults will experience from participation in your programs?

  14. Measurable Outcomes • Measurable outcomes will translate into more effective questions. • Resources • Handout “Verbs for Writing Measurable Objectives” • Packet – Building Capacity in Evaluating Outcomes

  15. Constructs • More than one question to measure the same outcome or concept. • Ask the same question in a different way • Ask multiple questions to show they gained knowledge (ex. Circle the 4 main functions of government)

  16. Examples –Measurable Outcomes & Constructs Program - Teach youth to make healthier food choices and to increase activity. • 2 Short-term Outcomes • Youth are able to identify healthy food choices • Youth plan to make more healthy choices

  17. Identify the intended outcomes • I know more about what foods are healthy. • I learned new ideas for being active. • I can identify healthy food options. • I have learned new things I can do to be healthier. • I plan to exercise more. • Decreasing screen time is a healthy choice.

  18. Intended Outcomes:Answers • I know more about what foods are healthy. • I learned new ideas for being active. • I can identify healthy food options. • I have learned new things I can do to be healthier. • I plan to exercise more. • Decreasing screen time is a healthy choice.

  19. Let’s go back-wards – What are the two outcomes? 1. The food in the cafeteria was fresh. 2. The dining area was clean. 3. There were a variety of fruits and vegetables. 4. Beverage selections included healthy options. 5. Vegetables were not over-cooked. 6. The servers were pleasant.

  20. What are the two Outcomes? • The cafeteria will provide a pleasant dining experience. • The cafeteria will provide healthy options for diners.

  21. The overly-busy 4-H Agent: Scene 4: Okay, now I get it. I don’t know where to start! Topics we will cover….. • Why evaluate? • Proper Usage of data (IRB) • Survey Templates • Data Tabulation Templates • Impact Statements • Tools for creating Surveys

  22. Things to consider • What do you want to know? • Does a survey already exit to measure what you want to know? • Who is your audience? Adults? Youth? • How much time do you have to administer survey? • Paper surveys or online? • How will you share the results?

  23. General Types of Surveys • Quantitative Data (Purse Strings) • Qualitative Data (Heart Strings) • Pretest and Posttest • Posttest Only • Retrospective Post then Pre

  24. The overly-busy 4-H Agent: Scene 5: Okay, well, I’m so busy! What kinds of questions should I ask? What should it look like? I wish there was a template I could use to get started….. Topics we will cover….. • Why evaluate? • Proper Usage of data (IRB) • Survey Templates • Data Tabulation Templates • Impact Statements • Tools for creating Surveys

  25. Our Survey Tools • Survey Templates • Likert Style - Pretest and/or Posttest • Likert Style - Retrospective Post then Pre • Combination Knowledge & Likert • Kid-friendly format • Program-specific Surveys • Ready to use with specified curricula • All surveys include open-ended questions • Most include Data Tabulation Spreadsheets

  26. Simple Surveys: Likert Style

  27. Simple Survey Templates – Likert Style • Standard Likert Questionnaire • Can be used as pretest/posttest or posttest only • Data Tabulation Spreadsheet & Directions • Retrospective Post-Then-Pre • Administered after intervention

  28. Points to Consider when Writing Likert Style Questions/Statements • Consist of a statement and a rating scale • Need to conform to one rating scale • One data point per question • Is what you’re measuring important to the intended audience of the evaluation? • Good for older youth and adults

  29. Simple Survey Templates – Likert Style • Adaptable Data Tabulation Spreadsheets with directions • Good for trainings, adults, older youth

  30. Combination Knowledge and Likert Style

  31. Quiz-style / Program or Topic Based • Include quiz-type questions (matching, definitions, true/false, etc.) • Can also include Likert Style questions • Pre and posttest or posttest only • Survey Answer Key & Data Tabulation Spreadsheet • Spreadsheet can include pre-made outcome statements

  32. Existing Surveys • CCRPI: Career Awareness • Marketing, Finance, Government/Public Administration, Information Technology • States of Matter (Paulding County) • Manners Worth Millions • Bullying (Cook County) • Farm to Fork • Less Stress on the Test

  33. Existing Surveys continued • Project Achievement* • Cloverleaf, Junior/Senior • Coming Soon: • Inherited traits • Barrier Islands • Mikey the Microbe • Plant ID *These are best suited for traditional demonstrations.

  34. The overly-busy 4-H Agent: Scene 6: I think I can write some questions and administer these surveys. Now what do I do with all these papers? Topics we will cover….. • Why evaluate? • Proper Usage of data (IRB) • Survey Templates • Data Tabulation Templates • Impact Statements • Tools for creating Surveys

  35. Tabulation Spreadsheets

  36. The overly-busy 4-H Agent: Scene 7: Great! I love these simple tools! What should I do with all this great data? Topics we will cover….. • Why evaluate? • Proper Usage of data (IRB) • Survey Templates • Data Tabulation Templates • Impact Statements • Tools for creating Surveys

  37. Impact Statements • Translate the numbers from your Data Tabulation Spreadsheet into Impact Statements. • Spreadsheets for existing surveys include impact statements. Let’s look at some examples!

  38. Impact Report Templates

  39. The overly-busy 4-H Agent: Final. Climactic Meeting with funders… Topics we will cover….. • Why evaluate? • Proper Usage of data (IRB) • Survey Templates • Data Tabulation Templates • Impact Statements • Tools for creating Surveys

  40. QUESTIONS??

  41. To Access Templates, go to…www.georgia4h.org/evaluationresources For more information contact:University of Georgia, State 4-H Office Jeff Buckley, jbuckley@uga.edu Jennifer Cantwell, jecantw@uga.edu

More Related