Sowing the Seeds : Program Evaluation that Works for You

1. Sowing the Seeds: Program Evaluation that Works for You Self-Guided Training Module Please press <Enter> or click the mouse to begin. Center drawing credit: Josh Jetson, 16, Australia

2. Introduction Welcome to the Sowing the Seeds self-guided training module on program evaluation.

3. Introduction The goal of this web-based module is to increase your capacity to evaluate your program. The examples used in this module focus on evaluation of youth programs but the principles of evaluation can be applied to any program.

4. Introduction Individuals who will find this module useful are program directors, staff members and volunteers, youth participants, parents, or anyone interested in quality programming. Please press <Enter> or click the mouse to begin.

5. Orientation This module is an interactive and flexible way for you to learn about the basics of program evaluation. It will take approximately 30 minutes for you to go through the core module. You will also have the option of going in-depth to learn more about some of the topics.

6. Orientation The module contains slides with descriptions and definitions. Also, you will have the opportunity to test your understanding with different activities. The module uses an example of a youth program to illustrate the steps of evaluation.

7. Orientation You also can move at your own pace– Just press <Enter>, click the mouse, or press the forward arrow when you are ready to continue or if you would like to move quickly through a section. Try it now to go to the next slide.

8. Outline of Training Module This module consists of five parts: I. Introduction to Evaluation II. Goals, Objectives, & Measures III. Data Collection Tools IV. Data Analysis & Reporting V. Evaluation Steps & Planning

9. I. Introduction to Evaluation So, let’s get started. Why do you want to evaluate your youth program?

10. I. Introduction to Evaluation Here are some reasons for conducting an evaluation. Program evaluation helps you: • assess changes in participant knowledge, attitudes, and skills; • secure and maintain program funding; • improve programming; and • justify program continuation and adoption. Are your reasons included?

11. I. Introduction to Evaluation Whatever your reasons for wanting to know more about program evaluation, this module will provide you with basic information and tools to think about and plan your own evaluation.

12. Types of Evaluation There are two types, or parts, of evaluation: Process Evaluation Outcome Evaluation &

13. Process Evaluation assesses what your program does. The process evaluation has to do with what you teach, how many sessions you hold, your attendance rates, the activities you conduct, etc. Process Evaluation

14. Outcome Evaluation describes how your participants will change as a result of the program. The outcome evaluation has to do with the knowledge, attitudes, and skills you anticipate your participants will gain. Outcome Evaluation

15. Types of Evaluation As you go through the module and the optional sections, you will learn more about how to conduct the process and outcome evaluations of your program.

16. Types of Evaluation If possible, it is important to conduct both types of evaluation. The information you get will help you to match up what you actually did in the program (process) with how participants changed as a result (outcome). The combination tells you what worked and what didn’t in your program.

17. Special Note: Youth Participatory Evaluation A growing number of youth serving organizations are working collaboratively with youth participants to design and conduct program evaluations, an approach called youth participatory evaluation. The results can be great for the program and for the youth! For more information on youth participatory evaluation, refer to the resource list at the end of the training module.

18. Welcome to Part II This part introduces the goals, objectives, and measures (optional). I. Introduction to Evaluation II. Goals, Objectives, & Measures III. Data Collection Tools IV. Data Analysis & Reporting V. Evaluation Steps & Planning

19. II. Goals & Objectives When you evaluate your program, you measure whether you are running the program and meeting those served (e.g., youth) as you had planned. Your program’s goals and objectives organize what you plan to achieve through your program.

20. II. Goals & Objectives Goals & objectives are important because they are the destination on your evaluation roadmap. Without them, it is hard to know what you want to accomplish and if you are succeeding! As you will soon learn, goals are more broad than objectives. Objectives are more detailed and measurable.

21. II. Goals & Objectives A goal is the broad and overarching purpose toward which your program is directed. For example, the goal of the YouthWorks program is “to contribute to a culture of free speech and social responsibility.”

22. Goals What is the goal of your youth program?

23. Objectives Your program’s objectives are the more specific things you plan to accomplish. For example, one objective of the YouthWorks program is to involve 25 youth in 10 media workshops over 3 months.

24. Goals & Objectives-Why both? You may be wondering: What is the purpose of having goals and objectives? It is important to have both because while objectives help you assess short-term steps or milestones in running your program successfully, a goal helps you to strive toward your ultimate vision for the program. Keeping your goal in mind can help you to write new objectives as your program changes and improves over time!

25. II. Goals & Objectives If you would like to learn more about goals and objectives now, click on this button. If you would like to continue with the core module, click on this button.

26. Objectives While a goal is general, an objective should be measurable, achievable, and specific. Remember this and you will get MAS (more) out of your objectives! Measurable * Achievable * Specific

27. Objectives Measurable means that you can set up a practical way to assess your objective. Achievable means your objective is realistic. Specific means that your objective is as detailed as possible.

28. Objectives Here is an example of an objective with MAS. One of YouthWorks program’s objectives is to involve 25 youth in 10 media workshops over 3 months. achievable measurable specific

29. Types of Objectives There are two types of objectives: Process Objectives Outcome Objectives &

30. Process Objectives describe what your program will do. A process objective has to do with what you will teach, how many sessions you will hold, your attendance rates, the activities you plan to conduct, etc. Process Objectives

31. Process Objectives Remember, one of YouthWorks program’s objectives is to involve 25 youth in 10 media workshops over 3 months. This is a process objective because it describes what the program will do. Process Objectives

32. Outcome Objectives describe how your participants will change as a result of the program. An outcome objective has to do with the knowledge, attitudes, and skills you anticipate your participants will accomplish. Outcome Objectives

33. Outcome Objectives Another objective of the YouthWorks program is to improve participants’ cooperation skills over the course of the 12-week program. This is an outcome objective. You will learn later how to make this objective measurable, by identifying how you will know if cooperation skills have improved. Outcome Objectives

34. Process or Outcome? Can you guess whether the following objectives are process or outcome? Guess and then click the mouse to see the answer. Objective: Instructors will cover material from at least 6 of the 8 lessons in the curriculum guide during one semester. Process Objective

35. Process or Outcome? Process or outcome? Guess and then click the mouse to see the answer. Objective: All 15 participants will develop 5 pictures in 3 weeks. Outcome Objective

36. Process or Outcome? Process or outcome? Guess and then click the mouse to see the answer. Objective: The program will have 85% attendance on average during the summer session. Process Objective

37. Goals & Objectives Review When you are clear about the goal and process and outcome objectives of your program, it will be much easier to evaluate your program. Take the next five minutes to read about the Better Bodies program and develop some process and outcome objectives for the program.

38. Goals & Objectives Review The Better Bodies program has as its goal, “to promote optimum lifelong physical health through weight management and education among youth.” The program involves about 50 youth (ages 9 to 12 years) yearly in an after school program. The program involves two two-hour sessions weekly over six months and includes lessons and hands-on activities focusing on nutrition education, food preparation, shopping skills, physical education, and games.

39. Goals & Objectives Review The program involves about 50 youth (ages 9 to 12 years) yearly in an after school program. The program involves two two-hour sessions weekly over six months and includes lessons and hands-on activities focusing on nutrition education, food preparation, shopping skills, physical education, and games. You already know the goal of the Better Bodies program. Can you come up with 1-2 process objectives and 1-2 outcome objectives for the program? STOP and take a minute to think about and write down these objectives. Save them to look at later.

40. Goals & Objectives Review Did you come up with 1-2 process objectives and 1-2 outcome objectives for the program? Save them to look at later.

41. II. Measures Measures are the pieces of information that tell you if you’re meeting your objectives. It is easy to remember what a measure is because it refers to how you will “size up” or measure if you are achieving your program objectives.

42. II. Measures The idea of “measures” can seem tricky, but we use this concept in our everyday life. For example, suppose you plan to get healthy this year. How would you keep track of whether you are meeting this goal?

43. II. Measures You might want to collect data in a couple of areas to chart your progress. These categories or measures may be: • Number of times per week you exercise for 30 or more minutes • Number of vegetables you eat each day • Number of pounds you lose each week

44. II. Measures If you would like to learn about measures now, click on this button. If you would like to continue with the core module, click on this button.

45. II. Measures Each measure is related to an objective. For example, if a process objective of the Better Bodies program is to involve the participants in 25 minutes of vigorous exercise at each session, then your measure for that objective would be the information that you record to know if youth are really getting opportunities for exercise. What would that be? The number of minutes provided for the youth to do vigorous exercise at each session.

46. Types of Measures Just like objectives, there are two types of measures: Process Measures Outcome Measures &

47. Process Measures are types of information that you collect about what your program is doing or has done. A process measure is a piece of information such as what you taught, how many sessions you held, your attendance rates, the activities you conducted, etc. Process measures

48. Process Measures Remember, one of YouthWorks program’s process objectives is to involve 25 youth in 10 media workshops over 3 months. The process measure for this objective is the number of youth in attendance at each media workshop. Process Measures

49. Outcome Measures are types of information that you collect to see how participants have changed as a result of the program. An outcome measure is a piece of information that has to do with the knowledge, attitudes, and skills your participants accomplish in the program. Outcome Measures

50. Knowledge Attitudes Skills Definition Participants’ understanding of concepts Participants’ feelings about topics Participants’ ability to perform activities Examples ·     Definitions of dance terms ·     Comfort level with using camera ·     Ability to develop photos independently ·     Components of the color wheel ·     Interest in sports history ·     Ability to assist others with homework tasks Outcome Measures Here are definitions and examples of typical outcome measures.