330 likes | 429 Views
Focused Program Evaluation. Randolph R. “Randy” Weigel & Susan James EPIC – 2004 Lusk, WY September 21. “Your tax dollars support this learning experience either directly or indirectly. Do you want your tax dollar to continue supporting this type of program?” Yes No
E N D
Focused Program Evaluation Randolph R. “Randy” Weigel & Susan James EPIC – 2004 Lusk, WY September 21
“Your tax dollars support this learning • experience either directly or indirectly. • Do you want your tax dollar to continue • supporting this type of program?” • Yes • No • Robert J. Fetsch, Extension Spec. • Colorado State University
Definition Of Evaluation “To evaluate is to make an explicit judgment about the worth of all or part of a program by collecting evidence to determine if acceptable standards have been met.”’ Andrews & Warner (1988)
Types of Evaluation • Process (Formative) • Impact (Summative)
When should we conduct an evaluation? Before? During? After?
When should we conduct an evaluation? • Project design • Start up • In-progress • End of program • Follow-up
When should we NOT conduct an evaluation? • No one cares • Not going to change the program • One time program • No time, $, resources or energy • No goals or objectives • To satisfy your “performance”
Evaluability Assessment What are you going to evaluate? What is the purpose of the evaluation? Who will use the evaluation What resources do you have and need? What questions will the evaluation hope to answer?
Knowledge Item How important is nitrogen fertilizer management for maintaining our water quality? ************* Farmers’ proper management of nitrogen fertilizer applications is essential to maintain good water quality. TRUE FALSE DON’T KNOW
Attitude Item How important is nitrogen fertilizer management for maintaining our water quality? ************* Farmer’s proper management of nitrogen fertilizer applications is essential to maintain good water quality. Strongly Strongly agree Agree Disagree disagree
Experience Item How important is nitrogen fertilizer management for maintaining our water quality? ************* • Do you follow recommendations for applying • nitrogen fertilizer when the recommendations are • based on deep soil sampling? • No • Yes—How much do you apply • 1. About 20% less than the amount recommended • 2. About 10% less than the amount recommended • 3. The exact amount recommended • 4. About 10% more than the amount recommended • 5. About 20% more than the amount recommended
Feeling Item How important is nitrogen fertilizer management for maintaining our water quality? ************* How do you feel when others imply that farmers must better manage nitrogen fertilizer applications to maintain good water quality? Extremely Extremely angry ___ ___ ___ ___ ___ calm
Collecting the Data: What sources will you use? • Existing Information • People • Observations • Pictorial Records
What data collection method(s) to use? • Which method will give you the information you • need? • Which method is appropriate for participants? • Which method is least disruptive? • Which method can you afford and manage?
Common Types of Data Collection Surveys Interviews Observation Case Study Group Techniques Tests Photos
Common Types of Data Collection (cont.) Document Review Testimonials Expert Panel Simulations Journals Unobtrusive measure “Triangulation”
Using the Information How will the data be analyzed? How will the information be interpreted? Who will receive the evaluation? How will the information be disseminated?
Avoid These Mistakes in Your Program Evaluation
Did today’s in-service provide you with information to review and if necessary improve your program evaluation skills? 1. Yes 2. No
Do you believe LISA research should or should not be funded by CSREES/USDA? 1. Should 2. Should not 3. Undecided or unsure
Please interpret each of these words below as a percentage. Example: Average = 50% Frequently Occasionally Seldom Sometimes
How much profit did your business earn in 2003? _____________ Dollars
http://cyfernet.ces.ncsu.edu/cyfres/browse_2.php?search=Evaluationhttp://cyfernet.ces.ncsu.edu/cyfres/browse_2.php?search=Evaluation