1 / 30

Thinking Evaluatively Evaluation Framework for Measuring Program Outcomes

Thinking Evaluatively Evaluation Framework for Measuring Program Outcomes. Nav Ghimire Agriculture Agent, Green Lake County Team Member, ELST. What is Evaluation ? . Judgment about how well your program objectives are being met….

garima
Download Presentation

Thinking Evaluatively Evaluation Framework for Measuring Program Outcomes

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Thinking Evaluatively Evaluation Framework for Measuring Program Outcomes Nav Ghimire Agriculture Agent, Green Lake County Team Member, ELST

  2. What is Evaluation ? Judgment about how well your program objectives are being met…. Nav Ghimire, Green Lake County Agriculture Agent, 2014

  3. Evaluation is……. Developing Criteria Gathering Evidence Forming Judgments Nav Ghimire, Green Lake County Agriculture Agent, 2014

  4. Criteria The measure against which judgments are made about.“Outcome/Program Objective / Evaluative Question” Evidence Data that is relevant to evaluative question to make decision. May be obtained from a variety of measurement tools, survey, interview, observations, reports, records, etc. Nav Ghimire, Green Lake County Agriculture Agent, 2014

  5. Judgment • Judgments should be made based upon the degree to which • the evidence indicates that the criteria are being met. • Judgment About What ? • * Appropriateness– Looks at process than products. • - Were participant needs met? - Was what learned important to them? - Were teaching methods appropriate? • * Effectiveness – Measures program accomplishments • in terms of changes in participant behavior. • - Knowledge, skills , attitude, and application of practice? • * Efficiency – Compares time, resources, and effort expended in • terms of accomplishments. • - Was accomplishment sufficient for resources used? Nav Ghimire, Green Lake County Agriculture Agent, 2014

  6. Steps in Evaluative Thinking Source: Journal of Agricultural Education Volume 53, Number 4, pp85–97; UW-Extension, PD& E Nav Ghimire, Green Lake County Agriculture Agent, 2014

  7. Nav Ghimire, Green Lake County Agriculture Agent, 2014

  8. Nav Ghimire, Green Lake County Agriculture Agent, 2014

  9. Nav Ghimire, Green Lake County Agriculture Agent, 2014

  10. How to Measure Program Outcomes List intended outcomes from the program. Write measurable program objective(s) to reach each outcome. List programactivitiesto achieve each objective. Write evaluativequestion(s) aligned to each objective. Identify key variable(s) that evaluation question(s) contain. Operationalize the variable(s) and identify indicator(s) to assess Identify source(s) of data. Determine data collection method(s). Nav Ghimire, Green Lake County Agriculture Agent, 2014

  11. How to Write a Good Objective Specifies a result, not an activity. Each objective describes just one result. Emphasize what will be done but does not explain how to do it. It is specific, measurable, achievable, possible, verifiable. Should have a subject, verb, and object (SVO) It is designed with and understood by those responsible for its implementation - Extension educators based on needs assessment. Nav Ghimire, Green Lake County Agriculture Agent, 2014

  12. Example of Nutrient Management Program Outcome: Number of farmers developing a nutrient management plan for their farm will be increased. Objective 1: Farmers will gain knowledge and skills about phosphorous management. Correct: Farmers will gain knowledge about the importance of phosphorous management in their farm /soil. Objective 2: Farmers will learn skills to manage phosphorous in their farm Objective 3: Farmers will learn skills to manage potassium in their soil. Objective 4: Farmers will gain knowledge about the importance of a nutrient management plan for their farm. Objective 5: Farmers will learn skills for writing a nutrient management plan for their farm. Nav Ghimire, Green Lake County Agriculture Agent, 2014

  13. Example of Family Living / Youth Program Outcome: Numbers of parents/caregivers developing nurturing abilities to effectively raise their children will be increased Objective 1: Parents/ caregivers will gain knowledge about the importance of nurturing to support their children. Objective 2: Parents/caregivers will learnnurturing skills to support their children / effectively raise their children. Outcome: Number of youth developing abilities to resist negative peer pressure will be increased. Objective 1: Youth will gain knowledge about the consequences of negative peer pressure in the school. Objective 2: Youth will learn skills to resist negative peer pressure to avoid bad habits in the school / life. Impact: Number of better citizen in the community will be increased. Nav Ghimire, Green Lake County Agriculture Agent, 2014

  14. Program objectives VsPrograms Outcomes Program Objective: They are learning objectives that you want your participants to learn as a result of your program. It guides program activities you want to implement as an educator to facilitate learning. Program Outcomes: What participants will achieve / are going to do as a result of your program / educational activities. Evaluation: Measures to what extents program achieved its objectives to produce a desired outcomes. Nav Ghimire, Green Lake County Agriculture Agent, 2014

  15. Evaluative Questions What is it that you want to know about your program? operationalize it (make it measurable) Do not move forward if you cannot answer this question. Nav Ghimire, Green Lake County Agriculture Agent, 2014

  16. Evaluative Questions • Provides guidelines to direct your evaluation • Demonstrates what area of programming outcomes you are interested to know about • Primarily not designed as a question to put in survey • Can be a part of the survey question, if needed Nav Ghimire, Green Lake County Agriculture Agent, 2014

  17. Evaluative Questions Nutrient Management Training Program Question 1: Did farmers increase knowledge about the importance of phosphorous management in their soil?. Correct Q.1: How much did program / training increased farmers’ knowledge about the importance of phosphorus management? Question 2: Did farmers develop skills for writing a nutrient management plan for their farm? Correct Q.2: To what extent the training / program developed farmers’ capacity to write a nutrient management plan for their farm? Question 3: How well did farmers correctly developed nutrient management plans for their farm? Question 4: How well did farmers applied nutrient management plan in their farm. Question 5: How much did farmers saved cost from improved nutrient management practices? Nav Ghimire, Green Lake County Agriculture Agent, 2014

  18. Evaluative Question Family Living / Youth Program Question 1: How much did program/training increased parents / caregivers knowledge about the importance of nurturing? Question 2: How well did parents / caregivers applied nurturing skills to raise their children? Question 3: To what extent youth developed their capacities / abilities to resist negative peer pressure? Question 3: How well did youth applied negative peer pressure skills to avoid bad habits? Question 4: How much could youth able to resist negative peer pressureto avoid bad habits ? Nav Ghimire, Green Lake County Agriculture Agent, 2014

  19. Examples of Good Evaluative Question(s) ? • How well the program / training was implemented? • To what extent has the program / training contributed to meet the needs of participants. • How much did program / training increased participants’ knowledge and skills? • To what extent the program / training was enough to solve participant’s farming problems? Nav Ghimire, Green Lake County Agriculture Agent, 2014

  20. Evaluative Question Contd… • How significant was the program’s contribution to strengthening the partner organization(s)? • To what extent did the program work well to all participants? • To what extent the possible program’s positive results are likely to last after the project ends? Nav Ghimire, Green Lake County Agriculture Agent, 2014

  21. Evaluation Framework Nutrient Management Training Program Nav Ghimire, Green Lake County Agriculture Agent, 2014

  22. Evaluation Framework Family Living /Youth Program Nav Ghimire, Green Lake County Agriculture Agent, 2014

  23. Questionnaire for Data Collection • Use self-constructed questionnaire • Each question should focus on one idea only • Easy to answer (Check rather than write), if possible • Keep the same scale throughout the questionnaire • Maintain floor and celling in the scale; very low – very high • Ask only what you need to meet your objective(s) • Use open ended questions when you want details • Give your friends / farmers to read and advise you Nav Ghimire, Green Lake County Agriculture Agent, 2014

  24. An Example of Questionnaire Nav Ghimire, Green Lake County Agriculture Agent, 2014

  25. Types of Question for A Program Evaluation Survey Nav Ghimire, Green Lake County Agriculture Agent, 2014

  26. Types of Question Contd….. Nav Ghimire, Green Lake County Agriculture Agent, 2014

  27. Types of Question Contd….. Nav Ghimire, Green Lake County Agriculture Agent, 2014

  28. Types of Question Contd….. Nav Ghimire, Green Lake County Agriculture Agent, 2014

  29. Types of Question Contd….. Nav Ghimire, Green Lake County Agriculture Agent, 2014

  30. Workshop • Identify a program. 2. Write 1 intended outcome. 3. Write 2 program objectives. • Write 1 evaluative question. 4. Display your work on the wall. Nav Ghimire, Green Lake County Agriculture Agent, 2014

More Related