EVALUATION PROGRAMS FOR GRANT PROJECTS

1. EVALUATION PROGRAMS FOR GRANT PROJECTS Linda Mason, Ed.D. OSRHE lmason@osrhe.edu

2. Evaluation • Accountability measure • Insurance policy for the grant funder • Shows if you did what you said you would do • Shows if the grant funder’s money created some good Not acceptable: “We are planning to evaluate….”

3. GRANT AGENCY CHOOSES • External Evaluation* - by someone not connected with your institution • Internal Evaluation – you or another project staff member

4. Internal Evaluation • Collect your own data • Simple annual evaluation process: “What did you get in last summer’s institute that you have been able to implement?” “What might we have done in the institute that would have helped you even more?”

5. Internal Evaluation • Prepare your own evaluation format • Use objectives and timeline • Keep records of EVERYTHING! • Photographs • Testimonies • Anecdotes • Write proposal as if you were evaluating it

6. Internal Evaluation • Summative evaluation – Outcome Evaluation Program effectiveness Impact on the problem

7. Internal Evaluation Summative evaluation: What did you accomplish? Were program objectives accomplished? Was the target population changed? Did you meet the timeline? Was the program cost effective? What new knowledge was generated?

8. Internal Evaluation Summative evaluation: Quantitative methods Formal approach Professional research procedures Control and comparison groups Compare to the big picture

10. Internal Evaluation • Formative evaluation – Use evaluation material to retool

11. Internal Evaluation • Formative evaluation – Process evaluation • Leads to changes mid-stream • Generally use qualitative methods • First year or first part • Throughout the life of program

12. Internal Evaluation • Formative evaluation – Ask questions during the project: How are we doing with the timeline? Do we need to change it? What does the recruitment tell us about group size? What did the first of 3 workshops lead us to determine about the next one? Any changes needed in format or length?

13. Internal Evaluation • Formative evaluation – Don’t ask questions about items you cannot or will not change.

14. Internal Evaluation - Tips • Send thorough reports to grant agency • Send more than requested • Send in format of summary and full report

15. External Evaluator • Consider hiring one even if not required • External evaluator adds credibility • Consider being an external evaluator to learn more about grant proposal production • Work with the evaluator at the proposal design to be sure evaluation fits. • Collect your own data

16. Evaluation - Types • You do not have to conduct fancy “two-sample t-tests” or “analysis of variance” • Pre- and post-tests • Attitude surveys to participants • Document interviews and focus groups • Make charts of results

17. EvaluationWays to share results - • Research Fair • Poster • Videoconference • Press release • Follow-up session after project • Project Results Meeting • Prepare a document

18. EvaluationWays to share results - • Email news release • Poster or brochure campaign • Prepare a video or CD • Go on the talk show circuit of TV or radio • Present at school assemblies • Present at faculty meetings • Share with similar or larger projects

19. EvaluationTraditional Ways to Share Results • Journal Article • Conference presentation

20. Evaluation in Grant Proposal • Depends on Grant Agency • Thoughtful examples of the plan • Reasonable

21. Evaluation in Grant Proposal • Linkages between activities and outcomes should be clear and explicit Example: 80% of the 21 to 24-year olds who participate in a comprehensive work-study program will pass their GED exams. • Show other support from larger or partnering programs.

22. Evaluation - Tips • Don’t tack on the evaluation at the end—Frontload it, build it in at the beginning. • Explain what measures you are going to use. • Use the evaluation section to go into detail. • Show sustainability of the project as part of the evaluation.

23. Evaluation - Tips • Negative findings are as useful as positive findings • Were there unexpected world, national or local events that seemed to affect the project? • Could the project be replicated?

24. Evaluation Summary • You WILL evaluate • Grant agency specifies internal or external • External strengthens credibility • Provide more data than requested • Use a variety of sharing techniques • Formative – changes during project • Summative - outcomes

25. Evaluation Summary • Frontload evaluation plan • Cost effective • Replicable • Renewed • Sustained