Using your evaluation: Communicating, reporting, improving

1. Using your evaluation: Communicating, reporting, improving Who What (Content) How (Format) When

2. Why communicate? Success is judged by how much our evaluations and the findings are USED. We don’t conduct evaluations for them to sit on the shelf. Evaluation provides opportunity for learning and we are educators! “The proper function of evaluation is to speed up the learning process by communicating what might otherwise be overlooked or wrongly perceived. The evaluator, then, is an educator. His success is to be judged by his success in communication; that is by what he leads others to understand and believe. Payoff comes from the insight that the evaluator’s work generates in others.” - L. J. Cronbach Cronbach, L.J. (1982). Designing evaluations of educational and social programs. San Francisco: Jossey-Bass, p. 8.

3. Myths One report is enough. People read written reports. Complex analysis and big words impress people. Oral reports have the same effect as written reports. Describing limitations weakens report. Everything should be reported. The audience knows why they are getting the report.

4. There are lots of ways to communicate our evaluation results. Which have you used? Written report - long, short, summary Success story Impact statement; spotlight Elevator story Press release Media appearance At a public meeting Memo, email, fax, postcard Newsletter Personal discussion Bulletin, brochure Display/exhibit Audio/video presentation

5. “Start with the end in mind” Hopefully, you had a plan for using your information when you started your evaluation. It would include: WHO? do you want to reach WHAT? will you report HOW? will you do it WHEN? will you report

6. # 1 WHO do you want to reach? Potential Users • Program staff • Youth participants • Parents • Volunteers • Your tenure committee • Funders • Program Committee • Collaborating agencies; partners • Schools boards, parent-teacher organizations • County board/City council (elected and appointed officials) • Community at large • Church organizations • State legislators • Professional organizations • The Extension and research community • Businesses; business groups • Police – law enforcement

7. #2 WHAT will you say, share, report, communicate?? What does the target audience care about? What do you want the audience to learn – understand? What type of information resonates with the audience: numbers, quotes, stories? What data will the target audience find compelling, trustworthy? What is your story?

8. # 3 HOW will you communicate – what format(s) will you use? Report Impact statement Executive summary Email, memo Personal discussion Display, exhibit Press release Newsletter, bulletin Poster Slide presentation Published article ??????

9. # 4 When will you communicate? • Monthly? • Quarterly? • Annually • Whenever I have a chance • When requested? • ???

10. Be prepared with an “Elevator story” A succinct, attention grabbing story that can be communicated in a short elevator ride. Be armed and ready with your best 30-second story! Your story is important. Your job is to make sure decision makers know it. You never know when opportunity might knock!

11. Success stories and annual accomplishment reports are ways we report in UWEX They demonstrate that we are responsible and accountable for our public dollars. See the guidelines and samples that are posted on this web site.

12. Success story and reporting outlines tie back to the logic model : SRRE University of Wisconsin-Extension, Program Development and Evaluation

13. Success Story Situation: Response: Results: Evidence: SRRE Situation What is the situation – the problem, concern, issue that needed to be addressed? Local, regional or state information about the issue. Succinct and compelling. • Response • Results • Evidence INPUTS and OUTPUTS: What did Extension do? What did we contribute? Who participated/benefited? #s of key demographics of participants OUTCOMES: What changed? For whom? What is the value of the change? What does this mean? What was learned? What evaluation did you conduct – how do you know that the results are accurate and credible?

14. Ingredients of a Good Success Story • Captures attention of reader • Provides a compelling, convincing story • Gives specific outcomes (e.g., money saved, skills increased, practices changed, actions taken) • Includes numbers and narrative • Based on reliable, credible information • Presents balanced, fair assessment • Uses succinct, clear writing

15. Use graphics and design to make the information interesting and easy to understand • Charts and graphs • Overheads • Pamphlets • Reports

16. Discuss limitations of the evaluation Written reports: • Be explicit about your limitations Oral reports: • Be prepared to discuss limitations • Be honest about limitations • Know the claims you cannot make • Do not claim causation without a true experimental design • Do not generalize to the population without random sample and quality administration (e.g., <60% response rate on a survey)

17. Formal evaluation reports typically include… • Abstract/executive summary • Introduction • Purpose of the evaluation; key questions • Program background, description • Methods/procedures • Data sources • Data collection procedures • Sampling • Limitations • Results • Discussion • Conclusions/recommendations • References • Appendices

18. Reporting results to the media All Media: • Avoid using too many statistics. • Focus on the key points. • For quotes, speak more globally about the issue. • Always give the source and timeliness of your stats. It’s the “news peg.” Steve Busalacchi Director, News & Information Wisconsin Medical Society

19. Reporting results to the media Radio and TV: • Do not offer exact statistics – ear cannot track. “73.6% of respondents” vs. “Nearly three quarters of those surveyed” • Don’t go into great detail. Have backup info ready. Steve Busalacchi Director, News & Information Wisconsin Medical Society

20. As you write and talk, remember these tips for effective communications Tailor the message to the topic and the audience Avoid jargon and technical terms Be clear, concise Use active voice Eliminate wordiness Check writing, grammar Be accurate, balanced, impartial Be timely Use graphics, quotes, photos, real stories Consult a communications specialist Write-rewrite-rewrite

21. You’ve spent time and resources doing your evaluation…make sure it isn’t ignored Here are some tips: • Engage stakeholders in doing the evaluation (in the design, data collection, in analysis, interpretation of results), then they “own” the evaluation and are more likely to use it and share it with others. • Get the information to the right people – target people who should care about the evaluation • Address issues that people think are important – while you may think everything in the evaluation is interesting and important, your audience may not. Customize your communications to each audience. • Keep it in front of people; keep talking about the evaluation and what was learned • Be timely: Share information when it is most likely to be used; when the ‘time is right’ and people are ‘ready to listen’.

22. And, remember… “Think like a wise man, but communicate in the language of the people.” − William Butler Yeats