Communication and Use of Evaluation Results

1. Communication and Use of Evaluation Results IbnSina IPHMS March 2009.

2. Why Do We Evaluate a Program? • To measure effectiveness • To ensure resources are being used to achieve planned results • To propose changes • To identify new directions and needs

3. What Do We Need To Do After an Evaluation Share information about analyzed data with identified stakeholders. HOW? Use Effective Communication Strategies to help stakeholders Understand the results and Use them.

4. Uses of Evaluation Results • Instrumental - to yield decisions • Conceptual – to generateideas and understanding • Symbolic/ Persuasive – to justify what decision makers want

5. Evaluation Findings May Be Used • For planning future interventions • For making changes or corrections to program, midterm • For lessons learned • For information

6. The Evaluation TOR • Communication and dissemination of the evaluation report will essentially follow the requirements and modalities agreed on between stakeholders, in the evaluation Terms Of Reference .

7. 4 Strategies for Effective Use of Findings and Recommendations: • Be intentional and purposeful about reporting • Focus reports on primary intended users • Share findings first in draft form (don’t surprise stakeholders) • Think positive about negatives

8. Presenting Evaluation Findings • Frame findings and recommendations as ‘lessons learned’ and ‘best practices’ • Use clear jargon-free style, • Use tables and figures to make information more understandable, • Communicate negative findings so that they can be used productively,

9. Presenting Findings of Evaluation…contd. • Summarize findings and interpretations appropriately for different audiences, • Ensure the format of each communication/report facilitates easy interpretation and assimilation of its contents.

10. Communication Strategy No one dissemination strategy is suited to all situations. • Identify the most effective means for disseminating findings and recommendations to various key stakeholders and audiences, so that their use is guaranteed within limits of time and budgets.

11. The ‘How To’ • You may have several intended audiences, each with different interests and preferences. • Before writing your report, consider the needs of your audience.

12. Be concise! • Content is not always best shared in long and complicated reports. • Three concise pages may have more impact than a fifty page report.

13. Simplify • Instead of a large document describing a complex set of ideas, consider dividing results into several smaller reports. • In determining your approach consider the easiest and clearest way to present information to your key stakeholders.

14. Method of Reporting Should Meet the Need • A report may be a simple verbal presentation, with supporting charts, case studies or targeted qualitative results. • Or may require more details such as the goals and history of the program, methods of evaluation, findings, interpretation, and conclusions and recommendations.

15. Parts of a Standard Evaluation Report… • Cover or Title Page • Table of Contents • List of Abbreviations • Executive Summary

16. …Parts of a Standard Evaluation Report • Introduction/ Background • Methodology • Findings • Conclusions and Recommendations • Annexures

17. What to Include… • Description of program, its goals, target population, and activities • Overview of evaluation questions • Description of evaluation participants (sample size, strategies used to obtain consent)

18. …What to Include • Methods and procedures used for collection and analysis of data • Outline of strengths and limitations of evaluation methodology

19. …What to Include • Evaluation findings including: -characteristics of participants -services provided by program -results for each major evaluation question.

20. …What to Include • Conclusions and Recommendations: - strengths of program revealed by evaluation findings - recommendations for improvement of services - other implications of findings such as policy implications - suggestions for improving future evaluation activities.

21. Tips for Writing Evaluation Reports Organize information – The aim in organizing is that readers can easily understand the structure of the report, and find the information that interests them.

22. To Organize Information • Organize by evaluation issues rather than by data collection. This will enable readers with different interests locate information quickly

23. To Organize Information • Provide an introduction that summarizes the format and content of the report • Use consistent headings and subheadings to help the reader.

24. Identify Key Findings In a long report, make your key findings stand out – accord them emphasis. • Within each section start with the most important information. • Present key findings in text and in tables • Use visual emphasis • Restate key findings in executive summary and conclusions.

25. Use an Executive Summary • For long reports, an executive summary can ensure that all kinds of readers can access significant information. • A shorter and simpler presentation ensures information is accessible to a wide range of readers.

26. Use of Appendices • To keep your report short and focused, consider using appendices for supplemental information. • Significant information should not be located in an appendix.

27. Use of Appendices Appendices can provide supplemental information to interested stakeholders, such as: • Additional information about program • Copies of focus group protocols, surveys etc. • Copies of consent forms, additional tables or figures.

28. Use Simple and Interesting Format • Keep report visually interesting • Leave “white space” – do not crowd the page • Use consistent stylistic and visual features.

29. Use Simple and Interesting Format • Use fonts, of legible size and shape • Use bullets and sidebars to emphasize key information.

30. Use Straightforward and Interesting Language… • Use familiar words rather than jargon • Use active verbs as much as possible • Delete unnecessary words and phrases • Keep sentences and paragraphs short

31. How Not To Write 1- Imagine a mental picture of someone engaged in the intellectual activity of trying to learn what the rules are for how to play the game of chess. 2- He was called to the office to be told about his father’s death by the headmaster.

32. How Not To Write 3- It is a tricky problem to find the particular calibration in timing that would be appropriate to stem the acceleration in risk premiums created by falling incomes without… aborting the decline in the inflation-generated risk premiums.

33. How Not To Write 4- His style of writing is magniloquent.

34. …Use Straightforward and Interesting Language • Keep sentences and paragraphs short • Avoid expressions that may be unfamiliar to readers across cultural communities (slang or idiomatic expression)

35. Straightforward versus Complex Language What do you think the following sentence means? It is indeed a not unsupportable inference that we have been unsuccessful in our attempt to forward the proposal to the proper agency in advance of the mandated date by which such proposals must be in receipt.

36. Be Objective • Refrain from any appearance of taking sides • Report both positive and negative findings • Discuss limitations of the evaluation fully and honestly

37. Be Objective • Be clear about what is a statistical finding versus your own interpretation. • Avoid emotionally charged language

38. Disseminating information • Why dissemination • Who (who is responsible) • What (what to disseminate, objective) • For Whom (select audience) • How (method)

39. Dissemination channels • Detailed written report • Summaries of evaluation findings • Brochure on lessons and recommendations • Annual report • Article in technical or organizational newsletter • News release • Press conference and Media appearance • Public meeting, public debate • Seminar, workshop, or group discussion • Electronically (e- mail, Internet, websites)

40. Effective Dissemination Strategies • As evaluators disseminate findings and recommendations in the most effective ways possible. WHY? • Because “evaluations are conducted primarily to provide decision makers with informed options for improving programs”.

41. Evaluators Should… • Report regularly and frequently, in person if at all possible, and target specific audiences and issues, • Focus on key points (simplify, simplify), • Know the audiences (tailor information to the audiences),

42. Evaluators Should…contd. Focus on actions that can be taken by the targeted audiences (doable recommendations), • Report in many different ways.

43. Review and discuss Results with Stakeholders • Before finalizing an evaluation report, consider reviewing results with other staff and colleagues. • Discussing a draft version of the report can provide additional viewpoints regarding meaning of data

44. Guidelines for Dissemination, to Increase Usefulness of Evaluation Results • Develop realistic recommendations focusing on improvement of evaluated program • Explore multiple uses of study data • Constantly remind decision makers of findings and recommendations • Share findings and recommendations with broad audiences.

45. Consider Creative Strategies for Disseminating Results Be innovative in reporting findings. • Use a variety of techniques such as visual displays, oral presentations, interim reports, and informal conversations.

46. Creative Strategies for Disseminating Results Other creative strategies may include: • Making a short video presenting the results • Sharing results with the media, through a press release and/or press conference

47. Creative Strategies for Disseminating Results • Making presentations to select groups • Writing separate summaries and articles targeting specific audiences • Staging a debate in which opposing points of view can be fully aired.

48. Effective dissemination of lessons learned can: • Help stakeholders at different levels understand relevance of other activities and achievements, thus improving collaboration and co-ordination • Inform decision makers to make changes and corrections, and avoid common mistakes, thus promoting a more enabling environment • Allow other practitioners to learn from previous experience and avoid ‘reinventing the wheel’

49. Acknowledgements • Communicating Evaluation Results - Issue 14, April 2008. • Bureau of Justice Assistance– Reporting and Using Evaluation Results – http://www.oip.usdoj.gov/BJA/evaluation/guide/ru2.htm • W.K. Kellog Foundation Evaluation Tool Kit: Reporting – http://www.wkkt.org/Default.aspx?tabid=90&CID=281&ItemID=2810021&NID=2820021&LanguageID=0