Multiple Indicator Cluster Surveys Data Dissemination and Further Analysis Workshop

1. Multiple Indicator Cluster SurveysData Dissemination and Further Analysis Workshop What Makes a Good MICS Report? Data Dissemination and Further Analysis Workshop

2. Session Objectives • To provide an overview of the different types of MICS report templates • To gain tips and advice on what needs to be paid attention to in order to use these templates to produce good quality high level reports.

3. The MICS Report Templates There are two templates available in MICS which can be customized by countries depending on the content of the survey: • Preliminary Findings Report • Final Report Templates are/will be available in English, French, Arabic, Russian and Spanish.

4. Why use the MICS templates? • To facilitate the efficient production of MICS reports so countries do not have to “reinvent the wheel” and spend time on the design of tables, indicator explanations etc. • To ensure the production of comparable reports that can be used easily to draw comparisons between countries. • To increase confidence among users in the survey; MICS is a recognised household survey programme with an internationally accepted methodology

5. The Global MICS Report Review Process

6. The Preliminary Findings Report Why do countries produce preliminary findings reports? • To disseminate a few results to stimulate interest in the survey. • To provide quick results on critical indicators such as mortality and malnutrition to allow for possible programme response as soon as possible. • To bide time while working on the final report (In MICS3 countries took over one year from the end of data collection to publication of final reports. Therefore a preliminary report was a useful measure to provide immediate results)

7. What is in the Preliminary Findings Report? • A very short and visual report that does not need a lot of text/writing. • More visual/less text than MICS3 • The tables and topics included in the report are a subset of the final report

8. Preliminary Findings Examples Currently on childinfo: • Iraq • Guinea-Bissau

9. The Final Report What is the Final Report Objective? • To convey the results of the survey to a wide audience which includes government agencies, non-governmental organizations, other multilateral donors, the press, and the general public: The current situation of children and women. What is in the final report? • The report describes the current situation of children and women taken from all indicator results included in the survey. It also includes detailed information on survey methodology and implementation. This information should be sufficient enough for readers to fully understand the methodology and tools used and how the findings are produced.

10. Who should write the Final Report? • Once tables are generated and finalized it should be quite straight forward to write the report. • Typically it is the MICS implementing agency and/or members of a stakeholder/technical committee who write the report. Chapters can be divided among different people. It is useful to have people familiar with subject matter. • Experts in various fields should be consulted and asked to review the tables in order that the key messages and any unusual findings are addressed and dealt with properly in the report and that technical terminology is appropriately used.

11. Content of the Final Report Template

12. Final Report Templates

13. Final Report Examples • Pakistan (Balochistan) • Togo • Serbia • Vietnam • Swaziland • Cuba • And many more…

14. The Executive Summary • In many cases people only have time or interest to read the executive summary, yet typically this part of the report is very weak. • The executive summary needs to highlight the key findings of the survey and present information to a potentially non-technical audience of decision and policy makers. • Poorly written executive summaries are usually a collection of bullet points that are simply a copy and paste of sentences in the main report, regardless of how important those findings are. • Strong executive summaries are those that give a brief description of the survey and go on to provide a short narrative for each theme highlighting indicator results at the national level and then providing the most severe disparities.

15. Introduction • A brief chapter providing background and objectives of the survey. • Relevant MICS indicators that have been excluded, perhaps due to inclusion in other recent surveys, should be mentioned and other sources referenced. • Country specific additions to questionnaires should be included as non-MICS additions.

16. The Sample and Survey Methodology • A well written and comprehensive description of the sample design and methodology will lend credibility to the results even if it raises some possible limitations. • Anyone reading the sample design, even if they have little knowledge of sampling issues, should be able to understand what was done at each stage of the design to select households. • Should not remain generic and only sticking to the template. Issues that arose in fieldwork or issues that may have caused some bias in the final results should be raised. • If a survey has not followed MICS standard protocols and guidelines fully then it is expected that this will be documented in the final report in detail.

17. The Results Chapters - Tables • Change from MICS3 - Tables are now in the results chapters and not at the end of the report. • Do not import all tables into the final report until they have been checked and verified through the global MICS program. From countless experiences, lots of unnecessary double work can be saved! • Export the data from SPSS to the excel tabulation plan first to check the tables format before exporting to the word document. All tables should be run weighted and un-weighted. The size of the un-weighted denominators should be checked for each table.

18. The Results Chapters - Table Conventions • Results based on denominators of 25-49 unweighted cases • should be put in parentheses, e.g. (43.2) • Results based on denominators of less than 25 unweighted • cases should be denoted by an asterisk: * • Background variables can be recoded into fewer categories if necessary i.e. Wealth quintiles – Richest 40%/Poorest 60% • Categories of background variables with very few cases i.e. less than 25 can be omitted from the table with a footnote provided to explain this i.e. • 3 unweighted cases with missing mother’s education not shown • All indicator numbers (both MICS and MDG indicators) should be provided as table footnotes.

21. The Results Chapters - Narrative Text • Begin by consulting other sources of data as much as possible to make sense of results, but do not present the other sources. • It is not recommended to use the report to perform and present trend analysis, but do keep trends and comparisons with other countries/surveys in mind. • Wherever possible, support findings from data with policy interventions – but only to the extent of a couple of sentences. • Indicate if there is an unexpected finding which was also observed in another survey. • It is not necessary to take every table and comment on them one at a time, use a narrative - keep all results of that section from all tables, or from other topics in mind

22. The Results Chapters - Narrative Text • Need to be careful with using background variables to describe data i.e. ethnicity is always of household head which should be borne in mind when describing women and child tables. Wealth index is of the whole household etc. • Always use the right and the same terminology for the variables and their categories names (e.g Wealth Index: The richest and not the richer) • A template provides suggestions for describing findings in the table but can only ever be used as a guide as sentences in the template may not match the country specific data.

23. Example of an inaccurate use of template Template: About [one in five] young women age 15-19 years is currently married ([%%] percent). This proportion [does not vary much] between urban ([%%] percent) and rural ([%%] percent), but is [strongly] related to the level of education. Example from a Final Report About one in five young women age 15-19 years is currently married. This proportion does not vary much between urban (27 percent) and rural (31 percent), but is strongly related to the level of education.

24. The Results Chapters - Narrative Text • When possible, sampling error tables should be used in order to make sure that differences being emphasised in the text from the tables are not within sampling errors. If need be, emphasise instead that results are within sampling errors – in text or in graphs – by using the calculated confidence interval. • Transparency is vital! Should not try to cover up any very irregular or surprising findings. These should be mentioned even if explanations cannot be given - a clause for further analysis and investigation should be included. Do not try to invent possible explanations! • If for any reason tables/indicators are not included in the main report yet the questions did appear in the questionnaire then the reason for their exclusion needs to be stated in the report.

25. Further tips! • It is usually good practice that a single committed individual is coordinating the report writing: It improves harmonisation of text and adherence to deadlines, as well as providing a clear link between table production and text production. • Get people who want to write (and can write). If you can, avoid people who must write. • Keep a list of issues as they appear. This will help keeping track and serve as a memory bank of the many topics for further analysis that always are inspired by report writing. • Try to finalise the chapters one by one. Report writing is straight-forward, but is also known to become delayed. By keeping a structured approach, the process towards a final report flows easier.

26. Further tips! • To increase the quality of the report it is worth considering employing a editor/proof reader. This will also minimise the time spent in reviews by all parties. • A map of the country showing cluster locations and a few photos from fieldwork can give the survey more life. • An important detail: Copying straight from SPSS Output (Crtl-C) to Excel can give problems between some versions (for instance, does not copy over table notes). Always check! • And finally: Take advantage of UNICEF’s Regional Office.