Multiple Indicator Cluster Surveys Survey Design Workshop

1. MICS Survey Design Workshop Multiple Indicator Cluster SurveysSurvey Design Workshop Overview of the MICS Process

2. Presentation overview • Decision process • Getting started • Survey Plan and Budget

3. Decision to conduct MICS • First step: Why is a survey needed? • In principle, a survey is conducted to satisfy data needs • There should be a clearly defined objective of the survey - how data will be used

4. Data needs assessment • To decide, a thorough data needs assessment needs to be carried out • Assess whether MICS (or another survey) will be the answer to fill the data gap(s) in question

5. UNICEF’s role • On monitoring-related issues, UNICEF works closely with NSOs and other relevant national organizations • UNICEF will be/is working closely with governments to assess data needs, on the basis of the MICS indicator list

6. Decision • If a MICS is decided, the process begins • Participation in the Global MICS Programme

7. Always remember • Survey planning, design, implementation, and dissemination is a continuous process • Decisions made in one area influence others • Some may be irreversible

8. Getting started • Establish steering and technical committees [Template ToRs] • Define roles, responsibilities, and accountabilities: the MoU (Memorandum of Understanding) [Template MoU] • Identify survey coordinator and key personnel [Template UNICEF MICS Consultant ToR]

9. Getting started • Decide on basic features: • Content [model questionnaires, indicator list] • data collecting tools (paper questionnaires or computer-assisted personal interviewing: CAPI) • reporting domains, sample size [sample calculators] • Fieldwork/data processing personnel needed [personnel calculators] • Produce survey plan and budget, including timetable [Template CSP&B available]

10. Steering Committee • Instrumental in full national ownership • Senior staff from implementing agency, UNICEF, other partners in development, relevant line ministries and stakeholders • Meet at critical points during implementation – funding and advocacy, review and approve MoU and survey plan and budget, review survey progress, etc.

11. Steering Committee • Objectives/Tasks • To ensure transparent decision-making • To identify a collaborative approach to fund-raising • To promote understanding for and utilisation of survey and results • To oversee smooth implementation • Appoint/invite technical experts to a Technical Committee

12. Technical Committee • Works under the Steering Committee • Composed of technical staff from organizations represented and other stakeholders • Advise Steering Committee on technical decisions and processes

13. Memorandum of Understanding • Between government and UNICEF (and other partners) • Defines roles and responsibilities, accountabilities, • Access and use of data, • Modes and areas of collaboration

14. Survey Coordinator • Technical person to coordinate and contribute to technical processes • Typically from the implementing agency • Ensures that processes run smoothly, protocols are followed, survey results are understood and used by stakeholders • Coordinate work of other technical staff

15. Other key technical personnel • Senior technical personnel/institution(s) • Sampling expert • Fieldwork coordinator • Data processing expert

16. UNICEF Support • Full commitment to work with government counterparts, support implementation, ensure compliance to protocols • Work closely with the survey coordinator • UNICEF MICS Consultant (under M&E)

17. The UNICEF/MICS Support and Coordination System • UNICEF MICS Consultant / UNICEF Focal Point • Regional Experts: In household surveys, sampling, data processing • Regional Office: MICS Coordinator/Specialist, M&E Sections • UNICEF New York: MICS Team, Statistics and Monitoring Section

18. Review Process • Participation in the Global MICS Programme • A continuous review process is in place with communication lines open [Technical Assistance Framework Presentation] • No one should (or can) do a survey alone • In-country assistance by UNICEF • Review of and feedback to all survey documents, tools, data, and results to ensure quality, timeliness, acceptance, and effective dissemination

19. Reporting Domains • Typically national, urban/rural and regional • Other domains may be defined or desired – special population groups, subnational estimation • Careful assessment of pros and cons needed

20. Country Survey Plan • The Country Survey Plan and Budget is the operationalisation of the MICS • It is the key document that describes all aspects of planning and implementation • It is an effective document of agreement and is often used for fund-raising • It is considered a “live” document that evolves through the planning and implementation

21. CSP should answer…. • Implementing agency, MICS/Survey Team and Governance Structure • Questionnaires • Which modules will be retained, which modules will not be used? • Any issues regarding specific questions or question groups?

22. CSP should answer…. • Sampling • What will be the sample size? [calculator] • Sample frame? Updating necessary? • What will be the general design characteristics of the sample? • What will be the domains?

23. CSP should answer…. • Survey instruments needed • Equipment and supply needs [Supply document available] • Training of Fieldwork Staff • Timing, participants, length [Calculator, model agenda, a.o. tools available] • Fieldwork • Timing, constraints, team composition, duration, logistics, monitoring [Calculator available]

24. CSP should answer…. • Data Entry and Data Processing • Timing, data entry team, duration, monitoring • Data Analysis and Reporting • Timing of production of Summary Findings Report and Final Report • Archiving and Dissemination • Plans for printing, launch and dissemination of the final Main Report and public sharing of SPSS data • The SPSS data and survey documents will be archived using the DDI Metadata Editor (Nesstar Publisher)

25. CSP should answer…. • Budget • How much will the major budget line items cost? • Available funding and funding needs [Budget template available] • Timetable • What is the timetable envisaged for MICS activities? [Timetable template available]

26. CSP should answer…. • Technical Guidance and Support • Detailing information on collaboration with the Global MICS Programme • Major Challenges and Support from UNICEF • What are the major challenges? • What kind of technical support will be needed? In what areas? Who from?

27. Timetable and budget • Golden rules: • Basic measures adhered to (e.g. simultaneous data entry) • Good planning is key to timeliness of production of results • Ensure that all activities are fully funded • A good survey plan of a necessary survey, with national ownership and international support, always attracts funding and is always funded

28. Major timetable components • Preparation, planning, governance, personnel • Adaptation of survey instruments (presentation) • Pre-test of survey instruments (presentation) • Sampling and Listing (presentation) • Main training (presentation) • Fieldwork (presentation) • Data processing* (presentation, workshop) • Data analysis and report preparation (presentation) • Dissemination of results (presentation, workshop) • Archiving and further analysis (workshop)

29. Budget and Timetable …A look at the templates

30. Computer – Assisted Interviewing (CAPI) • MICS started implementing surveys using computer-assisted personal interviewing (CAPI) in 2011 • The CAPI approach uses Tablet Personal Computers, Pocket PC personal digital assistants (PDAs) or smartphones as data collection tools

31. Potential Benefits • Positive improvements, but only with good quality control in place and without underestimating how much work it takes to properly implement a survey. • Checking of data in real time during interview • Considerably reduced levels of missing data • Shorter interview durations • Data ready almost immediately • Survey indicators monitored early in fieldwork

32. CAPI Specific Survey Planning, Design and Implementation Process • Equipment procurement and cost • Needed for all interviewers and supervisors planned for fieldwork, plus IT staff • Extra tablets/PDAs in case of problems/loss in the field • Basic PDAs/Tablet cost approx $450 with no accessories (plus approx $100 in needed accessories)

33. CAPI Specific Survey Planning, Design and Implementation Process • Technical assistance, Preparation and Pretesting • Significant extra programming compared with classic data entry • Full pretest required to properly test all aspects of the system • Programs must be ready and thoroughly tested before training begins • Difficulties in making revisions to the programming once data collection has started

34. CAPI Specific Survey Planning, Design and Implementation Process • Training • Paper questionnaires still important • Train on paper first, then Tablet / PDA • Extensive practice during training is needed • Data transfer and logistics of transferring data to central office • Field staff: computer literate and comfortable with applications • Balance between good interview experience and good computer knowledge

35. Country Survey Plan • Country teams planning or considering a MICS will be required to draft and present a ‘Country Survey Plan’ on the last day of the workshop

36. Country Survey Plan • 5-10 minutes presentations • Followed by short general discussion • There will be preparatory discussions and group work

37. Country Survey Plan • Power point presentation is encouraged • Handouts are optional • Please copy all power point presentations to the workshop laptop

38. Country Survey Plan • Between now and the time of your presentation, facilitators will work with groups of countries both during workshop hours and after the workshop hours, if needed • Bilateral meetings will also be organized to meet with country teams, if needed • Please take advantage of facilitators. That is why we are here