PLANNING FOR AN APRM NATIONAL SURVEY: KENYA S EXPERIENCE. BY WAFULA MASAI AFRICAN CENTRE FOR ECONOMIC GROWTH AND SCHOO

1. PLANNING FOR AN APRM NATIONAL SURVEY: KENYA’S EXPERIENCE.BYWAFULA MASAIAFRICAN CENTRE FOR ECONOMIC GROWTHAND SCHOOL OF ECONOMICS, UNIVERSITY OF NAIROBI.

2. OUTLINE Overview Developing the Survey Instruments (4) Nature of the Training and Enumerators’ Field Manual Selection and Training of Enumerators Study Design (sampling approach, field logistics, the technical activities, the budget, field logistics) Data Collection and Processing Activities and Timing Indicative Budget

3. 1. Overview: National Coordinating Structures

4. 2 Overview: Methodological Framework  Pre-field Methodology comprised of information, education and communication activities and included roll-out sensitization activities at provincial administration level Self Assessment Methodology by Technical Agencies Four Principal Instruments: Desk Research, Expert Panel, National Household Survey, Focus Group Discussions Syndicate reports (Trade Unions, Journalists, Professionals, Faith Based Institutions, Umbrella Organizations such as Private Sector Associations, Women’s Organizations, Youth Organizations, etc) Post-field Methodology (experts to interrogate the draft reports regarding integrating findings from the four instruments for each pillar, ensuring a flow in the reporting style, validating the contents of the draft report to ensure credibility, legitimacy, authority and acceptability. The last consultations included the Stakeholder Workshop and a retreat with Permanent Secretaries.

5. 3. Overview: Preliminaries(1) Kenya started by domesticating the NEPAD APRM Master Questionnaire (adding, subtracting, and clarifying some of the questions/issues) This was followed by recasting the domesticated Questionnaire into Survey Instruments (in this case four survey instruments), with each instrument containing questions on the four pillars The instruments included Desk Research (using a good Desk Research Instrument, competent scholars to conduct the survey, and relevant literature providing factual and verifiable information). Expert Respondents (using a good Expert Panel Instrument, carefully identified expert respondents based on criteria of age, education, gender, profession, ethnic background, region, etc., and carefully identified and trained interviewers)

6. Overview: Preliminaries(2) National Household Sample Survey (using the Sample Survey Instrument) Focus Group Discussions using FGD Checklist Carefully selected participants Competent facilitators Memoranda, Reports, and other written submissions from a diverse range of stakeholders

7. 4 Overview: National Survey Planning Team Subject Specialist (for the four pillars of governance) Senior Statistician preferably from the National Bureau of Statistics Data Analyst Logistician

9. (II) Developing the Survey Instrument (1) This was done in close consultation with the Central Bureau of Statistics It was necessary to develop the National Household Sample Survey Questionnaire from the NEPAD APRM Master Questionnaire comprising of: Identification and demographic information for each respondent Questions designed to capture perceptions in a quantifiable manner (Yes/No, Ranking from highest to lowest, selecting options), with minimal open ended questions.

10. Developing the Survey Instrument (2) Questions were asked on each of the objectives in the four pillars (See copy of questionnaire in the folder) Democracy and Political Governance (9 objectives) Economic Governance and Management (5 objectives) Corporate Governance (5 objectives) Socioeconomic development (6 objectives)

11. (III) Training and Enumerators’ Manual (1) The manual focused on principles of interviewing: Role of Enumerator Handling the respondent - including reluctant respondents Confidentiality Neutrality Probing Recording answers Call back procedures Language translation Ending the interview

12. (IV) Training Enumerators and Pre-testing the Questionnaire Selection of Enumerators was on the basis of academic qualifications, experience, fluency in Kiswahili, and region of origin (knowledge of local dialect) Training of Enumerators was based on the Manual and centred around concepts and definitions, and how to ask each question in English, and Kiswahili and how to translate it into the local dialect Survey instrument pre-testing, sharing of experiences from the Pre-Test and revision of questions for clarity and simplicity Learning how to refer to the Survey Manual in the event of not being sure of a question

13. (V) Study Design (1) The Sample for the survey was selected from the Sampling Frame - National Sample Survey and Evaluation Programme III (NASSEP III). The Sampling Frame is maintained and used by the Central Bureau of Statistics of Kenya to conduct household based sample surveys Sample Frame was developed from the 1999 Population and Housing Census and contained 1,133 clusters (of which 930 were rural and 203 were urban), with each cluster having approximately 100 households Each household in the cluster is identified by a number, the name of the household head and the exact village location. There were Cartographic maps to show location of each household in the cluster. The maps and household listings by name of household head are updated every 2 years and were therefore current.

14. Study Design (2) It was decided that all the 69 districts in Kenya were to be covered and hence stratification was on the basis of the districts Going by the district strata, 225 clusters were selected using the Probability Proportional to Size (PPS) Method. Then using systematic sampling with a random start, 10 households were drawn from each of the selected clusters. A sample size of 2250 households was estimated to be adequate for estimates at the national level. Each of the 69 districts was visited with some districts having more clusters and respondents than others depending on population size. District strata were further sub-stratified into urban and rural components taking into account rural-urban population proportions in the district

15. Study Design (3) Replacement of a household in the survey could only be done in consultation with Central Bureau of Statistics Headquarters. In each household, the household head was the respondent and if absent, the next senior member. From the data, it was possible to estimate the population parameters or indicators using proportions, ratios, means, and totals. The estimation process involved multiplying the weighting factor with the sample value per household and summing up the products for all the households in Kenya. Estimation of sampling errors and variance and statistical significance tests were done.

16. (VI) Data Collection and Processing (1) Administration of the interviews was through personal interviews of household heads by enumerators. Interviews were conducted using the language the respondent was more conversant with. Supervisors in the field edited the completed questionnaires (checking for completeness of the questionnaire and consistence of responses)

17. Data Collection and Processing (2) The questionnaires were then dispatched by Courier to the CBS (Nairobi) where they were received using a specifically designed checklist (district, cluster, sampled household) Data entry followed using data entry screens that had been developed and tested earlier. Information captured on the computer was verified and validated using inbuilt controls on variable ranges. Data was then weighted (using selection probabilities from the NASSEP III master frame) to enable estimation of population parameters. Necessary adjustments for population change and non response was then done Analysis involved production of frequency tables for selected variables. This information was then used during report writing

19. 8 Indicative Budget