Questionnaire design as related to analysis

1. Questionnaire design as related to analysis Intermediate Training in Quantitative Analysis Bangkok 19-23 November 2007

2. Objectives • Understand the implications of questionnaire design on the analysis • Illustrate examples and detect shortcomings of questions in different questionnaires • Share experience by participants in their surveys

3. Generalremarks • First think about the objective of the study and the analysis ( what and how do you want to know) – then the questionnaire • Difference between kind of survey CFSVA and EFSA • CFSVA- provides baseline information that can feed into monitoring systems, not emergency related information, has more time to be collected and analysed; • EFS(N)A- needs results within short period, 10-15 days vs. 4 weeks plus.

4. Generalremarks • Partner involvement • Partners are important to have a wide buy-in into the results, synergy effects, cost sharing etc. but also might add or change type of information that is collected beyond the need of WFP. • Quality of collected data: • Length of the questionnaire - shorter is usually better- if you don’t sacrifice important details. • Sacrifice details that are not analysed to avoid response fatigue

5. General remarks cont. • Language and translation • Differences between original and translated version (timing) • Do the questions in the original language mean the same thing as the working language? If not- the analyst can mis-interpret the results. • Number of categories for responses used in the questions. • Recode later or maintain the same categories? • Present graph/table with many categories • Homogeneity of Numbering of questions (letters or number)

6. Open vs closed question • Open question • can be answered with either a single word or a short phrase. • Closed question Can be answered with one of the categories/options included in the question ex. What is the major material of the roof? Observe and record. Do not ask question! Circle one 1Straw / thatch 2Earth / mud 3Concrete 4Tiles 5CGI sheet 6Other, specify ____________________

7. Open question • When to use an open question • Names (household head, village, unit measure) • Other (when the category is not included in the question) • Community / focus groups questionnaire • Small survey (max 50 households ) • When not to use an open question • When we have an exhaustive list of categories (crops, livelihoods) • ‘Other’ should not be used as alternative to a category (ex. in Sudan 23% of the pop answer ‘other’ to livelihood activities and we were not able to specify what ‘other’ meant)

8. Closed questions • When to use a closed question • When we know the categories - especially for question related with materials (roof, floor) and questions related with the context (ex. crops, livelihood activities etc.). • When we are not interested in a continuous variable (ex. age) and we want to collect it in categorical variable.

9. Example

11. Continuous vs categorical When to recode a continuous variable to minimize errors • Demography of the households • Land size • Stocks and agricultural production • Other?

12. When to use a continuous variable • FCS (1 to 7) • Number of animals • Proportions (proportional piling) • Expenditure • Measurement (height, weight, muac, child age in months)

13. Yes/no questions • Coding of a yes/ no question • It is helpful to code yes=1 and no =0, this allows the analysis to check the % of yes or no running a simple mean.

14. Skip rule • It is important that the skips for the questions are correct, if not the analyst will have problem in deciding which is the right variable and in the majority of the case he can not use both of them.

15. Skip rule- example

16. Missing values • How to recode missing values • Difference between missing and not applicable. • Be sure that you know the difference in the analysis! • Negative coding • For values as expenditure or income, the value 999 or 888 can be a real value. In these cases might be better to code the missing or not applicable as a negative number -999

17. Household ID • Importance of HHID in linking one module with different modules of the questionnaire and with different questionnaires • ex. household /child / mother, household / village • Importance of the coding (village, cluster, state, community) • It always has to be unique!

18. ID-Section – Darfur

20. Household ID - Exercise • What are the important elements? • How can we ensure this part is done correctly?

21. Modules Now we are going to see some examples of modules of the questionnaire and how they are linked with the analysis

22. Demographics – example of indicators • Average size of household • Number of educated people in an household • Incidence of absenteeism amongst school-going children; enrolment ratio, drop-out • Literacy of household heads • Percentage of male, female and children-headed households • Percentage of disabled/chronically ill in the households • Dependency rate

24. Demographic - issues • Age as continuous or categorical variables? • Age categories should be related to standards • School age, productive members, children, etc. • Polygamy / number of wives • Household size (1.1 & 1.7 should be the same) • Number of categories in the education level • Education of the mother of children as opposed to simply spouse’s education

25. Housing – example of indicators • Crowding (how many people sleep in the house) • Most common building materials used in housing (of floors, roofs and walls) • Availability of toilet facilities and type • Source of lighting, cooking fuel and water • Wealth index

26. Housing example From Uganda database

27. Housing - issues • Pilot testing the questionnaire – it’s useful to explore possible answers to a question • After the pilot the possible answers are included with codes, so that the ‘other’ will not be as necessary • recode the meaning of “other” when you have a lot of them (when the enumerator has entered in a string response) • Exclude the possibility of other for material questions (ex. Housing)

28. Housing - issues • The number of digits should be limited for any figure through boxes |_| (helps in data entry and cleaning) • Distance in km or minutes? • To water source, market, school, health centre - HH vs. community? • One way vs round trip, waiting time and means of transport

29. Agricultural – example of indicators • Percentage of households having access to land • Most common types / methods of land access • Common crops cultivated and amount • Source of seeds • People involved in agricultural activities • Stocks and agriculture production

30. Agriculture - issues • Units of land measurement • Acres, hectares, parcels, etc. • Land size in absolute value or in categories? What is more relevant in the analysis: the mean land size or the division in categories? • Mis-leading cash crop definition

31. Income – example of indicators • Income diversification • The most common activities • Average contribution of each of the income generating activities to a household’s income

32. Expenditure – example of indicators • The most common expenditure items- food & non-food • The average monthly expenditure of a household or per capita for each of the above items • Food /non food expenditure quintiles • Proportion of food expenditure versus non food expenditure

33. Income and expenditure – issues • Proportional piling (100%) • Income in absolute real value or express in categories • The number of digits should be limited for any figure through boxes |_| for data cleaning and entry • Different recall period for expenditures are often used- so this means it’s necessary to carefully calculate the monthly expenditure values in the analysis.

34. Food consumption – example of indicators • Average number of meals an adult and a child ate the previous day • Diet Diversity and Food Frequency • Food consumption profiles • Source of foods

35. Food consumption – issues • Collection of gender disaggregated data (meals per day) • Specify the child age range (infant vs children) • Don’t consider 0 if the household has no children • Rank the sources of food (main and second)

36. Maternal health and nutrition – example of indicators • Percentage of households with children aged between 6 – 59 months • Malnutrition indicators for: • children (waz, haz, whz) • Mother ( bmi) • Incidence of miscarriages / still-births (averaged for the sample) • Percentage of mothers who breast-fed their children • Information on prenatal and antenatal care available and used by mothers • Information on incidence and treatment of diseases such as malaria, diarrhea, fever, cholera, measles, cough etc • Information on prevalent hygienic practices followed

37. Maternal and child - issues • Link mother with child database • Date of birth – local calendar • Fever and diarrhoea separate question • Child size at birth, continuous or categorical? (subjective) • Mosquito net only for the mother or even for the child?

38. Conclusions • Data analysts must participate in the design of the questionnaire to avoid difficulties or missing information in the analysis • Even if PDAs are used, the analyst should carefully examine all the skip rules to be sure the correct information will be collected • The questionnaire designers and enumerator trainers should be involved in the analysis (if the analyst him/herself was not) to be sure the questions are understood by the analyst. • Information should be collected in order to calculate key indicators during analysis- questions that are not necessary in the analysis should not be included.