Questionnaire Design Part II

1. Questionnaire Design Part II Disclaimer: The questions shown in this section are not necessarily good or appropriate for a labour force survey. Some are shown as examples of what not to do. Do not take questions shown in this sections as recommendations for a survey.

2. Problems in answering questions • Information sought was never stored in long term memory • Forgetting or other memory problems • Misunderstanding of the question • Translation issues • Inaccurate estimation strategies • Problems in formatting an answer • Deliberate misreporting • Giving socially desirable answers • Failure to follow instructions

3. Solutions Avoid technical vocabulary. Use common terms. Problematic question: How strong is your fertility motivation? Revision: How sure are you that you want another child? Be very specific and use common terms. Don’t make the respondent guess what you are asking. Ask one question at a time. Avoid double barreled questions. Avoid presuppositions. Only ask appropriate questions. Avoid long or complex question wording. Be consistent in the terms you use.

4. Solutions Avoid bias. Don’t make the respondent think they should give a certain answer. Avoid objectionable or sensitive questions. May cause survey drop-outs or misreporting. Use a consistent reference period and repeat it frequently. Include instructions in the lead-in to the question. Provide an appropriate answer format. Don’t ask hypothetical questions. They can’t be answered accurately.

5. Response Formats The way you ask a question leads to a certain type of answer being given. • Design the question to give the form of answer you want. • Answers can be: • Open ended – Respondent answers in her own words. • Closed ended – Answer fits one of the specified formats. • Yes/No questions • Identification questions – give a specific number, amount, year, or category, but categories are not provided • Selection questions – select one of the provided response categories

6. Closed ended responses • Make sure the response categories match the question. • Make the response categories exhaustive and mutually exclusive. • Exhaustive categories cover all possible responses. Mutually exclusive categories do not overlap, so a response fits in only one category. • For selection questions, specify if the respondent is to choose only one or more than one category.

7. Reference Period • Make the reference period explicit • Announce the reference period in the lead-in or in the question. Repeat a brief version of it in subsequent questions. • Use specific boundaries for the time period. • People may understand the same phrase several ways, so being very clear is good.

8. Response Scales • Number of points on scale • Enough for meaningful separation but not so many as to cause confusion. 5 to 7 is probably good. • Even or odd number of scale points? • Odd if there is interest in the middle position. • Even if you want the person to commit to a point to a side of it. • Verbal descriptions for points • People will understand words differently • Vague versus specific quantifiers • Vague - rarely, occasionally, a few • Specific – never, daily, about once a month, 3 to 4 hours

9. Response Scales • More specific descriptions of points can yield more useful data. • How often did you attend religious services in the past year? • Not at all, A few times, About once a month, About two or three times a month, About once a week, More than once a week • Vague terms can mean different things to different people and you don’t know what your respondent meant by them. • Regularly might mean “weekly” to one respondent and mean “every time a relative got married” to another respondent.

10. Response Categories • Should be balanced • Unbalanced- Increase, Stay the same, Decrease a little, Decrease quite a bit, Decrease a great deal • Balanced- Increase greatly, Increase slightly, Stay the same, Decrease slightly, Decrease greatly • Should avoid loaded terms • Should not introduce bias or suggest socially desirable answers

11. Reference Period • Reference periods should be an appropriate length for the subject matter • People have trouble accurately remembering when things happened. • Omission – failure to remember or remembering inaccurately • Rounding/averaging – coming up with an answer that is “good enough” • Telescoping – including more or fewer events in the reference period depending on whether the events are closer to, or more distant from, the present in memory

12. Reference Period • Reducing omissions • Relevant contextual clues or prompts help stimulate accurate memory. • Coping with rounding/averaging • Respondents often use strategies such as multiplication to estimate (and often do it wrong or magnify the degree by which their estimate is off.) • For frequent events, shorter reference periods are better. • Minimizing over/under reports • Keep the reference period short enough that it can be remembered but long enough that the event will be captured.