Training Questionnaire design :

1. Training Questionnaire design : March 2009

2. Objetives of the training • Objectives of the session is to understand: • Objectives of questionnaires • Advantages and disadvantages • Design of questionnaires • Type of questions used • Common problems and pitfalls

3. Training plan: • Presentation plan: • Types of questionnaires • Stages for design • Presentation and layout • Type of questions • Closed questions • Problems and pitfalls • Additional information Intervention Impact

4. Training questionnaire design I:Types of questionnaires

5. 1.0 Stages of a survey design • Establish the goals of the project • Determine your sample - Whom to interview • Choose interviewing methodology - How to interview • Create your questionnaire

6. 1.1 What is a questionnaire • A series of written questions/items in a fixed, rational order • A questionnaire is an instrument (form) to: • Collect answers to questions • Collect factual data • Gathers information or measures • A well designed questionnaire: • Gives accurateand relevant information to your research question • Minimises potential sources of bias • Will more likely be completed

7. 1.2 Advantages /disadvantages • Provides only limited insight into problem: • Limited response allowed by questions • Maybe not the right questions are asked • Varying response: • Misunderstanding/misinterpretation • Need to get it right first time: • Hard to chase after missing data • Can reach a large number of people relatively easily and economically • Provide quantifiable answers • Relatively easy to analyse

8. 1.3 Types of questionnaires Interviewer-administrated Vs Self-administrated • Face to Face: • Telephone: • By post: • Email/internet:

9. 1.4 Advantages /disadvantages • Self-administered questionnaire: • Advantages: • Cheap and easy to administer • Preserves confidentiality • Completed at respondent's convenience • No influence by interviewer • Disadvantages: • Low response rate • Questions can be misunderstood • No control by interviewer • Time and resouces loss • Interview-administered questionnaire: • Advantages: • Participation by illiterate people • Clarification of ambiguity • Quick answers • Disadvantages: • Interviewer bias • Needs more resources • Only short questionnaires possible • Especially on telephone • Difficult for sensitive issues

10. 1.5 Choice of the type of questionnaire • Choice of the questionnaire type will depend on several factors such as: • Speed • Cost • Internet Usage • Literacy Levels • Sensitive Questions

11. Training questionnaire design II:Stages for design

12. 2.1 Stages in designing the questionnaire I • Design your survey plan: • Decide on goals: • Identify risk factors for population affected by the earthquake • Know the subject: • Lessons learnt, secondary data review • Formulate a hypothesis: • People affected by the earthquake will face food security and water access issue in the coming weeks • Define information needed to test hypothesis: • Current access to water, changes in food consumption, etc..

13. 2.2 Stages in designing the questionnaire II • Determine study population: sampling • Know the respondents: • Occupation • Special sensitivities • Education • Ethnicity / Language Questionnaire needs to be adapted to your population, not the opposite! • Design variables and questions: • Content of the questions • Format of the questions • Presentation and layout • Codingschedule (if appropriate) • Pilot and refine questionnaire

14. Training questionnaire design III:Presentation and layout

15. 3.1 Presentation and layout • Clear consistent layout: • Adequate space to answer / Large font size • Appropriate page breaks • Avoid • experimental layouts • fancy logos • printed on recycled paper/is an equal opportunity employer etc • Using colour or printing questionnaire on coloured paper may help • Use filter questions, if necessary • Give clear instructions about how to answer the questions (design guideline if necessary)

16. 3.2 Good practice for layout • Good appearance / easy on the eye • Short and simple • Relevant and logical • High response rate • Easy data summarisation and analysis

17. 3.3 Organizing questions • Decide on order of items/questions : • Easy  difficult • General  particular • Factual  abstract • Where to place sensitive questions? Be aware of ordering effects! • Group questions by topic/ response options • Starting questions: • Simple • With closed format • Relevant to main subject • Non-offending • Neither demographic nor personal questions Don’t put most important item last!

18. Training questionnaire design IV: Type of questions

19. 4.1 Contents of questions • Clear focus on research question • Avoid sidetracking • Avoid unnecessary information • Demographic information • Contact information (if non-anonymised)

20. 4.2 Format of questions • Adjust to responding audience: • Professionals vs. public • Middle class vs. prisoners • Keep sentences simple and short • Define key words (“fully vaccinated”) • Remember option “don’t know” • Ask forone information at a time: • Use mutually exclusive and exhaustiveanswer options • Vertical order of answer options Do you own cattle or have frequent contacts with cattle? Yes  No 

21. 4.3 Be accurate • Do you often touch cattle? Yes  No  • How often did you touch a cattle during the past 3 months? Once  Twice  Three times or more  Not at all  Don´t know  The range of error due to use of imprecise words may be as high as 20 to 30 percent.

22. 4.4 Be objective • Did you drink the strange brownish drink in Rumania? Yes  No  • Which beverage did you consume? Water  Beer  Wine  Karkadé  None of them  Don´t know 

23. 4.5 Be simple • Did you smoke not less than a mean amountof 7 cigarettes/2 days from 1999 onwards? Yes  No  • Did you smoke an average of 2 pack of cigarettes/week for the last 5 years? Yes  No  Don´t know 

24. 4.6 Bias • Bias = systematic differences in the measurement of a response • Recall bias: • Cases more likely to remember than controls • Observer bias: • Different interviewer – different interpretations • Different interpretation of similar questions • Reduce by structured questionnaire • Non response bias: • Those who respond are different from those who do not • Telephone interviews: more females, elderly • Reduce • Ensure high response rate • Random choice of interview partners • Correct during analysis (eg age, sex)

25. 4.7 Format of questions Two main question formats: Closed format  forced choice: • Yes  • No  • Don’t know  • Always  • Sometimes  • Never  Open format  free text: • Why do you feel concern about food accessibility during the next month? Please describe:__________________________________________________________________________________________________________________________________________________________ • Multiple choice • Numeric open end • Rating scale • Agreement scale • Text open end

26. 4.8 Closed or Open questions? • Advantages: • Not directive • Allows exploration of issuesto generate hypothesis, qualitative research, focus groups, trawling questionnaires • Used even if no comprehensive range of alternative choices • Good for exploring knowledge and attitudes • Detailed and unexpected answers possible • Disadvantages: • Interviewer bias • Time-consuming • Coding problems • Difficult to analyse! • Difficult to compare groups • Advantages: • Simple and quick • Reduces discrimination against less literate • Easy to code, record, analyse • Easy to compare • Easy to report results • Disadvantages: • Restricted number of possible answers • Loss of information • Possible compromise: • Insert field “others“

27. Training questionnaire design V: Closed questions

28. 5.1 Closed questions Straightforward response: What is your age in years? ___ years How long have you owned a cheap? ___ years What is your sex (gender)? Male  Female Did you stay in your house when the earthquake occurred? Yes  No  Don’t know 

29. 5.2 Closed questions Checklist: Which of the following agricultural activities did you do last week? • Buying seeds  • Preparing your field  • Planting  • Harvesting 

30. 5.3 Closed questions Rating scale - Nominal: Did you do use soap during the following domestic activities during the past six months? Always Sometimes Seldomly Never • Cooking     • After toilets     • After fieldwork     • Indicate your sex: ___Male ___Female

31. 5.4 Closed questions • Rating scale – Numerical: How useful would you think that information on the risk of being strike by earthquake? (please circle) 1 2 3 4 5 6 7 Not at all useful Very useful • Analogue: How much is your food insecurity severe (put the tick on the line) 0 10

32. 5.6 Closed questions Scales for measuring attitude (Lickert) Stray dogs carry a higher risk of rabies?: • No, I strongly disagree • No, I disagree quite a lot • No, I disagree just a little  • I’m not sure about this • Yes, I agree just a little • Yes, I agree quite a lot • Yes, I strongly agree

33. 5.7 Coding schedule • Questionnaire can be pre-coded • Quicker and easier data entry • Examples: • Male  1 ill  1 • Female  2 Not ill  0 • Don’t know  3 Don’t know  9 • Single  1 Separated  3 • Married  2 Divorced  4 • Widowed  5 Don’t know  9

34. Training questionnaire design VI:Problems and pitfalls

35. 6.1 Problems and pitfalls • Avoid questions that ask two things at once - you won’t know which ‘bit’ people are answering: • Have you ever had stomach ache and diarrhoea? • Avoid ambiguity..... • Do you go to the woods a lot? • Avoid jargon/abbreviations/slang: • How often do you get up at night to PU? (pass urine) • Should IVDUs be treated in the community?

36. 6.2 Problems and pitfalls • Avoid not mutually exclusive options: What age are you? • 16-20 • 20-25  • 25-30  • 35-40  • Avoid leading questions Do you think that the food in the hotel made you sick? Did the hotel staff seem unhygenic to you? Do you agree that the hospital staff were close to exhaustion? • Typographical / spelling errors

37. 6.3 Length of a questionnaire • Sufficient to capture needed data • Short enough to hold participants’ attention • Type of survey affects length • Types of questions affect length • Quantitative/Qualitative/Mixed approach affects length • Participant characteristics affect length

38. 6.3 Questionnaire validation • Use or adapt existing questionnaires • Validated (and possibly harmonised) • New questionnaires • Not validated • Needs to be tested (pilot) • Pilot with a similar group of people to your intended subjects • Highlights problems before starting • Effects of alternative wording • Overall impression on respondents and interviewers • Final polishing after several amendments

39. 6.4 Questionnaire introduction • Covering letter/ interview introduction: • Who you are/ you work for • Why you are investigating (purpose of the survey) • Why it is important to hear from the respondent • What may be done with the results and what possible impacts may occur with the results. • Where you obtained the respondent’s name • How and where you can be contacted • Guarantee of confidentiality • Length of interview (be honest) • Due date for response Usefulness of study should be clear to all respondents

40. Training questionnaire design VII:Summary

41. 7.1 Designing a questionnaire • It’s all about the questions… • Effect of a given word • Balance in question wording • Don’t know answers – offered or volunteered? • Using scales • Question order • Pre-testing

42. 7.2: Designing questions • When devising your questions consider the following: • What is essential to know? • What would be useful to know? • What would be unnecessary? • Retain the former, keep the useful to a minimum and discard the rest

43. 7.3 Best practices A well designed questionnaire: • Will give appropriate data which allow to answer your research question • Will minimise potential sources of bias, thus increasing the validity of the questionnaire • Will much more likely be tested and completed • FINALLY, keep your questionnaire short and the questions simple, focused and appropriate

44. Training questionnaire design VIII:Additional information

45. 7.1 Documentación Questionnaire design • Survey method: