Designing a questionnaire

1. Designing a questionnaire Dr Fortune Sithole University of Pretoria, South Africa

2. UNIVERSITY OF PRETORIA

3. Definitions • Questionnaire: A data collection tool that can be used in a wide variety of clinical and epidemiological research settings. • Survey: An observational study designed to collect descriptive information about an animal population (such as disease prevalence, level of production etc). Surveys often use questionnaires.

4. Study Types • Descriptive • Case report • Case series • Survey • Analytic • Observational • Cohort • Case control • Cross sectional • Hybrid • Experimental

5. Questionnaire design • Study Objectives • Focus groups • Pretesting • Validation • Types of questionnaires • Designing the question • Open question • Closed question • Wording the question • Methods of administration • Response rate • Data coding and editing

6. Open question • Mostly used in qualitative research • Allows people to express their opinions • Can also be used in quantitative research: • Fill in the blank question eg weight • Unknown categorical data e.g. Breed of cow

7. Closed questions • Closed ended, e.g. checklist, multiple choice, rating scale, ranking questions • Advantages: • Generally easier to answer • Easy to code the responses prior to data entry • Disadvantages: • Can be difficult to design • Can oversimplify an issue • May request information in a way that's different to respondent. E.g. Herd av. Milk production/cow/day VS average 305 day production.

8. Study Objectives • Base your questions on objectives. • Identify predictors (risk factors) and outcomes • Also identify confounders • Your questions should be a reflection of the above. • Avoid asking unnecessary and unrelated questions. • Preferable to use tested questions from instruments that have been used before.

9. Focus groups • A sample of the target population • To ensure that you are asking the right questions • To get all possible options for closed questions

10. Pretesting • After developing questions, read them carefully so they make sense • Give questionnaire to experts • Then pretest it to a sample of the target population • They should demonstrate understanding the questions (non ambiguity) • Record the time it takes to complete questionnaire

11. Validation • Responses from questionnaire could be directly compared with direct measures (e.g. disease status or food consumption) where possible. • Or responses could be compared with those from established questionnaires. • Also you can target groups that are known to have a certain exposure of interest and compare responses between these groups. • If its not possible, assess for repeatability (questionnaire administered to the same individuals at different times)

12. Types of questionnaires • Qualitative • Exploratory • Normally done by interviews • Quantitative • Structured • Used more often in veterinary epidemiology

13. Designing the question • Responding to a question (4 phases) • Understanding the question • Retrieving information (from memory or records) • Thinking or making a judgement if the question is subjective. • Communicating the answer • After drafting a question ask yourself: • Will question be understood? • Will person know answer right away? (could skip question or make up an answer) • How subjective is question (the less the better) • Are possible responses clear?

14. Wording • Clarity (be clear and specific) • How much exercise do you usually get? OR • During a typical week, how many hours do you spend exercising? • Simplicity (avoid technical jargon) • Drugs you can buy without a doctor’s prescription OR • Over-the-counter medications • Neutrality (avoid stereotypes that suggest a more desirable answer) • How often do you drink too much alcohol? OR • How often do you drink more than 5 drinks in one day?

15. Questions to avoid • Double barrelled (use of words ororand) • How many cups of coffee or tea do you drink during a day? • Hidden assumptions (may not apply to all people) • I felt I could not shake off the depression even with the help of my family – yes/no • Question and answer do not match • Q: Have you had pain in the last week? A: never, seldom, often, very often.

16. Checklist • Similar to multiple choice but may not be mutually exclusive • Tick all those that apply • Make sure you have yes/no in each option so that respondents are forced to respond to each. • What type of animals do you keep on your farm? • Cattle (Y/N) • Pigs (Y/N) • Poultry (Y/N) • Sheep (Y/N) • Goats (Y/N)

17. Multiple choice • Should be: • Mutually exclusive (no overlap) • Jointly exhaustive (cover all possibilities) • More than 5 options not recommended. • Type of practice (check one only) • Mixed • Small animal exclusively • Feline exclusively • Other

18. Rating question • Assigning a value on a predefined scale • Likert scale: strongly agree, agree, neither, disagree, strongly disagree • More continuous numerical scale (eg 1-10) • In your opinion, how severe would pain in dogs in the first 12 hours after each of the surgeries if no analgesics are given? • Orthopaedic 12345678910 don't know • Ruptured cruciate 12345678910 dont know • Abdominal .................................. • Castration .................................... • Dental ............................................

19. Rating question • Visual Analog Scale: suited for capturing subjective quantitative information • Indicate by placing an X on the following line how you would rate your work habits with regard to how you milk your cows: meticulous____________________quick

20. Ranking Question • Can be challenging for respondents especially with many options. • Please rank the following as sources of your knowledge of recognition and control of post operative pain in dogs and cats (1 = most important, 6 = least important) • Undergraduate vet school • Post graduate training • Journals • Continuing education courses • Experience

21. Methods of administration • In person interview • Phone interview • Postal • Email • Web based e.g. Survey monkey

22. In person interview • Advantages: • Full explanation of study purpose • High participation rate • Audio visual aids • Builds good relationship with participants • Lower proportion of missing values • Disadvantages: • Time consuming • Expensive • Limited to areas close to interviewers • Subject to interviewer bias

23. Telephone • Advantages; • Share many of advantages of in person • Less time consuming • Less expensive • Less susceptible to interview bias (no visual cues) • More complete reporting of sensitive information. • Disadvantages • Time limitation for responding

24. Postal • Advantages • Less expensive • No interviewer bias • Disadvantages • Low response rate (prone to selection bias) • Inappropriate if respondents have low literacy

25. Web based • Advantages: • Less expensive • No data coding and entry required • Disadvantages: • Suffer some drawback as postal questionnaires • Applicable only to respondents with access to internet • Multiple copies per individual (this can be avoided)

26. Response rate • Maximise response rate by: • A very clear objective • Clear structure and layout • Good pre-test and estimate of completion time • Follow up contact with respondents • Minimise length of questionnaire

27. Response rate • Other factors: • Incentives for completion e.g. T-shirt, cap, enter a draw • Include a pen • Financial incentives – to a certain limit • Stamped return envelope • Advance notice about questionnaire • Personalize questionnaire and cover letter • Deliver questionnaire by hand or courier • Avoid asking for sensitive information unless necessary

28. Data coding and editing • As you develop questionnaire start to think about data entry and analysis (this is very important!!!!!!) • Code your data before entry. • Data entry: • Specialised software: Epidata, Epi-Info (free) – these transfer coded data directly into analysis software. Also, they have feature to ensure quality control e.g. Avoidance of entering illogical values by imposing data ranges. • Spread sheets (e.g. Excel): discouraged especially with large data sets.