Introduction to Questionnaire Design

1. Introduction toQuestionnaire Design

2. Questionnaire refers to any list of questions, which can be self-administered by the respondent or read to the respondent by an interviewer

3. Process of Constructing the Questionnaire • Decide on the type of questionnaire that will be constructed. • Write down the questions. • Combine all questions and decide on which order to present them. • Write the introductory statement or cover letter. • Write the instructions to interviewers or respondents. • Design the Form. • If the study is a nationwide survey, translate the questionnaire to other major languages. • Pretest the questionnaire.

4. Strategies in Writing the Questions • Use the research objectives as the basis for your questions. • You may borrow questions that were already asked in past studies. • Anticipate the possible problem that you might encounter in extracting information from the respondents. • Consider the resources available. • Consider the medium used in acquiring the responses • Consider the level of measurement required by the analysis that you have in mind.

5. Types of Questionnaire Structured wording and order of questions are uniform for all respondents Unstructured wording and order of questions can vary for different subjects; usually used for qualitative studies like FGDs, case studies, etc.

6. Types of Questions in Survey Research • Open-Ended Questions • Closed-Ended Questions

7. Types of Questions Open – ended :response categories are not specified; the respondents are free to answer as they please Example: We would like to get your opinion regarding the effectiveness of the project in your barangay. a. What do you think are the strengths of the project? Why did you say so? b. How do you think can the project still be improved to meet its objective?

8. Types of Questions Closed - ended respondent selects one or more of the specific categories provided by the researcher or the question is asked as an open question and the interviewer allocates the answer to the appropriate code category. Example: Do you practice family planning? __ Yes __ No If YES, what methods did you use? __ Condom __ Pills __ Ligation __ IUD __ Injectables __ Natural __ Others, specify ____________________

9. Advantages of Closed-Ended Questions • The answers are standard, and can be compared from person to person. • The answers are much easier to code and analyze, and often can be coded directly from the questionnaire. • A respondent who is unsure about the meaning of the question can often tell from the answer categories what is expected. • Irrelevant responses are avoided.

10. Disadvantages of Closed-Ended Questions • It is very easy for a respondent who does not know the answer or has no opinion to try to guess the appropriate answer or even to answer randomly. • The respondent may feel frustrated because the appropriate category for his/her answer either is not provided at all or is not provided in sufficient detail . • Variations in answers among the different respondents may be eliminated artificially by forced-choice responses.

11. Advantages of Open-Ended Questions • They can be used when all of the possible answer categories are not known, or when the investigator wishes to see what the respondent views as appropriate answer categories. • They allow the respondent to answer adequately, in all the detail he/she likes, and to qualify and clarify his/her answer. • They can be used when there are too many potential answer categories to list on the questionnaire. • They allow more opportunity for creativity and self-expression by the respondent.

12. Disadvantages of Open-Ended Questions • They may lead to collection of worthless and irrelevant information. • Coding is often very difficult and subjective, leading to low inter-coder reliability. • Open-ended questions require superior writing skills, better ability to express one’s feelings verbally, and generally a higher educational level than do closed-ended questions. • Open-ended questions require much more of the respondent’s time and effort, and may engender a high refusal rate.

13. Examples of Pitfalls in Question Construction • Incorrect ordering of questions Q7. How would you rate the scent of lotion X? Q8. What do you think of lotion X? Improvement: ________________________________________ ________________________________________ • Double-barreled questions “Does your department have a special recruitment policy for ethnic minorities and women?” Improvement: ____________________________________ _____________________________________

14. Examples of Pitfalls in Question Construction • Words with double meaning “Does your household participate in the activities of the barangay?” The National Statistics Office defines the household as an aggregate of persons generally but not necessarily bound by ties of kinship, who live together under the same roof and eat together or share common the household food.

15. Examples of Pitfalls in Question Construction • Difficult words or technical terms “What proportion of your evening viewing time do you spend watching news programs?” Improvement: ___________________________________________ ___________________________________________

16. Examples of Pitfalls in Question Construction • Presuming questions “Do you think the Clean Air Act is being implemented properly by local authorities?” Improvement: _____________________________________ ____________________________________

17. Examples of Pitfalls in Question Construction • Sensitive or Threatening Questions “Do you masturbate? Yes _____ No _____ If yes, how often? Once a week, once a month, everyday?” Improvement: ___________________________________________ ________________________________________

18. Examples of Pitfalls in Question Construction • Unrealistic questions “What brand of perfume do you think you will be using three years from now?” Improvement: ___________________________________________ __________________________________________

19. Examples of Pitfalls in Question Construction • Incomplete/non-exhaustive listing “Did you learn about the brand from TV, radio, newspaper, or friends?” Improvement: ____________________________________________ ____________________________________________

20. Examples of Pitfalls in Question Construction • Biased Wording “Do you think that decent, low-cost funerals are sensible? Improvement: ___________________________________________ _________________________________________

21. Examples of Pitfalls in Question Construction • Leading questions “The majority of physicians in the Philippines feel that smoking is harmful; do you agree?” Improvement:__________________________________ ______________________________________

22. Examples of Pitfalls in Question Construction • Double Negatives especially in agree/disagree items “Please tell me whether you agree or disagree with the following statement about teachers in public schools: Teachers should not be required to supervise students in the halls, the lunchroom and school parking lots.” Improvement: ____________________________________________ ____________________________________________

23. Examples of Pitfalls in Question Construction • Dangling Alternatives “Would you say that it is very often, frequently, seldom, or never that you and your boss disagree about advertising campaigns?” Improvement: _______________________________________________ _______________________________________

24. Examples of Pitfalls in Question Construction • Insufficiently Specific or Unidimensional “Are you satisfied with your canteen?” Improvement: ___________________________________________ __________________________________________

25. Examples of Pitfalls in Question Construction • Hypothetical Questions “If you are the president of the Philippines, would you push through with the Charter Change?” Improvement: _______________________________________________ _______________________________________

26. Examples of Pitfalls in Question Construction • Vague questions “How often do you wash your hair?” Very often  Often  Not too often  Never Imrovement: ______________________________________ __________________________________________

27. Example to Avoid Unrealistic Questions Getting information on number of hours spent shopping • Did you go shopping in the past week? (IF YES) How many hours did you spend shopping in the past week? • On a typical week, how many hours did you spend shopping? Or, on the average, how many hours do you spend shopping in a week? • Do you go shopping? (IF YES) On your last shopping trip, how many hours did you spend shopping?

28. Use of Simple Words • WORD SIMPLER WORD • acquaint inform, tell • assist help • consider think • initiate begin, start • major important, chief, main, principal • materialize come about, happen, occur • purchase buy • sufficient enough • terminate end

29. Exercise: Evaluate the following statements: • According to the Chief Justice of the Supreme Court, the court should be free of bribery and corruption. • Do you think that the Congress, as an institution, is definitely free from external controls, influence or pressure? • Does the government agency have criminal and civil cases? • What is the nature of the complaint against the Chief of the Division? Have you ever filed an administrative complaint against a Division Chief? • Do you accept money for a favorable decision? If yes, how often? • Approximately how many administrative cases did you have in your agency from 2005 to 2008?

30. Filter and Contingency Questions A contingency question is a question whose relevance to the respondent is determined by the response to an earlier filter question. • There are some questions that are not relevant to all of the respondents in the study, particular if the population under study is very heterogeneous. • Instead on using separate questionnaires for the different subgroups, only one questionnaire containing filter and contingency questions is used.

31. Possible Format of Filter and Contingency Questions • Instructions telling which to answer next in the event of the category being chosen are placed beside each response category. In the past two months, did you have any illness that kept you in bed, indoors, or away from your usual activities? Yes …………(ASK A)……………1 No…………..(SKIP TO Q.25)……2 Or Yes…………………….1 ASK A No ……………………. 2 SKIP to Q.25

32. Possible Format of Filter and Contingency Questions • Used boxed contingency questions with no arrows leading to them. Have you ever had a miscarriage? ( ) Yes ( ) No If YES: How old were you when you had your first miscarriage? ____________

33. Response Category Format • Response categories should be exhaustive. • Response categories should be mutually exclusive. • Response categories should be made easy to data collectors, respondents, or coders to select appropriate options. • Response categories should include the “Other” option with sufficient space for writing answers. • Response categories should anticipate analytical needs and enable the collection of data that are suitable for those analyses. • It should be decided whether single or multiple answers are to be allowed.

34. Response Category Format Number of Categories • There is no optimal number of categories. • 5 to 6 categories are commonly used Odd or Even Number of Categories • If odd, usually the middle category is the neutral position. • If even, there will be no neutral category. Thus, the respondent is forced to take a position.

35. Some Types of Category Formats Verbal rating scale • This requires a respondent to indicate their position by selecting among verbally identified categories. • Examples: Strongly agree, Agree, Neither Agree nor Disagree, Disagree, Strongly Disagree Very important, Important, Somewhat Important, Not at all Important Excellent, Good, All right, Poor, Bad

36. Some Types of Category Formats Graphic Rating Scale • This requires a respondent to indicate his position on a continuum which ranges from one extreme of the attitude in question to the other extreme. • Since the respondent is not using a predetermined set of response categories, the researcher cannot determine how a particular respondent has subdivided the continuum. • Example: “To what extent do you feel employees should have a voice in determining performance evaluation content?” Employees should make Performance evaluation matters the decisions about 0 1 2 3 4 5 6 7 8 9 10 should be left entirely performance to the Administration evaluation matters

37. Some Types of Category Formats Rank-Order Scale • This involves having the respondent rank various objects with respect to the attitude in question. • This is actually a forced scale. • It is possible that respondent actually dislikes all of the objects they were asked to rank according to a favorable characteristic. This attitude is not captured by the scale. • Example: The following are some of the problems faced by the local government. Please rank them in terms of importance, from 1 (most important) to 5 (least important). _____ Pollution _____ Garbage _____ Traffic _____ Drug Addiction _____ Crime

38. Some Types of Category Formats Checklist Question • provides more than two alternatives from which the respondent can choose as many responses that apply to them. • Example: What kind of ball sports do you like to play? basketball volleyball football tennis baseball water polo others, pls. specify _____________

39. Question Order • The order of questions should be in logical sequence so that the trend of thought of the respondent is not broken. • The first part of the questionnaire are on Identification Information consisting of the geographic identification, interviewer’s record and the information about the sample. • Sensitive questions are placed towards the end of the interview schedule or questionnaire.

40. Approaches to Question Sequence Funnel Approach • Uses a general-to-specific question flow. Work Approach • Requires that difficult to answer questions are placed deep inside the questionnaire. Sections Approach • Arranges the questionnaire into sections or topic category divisions that are dictated by the objectives of the study

41. Approaches to Question Sequence Quintamensional Design • This 5-question approach was developed by George Gallup to cover the most essential features of an opinion: • Open-ended question on awareness/knowledge of the issue • Open-ended question on the attitudes toward the issue • Closed-ended question on attitude towards specific issues • Open-ended question on reason behind attitude • Closed-ended question on the intensity of attitudes

42. Questionnaire Lay-out & Design • Flow of questions should be respondent-friendly . • Vary questions by length and type. • Place scale items according to response required. • Separate reliability-check question pairs.

43. Physical Design of a Questionnaire • The appearance of the form should be simple and well spaced; • The size and type of font used should be readable; • The font size and type used for questions and directions should be different.

44. Basic Contents of the Cover Letter/Introduction • Identification of the persons or organization conducting the study • Explanation of the importance of the study and reason/s why it is being conducted • Reason why the respondent should be the one to answer the questionnaire • Assurance of no right or wrong answers • Guarantee the confidentiality of answers

45. Example of an Introduction Good morning/afternoon/evening. I am Josie Almeda from the UP School of Statistics Research Foundation. This study seeks to know how Filipinos feel about various issues and problems affecting their lives. This information is valuable both for planning and for scientific research. Your household is very important to this survey because it represents hundreds of others which are not in our sample. Everything you tell us will be strictly confidential. Your name will be in no way connected to the findings of this important study.

46. The Pretest Objective: To determine if there are errors in the questionnaires or areas that need improvement such as ordering of questions, wording, and the natural flow from one section to another. Respondents: No rules on who the respondents should be in the pretest and on the size of the sample.

47. Types of Pretest Participating Pretest • Respondents are told that this is a practice run, and are asked to explain their reactions and answers. • This allows for a very detailed probe on certain questions such as: What did the whole question mean to you? What did the (term) make you think of? Undeclared Pretest • The respondent is not told that this is a questionnaire under construction, and the interviewer plays it straight.

48. Evaluating the Pretest • Before the pretest, tell interviewers to write marginal comments on the questionnaire. • Conduct an oral debriefing after the pretest so that the interviewers can report on their experience. Some of the issues that can be discussed are: • Were there questions that made the respondent uncomfortable • Which questions have to be repeated • Did the respondent misinterpret any question • Which questions were the most difficult or awkward to read • Did any of the sections bore the respondent • Were there questions that respondent found stupid • Were there any section in which the respondent would have liked the opportunity to say more

49. Evaluating the Pretest • Review answers to open-ended questions • Tabulate the number of no responses, unable to answer, and don’t knows for each question. • Investigate questions that yield too many such answers. • Watch out for response sets. • Look at the variability of responses.

50. WORKSHOP Write a short questionnaire to measure employees’ job satisfaction on their current position. Incorporate topics on the employees’ preferences. Put questions regarding the personal characteristics of employees like age, sex, department, position, and others. Maximum of 15 questions only. Keep in mind the guidelines on questionnaire construction. Include a question that allows for multiple response.