Chapter 14 Questionnaire Development

1. Chapter 14Questionnaire Development Winston Jackson and Norine Verberg Methods: Doing Social Research, 4e

2. Questionnaire Design • Questionnaires have many applications and are widely used • The textbook covers: • General guidelines • Types of questions illustrated • Steps and rules for questionnaire design • Rules for ordering, formatting, and presenting © 2007 Pearson Education Canada

3. General Guidelines • Consult the respondent • Able to express opinion freely, opinions valued • Keep it short • Ask only pertinent questions • Achieve precise measurement • Collect data in its rawest form (income in dollars, precise occupation, age to the year, not age category) • highest level of measurement possible © 2007 Pearson Education Canada

4. Types of Questions Illustrated • Pre-coded, single-choice questions • Open-ended questions • Presence-absence questions • Rank-ordering questions • Likert-type questions • Index development © 2007 Pearson Education Canada

5. Types: Pre-Coded, Single-Choice • Pre-coded, single-choice questions ask respondents to indicate which one category applies • Answers are pre-coded for easy data entry • If not all options can be listed, include a category entitled “Other” with a space to indicate what the “Other” category implies • “Please specify” ______________ • Categories should be mutually-exclusive • i.e., no categories overlap with one another © 2007 Pearson Education Canada

6. Pre-Coded, Single-Choice: Example 4. What year are you in? Freshman 1 Sophomore 2 Junior 3 Senior 4 See Box 14.1 (p. 383) for more examples © 2007 Pearson Education Canada

7. Types: Open-Ended Questions • Open-ended questions do not have pre-set answers. Excellent way to explore new areas • Use open-ended questions when: • Too many response categories (year of birth) • You don’t wish to impose categories on respondents • “Really” consulting respondents • Qualitative – source of quotations • Determining appropriate categorization • You want a change in pace for respondents © 2007 Pearson Education Canada

8. Open-Ended: Example 21. What is (or was) your father’s occupation (e.g., supervisor, railway machine shop … supervises work of about 25 people)? Job ________________________________________ Brief Job Description __________________________ ___________________________________________ 23. What is the one thing you would like to see changed at the university? ____________________________________________ ____________________________________________ More examples in Box 14.3 on page 385 © 2007 Pearson Education Canada

9. Open-Ended: Caution • Minimize the number of open-ended questions because they: • Are time-consuming to code • May generate responses that are inconsistent • Are more likely to be left blank • May decrease response rate • Takes more time to complete the questionnaire • Use opinion-seeking questions sparingly • Place open-ended questions at 2/3 mark © 2007 Pearson Education Canada

10. Types: Presence-Absence Questions • Presence-absence questions ask respondents to check off which items in a list do or do not apply to them • Less commonly used than other types of questions © 2007 Pearson Education Canada

11. Presence-Absence: Example 23. Have you ever had contact with a physically handicapped person in any of these groups? (Circle to indicate “yes” or “no” for each group) Yes No Community ----------- 1 0 Family ----------------- 1 0 Relatives -------------- 1 0 Elementary school -- 1 0 High school ----------- 1 0 University class ------ 1 0 As a co-worker ------- 1 0 © 2007 Pearson Education Canada

12. Types: Rank-Ordering Questions • Rank-order questions ask respondents to indicate an ordering of response items, usually from most preferred to least preferred • Must be done with great care • Ask for only three most important items • Must make instructions explicit • These types of questions should be avoided or minimized because they take time © 2007 Pearson Education Canada

13. Rank-Ordering Questions: Example 31. Rank-order the three most important characteristics you want in the job you make your life’s work. (Place a 1 beside the most important one; a 2 beside the second most important one; and a 3 beside the next most important one.) High salary.…………. ____ Security.…………….. ____ Continued interest….. ____ Power……………….. ____ Prestige……………… ____ Excitement………….. ____ © 2007 Pearson Education Canada

14. Types: Likert-Type Questions • Likert-type questions ask respondents to indicate how much they agree or disagree with a statement • Response options originally included: strongly disagree, disagree, are undecided or neutral, agree, strongly agree • Today often used with numbered response options (see example, next slide) • Used in the development of indexes (see Chapter 13) © 2007 Pearson Education Canada

15. Likert-Type Questions: Example In the following, circle a number to indicate the extent to which you agree or disagree with each statement: 52. I believe capital punishment represents the most effective deterrent to murder. Strongly disagree 1 2 3 4 5 6 7 8 9 Strongly agree 53. I believe a murderer can be rehabilitated to become a resonsible, functioning member of society. Strongly disagree 1 2 3 4 5 6 7 8 9 Strongly agree 54. I believe a life sentence is a satisfactory penalty for murder. Strongly disagree 1 2 3 4 5 6 7 8 9 Strongly agree © 2007 Pearson Education Canada

16. Tips for Likert-Style Questions • Avoid the word “and” • Makes the questions multidimensional • Always place “strongly agree” on right side, with 9 indicating strong agreement • Response set, a situation in which the respondent answers similarly to all answers, is best avoided by wording some statements positively, others negatively • Avoid negatives that can confuse respondents • Use direct negative statements © 2007 Pearson Education Canada

17. Tips for Likert-Style Questions (cont’d) • Vary “strength of wording” to produce variation in response • The nursing care I received was good. • The nursing care I received was perfect in every instance • Before the first Likert-type item, provide a brief explanation of how respondents are to to indicate their answers © 2007 Pearson Education Canada

18. Steps in Questionnaire Development • Make a list of variables. Usually includes: • Background variables • Dependent variable(s) • Independent variables • Others: intervening, antecedent, spurious • Anticipate how data will be analyzed • Procedures depend on level of measurement • Write the proposed questions on index cards • Facilitates editing and re-arranging order • Double check: do you have all the variables? © 2007 Pearson Education Canada

19. Steps (cont’d) • Review the conceptual definitions of variables • Done in anticipation of step 6 - developing the wording • Example: how to measure socioeconomic status? • As reflecting various levels of respect and prestige: use Pineo and Porter’s scale of occupational prestige to measure SES • As reflecting variation in access to scarce resources: use total income to measure SES © 2007 Pearson Education Canada

20. Steps (cont’d) • Develop wordings for questions • After considering the conceptual definition, also: • Use simple words: Grade 7 or 8 level • Avoid “and”: make sure only one question • Vary wording to produce variation in replies • Avoid complexity – make it simple • Use existing wording if comparative study • Take the edge off sensitive questions • Be precise, highly specific when choosing wordings © 2007 Pearson Education Canada

21. Steps (cont’d) • Pre-test the questionnaire • Start by filling out the survey yourself, then ask individuals to do so • Ask for feedback on the questionnaire • Is it too long? Any problem with wording? • Pilot study • Send questionnaire to a small sample of respondents • Use data to determine which items will be used for indexes, modify unclear questions, create pre-coded, single-choice questions based on responses to open-ended questions © 2007 Pearson Education Canada

22. Ordering the Questions • Use a brief statement to introduce the survey to the respondent – re: legitimacy of study • Say who is conducting/sponsoring the study • Assure confidentiality and anonymity • Ease them into it • Place easy to answer/salient questions at the beginning • Place key and repeated questions at 1/3 point • Group questions by type • E.g., put Likert-type questions together © 2007 Pearson Education Canada

23. Formatting the Questions • Begin conditioning respondents • E.g., have consistent formatting so respondents can get used to how to answer (format types shown in next two slides) • Anticipate computer data entry • Pre-code questions before data collection • Vary placement of response categories • Response options are appealing and easy to understand • Clearly indicate branching (shown on slide 14-26) • Clear respondents away from irrelevant questions • Make it easy to fill out; will increase response rate © 2007 Pearson Education Canada

24. Four Format Styles Illustrated 1. Babbie Format Style for Fixed-Choice Responses: 23. Have you ever smoked marijuana? [ ] Yes [ ] No 2. Dillman Format Style for Fixed-Choice Responses: Q-22 Your Sex (Circle number of your answer) 1 MALE 2 FEMALE Sources: Earl Babbie (1992) The Practice of Social Research, 6th ed. Belmont, CA: Wadsworth Publishing Company. p. 155. Don A. Dillman (1978). Mail and Telephone Surveys: the Total Design Method. New York: John Wiley and Sons. p. 134. © 2007 Pearson Education Canada

25. Four Format Styles Illustrated 3. Krahn Format Style for Fixed-Choice Responses: 54. In the past year, has any member of your immediate family (not counting yourself) been unemployed (out of work and not looking for work)? No 1 Yes 2 4. Jackson Format Style for Fixed-Choice Responses: 4. What year are you in? Freshman ------------- 1  Sophomore ----------- 2  Junior ------------------ 3  Senior ----------------- 4  Sources: Harvey Krahn (1991). “Sociological Methods of Research.” In Lorne Tepperman and R. Jack Richardson, eds., The Social World: An Introduction to Sociology. Toronto: McGraw-Hill Ryerson. p. 51. Winston Jackson (1995). Methods: Doing Social Research. Toronto: Prentice Hall. p. 388. © 2007 Pearson Education Canada

26. Branching Question Illustrated © 2007 Pearson Education Canada

27. Presentation of Questionnaire • Have a distinctive look • E.g., coloured paper, graphics for questions is appealing • Can increase the appeal of doing the questionnaire • Don’t squeeze too much onto one page • Crowding questions on a page can make the questionnaire appear too long © 2007 Pearson Education Canada