Guidelines for Writing Surveys

1. Guidelines for Writing Surveys • A number of resources available • Survey Research by Backstrom and Hursh-Casar • Survey Research Methods by Fowler

2. Survey Terms • closed question: A survey question that offers response categories. • context effects: The effects that prior questions have on subsequent responses. • open question: A survey question that does not offer response categories. • recency effect: Over reporting events in the most recent portion of a reference period, or a tendency to select the last-presented response alternative in a list. • response effects: The effects of variations in question wording, order, instructions, format, etc. on responses. • screening questions: Questions designed to identify specific conditions or events.

3. General Guidelines • Keep the survey as short as possible • Ask only what you need • The more white space on the survey instrument the better • Typically, surveys begin with demographic items then get into specific research areas • Save the more “delicate” questions for the end of the survey

4. Consistency • Instructions should be provided for the survey and unique types of items • Provide definitions if necessary • The survey instrument should have like types of items grouped together • Keep scales in the same direction

5. Closed Response Questions • Agree–disagree • Forced choice • Ordered response categories or scales

6. Question Order • Question order changes the context in which a • particular question is asked. • Prior questions can influence answers to subsequent questions through several mechanisms. • For example, an obscure “monetary control bill” was more likely to be supported when a question about it appeared after questions on inflation, which presumably led respondents to infer the bill was an anti-inflation measure.

7. Terminology • “Avoid ambiguity” is a truism of questionnaire design. • Language is inherently ambiguous, and seemingly simple words may have multiple meanings. • Research by Belson and others demonstrates that ordinary words and phrases, such as “you,” “children,” and “work”are interpreted very differently by different respondents.

8. Terminology • In a national sample, respondents were randomly assigned to be asked one of two questions: • 1. “Do you think the United States should allow public speeches against democracy?” • 2. “Do you think the United States should forbid public speeches against democracy?” • Support for free speech is greater–by more than 20 percentage points–if respondents answer question 2 rather than question 1. • That is, more people answer “no” to question 2 than answer “yes” to question 1; “not allowing” speeches is not the same as “forbidding” them, even though it might seem to be the same.

9. Leading Questions • A leading question is simply one that, either by its form or content, suggests to the subject what answer is desired or leads him/her to the desired answer (Loftus & Palmer, 1974). • How fast were the cars going when they hit each other?“ • In this question the word hit would be replaced with other words such as smashed, collided, bumped, and contacted. • When the word smashed was used, results showed that the subject estimated the car to be traveling at a faster speed then when the word bumped was used.

10. Don’t Know • Giving “don’t know” as an explicit response option? • On one hand, this has been advocated as a way of filtering out respondents who do not have an opinion and whose responses might therefore be meaningless. • On the other hand, it increases the number of respondents who say “don’t know,” resulting in loss of data.

11. Likert Scales • There are correct and incorrect ways of setting up a Likert Scale • Used to collect data about respondents' feelings or attitudes • The number at one end of the scale represents least agreement, or "Strongly Disagree," and the number at the other end of the scale represents most agreement, or "Strongly Agree." • Ends of a Likert Scale are opposites • Label each scale item (Don’t place one at the top of the page and require the respondent to refer back to it) • The minimum number of points on the scale with descriptors should be both ends and the middle (If an odd number scale)

12. Odd Versus Even Number Likert Scales • When using an odd number of points on the scale (ie: 5 point scale), ends are opposite and the middle is “neutral” • Don’t know or NA is not neutral • When using an even number of points on the scale (ie: 4 point scale), end points are opposite. • Even numbered scales force the respondents to make decisions.

13. Available Choices • When asking a question with boxes to check, provide instructions as to “check one box”, check all that apply, etc. • Be sure to eliminate overlapping categories • How many years have your worked here? • 0-1 year 1-3 years 3-5 years • No possible answer • How many years have your worked here? • 0-1 year 1-3 years 3-5 years • What does someone check if they worked 7 years?

14. Thank You and Return Instructions • Thank the participant • Provide instruction on how to return the survey

15. Pilot Testing • Before a survey instrument is ever used in a real study, it must be pilot tested!!