Survey Research

1. 12 Survey Research

2. Survey Research • Nonexperimental method using interviews or questionnaires to assess attitudes, activities, opinions, or beliefs • Surveys often used to • assess changes in attitudes over time • test theoretical models • describe and predict behavior • To insure high external validity, random samples should be used

3. Survey Research

4. Steps in Conducting Survey Research • plan and design the survey research study • determine what issues you want to survey • determine whether a cross-sectional or longitudinal design will be used • identify the target population and select the sample(s) • construct and refine the survey instrument • collect the survey data • enter and “clean” the data • locate and eliminate errors where possible • analyze the survey data • interpret and report the results

5. Survey Research

6. Cross-Sectional Designs • Cross-sectional studies • collecting data in a single, brief time period • typically from multiple groups in survey research • examples • Whisman (2007) • “Marital Distress and DSM-IV Psychiatric Disorders in a Population-Based National Survey” • a national survey research study with a representative sample of English-speaking adults (18 years or older) in the United States • found that marital distress was associated with anxiety, mood, and substance disorders • the association between marital distress and depression was stronger when one moved from younger to older age groups • Plous (1996) • surveyed APA members to determine members’ attitudes toward the use of animals in research • the majority of respondents approved the use of animals, but wanted to eliminate or minimize the pain experienced by research animals and the number of animals euthanized

7. Longitudinal Designs • Longitudinal studies • collecting data from the same participants at more than one point in time • can be time consuming and expensive • in survey research, longitudinal studies can be called panel studies • type of longitudinal design in which the same individuals are surveyed multiple times over time • example • Moskowitz and Wrubel (2005) • wanted to gain a more in-depth understanding of the meaning of having contracted HIV • participants included 57 gay men, ranging in ages from 24 to 48, who tested positive for HIV • researchers conducted bimonthly interviews over the course of 2 years to identify how these individuals appraised their HIV related changes over time

8. Trend Studies • Independent samples are taken successively from a population over time and the same questions are asked • i.e., same survey questions are asked of different samples over time • example • General Social Survey • conducted by the National Opinion Research Center (at the University of Chicago) • each year, a different sample of U.S. citizens who are 18 years or older are asked questions about many social, psychological, and demographic variables

9. Survey Data Collection Method • Interview • verbal self-report data are collected from interviewees by an interviewer • types • face-to-face or personal interview • advantages • ability to clear up ambiguities and higher completion rate • disadvantage • expense and participants may be uncomfortable discussing private issues • telephone interview • less expensive than face to face and comparable data • can utilize random digit dialing for random samples

11. Survey Data Collection Method • Questionnaire • self-report data collection instrument filled out by research participants • Mail questionnaires • advantage • low cost • disadvantage • low return rate, typically 20-30% • Group-administered questionnaire • advantage • quick and efficient • disadvantage • cannot be used if participants are spread out across locations

12. Survey Data Collection Method • Electronic survey • e-mail and Web-based • advantages of electronic surveys • low cost • instant access to wide audience • data in form easy for analysis • flexible in layout – especially web-based survey • disadvantages of electronic surveys • privacy and anonymity may not be upheld • sample may not be representative of population because of volunteer sampling

13. Constructing and Refining a Survey Instrument

14. Constructing and Refining a Survey Instrument • Principle 1. Write items to match the research objectives • construct items that cover the different areas and content needed to fulfill your objectives • conduct an extensive review of the literature to make sure you have identified all areas that you need to cover • write items and construct a questionnaire that will have the psychometric properties of providing reliable and valid data • content and construct validity are especially relevant

15. Constructing and Refining a Survey Instrument • Principle 2. Write items that are appropriate for the respondents to be surveyed • who will be completing the questionnaire? • you need to consider, empathetically, how your participants will view what you write • don’t use stilted or pretentious language • consider reading level and the demographic and cultural characteristics of your participants • write items that are understandable and meaningful to participants • use natural and familiar language

16. Constructing and Refining a Survey Instrument • Principle 3. Write short, simple questions • survey questionnaire items should be short, clear, and precise • use simple language and avoid jargon • write items that are unambiguous and easy to answer

17. Constructing and Refining a Survey Instrument • Principle 4. Avoid loaded or leading questions • loaded term • a word that produces an emotionally charged reaction • example – “liberal” • may have political connotations even when used in the looking question • “I like a liberal amount of peanut butter on my sandwich” • leading question • suggests to the respondent how they should respond • example

18. Constructing and Refining a Survey Instrument • Principle 5. Avoid double-barreled questions • double-barreled questions ask about two or more issues in a single question • example • “Do you agree that President Obama should focus his primary attention on the economy and foreign affairs?” • Principle 6. Avoid double negatives • double negative • a sentence construction that contains two negatives • example • Do you agree or disagree with the following statement? • psychology professors should not be allowed to conduct research during their office hours

19. Constructing and Refining a Survey Instrument • Principle 7. Determine whether closed-ended or open-ended questions are needed • open-ended question • a question that allows participants to respond in their own words • example • “What do you do most often when you feel depressed?” • open-ended better if researcher is unsure what respondent is thinking or variable is ill-defined • commonly used in exploratory or qualitative research • responses to open-ended questions must be coded and categorized

20. Constructing and Refining a Survey Instrument • Principle 7. Determine whether closed-ended or open-ended questions are needed • closed-ended question • a question where participants must select their answer from a set of predetermined response categories • closed-ended are easier to code and provide more standardized data • example

21. Constructing and Refining a Survey Instrument • Principle 7. Determine whether closed-ended or open-ended questions are needed • mixed-question format • a combination of both open- and closed-ended questions • example

22. Constructing and Refining a Survey Instrument not mutually exclusive mutually exclusive • Principle 8. Construct mutually exclusive and exhaustive categories • mutually exclusive • the categories do not overlap

23. Constructing and Refining a Survey Instrument • Principle 8. Construct mutually exclusive and exhaustive categories • exhaustive • categories include all possible responses

24. Constructing and Refining a Survey Instrument • Principle 9. Consider the different types of closed-ended response categories • rating scales • dichotomous • two choices (e.g., yes and no) • multichotomous • more than two choices (usually preferred) • ability to measure direction and strength of attitude • distance between each descriptor should be the same • anchors • descriptors placed on points on a rating scale

25. Examples  dichotomous multichotomous 

26. Examples of Response Categories for Popular Rating Scales

27. Constructing and Refining a Survey Instrument • Principle 9. Consider the different types of closed-ended response categories • binary forced choice • participant chooses one of a pair of attitudinal objects • can reduce response set • can be difficult for item analysis • typically not recommended

28. Binary Forced Choice Example • Example • Narcissistic Personality Inventory (NPI) (Foster & Campbell, 2007) • used to measure “normal” narcissism in personality and social psychological research

29. Constructing and Refining a Survey Instrument • Principle 9. Consider the different types of closed-ended response categories • rankings • participants asked to put their responses in ascending or descending order • can be open or closed ended • typically rank 3-5 objects • example

30. Constructing and Refining a Survey Instrument • Principle 9. Consider the different types of closed-ended response categories • checklists • participants asked to check all response categories that apply • example

31. Constructing and Refining a Survey Instrument • Principle 10. Use multiple items to measure complex or abstract constructs • variables like gender, weight, or ethnicity can be easy to measure • complex or abstract constructs such as self-esteem, intelligence, or locus of control can be harder • multiple items needed to measure these constructs • semantic differential • scaling method in which participants rate an object on a series of bipolar rating scales

32. Semantic Differential Example • “Occupation and Social Experience: Factors Influencing Attitude Towards People with Schizophrenia” (Ishige & Hayashi, 2005) • measured the participants’ attitudes using 20 bipolar adjectives • adjective pairs used • safe vs. harmful, bad vs. good, fierce vs. gentle, shallow vs. deep, active vs. inert, lonely vs. jolly, simple vs. complicated, dirty vs. clean, distant vs. near

33. Constructing and Refining a Survey Instrument • Principle 10. Use multiple items to measure complex or abstract constructs • Likert scaling • a multi-item scale is used to measure a single construct by summing each participant’s responses to the items on the scale • questions can be positively or negatively worded • statistically analyzed using coefficient alpha

34. Likert Scale Example • Five items each positively and negatively worded

35. Constructing and Refining a Survey Instrument • Principle 11. Make sure the questionnaire is easy to use from beginning to end • ordering of questions • if using positive and negative questions, ask positive questions first • ask interesting questions first to capture participants’ attention • demographic questions last • questionnaire length • questionnaire optimal length unknown • mail questionnaires should be short • telephone interviews should be less than 15 minutes • face-to-face interviews can be longer

36. Constructing and Refining a Survey Instrument • Principle 11. Make sure the questionnaire is easy to use from beginning to end • contingency questions • an item directing the participant to different follow-up questions depending on the initial response • to many can be confusing for participant • example

37. Constructing and Refining a Survey Instrument • Principle 11. Make sure the questionnaire is easy to use from beginning to end • response bias • social desirability bias • occurs when participants respond in a way to make themselves look good • can minimize by insuring anonymity • if using binary forced choice questions, make each choice equally desirable • response set • tendency to respond in a specific way • participant may not want to pick extremes and always choose middle choice • solution: use even number of response categories on rating scale • including multiple question types helps to reduce response set, but can also reduce reliability

38. Constructing and Refining a Survey Instrument • Principle 12. Pilot test the questionnaire until it is perfected • to identify and fix problems • to practice protocols • clear up ambiguity • use think aloud technique

39. Selecting Your Survey Sample From the Population • If primary goal is to explore relationship between variables rather than generalization, convenience sample is acceptable • If generalization to population is needed, a random sampling method should be used

40. Preparing and Analyzing Survey Data • Check for errors • examples • participants answers 7 on a Likert scales that is 1-4 • participant does not answer a question, data missing • Analyze quantitative data with statistical analysis