1 / 95

RMK: Survey Research

RMK: Survey Research. Types of Surveys (Q & I) Types of Questions Decisions about Question Content Decisions about Response Format Decisions about Question Wording Decisions about Question Placement Conducting Interviews Advantages / Disadvantages of Methods.

vbarnes
Download Presentation

RMK: Survey Research

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. RMK: Survey Research • Types of Surveys (Q & I) • Types of Questions • Decisions about Question Content • Decisions about Response Format • Decisions about Question Wording • Decisions about Question Placement • Conducting Interviews • Advantages / Disadvantages of Methods

  2. Types of SurveysQuestionnaires and Interviews

  3. Questionnaires Group administration Mail administration Household drop off Web-based

  4. Interviews Personal Telephone

  5. Types of Questions

  6. Type of Questions • Unstructured • Open ended / short comment box / transcript of interview / narrative • Structured • Details in following slides

  7. Structured Questions Dichotomous ____Yes ____No Male Female

  8. Structured Questions Nominal Occupational Class: 1 = Truck driver 2 = Lawyer 3 = etc.

  9. Structured Questions Ordinal Rank the candidates in order of preference from best to worst... ___ Bob Dole ___ Bill Clinton ___ Newt Gingrich ___ Al Gore

  10. Structured Questions Interval Likert Response Scale 1 2 3 4 5 strongly disagree neutral agree strongly disagree agree

  11. Structured Questions Interval Semantic Differential very some- neither some- very much what what much interesting boring simple complex uncaring caring useful useless etc.

  12. Structured Questions Interval Guttman (Cumulative) Scale ___Are you willing to permit immigrants to live in your country? ___Are you willing to permit immigrants to live in your community? ___Are you willing to permit immigrants to live in your neighborhood? ___Would you be willing to have an immigrant live next door to you? ___Would you let your child marry an immigrant?

  13. Filter or Contingency Questions Have you ever smoked marijuana? yes no

  14. Filter or Contingency Questions Have you ever smoked marijuana? yes no If yes, about how many times have you smoked marijuana? once 2 to 5 times 6 to 10 times 11 to 20 times more than 20 times

  15. Filter or Contingency Questions • Try to avoid having more than three levels (two jumps) for any question. • If only two levels, use graphic to jump (for example, arrow and box). • If all responses can’t fit on the page, then jump to a new page. • Electronic advantages (web pages) • Can be programs to move forward in a specific way based on response.

  16. Decisions About Question Content

  17. Decisions About Question Content • Example • Do you need the age of each child or just the number of children under 16? • Example • Do you need to ask income or can you estimate? Is the question necessary / useful?

  18. Decisions About Question Content • Examples: Bad Question • What are your feelings towards African-Americans and Hispanic-Americans? • What do you think of proposed changes in benefits and hours? Are several questions needed? Don’t combine two issues in one...

  19. Decisions About Question Content • If you ask about earnings, might not mention all income • If ask if they’re in favor of public TV, ask for whom (might not mention it’s because of Sesame Street for the kids) Are several questions needed? Did you get all the info desired...

  20. Decisions About Question Content • If you ask research participants about attitudes towards a specific religion, can you interpret the results without finding out about their attitudes towards religion? Are several questions needed? Do you need more context to interpret answer...

  21. Decisions About Question Content • If they say they support public TV … • do they watch it? • would they be willing to have their tax dollars spent on it? Are several questions needed? Need more info to determine the intensity...

  22. Decisions About Question Content • Examples • ‘Do you think Dean Rusk acted correctly in the Bay of Pigs crisis?’ -- can’t say if they don’t know who he is. • In TV surveys, ‘regular’ watchers can be asked about program content, ‘never watch’ should not be asked. Do respondents have the needed info?

  23. Decisions About Question Content • How well did you like the book? Versus... • Did you recommend the book to others? • Did you look for other books by that author? Does question need to be more specific?

  24. Decisions About Question Content • For instance, seasonal specificity. • If you ask what they usually watch and it’s summer, you’ll get a different answer than in the winter. Is question sufficiently general?

  25. Decisions About Question Content • Ask about the benefits of a tax cut without asking about the disadvantages • Ask about the disadvantages of eliminating welfare without asking about the benefits Is question biased or loaded?

  26. Decisions About Question Content • To find out age or income, give brackets. • [18-22], [23-27], etc. • To find out about charitable contributions, ask how much “people you know” typically give. • To find out what magazines they read, offer to buy used ones. Will respondent answer truthfully?

  27. Decisions About the Response Format

  28. Should the Response Format be... • Single option vs. multi-option • Structured • Dichotomous (two possibilities) • Multiple choice • Fill-in-the-blank • Scale - interval (e.g., Likert response format, semantic differential, Guttmann scale) • Checklist – see next slide… • Unstructured • Open ended / free answer / narrative

  29. If Checklist... • are all alternatives covered? • is it of reasonable length? • is the wording impartial? • is the form of the response easy, uniform? • If its possible that there may be some other alternatives, provide “other” option and allow open ended response.

  30. Decisions About Question Wording

  31. Decisions About Question Wording • Questions about ‘nationality’ or ‘marital status are too vague. • Questions about ‘mass media’ are too general. • What kind of headache remedy do you use? • Want brand name? • Want type? (Aspirin, pill, or capsule?) Can the question be misunderstood?

  32. Decisions About Question Wording • If you ask what social class someone’s in, you assume that they know what social class is and that they think of themselves as being in one. • Check assumption in a previous question. What assumptions does question make?

  33. Decisions About Question Wording • Do you think Congress will cut taxes? • Do you think Congress will successfully resist tax cuts? • Neither of these specifies a time frame. Is the time frame specified?

  34. Decisions About Question Wording How personal is the wording? Ensure Appropriate Context for the study • Are working conditions satisfactory or not satisfactory in the plant where you work? • Objective • Do you feel that working conditions are satisfactory or not satisfactory in the plant where you work? • Feeling • Are you personally satisfied with working conditions in the plant where you work? • Personal satisfaction

  35. Decisions About Question Wording • How did you feel about being in the war? - May be too direct • How well did the equipment hold up in the field? • How well were new recruits trained? Is the wording too direct?

  36. Decisions About Question Wording • Does question contain difficult or unclear terminology? • Does the question make each alternative explicit? • Is the wording objectionable? • Is the wording loaded or slanted? Some Additional Issues...

  37. Question Placement and Sequence

  38. Decisions About Placement • Is the answer influenced by prior questions? • Does question come too early or too late to arouse interest? • Does the question receive sufficient attention?

  39. The Opening Questions • These are particularly true in the case of an interview. • Should be easy to answer • Should not be sensitive material • Should get the respondent “rolling” • In a survey, consider • putting basic demographic questions towards the end of a survey because these are easy to complete and thus it may overcome “survey fatigue.”

  40. Sensitive Questions • Only after trust is developed • Should make sense in that section of the survey (not “out of left field”) • Precede with warm-up questions

  41. A Checklist of Considerations (slide 1) • Start with easy, nonthreatening questions. • Put more difficult, threatening questions near end. • Never start mail survey with an open-ended question. • Put demographics at end (unless needed to screen).

  42. A Checklist of Considerations (slide 2) • Avoid demographics at beginning. • For historical demographics, follow chronological order. • Ask about one topic at a time. • When switching topics, use a transition. • Reduce response set. • For filter or contingency questions, make a flowchart.

  43. Important Rule Keep questionnaire as short as possible!

More Related