1 / 5

Surveys

Surveys. Alfred Kobsa University of California, Irvine. When to chose a survey. Scope of problem very clear Questions can be clearly formulated at a concrete level

mrule
Download Presentation

Surveys

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Surveys Alfred Kobsa University of California, Irvine

  2. When to chose a survey • Scope of problem very clear • Questions can be clearly formulated at a concrete level • Complex/holistic answers not needed. Checking pre-formulated answers is sufficient (plus occasionally one or two free-text sentences). • Mailing lists with current addresses available • Specific location not required • Answers from many users needed • Answers needed quickly • Broad geographic distribution of respondents needed • Anonymity required

  3. Steps • Identify the objectives of the study (in writing) • Select the target audience • Select the form of announcement and response collection (email, physical mail, web [☛survey websites], in person, phone) • Decide how to analyze the data (software, statistical tests if any) • Brainstorm questions (closed questions, open questions) • Formulate questions (mind double-barreled, double negatives, leading or loaded questions, and questions with self-image, acquiescence and social desirability bias) • Use existing surveys (e.g. System Usability Scale SUS ) • Consider capabilities of survey site • Reduce question set • Pilot test and revise your questions!

  4. Problems to consider • Non-representative sample population (☛ compare demographics of respondents with demographics of names in mailing list and normalize answers accordingly) • Low response rate (☛ pre-announcement, personalized cover letter, reward, short survey, few open-ended questions, self-addressed envelope w/ stamp, multi-mode reminders) • Self-selection bias of respondents based on topic (☛ hide your study goal among more general questions) • Attention, honesty (☛ reverse-code questions, check consistency, discard data) • Primacy effect for unrelated answer options (☛randomize them) • Self-incrimination (☛ emphasize “harmlessness”, impunity, anonymity) • Social desirability bias, self-image bias (☛ phrase questions carefully, use several questions for same concept, go for facts and not habits, attitudes or intentions) • Sensitive questions (☛ explain their purpose, do not make answers mandatory, offer broad answer bins)

  5. Response format • Closed-ended questions • Single-choice or multiple-choice (include “other” + “explain”) • Ratings (“on a scale from 1 to 5, how would you rate…”) Likert* scale: [1 … n] or [-n -n+1…. n-1 n] (typically n = 5, 6 or 7) • with mid-point or without (forces respondents to take a stance) • with verbal “anchors” for first and last value only, or forall/some values including first and last** • Ranking scales (e.g., sorting by priority) • Open-ended questionsShould be kept to a minimum and optional since • they may cause users not to fill in or abandon a questionnaire • if users answer them, they only provide brief answers (max textbox size) • answers are hard to evaluate *) pronounced like “ike” in the US, but like “lickert” elsewhere (also by Mr Likert himself and his son) **) First and last value anchors should be “extreme”

More Related