Surveys: What Are They?

2. Surveys: What Are They?

3. Surveys: What Are They Good For? A survey is a collection of questions asked repetitively to a sample of a population to mathematically derive characteristics of the total population.

4. Why is This Cycle Important? It’s a framework It provides guidelines and reminders as you work with clients and stake-holders You’re likely doing parts of it already Those are likely the parts of your process that work!

5. Great Survey Design Cycle

6. The Trifecta: Need, Design & Act

7. Unit 1: Need

9. Set Your Goals and Objectives

10. Needs: We All Have Them • Questions to ask: • What are we trying to figure out? • What kinds of reports or data do we want or expect? • What will we do with this data when we’re done? • Who is our intended audience or population? • How are you going to access the target audience?

11. Examples of Need • How well known is my brand? • Will customers buy this product? • If we offer X benefit, will our employee happiness go up? • Why are my customers not converting? • Will my product do well in a new market?

12. Set a Survey Goal • Setting goals and objectives for a survey • Define your goal. A goal is not a single learning point – a goal is what you are going to do with this data, and why. • Good goal: grow your company into new markets. (“A survey will determine which markets are good for our existing products.”) • Bad goal: make more money for your business.

13. Learning Objectives • Determine your learning objectives • These should all support your overall need and goal • A good amount of learning objectives: three • You should have no more than five!

14. Brainstorm Your Questions

15. Selection and Refinement

16. Eye on the Prize: ROI • If you are going to spend more time and money on running the research for this need than the overall completion of goals would generate, it’s a waste of time and money • If there is no ROI measurement, there is no encouragement to take action • Without communication from the start about possible actions to take, survey results may have no obvious meaning

17. Unit 2: Design

19. Organize Brainstorm

20. Refine Brainstorm Ideas into Questions

21. Guide: Writing Questions • Multiple choice versus open-text questions • Quantitative versus qualitative • Phrasing and language use – unclear language, grammar, ambiguity can all be issues • Remember that language can differ between demographic groups • Keep your questions: • Brief - Simple • Relevant - Specific and direct

22. Qualitative Versus Quantitative You may introduce bias into your survey with every qualitative answer you ask, unless the resulting answers are discrete. This data should never be added to quantitative data without the information being entirely clear in all reporting.

23. Guide: Exploratory Studies Open-text questions You should never have a required question that does not have an opt-out option (this creates bad data)

24. The Four Horsemen of the Surveypocalypse

25. Emotional Bias Asking loaded questions Asking neutral-seeming questions on a loaded topic

26. Identity Bias “How much do you love SurveyGizmo?” Asking “Do you like SurveyGizmo?” with a SurveyGizmo logo in the corner of the survey

27. Option Bias Required, non-applicable questions Leading or restrictive options Different types of scales Option lists of death

28. Conversational Bias Surveys as a conversation Respondents giving the answer they think you want to hear

29. Lack of Focus Covering too many diverse topics Additional questions that do not meet the survey goal Questions that are not inline with the learning objectives Questions that do not derive actionable results

30. Miscommunication • Know your audience and the language that they use and understand • Avoid technical terms unless it is appropriate • Define terms if necessary • Remember to speak in your company’s voice • Have a peer review for clarity

31. Survey fatigue as a cultural trend • Cultural survey fatigue • The average respondent is fatigued already, just by nature of: • Receiving emails from organizations • Suggestions on receipts and from cashiers

32. The Wrap-Up: Question Mistakes to Avoid • Try to avoid… • Leading questions • Loaded or suggestive questions (like our star rankings) • Fatiguing question types – large tables, lots of open-text or essay questions • Sensitive questions • Highly technical language

33. Unit 3: Build

34. How are Design and Build different? Design: Involves thinking about psychology, emotions and words. It is the more abstract phase. Build: Involves taking into account security walls, logic, combatting fatigue, bias, and poor data collection; It is the more active phase.

35. Stages of Build

36. The Usual Suspects

37. The Radio Button • Quantitative • Scale (should be horizontal) • Categorical (should be vertical) • “All of the above” is a no-no!

38. The Scale: In its Natural Environment

39. Scale questions: The controversy Neutral or not?

40. NPS: Net Promoter Score Neutral or not?

41. The Checkbox: Choose All That Apply

42. The Checkbox: Beware! Choosing more than one option changes statistical reporting a lot!

43. Multi-Text Questions Please list the names of phone providers that you have seen or heard advertised. Qualitative Explorative or un-aided response; used for lists

44. Essay Questions 3. What is your favorite thing about SurveyGizmo? Qualitative and explorative This is a way to gather unaided responses for your survey

45. Essay Questions: Your Friend? Maybe.

46. Table Questions Do NOT use as a space-saver – these are fatiguing!

47. Table Questions: What’s Totally Okay

48. Build your survey. Test It. Get buy-in from your stakeholders.

49. Why Validation Matters:

50. Are your questions reporting the way you expect? • Are you able to create the reports you need using the data you are collecting? • Is the data in the format you need? Test Reports