1 / 32

On the road to clarity: Differences between sample sources

On the road to clarity: Differences between sample sources. Steven Gittelman. Problem Respondents: are they at the core of the issue?. Our data shows that the frequency of problem respondents (professionals, speeders, etc.) is far lower in Europe than in the United States.

heidi
Download Presentation

On the road to clarity: Differences between sample sources

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. On the road to clarity: Differences between sample sources Steven Gittelman

  2. Problem Respondents:are they at the core of the issue? • Our data shows that the frequency of problem respondents (professionals, speeders, etc.) is far lower in Europe than in the United States. • Respondents are not problems unless they bias purchasing behavior in our data.

  3. Objectives • Do problem respondents impact purchasing data? • Who potentially are the problem respondents? (professionals, speeders, satisficers, logic errors, validity checks) • How do they impact our sampling universe? • Arrive at workable solutions.

  4. The road we took... • Compared survey results from 17 sets of US Consumer Panels, 1 UK and 1 CATI. 400 completes per source. Dec. 2007- Dec. 2008. • Demographic quotas (age, income, gender, ethnicity) were used to simulate census. • Median length was 13 minutes. • Questions covered: Technology and the media, Participation in market research, Buyer Behavior, Values and lifestyle, Demographics, Questionnaire Satisfaction.

  5. Respondent Types • Professional Respondents were defined in three ways: • (1) Self report taking on-line Surveys “practically every day” (25% of Total). • (2) Self report (open ended) taking over 30 online surveys “in the past month” ( 15% of Total). • (3) Multiple panel membership >5panels. (36%) • Inconsistency: Brand vs. Price, Price vs. Brand, Happy with standard of living vs. unhappy. • Failure to follow instructions: Instructed to enter a predetermined answer, also known as a trap question. • Speeders: survey times in the bottom 10 percent.

  6. Professional RespondentsBy Sample Source Panel M1 and M2 were not asked number of panels.

  7. Professional RespondentsSample sources grouped by type. Measures of Professionalism 70% Access Panels 60% 50% 40% Percent of Respondents 30% 20% 10% 0% M1 M9 M7 M5 M6 M8 M12 M13 M14 M17 M18 M15 - UK All Panels M3 - River M4 - River M11 - River M2 - Point System Panels M16 - Point System M10 - Social Network >= 5 Panels Every Day >30 Surveys

  8. Distribution of Survey Flaws, Inconsistencies (Brand over Price)

  9. Distribution of Survey Flaws, Failure to follow instructions

  10. Distribution of SpeedersShortest 10%

  11. Demographics • Gender, Age, Income, and Ethnicity set by Quota • Compare Distributions by: • Education • Having Children under 18 • Employment • US Census

  12. Education Distribution Professional

  13. Education Distribution by Panel No Quotas Set Point System UK

  14. Having Children Under 18No Quotas Set Professionals

  15. Having Children Under 18 No Quotas Set

  16. Employment DistributionNo Quotas Set Professionals

  17. Employment Distribution No Quotas Set Social Network Point System Access Panels River UK

  18. Variation in Buyer Behavior • Measuring buyer behavior is the objective of most marketing research. And therefore, consistency of those measurements are critical. • Variables • Number of High Tech Items Purchased • Internet Purchase behavior • Purchasing Opinions • Credit card Usage • Coupons • Price vs Brand / Brand vs Price

  19. Buyer Behavior Segments by Panel Social Network Point System Access Panels River UK

  20. Statistical Panel Profiles Against Buyer Segments

  21. Buyer Behavior Segments by Respondent Type

  22. MDSPosition based on Buyer Behavior UK River Social Network

  23. Principal Buyer Behavior vs. Professionals (>30 Surveys/Month) M1-5 (US) Deleted M15 (UK) Deleted

  24. Stepwise Regression

  25. Principal Buyer Behavior vs. Speeders (<10 Percentile of Completion Speed)

  26. Optimum Sampling • Goal is to: • Minimize “Error” on objective, which in this case, we chose education as represented by the panels themselves. • Minimize number of panels in the solution. • Use Microsoft Excel Solver • The solution we obtained was: 40% M3 – River 40% M6 – Access Panel 20% M14 – Access Panel

  27. Optimum Education Attainment Distribution Panel Panels Panels Panels Panels Panels Panels Panels

  28. Improve the Optimum with increased Panels

  29. A Look Back • Panels vary widely between sourcing mode. • Professional Respondents appear to be a strong driver. • So far our analysis has been somewhat granular…..

  30. Future Discussions… Let’s take a second look at the access panels. Social Network Point System , Stability • Homogeneity • , Predictability • and Reliability

  31. Think of Global Panels…. We are currently collecting data in 29 Global Markets. The Take Home Message...

  32. THANK YOU! 200 Carleton Ave. East Islip, New York 11730 631-277-7000 800-645-9850

More Related