450 likes | 533 Views
What’s all the noise about?. Steven Gittelman, Ph.D. Our data shows that the frequency of problem responders (professionals, speeders, etc.) is far lower in Europe than in the United States: we have acculturated our respondent pool.
E N D
What’s all the noise about? Steven Gittelman, Ph.D.
Our data shows that the frequency of problem responders (professionals, speeders, etc.) is far lower in Europe than in the United States: we have acculturated our respondent pool. Problem respondents may be defined as those who bias market research data so that it misrepresents information needed for business decisions. Thus, speeders, professionals, satisficers, etc. matter little unless their purchasing behavior is different. So are the variations noise or signal? Is it noise or signal?
We need to measure it. To measure it we need to agree upon metrics. What are the important parameters--- - Psychographics? - Demographics? - Purchasing Behaviors? - Or should we hang our hats on “problem respondents?” - What about duplication technologies? What can we do about it?
Once upon a time we lived in a world of probability samples. There was a standard, consistency in our lives. Now we are in a non-probability universe online. But somewhere there is a platform—a theoretical population online. We need a new basis for comparison, a new standard. We cannot find ourselves in typologies of problem respondents. We have tried post survey to anchor ourselves in a collective standard-it is retrogressive. But we need a progressive standard that changes with each new sample population that we add. To build such a model we need more data that provides a broader anchor. Brave new world
Respondent Types Professional Respondents fall into four categories: (1) Self report taking on-line Surveys “practically every day” (25% of Total). (2) Self report (open ended) taking over 30 online surveys “in the past month” ( 15% of Total). (3) Multiple panel membership >panels. (4) Panel members of long vs. short duration. Inconsistency: Brand vs. Price, Price vs. Brand, Happy with standard of living vs. unhappy. Failure to follow instructions: Instructed to enter a predetermined answer, also known as a trap question. Speeders: survey times in the bottom 10 percent. Straight liners: low variance in answers to grid questions. Duplication technology
I can see clearly now! Compared survey results from 17 sets of US Consumer Panels, 1 UK and CATI. 400 completes per source. Dec. 2007- Dec. 2008. Selected demographic quotas (age, income, gender, ethnicity) were used to simulate census. Median length was 13 minutes. Questions covered: Technology and the media, Participation in market research, Buyer Behavior, Values and lifestyle, Demographics, Questionnaire Satisfaction.
Average Number of Panel Memberships M1 and M2 were not asked the number of panels.
Professional RespondentsBy Sample Source Panel M1 and M2 were not asked number of panels.
Impact of Max Panel Age----Just when you thought it was safe to go into the water.
Max age on panel by panel (Eliminated M1-M5)
Distribution of Survey Flaws, Inconsistencies (Brand over Price)
Distribution of Survey Flaws, Failure to follow instructions
Straight-Liner Distribution for RDD Phone Sample
Straight Line Responses as Standard Errors (Variance) -Online
Gender, Age, Income, and Ethnicity set by Quota Compare Distributions by: Education Having Children under 18 Employment Compare Across Panels Telephone---what do we do about cord cutters? Census Demographics
Education Distribution by Panel –No Quotas Set No Quotas Set Point System UK
Having Children Under 18No Quotas Set Professionals
Having Children Under 18 No Quotas Set
Employment Distribution No Quotas Set Social Network Point System Access Panels River UK
Let’s go to the videotape….We’ve examined panels in the usual format, demographics and respondent typologies.But where does that leave us?------Do we have a model for the future as yet? So we will give it another try---segmentation and cluster analysis bring variables together.
Social opinions and behavior can be expected to drive purchasing behavior or at least provide a basis for segmenting the market. Consistency of these measurement may likewise be critical. Variables Groups Internet Use. Taking Surveys Having a Passport Social Characteristics Measures: Driving Variables Variation in Opinions
Key Social Variables • Selected key variables determining Social Clusters. • Social Behavior, Unconventional, Passports, Time over Money, Risk Avoidance
Measuring buyer behavior is the objective of most marketing research. And therefore, consistency of those measurement are critical. Variables Number of High Tech Items Purchased. Internet Purchase behavior Purchasing Opinions Measures: Clusters (Segments) Driving Variables Variation in Buyer Behavior
Buyer Behavior Segments by Panel Social Network Point System Access Panels River UK
MDS (Multi Dimensional Scaling)Position based on Buyer Behavior
Principal Buyer Behavior vs. Professionals (>30 Surveys/Month) M1-5 (US) Deleted M15 (UK) Deleted
Principal Buyer Behavior vs. Speeders (<10 Percentile of Completion Speed) M1-5 (US) Deleted M15 (UK) Deleted
Are we there yet? Are we there yet?Our discussions have been retrogressive, looking back at cumulative data collected. For us to provide workable tools they must be progressive, not retrogressive. After all, we don’t want to go through this every time we add a new panel. Do we? We don’t want to re-segment each time there is a new panel on the block.
In Progress….Currently collecting data in 40 Global Markets.
When we look at global panels we have a whole new world. Developing markets show us the universe in creation. Ageing panels help us forecast the future. As we create our model we can add within country comparisons just as if they were within mode comparisons here in the states. If our model is truly progressive we will add each new market and compare it to the whole. All we need is the data………… It’s like looking at the stars…..
Distribution of Buyer Behavior Segments Homogeneity, Stability, Predictability and Reliability
Comparison of Buyer Behavior Segments Among Panels by Country
THANK YOU! 200 Carleton Ave. East Islip, New York 11730 631-277-7000 800-645-9850