330 likes | 445 Views
To Cell or not to Cell. Presentation to Presentation to the Ottawa Chapter of the Marketing Research and Intelligence Association June 11, 2008 the Ottawa Chapter of the Marketing Research and Intelligence Association June 11, 2009 .
E N D
To Cell or not to Cell Presentation to Presentation to the Ottawa Chapter of the Marketing Research and Intelligence Association June 11, 2008 the Ottawa Chapter of the Marketing Research and Intelligence Association June 11, 2009 Presentation to the MRIA Ottawa Chapter June 11, 2009 Randa Bell ASDE Survey Sampler, Inc. We get what you need
Current RDD sample does not include cell phones
Residential Telephone Service Survey RTSS The RTSS monitors residential phone penetration rates and reasons for non-subscribing Since 2005, the RTSS has been conducted in December only The collection period is the Labour Force Survey (LFS) interview week
Household composition of the 6.3% cell only households in Canada *Source: Statistics Canada: Residential Telephone Service Survey (RTSS) Dec 2007
Household composition of the 6.3% cell only households in Canada Source: Statistics Canada: Residential Telephone Service Survey (RTSS) Dec 2007
Using IVR to screen for cell only households Programmed and recorded survey with one question only Nature of cell phone we were reaching: Cell only household or in addition to landline Simplified wording
The questionnaire 20 seconds, National Study Only one question, nothing to sell Phone line which is not cell phone = PRESS 1 Cell phone only household = PRESS 2 Reimbursement of call = PRESS 3 Repeat question = PRESS 9
Two days dialing results of using IVR to screen for cell only households
BBM inclusion of cell phones onlyhouseholds (MO) Assess the viability of recruiting into the TV panel Recruiting a subset from the establishment survey to participate in phase 2
Household Technology Inclusion of Mobile-Only Households in Canadian Radio & Television Ratings PanelsApril 2009 *Significant at p<0.05, ^Quebec Sample Only, ^^May Establishment Survey
BBM Canada Household Technology • BBM Canada, Sondages BBM in Québec: not for profit, member-owned tripartite industry organization, operating since 1944 • Provide broadcast measurement and consumer behaviour data, as well as industry-leading intelligence to broadcasters, advertisers and agencies • Services include a digital broadcast-ready TV & Radio people meter system, diary surveys for 100+ radio and television markets, and a variety of syndicated and custom research studies *Significant at p<0.05, ^Quebec Sample Only, ^^May Establishment Survey
Panel Establishment Household Technology • Establishment survey conducted monthly (except Dec, Jul and Aug) for each panel • Large scale telephone survey • Establish sample frame (households) for recruitment into Panel *Significant at p<0.05, ^Quebec Sample Only, ^^May Establishment Survey
Mobile-Only Test Household Technology • To overcome under-coverage in sample frame, the addition of mobile-only homes was tested in two phases • Existing Panel Establishment survey to assess participation by mobile-only homes • First three weeks of May, 2008, concurrent with the regular May Establishment • National sample of 4,316 mobile-only homes provided by ASDE • Safety assured at beginning of call. A few questions regarding mobile use were added to survey. • All respondents, regardless of completion of survey, were offered $5 as compensation for air time if address provided. *Significant at p<0.05, ^Quebec Sample Only, ^^May Establishment Survey
Establishment Survey Response Metrics – Cell-Only Households Nationally Establishment Survey Response Metrics – Cell-Only Households Nationally
Establishment Survey Response Metrics – Total Quebec Establishment Survey Response Metrics – Total Quebec Significant at p<0.05
Establishment Survey Response Metrics – Montreal EM Establishment Survey Response Metrics – Montreal EM Significant at p<0.05
Establishment Survey Response Metrics – Quebec Balance Establishment Survey Response Metrics – Quebec Balance Significant at p<0.05
Household and Respondent Data Household and Respondent Data *Significant at p<0.05, ^Quebec Sample Only, ^^May Establishment Survey *Significant at p<0.05, ^Quebec Sample Only, ^^May Establishment Survey
Household Technology Household Technology *Significant at p<0.05, ^Quebec Sample Only, ^^May Establishment Survey *Significant at p<0.05, ^Quebec Sample Only, ^^May Establishment Survey
Other Points of Interest Household Technology • Respondents did not call back to call centre based on • number on call display (as experienced in US) • Only 1% of respondents did not live in same geography • as area code indicated • 88% indicated they were contacted in the city • in which they live • Reimbursement for air time did not appear to be factor. • Very few (less than 1%) of non-completes • requested reimbursement *Significant at p<0.05, ^Quebec Sample Only, ^^May Establishment Survey
Harris-Decima Benchmark study • Numbers were randomly generated for Canada • The sample was to be used as a benchmark → Wanted to analyze: Age composition Male/Female composition Cost and performance Cooperation rates We get what you need
No changes to the standard screener no special accommodations for providing incentives because they are reaching people on their cell phones • 2 separate call dispositions. We get what you need
At first glance, it looks like the cell phone sample performed considerably worse than the RDD sample 4.6 times the ratio of sample to complete We get what you need
Comparing the disposition of all numbers used • RDD sample vs. cell phone sample 20% of the RDD VS. 55% of the cell sample was NIS sample was NIS The use of a dialer means that the extra numbers cost time on the dialer and not on interviewing costs. We get what you need
The next step was to exclude the NIS numbers from the call disposition Ratio of sample to completes dropped by 56% for the cell sample and by 20% for the RDD sample • No answer/ Answering Machine/ Busy was 65.2% for cell numbers and 51.8% for RDD sample. • Refusal rate on cell phone was 25.7% and on RDD was 34% We get what you need
Benefits Looking at the actual cell phone sample completes: 41.1%58.9% 18-34: 53.4% 35-54: 31.5% 55+: 9.6% We get what you need
Conclusion Preliminary results show that people are just as responsive to being called on their cell phone if not more than on their home line One hypothesis could be the novelty of getting a call on their cell phones. We get what you need
Futuremoving on Focus on testing for various scenarios: Response rate for various length surveys Incidence of cell only and cell mostly Incentives vs. reimbursement Respondent experience
To include or not to include With cell only households at 6.3% Including them in a national sample may not substantially affect survey findings BUT It does improve the overall representativeness of the sample Reduces sampling error for these low incidence groups
Other considerations Under 18 with cell phones Reimbursement Safety and scheduling for call backs Geographical location vs. area code location Quality of responses, better or worse
How to incorporate cell only households sample into RDD Blended frame sampling Portion from cell phones only households Portion from RDD Augment traditional sample with cell phone numbers (no overlap) Dual Frame sampling Portion from cell phones Portion from RDD Qualify if overlap
Thank you Randa Bell rbell@surveysampler.com www.surveysampler.com 819-770-3651 Sorry no cell number provided