THE Market Research Technology Event Las Vegas, NV May, 2012

1. Migrating Surveys from Desktop to Mobile and Touch-based Interfaces…where are we?Dr. Bill MacElroyChairman, Socratic Technologies, Inc. THE Market Research TechnologyEvent Las Vegas, NV May, 2012

2. History of Survey Data Collection • Since the early days of survey work, the mode of data collection has had a tremendous impact on the quality of data and the types of biases encountered Creative Destruction (Schumpeter, 1942) Disruptive Innovation (Christensen, 2003)

3. Data Quality Effects • As technology becomes more embedded, it is creating a ubiquitous environment for higher degrees of data quality and candor in data collection • Lower the Perceived Human Presence  the Greater Candor

4. Variables Influencing Dropout Rates in Web-based Surveys • Why are we interested in the effects of engagement and new interface opportunities? • Design of surveys must take into account the interaction between burden and personal return variables • The goal: to enhance salience • Length of survey (both in terms of time to complete and number of questions) • Incentive (either total incentive offered as a prize package or the approximate value of the incentive on an individual basis) • Engagement Level lessens the perception of burden • A combination of these factors influences the number and proportion of mid-survey abandoners (mid-terms)

5. Engagement Versus Other Survey Characteristics • Mid-terminates -- an indication of the point at which respondent fatigue, boredom or lack of perceived value becomes "critical" • Note: Critical  "pleasurable" • As a rule of thumb, when surveys have a mid-terminate rate of more than 30% a post-hoc evaluation of factors leading to the problem is probably a prudent decision • Findings from these studies indicated that no significant differences can be found across different geographies Findings from 19 Web-based studies • All of the studies were with business-to-business, technology-related decision makers • Surveys included U.S., European and Asian respondents (all surveys conducted in English) • The total number of respondents included in these surveys = 21,867 • Median sample size = 473

6. Number of Screens • The more screens/questions, the greater the number of the mid-terminates • Surveys that exceed 30 screens/questions are predicted to exceed the maximum acceptable level of dropouts

7. Total Average Time of Survey • Surveys that took more than 17.5 minutes led to predicted completion rates of less than 70%

8. Time Tolerance for Online Surveys At 15 minutes, over 70% find a typical surveylength “acceptable.” At 25 minutes,over 70% find surveylength “unacceptable.”

9. Known Value of the Incentive • A cash-equivalent value of only $5.00 would still leave a predicted 78% completion rate • Once the value hit $22.00, the curve flattened noticeably • Increasing the individual incentive rate above a certain level does little to influence mid-terms

10. Incentive Rate Tolerances in Online Surveys For a survey of 12-15 minutesa $5 incentive would beacceptable to 65% Increasing the incentive to $10 would be acceptable to 85%

11. Engagement Factor • But engagement works to improve focus only up to a point… • < 10 minutes, no sig. effect • 17 to 42 minutes, improved completion rates • > 45 minutes, no sig. effect With Animation No Animation Zone of Effect 14% 20%

12. Engagement: Technology versus Engagement • New technologies tend to create a sense of excitement for survey takers due to the novelty of the new medium • However, the novelty is wearing off more quickly as people become used to more-and-more engaging technologies in their environment • Sustaining engagementwill take more thantechnology…it willrequire content andinteractivity beyondthe current paradigmof just “asking questions”

13. The Next Generation of Data Collection Technology

14. Moving from the Keyboard/Mouse to Touchscreen • We are about to enter another period of rapid technology change • The way people interact with surveys is about to shift radically as the Keyboard and Mouse paradigm (dominant since the inception of the personal computer revolution) is giving way to less manual input methods

15. Moving from the Keyboard/Mouse to Touchscreen • Interface Challenges: • Size: Tablets versus SmartPhones • Speed: Click, Drag and Drop versus Activate, Define, Drag and Drop

16. Moving from the Keyboard/Mouse to Touchscreen • Interface Challenges: • Accuracy: Mouse Pointer versus Stylus versus Finger • Finger Structures • Non-structured Entry: Typing versus Virtual Keyboards/Writing Recognition

17. Moving from the Keyboard/Mouse to Touchscreen • Interface Improvements: • Technical Engagement • Greater Flexibility to Customize Interface (Grow, Hide, Organize, Arrange) • Promise of Improved Voice-controlled Input

18. Moving from the Keyboard/Mouse to Touchscreen • Biggest Challenge: User Experience • Bridging Mechanisms: • Demos • Instructions • Diagrams

19. Example: Scalar Ratings • Demo

20. Example: Sorting • Demo

21. Example Complex Interactions • Tools versus Direct Manipulation • Grow • Rotate • Review/Swipe • Arrange • Demo

22. Research-on-Research

23. Background • Since the late 1990's, "mobile technology" has been predicted to significantly overtake the PC as the dominant data collection mode • Panels consisting of mobile users have been recruited and have been "survey ready" for years (most of these have not been profitable) • Software (and apps) for conducting research have been available since the first smartphones and tablets • So why aren't more research agencies pushing the mobile platform for new, geo-locational and event-based research? Some observations: • Research companies, by nature, are not early adopters • Many companies have had poor to mixed results using mobile platforms; clients don’t want to pay to experiment • Limited screen space lessens ability for complex questions • Mobile technology is still in "flux" meaning that surveys have to be programmed at the "lowest common denominator" for cross-platform compatibility • Lack of Flash or other similar application can limit survey interactivity and engagement • Researcher's doubt the "convenience" factor is really at play in driving surveys to mobile • Panelists have to pay for data transmission vs. "free" cost of internet connection

24. Hypotheses • Research Questions… • Is mobile perceived to be more convenient? • Wouldn't people actually prefer to do survey via mobile? • Is the mobile experience really more engaging, interesting, timely and user time friendly? • Does research "in the moment" produce higher quality data? • For this conference, we looked at the results from a number of studies… • Dunkin’ Youth Panel – Mobile (previously presented in public) • IIR Mobile Evaluations (8 Studies) • TMRTE Mobile Research -- designed specially for discussion at this conference

25. Dunkin’ Youth Panel – Mobile • (Fielded June 29th to August 30th 2007) • Dunkin’ Youth Panel Recruitment Project • Targeted 16-24 year olds in the Boston area • Part 1: Campaign to get folks to take a 1-question satisfaction survey on mobile device and provide email address for follow-up survey OR go to a listed Website • Part 2: Participate in a follow-up survey • Part 1: • Total Mobile Respondents 958 (took more than a month) • % Providing Name 73% • % Providing Age 72% • % Providing Email 69% • % Providing Rating 70% • Part 2: • Socratic sent follow-up emails to all providing emails and got 34 responses in return • None opted to join panel • Web-based Comparison • Those going to the Web site produced hundreds of survey takers • 85% of those responding via Web volunteered to join Dunkin' VIP panel • Cost of responses by mobile were 14.6X that of recruitment to Web

26. IIR Mobile Evaluations • July 2011 and continued through Dec 2011 • IIR interested in moving away from paper questionnaires toward real-time evaluations • 8 conferences • 325 completes from a total of 2,556 attendees (13%) • Stats… • 85% on average completed following conference vs. completing over the course of the conference • Respondents rated an average of 11 speakers (range was 26 to 181 possible speakers) • In general, less than half of reasonable number of evaluations were completed… • Twice as many done via PC • 221 of 325 completed on computer (8.8%) • 104 on mobile device (4%)

27. TMRTE Mobile Research • January 30, 2012 and continued through February 10, 2012 • Project to compare response rates, survey stats and satisfaction ratings between modes • First step: Screener to identify owners of computers and smartphones/tablets with internet connectivity • n=1,458 completes • Total invites sent: 24,400 • Invites sent before 2/9 = 11,664 • Invites sent after 2/9 or later = 12,736 (sent additional invites to target and reach quotas for smartphone/tablet group – main survey) • Mean survey length = 1.6 minutes

28. TMRTE Mobile Research Main Survey • Recruitment • Much easier to get people to do survey work on PCs versus Mobile Devices • 87% indicated preference for computer, 13% for tablet/smartphone *98% completed on computer when instructed to use smartphone/tablet, 2% used smartphone/tablet instead of PC

29. Speed & Quality • PC users were able to complete up to 20% faster than Smartphone/Tablet users • Both modes answered open-ended questions equally well • However, a few Mobile users wrote a LOT more than average

30. Demographics • Demographically, the samples were not significantly different for gender or age

31. Demographics • However, education and income levels did vary significantly; lower income and education responded via PC-based survey

32. Survey Experience • Computer users reported generally higher levels of satisfaction and enjoyment

33. Online Activities • Interestingly, PC users are more likely to share thoughts via social media *Twitter, LinkedIn, Facebook, Other

34. Summary • Research technology is again at an inflection point where innovations may fundamentally change the way we collect data • Mobile and Tablet interfaces have not yet stabilized…users still have to "learn" to use surveys in these new modes • The general survey taker still far prefers the PC to mobile platforms • Most surveys on the PC are perceived to be: • Faster • Easier • Less burdensome • More interesting • "Better use of my time" • Both interfaces and public experience will have to mature in coming years before Mobile becomes a preferred/acceptable mode of standard research data collection

35. Questions and DiscussionDr. Bill MacElroyChairman, Socratic Technologies, Inc.2505 Mariposa StreetSan Francisco, CA 94110415-430-2200bill.macelroy@sotech.com