Download
1 / 19
190 likes | 208 Views
Explore differences in respondents of Big Ears telephone survey and online surveys, plus insights on college students' feelings about cell phones. Detailed findings and executive summary included.
E N D
Top-line Report:Automated Telephone VS Online HTML Survey(A College Cell Phone Case Study) October 25, 2006
Contents • Background & Objectives • Methodology • Executive Summary • Detail Findings
Background & Objectives The focus of this qualitative study is twofold: • The primary objective is “Research on Research” --- To compare the differences between respondents of online surveys and respondents of a Big Ears’s telephone survey. • A secondary objective, is to understand how college students feel about their cell phones in general.
Background & Objectives • Big Ears is a revolutionary new type of survey system for conducting fully automated telephone surveys. The tool converts survey questions to computerized audio clips for telephone surveys. • Big Ears can also take advantage of VOIP technology such as Skype to significantly reduce costs associated with traditional telephone interviewing, and make it simple to do oral interviews directly from a web page. • In addition to common survey features such as skip logic and output to common data files, Big Ears archives recordings of open ended response to a .wav file. The recording of open-ended responses which can be browsed via the web interface, is a key strength of the BigEars approach. [NOTE: Within this report example audio files have been embedded, simply click on the speaker image to listen]
Methodology • 300 college students from Anderson Analytics were sampled. Response rates were >60% with 187 completes. • The students were divided into two groups, A & B. • Group A was then divided again; half of the group was assigned to take the survey online and the other half was assigned to take the survey via Big Ears. • Group B was given the choice to take the survey using either methods.
Executive Summary • The college mobile phone market is a saturated and mature market. Only 2 out 187 students surveyed do not have a cell phone. • Most students associate their cell phone with convenience overall satisfaction with products and services is high • Cost is the primary issue when selecting a service provider • Free minutes and free phone are commonly mentioned as incentives • 1/3 of the students agreed that their cell phone is a reflection of their self image
Executive Summary • Positive Aspects of the Big Ears Surveys • Respondents give more detailed answers to open-ended questions • More convenient in some situations: • When top-of-mind is important (no opportunity to go back/change) • For people on the go • Answers can be given w/o using the keypad • No worries about literacy, typing skills, etc. • A viable alternative for participants who encounter difficulties with online surveys (or without internet connections) • Key Items to Consider When Using Big Ears • The surveys take somewhat longer to complete than online surveys • Computerized audio should be interesting and be as clear as possible • Ideal for some open ended questions • If you want information that is not top of mind, supply the questions ahead of the call. • Consider using 800 access # to maximize participation (until VOIP use increases)
Executive Summary • Differences in Online and Phone Survey Responses • Overall, there are no glaring differences in terms of the attitude towards cell phone between the phone survey participants and the online survey participants • However, people who select to use the phone survey exhibit less concern about the phone cost and generally believe that they have a good phone service.
Open-ended input comparison Open-ended questions via the phone survey allow the respondents to freely express their thoughts. They tend to give longer answers by phone; on average, 13 more words than online surveys* Word Count Audio clip example *Due to uneven and small sample size, the confidence level is at mid 80%.
Ease of use • While 94.6% of the students who took the online survey found the process easy, only 69.0% of phone respondents found their surveys easy to complete. • The difficulty is generally attributed to temporary technical issues effecting the performance of the phone survey, and the longer time to complete the phone survey. Q: How easy, or hard, did you find it to complete this survey? Significantly higher than total Significantly lower than total
Ease of use • However, when asked to compare online survey with phone survey, students who took the phone survey found it surprisingly easy. Q: Do you agree, or disagree, with the following statement. "web surveys are more convenient than phone surveys"? Audio clips “Just because I don't like looking at a computer screen for too long when given a survey. This is pretty easy, I'm just sitting in the park and I'm enjoying it a lot more.” “Seems like a telephone survey like this one would be just as easy or simple as an online survey.” “It was just easy to say my answers, it was fast, it was easy to do. I didn't have to click anything, I just had to listen. There was only one time where I couldn't understand what the lady was saying but other than that it was perfectly easy..” Significantly higher than total Significantly lower than total
Voice & Online Survey Response Difference • Students were contacted via email about the participation of this survey. They all have email accounts and easy access to internet. There was also no 800 access, so many would have incurred a cost if using the phone survey. However, a number of them still chose the phone survey • The reasons that these students rather use their phones might be: • A. Fewer of them thinks phone/service cost is an important factor when purchasing a phone/plan. • B. More of them believe that their current phone service provide is a better choice. Q: How important is cost? Q: Do you agree, or disagree with this statement, “My cellular phone service provider is a better choice for me than any other.” • Generally speaking, and judging from their recorded responses, phone survey participants are quite fond of their cell phones. Significantly higher than total Significantly lower than total
Choosing Phone over the Internet • Some of the reasons for choosing phone over online survey are due to technical issues • “The online survey was inaccessible” • “I was busy doing some stuff online, and I had my cell phone on hand so I decided to do the survey by phone and finish off what I was doing online at the same time.” • In addition, the students also pointed out areas that could make the phone experience better: • “…this survey might be taking up my minutes” • “My answers weren't recognized” • “It is hard to discern what the speaker is saying” • “To difficult to talk to an automated voice.”
Importance of phone features • This survey only specifically asked the students 3 factors regarding their phone/service selection process: Cost, Text Messaging, and Choice of Games. Q: how important is each feature to you in your choice of cellular phone service? • Judging simply by these 3 pre-selected factors, it might seems that the cost is the most important factor in this demographic. Q: Assuming you were purchasing a new cellular phone, what is the main thing that you would look for when comparing different cellular phones? (open-ended question. Voice recorded & analyzed) • However, the open-ended responses fielded by Big Ears, in conjunction with text mining techniques, reveal that “looks, camera, and phone sizes” are more relevant than cost.
Investigating Outliers • The average phone bill reported in this survey is $63. However, as with any survey, there are outliers presents in the survey results. Typically, these outliers can either be legitimate data point, or could be survey/data entry errors. • With the Big Ears recording, some of these outliers can be investigated. • For example, Big Ears was able to provide the voice recording of a student answering the phone bill question with conviction (“200”), therefore concluding that the outlier is not the result of data entry error.
Cell Phone & Self Image • 1/3 of the students agreed that their cell phone is a reflection of their personality • Using the Big Ears wav files, in conjunction with text analysis, the survey reveals that cell phones reflect these students’ “personal style.” • 40% of the students in the top two boxes specifically expressed that the cell phone reflects their personal style. Q: Do you think the model, or brand of cellular phone a person has, says something about their image? Example: • “Feminine, up-to-date.” • “I like sleek, easy to use items” • “I think it says that I am up to date with technology and hip/stylish.” Other words mentioned includes: fashion conscious, pretty hip, tech savvy, personal touch, personality, etc.
First Thoughts About Cell Phones • When asked about cell phone, the students first think of the convenience the phone provide, and also about the communication they can have with family and friends. The open-end responses are filled with examples such as: “… being able to keep in touch with your girlfriend and family. That's the only way people keep in touch with me. Another way is that just being able to be contacted at all times, anywhere, anytime.” “It is very convenient for people to reach one another.” Other positive thoughts about cell phones include: good for emergency cool ringtones Some negative thought about cell phones include: bad reception people talking on the train Q: What is the first thing that comes to mind when you think of cellular or mobile phones? (open-ended question)
The Strength of Big Ears The real strength of phone surveys lies in short (5-10 questions) surveys with scales and skip logic used to target insightful responses to open-ended questions. It is very important that the experience be a quick and convenient one. On the open-ended questions, the phone surveys allows the participants to freely express their opinions in detail, without the self-censoring and summarization that can occur if writing is considered a chore. Advanced text mining software available in the market today can work very well with Big Ears. In-depth studies and studies that are a hybrid of qualitative and quantitative techniques can be performed with data collected using Big Ears.
