How to Conduct Successful Market Research Projects

1. How to Conduct Successful Market Research Projects September 2000 Gobde Research and Analysis

2. Contents of the Library Research Kit Survey Research Process Survey Instrument Mail Survey Research Ballot Measure Feasibility Research Qualitative Research Appendices A guide to creating an RFP Layout options for mail survey CD-ROM

3. Survey Research Process

4. Prior to developing specific survey questions and the sampling frame. What needs to be accomplished by conducting the survey? Objectives: assess support level for a ballot measure vs. gather opinions about current and potential services. Survey Research Process IDefining Research Objectives

5. Primary reason for sampling: reliably representing the universe with a smaller group of individuals. Representativeness of a sample is dependent upon the degree of randomization of the sample: every individual in the universe has equal chance of being included. Survey Research Process IIDeveloping the Sample Frame

6. Sampling frame must be appropriate for the research objectives: RDD, voter, or card holder. Sampling frame must take into account all potential factors that might bias the sample. Sampling frame should be relatively easy to administer. Sample size should be sufficient to provide an acceptable margin or error for important analyses. Survey Research Process IIDeveloping the Sample Frame

7. Random Digit Dial (RDD) Drawn by determining the active phone exchanges in a given sampling area. Appropriate for gathering information about an entire population. Incidence rate. Tendency to under-represent minorities and over-represent older women. Telephone book not an acceptable substitute for RDD. Survey Research Process IIDeveloping the Sample Frame

8. Voter File Sample: Appropriate for gathering information among registered voters. Stratify and cluster by demographic characteristics. Each cluster represents individuals sharing a similar demographic profile. Generally no incidence rate issue. Sample likely voters to increase prediction accuracy. Survey Research Process IIDeveloping the Sample Frame

9. Reliability and validity Sources of measurement error: Question wording effects. Question order effects. Response category effects. Interviewer effects. Panel effects. Learning effects. Survey Research Process IIIDeveloping the Questionnaire

10. General rules: Keep the order of questions logical, with smooth transitions. Move from general to specific. Make sure respondents understand the wording. Be conversational. Avoid monotony. Include exhaustive and non-overlapping response categories. Survey Research Process IVDeveloping the Questionnaire

11. Estimate the length of the questionnaire. Ensure that words, phrases, and subjects are easily understood by the respondents. Ensure that answer categories match with what the respondents have to say. Ensure that the questionnaire achieves the research objectives. Survey Research Process VPre-testing the Questionnaire

12. Appropriate when the universe contains a significant number of linguistically isolated respondents. Appropriate when the universe contains a significant number of bi-lingual respondents who are more comfortable in their first language. Overriding goal: translation maintains the exact meaning of the questions. Survey Research Process VILanguage Translation

13. Telephone surveys: Telephone interviewing is efficient and relatively cost-effective. Conducted on weekdays (evening) and weekends (late morning/afternoon). CATI programming facilitates interviewing process. Survey Research Process VIIData Collection

14. Survey Research Process VIIData Collection Volunteers useful for: Short, straightforward questionnaires. Cost savings. Potential bias. Disadvantages of using volunteers: Higher refusal rate. Not monitored and more interviewer errors. Not equipped with CATI programming technologies and skills.

15. Survey Research Process VIIData Collection Criteria for selecting a high-quality data collection facility: Demographics of the interviewing staff. Supervisors’ and project managers’ experience. Ability to monitor remotely. Multi-lingual interviewing ability. Proficiency working with different types of samples. CATI and turnaround time.

16. Mail surveys: Appropriate for reaching certain individuals (e.g., individuals without telephones or without identifiable numbers). Disadvantages: lack of control over respondents, low response rate, long data collection period, and bias based on respondents’ self-selection. Need for incentives. Survey Research Process VIIData Collection

17. Intercept surveys: Administered face-to-face. Effective for reaching individuals concentrated at a specific location. Attempt to contact every ‘nth’ person. Higher response rate than mail surveys but a sound sampling design for an intercept survey can be challenging. Survey Research Process VIIData Collection

18. Internet surveys: Represent only a select group of individuals: those with a known email address or visitors to a site. HTML questionnaire at a selected URL address. Sending the URL to potential respondents via email. Respondents click on the URL link and complete the survey. Efficient, accurate, and cost-effective. Survey Research Process VIIData Collection

19. When using a professional data collection facility: Data normally are provided as a ‘flat’ ASCII file. Data map is needed to locate variable values and locations. An example of a ‘flat’ ASCII file is provided on the next slide. Survey Research Process VIIIData Processing

20. Survey Research Process VIIIData Processing Respondent# 10 Column 6

21. When conducting the survey with volunteers: Record all responses on paper. Keypunch responses into computer for data processing. Software packages to use for keypunching: Excel, SPSS, or SNAP. Each column is a variable and each row is a respondent. Survey Research Process VIIIData Processing

22. Tasks involved: Merging multiple data sets into one. Checking and cleaning data. Coding and re-coding verbatim responses into numerical values. Aggregating all responses to interpret the population as a whole. Deriving variables to perform necessary analyses. Survey Research Process VIIIData Processing

23. Weighting data: Adjust a known bias in the sample. Ensure the representativeness of the sample. Weight data on a single dimension. Weight data on multiple dimensions. Survey Research Process VIIIData Processing

24. Statistical software packages to use: SPSS, SAS, STATA, or SNAP. Statistical techniques: mean, mode, median, crosstabulation, correlation, factor analysis, and regression analysis. Survey Research Process IXData Analysis

25. Recap of objectives and methodology Detailed question-by-question analysis. Topline results. Segmentation of key audiences. Key findings. Conclusions and recommendations. Note: Examples of GRA’s final reports are provided on the CD-ROM. Survey Research Process XReport Writing

26. Sample Questions: Customer Satisfaction

27. Brief introduction and do not reveal the specific purpose of the survey. Qi. For statistical reasons, I would like to speak to the youngest male currently at home that is at least 18 years of age. (IF NO MALE AT LEAST 18 AVAILABLE, THEN ASK): Ok, then I’d like to speak to the youngest female currently at home that is at least 18 years of age (IF NO FEMALE AT LEAST 18 AVAILABLE, THEN ASK FOR CALLBACK TIME) Qii. In what city do you live? Survey Instrument Examples

28. Q1. How long have you lived in <City that library services>? Q2. What information sources do you use to find out about <City name> news, information and events? (DON’T READ LIST, RECORD FIRST TWO RESPONSES) Q3. In the last three months have you ________? (CONTINUE UNTIL ALL ITEMS ARE READ) Example of skip pattern: If Respondents indicated ‘2’ for Q3A, then ask Q4, otherwise, skip to Q5. Survey Instrument Examples

29. Q4. What is the main reason you have not used your public library recently? (DON’T READ LIST, ONE RESPONSE ONLY) Q5. Your local library offers many programs and services. I’m going to read several of these and I’d like you to tell me if you are aware that they are available. Are you aware that ____ (is/are) available at your local branch library? Survey Instrument Examples

30. Q6. In your opinion, how important is it for libraries to ______? Is it very important, somewhat important, not too important, or not at all important? (CONTINUE UNTIL ALL ITEMS ARE READ). Q7. Are you satisfied or dissatisfied with your local library’s efforts to _____? (GET ANSWER, THEN ASK): Is that very (satisfied/dissatisfied) or somewhat (satisfied/dissatisfied)? Survey Instrument Examples

31. Q8 & Q9. If your library: ______, would you be more or less likely to use the library? (GET ANSWER, THEN ASK): Is that much (more/less) likely or somewhat (more/less) likely? Q10. Did you know the <library name> maintains its own website? Q11. Have you ever visited your local library’s website? Q12.What have you used the website for? Q13. Are you generally satisfied or dissatisfied with the resources available on the library’s website? Survey Instrument Examples

32. Q14. If your library _________, would you very interested, somewhat interested, not too interested, or not at all interested in this service? Q15. If you knew that, with a personalized online library account, you could _________, would you be very interested, somewhat interested, not too interested, or not at all interested in using this service? Demographic questions Survey Instrument Examples

33. Using Mail Surveys

34. When a particular group is difficult or impossible to reach by telephone. When the group of interest is only library users: intercept combined with mail surveys. When there is not an immediate deadline. Mail Survey Research IWhen Are Mail Surveys Useful?

35. A complete list of potential respondents. Contact information including, at minimum, the mailing address should be provided. Additional information preferred: name, geographic location, gender, and etc. Mail Survey Research IIAcquiring the Sample

36. Visually appealing and reader-friendly. Use graphic design to increase response rate. Professional graphic designers. Mail Survey Research IIIGraphic Design

37. Envelop is the first attempt to get potential respondents to open the survey. Distinguish the survey from other direct mail or junk. If possible, specify the name of the recipient. Take advantage of a cover letter to introduce potential respondents to the survey. Mail Survey Research IVEnvelope and Cover Letter

38. First class mail: 33 cents per mail piece. 1-6 days delivery time. Zip codes not required for delivery. Mail forwarding free for 6 to 12 months. Standard rate mail: 18 cents per piece. Up to 21 days delivery time. Valid zip codes required. Mail Survey Research VPostage

39. To increase response rate. An effective yet economical incentive: eligibility in a drawing for a prize such as a gift certificate or a cash prize. Mail Survey Research VIIncentives

40. Two other approaches to increase response rate. Pre-notification letters notify respondents of the survey and explains the objectives. Follow-up letters are sent one week after the surveys have been mailed out to kindly encourage respondents to complete and return the survey in time. Mail Survey Research VIIPre-notification and Follow-up Letters

41. Ballot Measure Research & Campaigns

42. Determine the level of support for a new tax and at what tax amount the support is greatest. Identify the spending projects that resonate best with voters. Determine the arguments for the measure that are most effective in persuading voters to support the measure as well as those that could weaken the support. Ballot Measure Feasibility Research IDeveloping the Feasibility Survey

43. No longer than 75 words. Basic approach: state the original problem, followed by features of the measure that will contribute to the proposed solution. Features should be limited to the compelling ones found in the survey. Tax level is mentioned as the necessary means to achieve the goal. Ballot Measure Feasibility Research IIWriting the Ballot Question

44. Identify and turnout voters whose support for the measure was greatest. Be offensive with benefits and supporting arguments. Avoid reiterating detracting arguments even in an attempt to refute those arguments. Seek assistance from a research consultant and a political consultant. Ballot Measure Feasibility Research IIIThe Information Campaign

45. Qualitative Research

46. Utilizes open-ended interviewing Explore and understand opinions and attitudes. Not representative. Very small sample size. Examples: focus groups and executive interviews. Qualitative Research IWhat Is Qualitative Research?

47. Pre-survey: developing hypotheses to be tested in a telephone or mail survey. Post-survey: in-depth exploration of previous quantitative findings. Exploration of potential service and program improvements. Identifying strengths and weaknesses. Exploring communication messages. Understanding usage motivations. Qualitative Research IICommon Uses

48. 8 to 10 participants. Typically takes two hours to conduct. Appropriate when research objectives require qualitative, in-depth responses from a select group of participants. Findings can not be generalized to the larger population. Recruiting, incentives, and facility rental. Qualitative Research IIIFocus Groups

49. Similar to focus groups, to explore issues and attitudes through in-depth discussions. One-on-one interviews. In-person or via telephone. Qualitative Research IVExecutive Interviews

50. Creating an Effective RFP