Research Design

1. Research Design Jeremy Kees, Ph.D.

2. Formulate Problem Stages in the Research Process Determine Research Design Design Data Collection Method and Forms Design Sample and Collect Data Analyze and Interpret the Data Prepare the Research Report

3. Overview of Research Design Exploratory • “discovery” Descriptive • “relationships” Causal • “cause-and-effect”

4. Overview of Research Design Uses Types • Formulate problems more precisely • Develop Hypotheses • Establish priorities for research • Eliminate impractical ideas • Clarify concepts • Literature search • Experience survey • Analysis of select cases • Interviews • Ethnographies • Focus groups • Etc. Exploratory Research • Describe segment characteristics • Estimate proportion of people who behave in a certain way • Make specific predictions Descriptive Research • Longitudinal study • Panels • Sample Survey • Provide evidence regarding causal relationships • Rule out all other explanations Causal Research • Laboratory experiment • Field experiment

5. Relationship Among Research Designs Descriptive Research ExploratoryResearch Causal Research 5

6. Qualitative versus Quantitative Research Data • Quantitative = numeric data • Qualitative = non-numeric data • Caveat – all qualitative data can be coded and all quantitative data is based on judgment • Common Assumption: • Qualitative Data = preliminary • Quantitative Data = confirmatory

7. Qualitative Research To gain a qualitative understanding of the underlying reasons and motivations Small number of non-representative cases Unstructured Non-statistical Develop an initial understanding Quantitative Research To quantify the data and generalize the results from the sample to the population of interest Large number of representative cases Structured Statistical Recommend a final courseof action Objective Sample Data Collection Data Analysis Outcome Qualitative vs. Quantitative Research 7

8. Qualitative vs. Quantitative Research • Detail versus Generalizability • Quantitative Methods • Generalizations to other populations and/or situations • Qualitative Methods • Rich Understanding

9. Exploratory research

10. Focus Groups • Focus groups: small group discussions led by a trained moderator • Objectives: • Generate ideas • Understand consumer vocabulary • Reveal consumer needs, motives, perceptions, and attitudes on products and services • Understand findings from quantitative studies 10

11. Focus Groups • Advantages: • Generation of fresh ideas • Client interaction • Versatility • Ability to tap special respondents • Disadvantages: • Representative of the population? • Subjective interpretation • High cost-per-participant 11

12. In-Depth Interviews (IDIs) • In-Depth interview • A set of probing questions posed one-on-one to a subject by a trained interviewer so as to gain an idea of what the subject thinks about something or why he or she behaves a certain way 12

13. Interviews • Not influenced by others • Max probing, great depth • Expensive • Time consuming • Candid, sensitive topics Popular Qualitative Research Methods Focus Groups • Qualitative • Flexible • Probing • Richness of data • Gets at the “Why” of customers’ behaviors • Generates ideas • Clarifies other project results • Group dynamics, expect more creative • Some probing • Relatively inexpensive • Ready industry 13

14. Still Other Qualitative Methods • Ethnographies • developing understandings of the everyday activities of people in local settings • Observation • Insight into actual, not reported, behaviors • Protocol analysis • involves placing a person in a decision making situation and asking him or her to verbalize everything he or she considers when making a decision • Projective Techniques • participants are placed in (projected into) simulated activities in the hopes that they will divulge things about themselves that they might not reveal under direct questioning

15. “Alternative” Techniques • Implicit Association Test • https://implicit.harvard.edu/implicit/ • Resistometer • Eye Tracking

16. Mystery Shopping • Mystery shoppers are people who pose as consumers and shop at a company’s own stores or those of its competitors • Purpose of Mystery Shopping • Evaluate customer service • Measure employee training • Recognize good employees • Monitoring the competition • Prepare for competition

17. descriptive research

18. Descriptive Research • For our purposes, “survey” research • Asking a sample of people from a population a set of questions • Using the answers to describe that population • Common Goals • Describe what is going on or exists • Estimate how groups of consumers might behave • Examine relationships between two or more variables • Predict

20. Descriptive Research • In contrast to exploratory (qualitative) techniques, the purpose of survey research is to produce statistics • In contrast to exploratory (qualitative) techniques, survey research is concerned with generalizability • Sampling becomes very important

21. Descriptive Research • Three key aspects • Designing questions • Sampling • Data collection • We want to have a study that is precise, credible, and accurate • A key issue is reducing error throughout the process (Fowler) • Poor attention to ANY of these aspects can result in poor results (Fowler, Ch. 13)

22. Descriptive Research

23. Descriptive Research • Two Basic Types • Longitudinal • Cross-Sectional

24. Limitations… • Correlation is NOT causation • Survey research is limited by what people are willing and able to tell us in the context of a survey • This limitation can be addressed using other methods (triangulation)

25. causal research

26. Causal Research • Helps us determine if one or more IVs (treatment, predictors) causes or affects one or more DVs (outcome variables) • Most demanding design—strongest conclusion • Requires the highest degree of understanding of the problem

27. Evidence that supports a causal inference… • Correlation • Observe the relationship (variability) between ad spend and sales • Independent variable (treatment) occurs before the dependent (outcome) variable • Change ad spend and look for an effect on sales. • Eliminate “Alternative Explanations” • If we observe an increase in sales when we increase ad spend…. • Were there changes in any other parts of our marketing mix? • What is going on with the economy? • Any changes in competition’s ad spend? • Etc, etc, etc……

28. Types of Experiments Laboratory Experiment Research investigation in which investigator creates a situation with exact conditions, so as to control some, and manipulate other, variables Experiment Scientific investigation in which an investigator manipulates and controls one or more independent variables and observes the dependent variable for variation concomitant to the manipulation of the independent variables Field Experiment Research study in a realistic situation in which one or more independent variables are manipulated by the experimenter under as carefully controlled conditions as the situation will permit 28

29. Definitions and Concepts • Independent variables (IV) are variables or alternatives that are manipulated and whose effects are measured and compared, e.g., price levels. • Test units are individuals, organizations, or other entities whose response to the independent variables or treatments is being examined, e.g., consumers or stores. • Dependent variables (DV) are the variables which measure the effect of the independent variables on the test units, e.g., sales, profits, and market shares. • Extraneous variables are all variables other than the independent variables that affect the response of the test units, e.g., store size, store location, and competitive effort. • Covariates

30. Validity • Internal validity refers to whether the manipulation of the independent variables or treatments actually caused the observed effects on the dependent variables. Control of extraneous variables is a necessary condition for establishing internal validity. • External validity refers to whether the cause-and-effect relationships found in the experiment can be generalized. To what populations, settings, times, independent variables and dependent variables can the results be projected?

31. Causal Research (Experimental Design) • Internal Validity

32. Causal Research (Experimental Design) • External Validity

33. Controlling Extraneous Variables • Randomization • random assignment of treatment conditions to experimental groups • Assumed to produce ‘balancing’ across groups • this is the key to internal validity (extraneous variables are equal across groups due to random assignment)

34. Characteristics of “Good” Experiments • Random assignment • Comparison group/control group • As a source of comparison • As a control for rival hypotheses • Generalizability/external validity • Random selection

35. Limitations of Experimentation • Experiments can be time consuming, particularly if the researcher is interested in measuring the long-term effects. • Experiments are often expensive. The requirements of experimental group, control group, and multiple measurements significantly add to the cost of research. • Experiments can be difficult to administer. It may be impossible to control for the effects of the extraneous variables, particularly in a field environment. • Competitors may deliberately contaminate the results of a field experiment.

36. To conclude… • Experiments are the only way to show causation • But often take a back seat to descriptive studies due to time, cost, and control issues • Exploratory and descriptive studies are useful, but be careful not to infer too much • Correlation is not causation • Again, let your research questions dictate your design!

37. PharmaFirm • Objective • Test the “fair balance” of different versions of websites • Print material was the “control” • Design • 4 (control, ws1, ws2, ws3) x 2 (GP, Surgeons)

38. Team Assignment #2 • Refine your research questions • Need to be clear, concise, and “testable” • Based on your research questions • Design 2 potential studies that could address your research questions • Explain the benefits and weaknesses of each approach • Pick the “best” design and explain your decision (Note: Don’t worry about measurement or sampling too much---you’ll have your chance to do that later)