Research design

1. Research design IMD09120: Collaborative Media Brian Davison 2010/11

2. Research design • Recap • Variables • Types of design • Hypotheses • Sampling • Your designs

3. Recap: tests you have done (1) • Can the number of social networking accounts predict the number of contacts a person has? • Do female students have significantly more social networking contacts than male students? • Does face-to-face differ significantly from WebCT chat as a channel of communication in terms of satisfaction? • Do students from different schools differ significantly in the number of social networking contacts they have?

4. Recap: tests you have done (2) • Do students differ significantly by age in the number of social networking contacts they have? • Does the quality of video and/or group size have a significant effect on satisfaction in mediated group communication? • Is the preference for Facebook statistically significant? • Is preferred social networking site related to gender?

5. Recap: tests you have used • Correlation • Independent t-test • Paired t-test • ANOVA • 2

6. Variables • Things you can measure • Height, weight, age • Number of car journeys • Gender, nationality • Attitude, belief, satisfaction

7. Types of variable • Continuous • Infinitely divisible • Discrete • Values come in defined ordered steps • Categorical • Values are categories with no ordering • Examples • Height • Number of car journeys • Gender • Nationality • Observed symptoms of disease • Attitude • Age • Belief • Weight • Satisfaction

8. Dependence • Independent variable • Manipulated by the investigator • Dependent variable • Measured for effect • Examples • Does face-to-face differ significantly from WebCT chat as a channel of communication in terms of satisfaction? • Is preferred social networking site related to gender?

9. Confounding variable • A factor that affects the dependent variable and which is not included in the design • What confounding variables might there be in the F2F-chat experiment?

10. Research designs • Correlational • Looking at how one variable changes in relation to another • eg smoking and incidence of cancer • Experimental and quasi-experimental • Looking for effects of one variable on another • IV and DV

11. Correlation designs • Collect data about both variables • Measure the extent to which their distributions match • Causation cannot be inferred from a correlation • Which is correlational: • Can the number of social networking accounts predict the number of contacts a person has? • Do female students have significantly more social networking contacts than male students? • Does face-to-face differ significantly from WebCT chat as a channel of communication in terms of satisfaction?

12. Experimental designs • Researcher manipulates IV • Subjects are randomly assigned to test conditions • Which is experimental: • Can the number of social networking accounts predict the number of contacts a person has? • Do female students have significantly more social networking contacts than male students? • Does face-to-face differ significantly from WebCT chat as a channel of communication in terms of satisfaction?

13. Quasi-experimental designs • Some variables cannot be manipulated • eg gender, occupation • Therefore random allocation to conditions is impossible • Which is quasi-experimental: • Do students differ significantly by age in the number of social networking contacts they have? • Does the quality of video and/or group size have a significant effect on satisfaction in mediated group communication? • Is the preference for Facebook statistically significant?

14. Between or within subjects • Between-subjects design • Different subjects randomly assigned to different conditions • Within-subjects design • Same subjects used in all conditions

15. Within-subjects • Also called related or repeated measures • Pro: • Using same subjects helps to eliminate confounding variables • Fewer participants needed • Con: • Subjects may perform differently on second test – ordering effects • Subjects more likely to guess the purpose of the study – demand effects • Not possible for quasi-experimental studies • Ordering effects can be eliminated by counterbalancing

16. Between-subjects • Also called independent or unrelated • May be significant variation within each group • Pro: • No ordering effects • Demand effects are less likely • Con: • More participants required • Less control over confounding variables

17. What kind of design? • Do female students have significantly more social networking contacts than male students? • Does the quality of video and/or group size have a significant effect on satisfaction in mediated group communication? • Does face-to-face differ significantly from WebCT chat as a channel of communication in terms of satisfaction?

18. Population Pop. mean

19. Sampling error Population Sample Pop. mean Spl. mean

20. What rule connects this series of numbers? 10, 20, 30 • Confirmation bias

21. Testing hypotheses • Null hypothesis • There is no relationship / difference between two variables • ie. All observed differences are the result of sampling error • Probability values in tests = probability of the observed result if the null hypothesis is true • Alternative hypothesis • There is a relationship / difference between two variables • Specific to the study you are doing

22. Logic of hypothesis testing • Formulate a hypothesis • Measure the variables and examine their relationship (using samples) • Calculate the probability of the result if the null hypothesis is true • If the calculated probability is small enough, reject the null hypothesis

23. Two types of test Is x significantly larger / smaller than expected? Is x significantly different from expected?

24. How many tails? • Do female students have significantly more social networking contacts than male students? • Does face-to-face differ significantly from webCT chat as a channel of communication in terms of satisfaction? • Do students from different schools differ significantly in the number of social networking contacts they have?

25. Short break

26. Effect size Small effect Large effect • A large effect size is easier to detect than a small one • The power of a test is a measure of its ability to detect an effect • More participants are required to detect a small effect size

27. Cohen’s power primer • Tabulates required sample size according to • Confidence interval • Statistical test • Effect size • Values for 95% confidence • Cohen (1992)

28. ANOVA sample sizes for 95% confidence

29. 2sample sizes for 95% confidence • More precise figures can be calculated using G*Power

30. How many variables do you have? Two More than two Are you looking for differences between conditions, or relationships among variables? Are you looking for differences between conditions, or relationships among variables? Differences Relationships Relationships Differences Do you have a between-participants or a within-participants design? Do you want a regression equation, or simply the strength of a relationship? Are you interested in a regression equation, or exploring clusters of correlations? Do you have more than one DV? Yes No MANOVA Between Within Strength Regression Regression Correlation Do you want to look for differences between conditions while controlling for the effects of another variable? Independent t-test Related t-test Pearson’s Product Moment Correlation Coefficient Linear regression Multiple regression Factor analysis No Yes Do you have more than one IV? Analysis of covariance No Yes Dancey & Reidy (2007) Statistics without maths for psychologists. Prentice Hall (p. 157) One-way ANOVA ANOVA for multiple IVs

31. How many variables do you have? Two More than two Are you looking for differences between conditions, or relationships among variables? Are you looking for differences between conditions, or relationships among variables? Differences Relationships Relationships Differences Do you have a between-participants or a within-participants design? Do you want a regression equation, or simply the strength of a relationship? Are you interested in a regression equation, or exploring clusters of correlations? Do you have more than one DV? Yes No MANOVA Between Within Strength Regression Regression Correlation Do you want to look for differences between conditions while controlling for the effects of another variable? Independent t-test Related t-test Pearson’s Product Moment Correlation Coefficient Linear regression Multiple regression Factor analysis No Yes Do you have more than one IV? Analysis of covariance No Yes Dancey & Reidy (2007) Statistics without maths for psychologists. Prentice Hall (p. 157) One-way ANOVA ANOVA for multiple IVs

32. Your turn • Design a study to test whether caffeine affects level of anxiety.

33. Your turn again • Design a study to test whether mathematical ability is related to musical ability

34. One more time • Design a study to test whether there is a relationship between doing sport and intention to vote

35. Coursework: variables and hypotheses • What are you trying to improve? • eg group identity, social presence, group effectiveness • This is your DV • What are you changing? • eg additional features, different layout, modified presentation • Your IV is WebCT version: may be old/new, or +/- combined factors • What do you expect? • Eg adding feature X leads to reduced social loafing • This is your hypothesis • Null hypothesis is “there is no difference”

36. Coursework: research design • Levels of IV • How to measure DV • Experimental groups • Within subjects or between subjects • Eliminating confounding variables and unwanted effects • Quality of system – create an “as-is” Powerpoint • Ordering effects • Demand effects • Researcher bias

37. Statistical test • Method 1 • What question are you asking? • Which practical example matches? • Method 2 • Follow the flowchart • Use one method to check the other