Research Methods: Midterm Review Part 2

1. Research Methods: Midterm Review Part 2 Dr. Dodge February 28, 2006

2. Variables

3. Variables • Variables: building blocks of hypotheses that are held together by the “glue” of the relationship we are studying. • Wide range of definitions and categories of variables. • Characteristics are not fixed but are able to vary (take on more than one value)

4. Variables Functions • Independent variable:“…is the factor that is manipulated or controlled by the researcher” • Variable that is “independent of the outcome being measured. What causes or influences the outcome” _________________________________________ • Dependent variable: “is a measure of the effect (if any) of the independent variable • Is influenced by the independent variable • Factor that is observed or measured to determine the effect of the independent variable

5. Variables: Measurement Scales • Two different scales for measurement of variables: • Continuous or categorical • Nominal, ordinal, interval, or ratio

6. Variables: Measurement Scales • Continuous or Categorical • Continuous variables: have an ordered set of values within a certain range. Values between two points (e.g., 4 and 5) on the range actually mean something. • Categorical variables (i.e., discrete variables): measured in categories. An observation is either in a category or it isn't. There is no meaningful “in between” option. • When planning data collection, always try to collect data in a continuous format

7. Variables: Measurement Scales • Nominal, Ordinal, Interval, or Ratio • Nominal: Names, classes, or symbols designating unique characteristics - simple classification, no order. • Ordinal: Assignment of numbers of symbols indicates order of relationship. Order only is indicated; there is no indication of amount. Ex: rank order data. • Interval: has the same ordering properties as ordinal data and it also has equal, meaningful intervals and an arbitrary zero point. • Ratio: has the same properties as interval data and also has an absolute zero point.

8. Variable Levels and Factors • The most basic experimental design has two variables • Independent Variable • Dependent Variable • The independent variable has two Levels • Experimental Group (Usually receives treatment) • Control Group (Usually does not receive treatment) • A grouping variable is called a “factor” • The number of groups are called “levels”

9. Levels and Factors(4 Level Factor)

10. Research Questions

11. Research Questions • Questions that guide your research • Should be debatable and of interest to both you and your potential readers • Should also be based on a narrow topic • Should guide your research • You can have more than one research question in a study

12. Defining Research Questions • To help define questions: • People, patients or population - who are you asking the question about? • Intervention - what intervention are you interested in? • Control or comparison - what are you comparing the intervention to? • Outcome - what outcome are you interested in measuring?

13. Hypotheses

14. Hypotheses • Hypotheses: predictions about the relationship among two or more variables or groups based on a theory or previous research • Assumptions or theories that a researcher makes and tests

15. Hypotheses • Hypotheses are important because they: • Direct our observation: identifies the variables examined and data to be collected • Describe a relationship among variables: state that as one variable increases, the other will decrease; as one variables increases, the other will increase … • Refer to populations: help researchers infer that results of a sample will translate to a population!

16. Hypotheses • Hypotheses have four functions: • Estimate population characteristics • Correlate variables • Display differences among two or more populations • Show possible cause and effect • Two types of hypotheses: • Research hypotheses • Statistical hypotheses

17. Research Hypotheses • Research Hypothesis: statement of the relationship among two or more variables or groups • Acceptance or non-acceptance is based on resolving a logical alternative with a null hypothesis. • Example: Students who attend school regularly will score higher on their FCAT exams than students who do not.

18. Research Hypotheses • Can be “directional” or “non-directional.” • Directional hypotheses: predict specific relationship among two or more variables or groups • Show possible cause and effect • Ex: IQ scores will correlate in a positive manner with shoe size. (And why would that be?  )

19. Research Hypotheses • Non-Directional Hypotheses: predict differences among two or more groups, but do not specify the direction of the differences • Ex: Men and women will differ on measures of sexual arousal when exposed to explicit auditory sexual stimuli.

20. Statistical Hypotheses • Statistical hypotheses: mathematical, or logical, statements that help researchers interpret the results of research • Statistical hypotheses consist of the Null Hypothesis (H0), the hypothesis of no difference, and the Alternative Hypothesis (H1 or HA) which is similar in form to the research hypothesis. Both can be expressed in alphanumerical formulae. • Null: (H0: µ1 - µ2 = 0 ) • Alternative: (H1: µ1-µ2 ≠ 0)

21. Statistical Hypotheses • In other words … • Null: There will be no difference on measures of aggression between students who have completed the research methods midterm review and students who have not completed the review. • Alternative: There will be a difference on measures of aggression between students who have completed the research methods midterm review and students who have not completed the review.

22. N.B.! • The null hypothesis always implies that there is no relation or statistical difference between variables or groups • The alternative hypothesis always implies that there is a meaningful relationship among variables or groups

23. Testing Hypotheses • We only test the null hypothesis • We do not test the research hypothesis • We use a variety of statistical procedures to test null hypotheses. Depends on a variety of factors including the research hypothesis, the data, the sampling strategy, and what we want to be able to say as a result of our testing.

24. Types of Tests • Statistical procedures: correlation, analysis of variance (ANOVA), analysis of covariance (ANCOVA), regression, multivariate analysis of variance (MANOVA), t-tests, and Chi-Square. • Each procedures has an associated test statistic used to determine significance. Ex: ANOVA, ANCOVA, and regression use F statistics and their associated p-values. • Important: All test statistics are eventually related to a probability distribution and a p-value. The p-values mean the same thing across test statistics.

25. Error Types • Type I and Type II Errors – inherent in hypothesis testing. Errors are mistakes that we can make when judging the null hypothesis. • Type I Error: the tested hypothesis is falsely rejected. (You say you found something, but that something is really an error.) A type I error is a false positive. • Type II Error: when a false tested hypothesis is not rejected (You do not find something that is, in fact, there.) A type II error is a false negative.

26. Alpha and Beta • Alpha: level of probability (pre-set by the researcher) that the tested hypothesis will be falsely rejected. Alpha is the pre-set risk of a Type I error. The degree of risk that you accept, in advance of conducting the study, that what you find will be an error. • Beta: level of probability that a false null hypothesis will not be rejected. The probability that you won’t find what you are looking for if, in fact, it is really there. • Probability (p) Value: Probability that observed relationships or differences are due to chance. Alpha is also known as significance level or rejection region.

27. Error Types Chart

28. Power, Effect Size, and Measurement • Statistical power: probability of rejecting a null hypothesis that is, in fact, false. The probability of finding relationships or differences that in fact exist • Statistical power is related to: • Sample size • Effect size • Statistical design • Significance criteria

29. Power, Effect Size, and Measurement • Effect size (ES): amount of variance between the independent variable(s) (IV) and the dependent variable(s) (DV). Degree to which changes in the IV(s) result in changes in the DV(s).

30. Power, Effect Size, and Measurement • Relationships of measurement, research design, and statistical power means that large treatment effects can actually be observed as small effects. • Even if an intervention is very effective, measurement and design complications may make the effect appear small and thus require high statistical power for detection. • We will discuss these issues in the second half of the semester.

31. Test Statistics, Probability, and Significance • WHETHER YOU ARE LOOKING AT OBTAINED VALUES OF TEST STATISTICS IN RELATION TO CRITICAL VALUES OR YOU ARE LOOKING AT ACTUAL PROBABILITY LEVELS, IT IS IMPORTANT TO NOTE THAT TEST STATISTICS AND THEIR ASSOCIATED PROBABILITIES ONLY TELL US THE PROBABILITY THAT A DIFFERENCE OR RELATIONSHIP OCCURRED BY CHANCE. • THESE STATISTICS DO NOT TELL US THE SIZE OF GROUP DIFFERENCES OR THE STRENGTH OF RELATIONSHIPS

32. Steps in Hypothesis Testing for Quantitative Research Designs • Hypothesis testing is a 4 phase procedure: • Phase I: Research Hypotheses, Design, and Variables • Phase II: Statistical Hypotheses • Phase III: Hypotheses Testing • Phase IV: Decision/Interpretation

33. Phase I: Research Hypotheses, Design, and Variables • State your research hypotheses. • Decide on a research design based on your research problem, your hypotheses, and what you really want to be able to say about your results (Ex: if you want to say that A caused B, you will need an experimental design). • Operationally define your variables. Recall that one variable can have more than one operational definition.

34. Phase II: Statistical Hypotheses • Consider your chosen statistical procedures. • Write one statistical null hypotheses for each operational definition of each variable that reflects that statistical operations to be performed.

35. Phase III: Hypotheses Testing • Select a significance level (alpha). • 2. Compute the value of the test statistic (e.g., F, r, t). • 3. Compare the obtained value of the test statistics with the critical value associated with the selected significance level orcompare the obtained p-value with the pre-selected alpha value. • 4. If the obtained value of the test statistic is greater than the critical value (or if the obtained p-value is less than the pre-selected alpha value), reject the null hypothesis. If the obtained value is less than the critical value of the test hypothesis, fail to rejectthe null hypothesis. • In other words: If p is less than or equal to alpha, reject the null hypothesis.

36. Phase IV: Decision/Interpretation • For each research hypothesis, consider the decisions regarding the statistical null hypotheses. • For each research hypothesis, consider qualitative contextual information relating potential plausibility. • Cautiously explain your findings with respect to the research hypotheses. • List and discuss the limitation.

37. Questions?