Hypothesis Testing

1. Hypothesis Testing Repeated-Measures Data with One Sample

2. Repeated-Measures Testing with One Sample • One-Sample Z/T-Test: • Looks at difference between one sample and its population at one point in time • What if we wanted to compare a sample with its population when we had data at multiple time points (i.e. one subject provided data for multiple groups/samples)?

3. Repeated-Measures Testing with One Sample • Why would we want to do this? • If we were devising a treatment for a particular disorder, we’d want to know how our subjects fared post-treatment as a function of their pre-treatment score • I.e. if we know that Bob has a score of 10/21 on a measure of anxiety after participating in a treatment for social phobia, treatment is only successful if Bob scored much less than this before treatment • Knowing Bob’s score at two time points (before and after treatment), helps us to make this determination

4. Repeated-Measures Testing with One Sample • Taking multiple assessments of one subject over time gives us more data from which to draw conclusions, allows us to use more powerful statistical tests, and therefore requires fewer subjects to be run • “Powerful” = the test can detect a real difference between two groups, or between two or more time points with fewer subjects (Remember – sample size effects the likelihood of obtaining a significant p-value)

5. Repeated-Measures Testing with One Sample • OK, well then why not just pretend the data from Time 1 is one sample, the data from Time 2 is another sample, and compare the means on the two? • This is a big statistical no-no – one of the assumptions of the Two-Independent Samples T-Test (covered in the next chapter) is that the samples are independent/unrelated • If subjects provide data at Time 1(pre-Tx sample) and Time 2 (post-Tx sample), these two samples are not independent, they are probably related. If Bob experiences the most anxiety (has the highest score on our anxiety scale) before treatment, he probably experiences the most anxiety after treatment (has the highest score after treatment)

6. Repeated-Measures Testing with One Sample • OK, so what do we do then? • Use the Two Related Samples/Repeated Measures/Matched Samples T-Test • How does this test differ from other t-tests? • It uses difference/gain scores • Difference Score = scores representing the difference between performance on two occasions • i.e. Time 2 – Time 1, or visa-versa

7. Repeated-Measures Testing with One Sample • Difference Scores

8. Repeated-Measures Testing with One Sample • Ho for Repeated Measures Tests: • For a Two-Tailed Test: • There is no difference in scores from Time 1 to Time 2 • AKA The average of the population of difference scores (μD) = 0 • AKA μD = μ1 (population mean of Time 1 scores)–μ2 (population mean of Time 2 scores) = 0 • H1 for Repeated Measures Tests: • For a Two-Tailed Test: • There is a difference in scores from Time 1 to Time 2 • μD = μ1–μ2 ≠ 0

9. Repeated-Measures Testing with One Sample • Ho for Repeated Measures Tests: • For a One-Tailed Test: • If the Time 1 scores are lower than Time 2 (if they’re higher than you would just use “>” instead of “<“) • μ1 (population mean of Time 1 scores)<μ2 (population mean of Time 2 scores) • H1 for Repeated Measures Tests: • For a Two-Tailed Test: • The Time 1 scores are equal to or higher than Time 2 (if not, use “” instead of “”) • μD = μ1μ2

10. Repeated-Measures Testing with One Sample • How do we calculate a Repeated Measures T-Test? • Same as previous t-tests, just using difference scores as opposed to raw scores

11. Repeated-Measures Testing with One Sample • = the mean of the difference scores • s = the standard deviation of difference scores • N = the number of difference scores (NOT the total number of scores, which is twice the number of difference scores) • df = N – 1 = number of pairs minus 1

12. Repeated-Measures Testing with One Sample • Review – When to use Repeated Measures T-Tests: • 1. One subject provides data for two time points • 2. One subjects provides data for two groups/samples

13. Repeated-Measures Testing with One Sample • Example #1: • The following data is from an experimental treatment that I conducted to help prevent depression by taking pessimistic people (who are at risk for developing depression) and making them more optimistic. The data are levels of depression as assessed with the Beck Depression Inventory-II (BDI-II), both before and after treatment, for a subset of those in the group receiving treatment (we also had a group that didn’t receive treatment). • Why did we have a no treatment group?

14. Repeated-Measures Testing with One Sample

15. Repeated-Measures Testing with One Sample • Example #1: • Difference Scores: 0, -1, 2, 0, 0, 4, 10, 7, -5, 0 • Diff. Scores2: 0, 1, 4, 0, 0, 16, 100, 49, 25, 0 • = 17/10 = 1.7 • s2 = = [195 - (172/10)]/9 = 18.46

16. Repeated-Measures Testing with One Sample • s = √18.46 = 4.30 • t = (1.7 – 0)/(4.30/√10) = 1.25 • Critical t (df = 9, two-tailed, p < .05) = 2.262 • We would fail to reject Ho

17. Repeated-Measures Testing with One Sample • Why not to use Repeated Measures data? • 1. If subjects are asked to take the same measures at two time points, they may respond similarly to Time 2 as to Time 1 if they can recall their earlier responses. • Solution: Create two measures that are similar, but not identical • Although this involves proving that they’re similar

18. Repeated-Measures Testing with One Sample • Why not to use Repeated Measures data? • 2. The questions asked on the test at Time 1 may tip off subjects to the point of the experiment, and contaminate the results. • Solution: Throw in “filler” items to the measures that have nothing to do with your experiment and that you’ll essentially ignore in data analysis to throw off your wiley subjects