1 / 58

Hypothesis Testing

Hypothesis Testing. Quantitative Methods in HPELS 440:210. Agenda. Introduction Hypothesis Testing  General Process Errors in Hypothesis Testing One vs. Two Tailed Tests Effect Size and Power Instat Example. Introduction  Hypothesis Testing. Recall:

rolf
Download Presentation

Hypothesis Testing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Hypothesis Testing Quantitative Methods in HPELS 440:210

  2. Agenda • Introduction • Hypothesis Testing  General Process • Errors in Hypothesis Testing • One vs. Two Tailed Tests • Effect Size and Power • Instat • Example

  3. Introduction  Hypothesis Testing • Recall: • Inferential Statistics: Calculation of sample statistic to make predictions about population parameter • Two potential problems with samples: • Sampling error • Variation between samples • Infinite # of samples  predictable pattern  sampling distribution • Normal • µ = µM • M = /√n

  4. Introduction  Hypothesis Testing • Common statistical procedure • Allows for comparison of means • General process: • State hypotheses • Set criteria for decision making • Collect data  calculate statistic • Make decision

  5. Introduction  Hypothesis Testing • Remainder of presentation will use following concepts to perform a hypothesis test: • Z-score • Probability • Sampling distribution

  6. Agenda • Introduction • Hypothesis Testing  General Process • Errors in Hypothesis Testing • One vs. Two Tailed Tests • Effect Size and Power • Instat • Example

  7. General Process of HT • Step 1: State hypotheses • Step 2: Set criteria for decision making • Step 3: Collect data and calculate statistic • Step 4: Make decision

  8. Step 1: State Hypotheses • Two types of hypotheses: • Null Hypothesis (H0): • Alternative Hypothesis (H1): • Directional • Non-directional • Only one can be true • Example 8.1, p 223

  9. Assume the following about 2-year olds: µ = 26  = 4 M = /√n = 1 n = 16 Researchers want to know if extra handling/stimulation of babies will result in increased body weight once the baby reaches 2 years of age

  10. Null Hypothesis: H0: Sample mean = 26 Alternative Hypothesis: H1: Sample mean ≠ 26

  11. Reality: Only ONE sample will be chose Assume that this distribution is the “TRUE” representation of the population µM Recall: If an INFINITE number of samples are taken, the SAMPLING DISTRIBUTION will be NORMAL with µ = µM and will be identical to the population distribution What is the probability of choosing a sample with a mean (M) that is 1, 2, or 3 SD above or below the mean (µM)?

  12. µM µM p(M > µM + 1 ) = 15.87% p(M > µM + 2 ) = 2.28% µM p(M > µM + 3 ) = 0.13%

  13. Inferential statistics is based on the assumption that our sample is PROBABLY representative of the population µM It is much more PROBABLE that our sample mean (M) will fall closer to the mean of the means (µM) as well as the population mean (µ)

  14. µM Our sample could be here, or here, or here, but we assume that it is here!

  15. H0: Sample mean = 26 If true (no effect): 1.) It is PROBABLE that the sample mean (M) will fall in the middle 2.) It is IMPROBABLE that the sample mean (M) will fall in the extreme edges H1: Sample mean ≠ 26 If true (effect): 1.) It is PROBABLE that the sample mean (M) will fall in the extreme edges 2.) It is IMPROBABLE that the sample mean (M) will fall in the middle

  16. µ = 26 M = 30 Assume that M = 30 lbs (n = 16) Accept or reject? H0: Sample mean = 26 What criteria do you use to make the decision?

  17. Step 2: Set Criteria for Decision • A sampling distribution can be divided into two sections: • Middle: Sample means likely to be obtained if H0 is accepted • Ends: Sample means not likely to be obtained if H0 is rejected • Alpha (a) is the criteria that defines the boundaries of each section

  18. Step 2: Set Criteria for Decision • Alpha: • AKA  level of significance • Ask this question: • What degree of certainty do I need to reject the H0? • 90% certain: a = 0.10 • 95% certain: a = 0.05 • 99% certain: a = 0.01

  19. Step 2: Set Criteria for Decision • As level of certainty increases: • a decreases • Middle section gets larger • Critical regions (edges) get smaller • Bottom line: A larger test statistic is needed to reject the H0

  20. Step 2: Set Criteria for Decision • Directional vs. non-directional alternative hypotheses • Directional: • H1: M > or < X • Non-directional: • H1: M ≠ X • Which is more difficult to reject H0?

  21. Step 2: Set Criteria for Decision • Z-scores represent boundaries that divide sampling distribution • Non-directional: • a = 0.10 defined by Z = 1.64 • a = 0.05 defined by Z = 1.96 • a = 0.01 defined by Z = 2.57 • Directional: • a = 0.10 defined by Z = 1.28 • a = 0.05 defined by Z = 1.64 • a = 0.01 defined by Z = 2.33

  22. Critical Z-Scores  Non-Directional Hypotheses Z=1.64 Z=1.64 Z=1.96 Z=1.96 Z=2.58 Z=2.58 90% 95% 99%

  23. Critical Z-Scores  Directional Hypotheses Z=1.28 Z=1.64 Z=2.34 90% 95% 99%

  24. Step 2: Set Criteria for Decision • Where should you set alpha? • Exploratory research  0.10 • Most common  0.05 • 0.01 or lower?

  25. Step 3: Collect Data/Calculate Statistic • Z = M - µ / M where: • M = sample mean • µ = value from the null hypothesis • H0: sample = X • M = /√n • Note: Population  must be known otherwise the Z-score is an inappropriate statistic!!!!!

  26. Step 3: Collect Data/Calculate Statistic • Example 8.1  Continued

  27. M = 30 Researchers want to know if extra handling/stimulation of babies will result in increased body weight once the baby reaches 2 years of age Assume the following about 2-year olds: µ = 26  = 4 M = /√n = 1 n = 16 Z = M - µ / M Z = 30 – 26 / 1 Z = 4 / 1 = 4.0

  28. Step 4: Make Decision • Process: • Draw a sketch with critical Z-score • Assume non-directional • Alpha = 0.05 • Plot Z-score statistic on sketch • Make decision

  29. Step 1: Draw sketch Critical Z-score Z = 1.96 Critical Z-score Z = 1.96 µ = 26 M = 30 Z = 4.0 Step 2: Plot Z-score Step 3: Make Decision: Z = 4.0  falls inside the critical region If H0 is false, it is PROBABLE that the Z-score will fall in the critical region ACCEPT OR REJECT?

  30. Agenda • Introduction • Hypothesis Testing  General Process • Errors in Hypothesis Testing • One vs. Two Tailed Tests • Effect Size and Power • Instat • Example

  31. Errors in Hypothesis Testing • Recall  Problems with samples: • Sampling error • Variability of samples • Inferential statistics use sample statistics to predict population parameters • There is ALWAYS chance for error

  32. Errors in Hypothesis Testing • There is potential for two kinds of error: • Type I error • Type II error

  33. Type I Error • Rejection of a true H0 • Recall  alpha = certainty of rejecting H0 • Example: • Alpha = 0.05 • 95% certain of correctly rejecting the H0 • Therefore 5% certain of incorrectly rejecting the H0 • Alpha maybe thought of as the “probability of making a Type I error • Consequences: • False report • Waste of time/resources

  34. Type II Error • Acceptance of a false H0 • Consequences: • Not as serious as Type I error • Researcher may repeat experiment if type II error is suspected

  35. Agenda • Introduction • Hypothesis Testing  General Process • Errors in Hypothesis Testing • One vs. Two Tailed Tests • Effect Size and Power • Instat • Example

  36. One vs. Two-Tailed Tests • One-Tailed (Directional) Tests: • Specify an increase or decrease in the alternative hypothesis • Advantage: More powerful • Disadvantage: Prior knowledge required

  37. One vs. Two-Tailed Tests • Two-Tailed (Non-Directional) Tests: • Do not specify an increase or decrease in the alternative hypothesis • Advantage: No prior knowledge required • Disadvantage: Less powerful

  38. Agenda • Introduction • Hypothesis Testing  General Process • Errors in Hypothesis Testing • One vs. Two Tailed Tests • Effect Size and Power • Instat • Example

  39. Recall  Step 4: Make a Decision • Statistical Software  p-value • The p-value is the probability of a type I error • Recall alpha (a)

  40. Recall  Step 4: Make a Decision • If the p-value > a accept the H0 • Probability of type I error is too high • Researcher is not “comfortable” stating that differences are real and not due to chance • If the p-value < a  reject the H0 • Probability of type I error is low enough • Researcher is “comfortable” stating that differences are real and not due to chance

  41. Statistical vs. Practical Significance • Distinction: • Statistical significance: There is an acceptably low chance of a type I error • Practical significance: The actual difference between the means are not trivial in their practical applications

  42. Practically Significant? • Knowledge and experience • Examine effect size • The magnitude of the effect • Examples of measures of effect size: • Eta-squared (h2) • Cohen’s d • R2 • Interpretation of effect size: • 0.0 – 0.2 = small effect • 0.21 – 0.8 = moderate effect • > 0.8 = large effect • Examine power of test

  43. Statistical Power • Statistical power: The probability that you will correctly reject a false H0 • Power = 1 – b where • b = probability of type II error • Example: Statistical power = 0.80 therefore: • 80% chance of correctly rejecting a false H0 • 20% of accepting a false H0 (type II error)

  44. Statistical Power • What influences power? • Sample size: As n increases, power increases - Under researcher’s control • Alpha: As a increases, b decreases therefore power increases - Under researcher’s control (to an extent) • Effect size: As ES increases, power increases - Not under researcher’s control

  45. Statistical Power • How much power is desirable? • General rule: Set b as 4*a • Example: • a = 0.05, therfore b = 4*0.05 = 0.20 • Power = 1 – b = 1 – 0.20 = 0.80

  46. Statistical Power • What if you don’t have enough power? • More subjects • What if you can’t recruit more subjects and you want to prevent not having enough power? • Estimate optimal sample size a priori • See statistician with following information: • Alpha • Desired power • Knowledge about effect size  what constitutes a small, moderate or large effect size relative to your dependent variable

  47. Statistical Power • Examples: • Novice athlete improves vertical jump height by 2 inches after 8 weeks of training • Elite athlete improves vertical jump height by 2 inches after 8 weeks of training

  48. Agenda • Introduction • Hypothesis Testing  General Process • Errors in Hypothesis Testing • One vs. Two Tailed Tests • Instat • Example

More Related