1 / 10

Statistical Significance

Statistical Significance. KNR 164. What is Statistical Significance?. A statistical hypothesis test is a method of making decisions using data from a scientific study

magee
Download Presentation

Statistical Significance

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Statistical Significance KNR 164

  2. What is Statistical Significance? • A statistical hypothesis test is a method of making decisions using data from a scientific study • In statistics, a result is called statistically significant if it has been predicted as unlikely to have occurred by chance alone, according to a pre-determined threshold probability, the significance level

  3. What is a p-value? • p-value is the probability that we would have seen our data (or something more unexpected) just by chance if the null hypothesis (null value) is true • Small p-values mean the null hypothesis is unlikely given the data

  4. What is a p-value? • Test to see if you can reject the null hypothesis • Many common statistical tests, such as chi-squared tests or Student's t-test, produce test statistics which can be interpreted using p-values • If you reject the null hypothesis, your results indicate that the observed result would be highly unlikely under the null hypothesis • that is, the observation is highly unlikely to be the result of random chance alone

  5. What is a p-value? • Our data are so unlikely given the null hypothesis that I’m going to reject the null hypothesis! • Don’t want to reject our data – we want to accept our hypothesis and reject the null as truth (and be confident in doing this)

  6. Alpha • Alpha () is the level of significance in hypothesis testing: • Alpha is a probability specified before the test is performed. • Alpha is the probability of rejecting the null hypothesis when it is true. • By convention, typical values of alpha specified in medical research are 0.05 and 0.01.

  7. What does Alpha Level Mean? • Alpha () level indicates at what p-value the null will be rejected • Example: • = 0.05; calculated p-value = 0.008; rejectnull hypothesis • = 0.05; calculated p-value = 0.110; do not reject null hypothesis

  8. The p-value • By convention, p-values of <.05 are often accepted as “statistically significant” in the medical literature; but this is an arbitrary cut-off • A cut-off of p<.05 ( = .05) means that in about 5 of 100 experiments, a result would appear significant just by chance

  9. Hypothesis Testing The Steps: 1. Define your hypotheses (null, alternative) 2. Specify your null distribution 3. Do an experiment 4. Calculate the p-value of what you observed 5. Reject or fail to reject (~accept) the null hypothesis

  10. How to interpret a p-value • P-value is significant or it is not • It is a all-or-none decision • Compared to the a priori p-value • How confident do you want to be given your result? p< .05 says 95% confident. • p = .001 is the same as p = .02 in terms of interpretation as (they are both less then p = .05) Gilbert PB, Berger JO, Stablein D, Becker S, Essex M, Hammer SM, Kim JH, DeGruttola VG. Statistical interpretation of the RV144 HIV vaccine efficacy trial in Thailand: a case study for statistical issues in efficacy trials. J Infect Dis 2011; 203: 969-975.

More Related