1 / 10

Type I & Type II errors

Type I & Type II errors. Brian Yuen 18 June 2013. Definition of Type I error ( α ). Concluded a statistical significant effect size from the sample when this does not exist in the whole population (H 0 is true) Reject the null hypothesis (H 0 ) when it is true False positive result

fiona
Download Presentation

Type I & Type II errors

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Type I & Type II errors Brian Yuen18 June 2013

  2. Definition of Type I error (α) • Concluded a statistical significant effect size from the sample when this does not exist in the whole population (H0 is true) • Reject the null hypothesis (H0) when it is true • False positive result • Level of significance - P value • Usually allowed for 5% - arbitrary but acceptable rate! • Repeatedly sample and test from the same population, you will expect to make 1 wrong conclusion (with statistically significant result) in 20 of these tests if the null hypothesis is in fact true

  3. Definition of Type II error (β) • Concluded a non-statistical significant effect size from the sample when this exists in the whole population (H1 true) • Fail to reject the null hypothesis (H0) when it is not true • False negative result • Power of study = 100% - False negative rate • Maximum accepted false negative rate at 20%  Minimum power of study at 80%

  4. What does a significant result mean? Chance findings fall into this region • Remember we are trying to find enough evidence to reject the null hypothesis • i.e. to show that the observed effect size has exceeded our pre-set threshold of expected events, hence a statistical significant result • Observed a significant result, are we • lucky to find out null hypothesis is indeed incorrect? • A genuine finding • unlucky to observed an unlikely result/event? • Type I error Findings fall in these 2 regions are too extreme to say these by chance

  5. Can we eliminate these errors completely? • Type I error • Not really, there is always a pre-set (5%) chance for a Type I error • You could minimise this by reducing the level of significance from 5% to 1%, but you would expect a larger sample size • Type II error • Not really, there is always a pre-set (20%) chance for a Type II error • You could minimise this by reducing the error rate from 20% to 10%, but you would compromise this with a larger sample size

  6. Can we ever work out if Type I error exist in published results? • Yes, if the result is statistically significant, then there is a possibility of a Type I error • Since we allowed for 5% chance to have this error, there is always a possibility of getting such error and there is no way we can get rid of this doubt • We could work out if there is any hint of an unusual/unlikely result by comparing results from other similar studies • Testing multiple outcomes issue • P<0.05 means it is unlikely that the observed effect size is by chance (less than 5%) • P<0.01 means it is unlikely that the observed effect size is by chance (less than 1%)

  7. Can we ever work out if Type II error exist in published results? • Yes, if the result is not statistically significant, then there is a possibility of a Type II error • The key point is to identify if the author had performed a sample size calculation • with a pre-defined power before the study starts • on the one primary outcome they made conclusive statement • whether they had recruited the desired number at the analysis stage • whether they had pre-defined a clinically worthwhile effect size to be detected and was observed • Then the chance of having a Type II error would be the pre-set value, i.e. 20%

  8. Type I error (α) & Type II error (β)

  9. Mind twisting quizzes! • In a published paper, a statistically significant result of a fully powered 2-group study was reported on a single primary outcome. What can you comment on this result regarding Type I/II error? • In a published paper, a non statistically significant result of a fully powered 2-group study was reported on a single primary outcome. What can you comment on this result regarding Type I/II error? • You have reviewed 20 studies of a similar kind regarding the same outcome measure. You have found 5 studies with statistically significant results, while the others were not. What is your view on these results?

  10. Reference • http://www.bmj.com/about-bmj/resources-readers/publications/statistics-square-one/5-differences-between-means-type-i-an

More Related