1 / 17

Statistical vs. Practical Significance

Statistical Significance. A finding is statistically significant when it allows the researcher to reject the null hypothesis at a pre-specified level of confidence or probability. This level is called the alpha level (?) and for educational purposes is usually set at .05.. Statistical Sign

derrick
Download Presentation

Statistical vs. Practical Significance

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

    1. Statistical vs. Practical Significance A Presentation by: Raymond Giovanelli, Melanie McKenzie, Tom Bulla, Jeff Cox, and Kim Mattox

    2. Statistical Significance A finding is statistically significant when it allows the researcher to reject the null hypothesis at a pre-specified level of confidence or probability. This level is called the alpha level (?) and for educational purposes is usually set at .05.

    3. Statistical Significance Answers the questions: Are the results beyond what would be expected due to sampling error alone? Are the results large enough that it is very unlikely that they are due to sampling error?

    4. Statistical Significance Some Tools of the Trade t-Tests One-way Analysis of Variance (ANOVA) Regression Analysis To learn more about t-tests click here

    5. Statistical Significance Findings are said to be statistically significant when the p value associated with the test statistic is smaller than the predetermined alpha level (usually .05). A p value of less than .05 would result in rejecting the null hypothesis.

    6. But does it really mean anything? Just because a finding is statistically significant doesn’t necessarily mean much to the practitioner or statistician.

    7. Practical Significance While a finding may be statistically significant it may or may not be practically significant. A researcher must contemplate the usefulness of the finding to determine its practical significance.

    8. Practical Significance When determining practical significance the researcher must consider the following: The quality of the research questions The relative size of the effect The size of the sample The importance of the finding Confidence intervals The link to previous research The strength of correlation

    9. Practical Significance Just knowing that A and B are different is not enough. The researcher needs to know the size of the difference and the error associated with the estimate.

    10. Practical Significance Effect sizes are utilized to let the researcher know how large the differences are and if the differences found have any practical significance.

    11. Practical Significance Cohen’s Effect Size Guidelines: < .2 Small .5 Moderate .8 Large

    12. Consider this example Of 200,000 students, the mean IQ scores (M = 100.15, SD = 15) from 12,000 students randomly assigned to one zip code were found to be statistically significantly (p<.05) higher than the IQ scores of the other 188,000 students (M = 99.85, SD = 15). Thompson, 2002

    13. Would you recommend implementing a special “gifted” program for these children? NO

    14. Clinical Significance Similar to practical significance, clinical significance addresses the following question: Are treated individuals as a group indistinguishable from normals with respect to the primary complaints following treatment? Thompson, 2002

    15. Clinical Significance Clinical significance refers to the practical or applied value or importance of the effect of the intervention--that is, whether the intervention makes a real (e.g., genuine, palpable, practical, noticeable) difference in everyday life to the clients or to others with whom the client interacts. Effect sizes are important. Thompson, 2002

    16. Review A test for statistical significance only tells you that the results are unlikely to have been caused by sampling error. Only you can determine whether the results have any practical significance

    17. Some keys for further information:

More Related