Statistics for science fairs

1. Statistics for science fairs How to make sure your students have the best chance of success at SSEF of Florida

2. My personal involvement Judging Captain of Senior Animal Sciences How we judge: First, divide projects in orbit and focus groups Use scores to group similar level projects in focus group Discuss top scoring projects to determine overall category winner Last year, much debate over specific project Scores ranked from 1st – 10th depending on judge Upon further discussion, deciding factor came down to use of statistics (incorrect statistics for experimental design) Likely would’ve placed higher, possibly 1st with correct statistics.

3. Judge’s perspective Are there statistics? Are they the right statistics? Does the student understand the statistics? Don’t need to know the underlying calculations being used, just how they work and what they mean

4. Common Pitfalls • Experimental design with small sample sizes • Using averages to make conclusions about data results without looking at the variability in the data • Correlation vs. Causation • Extrapolation of data • Off-the-shelf programs (i.e. Excel) make statistics easy to use, therefore easy to use incorrectly • No statistics at all

5. Role of teachers/advisors • Students are not often familiar with statistics – initial exposure • Preliminary discussion prior to starting project • Good statistical analysis is derived during experimental design • Make sure sample size is adequate • Understand what tests will help determine significance/validity • Reassess statistics as first collecting data • Modifications to design may be necessary • Find mentor to assist with statistics, if needed

6. Experimental design • Keys to designing project with statistics in mind: • Accurately and clearly define variables and sample space • Accurately define factors and levels of factors • Identify the type of experiment, making sure to use appropriate controls • Making sure to perform enough replications • Making sure to understand the likely distribution of data • Awareness of types of exploratory and inferential analyses used in your field of science. Look to journal articles.

7. Variables Quantitative variables – differ in magnitude, can be measured Qualitative variables - categorical, observations differ in kind (nominal) Rank qualitative variables – ordinal variables (allows for mathematical analysis) Pre-planning allows to choose what kind of data you will collect, what statistics you can use.

8. Reducing noise in results • Experimental Observations = combination • Signal – true effects of variable/outcome • Noise – random error introduced by experimental design • Increase signal-to-noise ratio (decrease noise) • Making repeated measurements of one item • Increasing sample size • Randomizing samples • Randomizing experiments • Repeating experiments • Including covariates (other variables that might impact results)

9. Exploratory Data Analysis Categorical Variables - bar graphs, pie charts, two-way tables Quantitative Variables – stem plots, histograms, relative cumulative frequency plots, time plots, scatterplots Calculate and compare mean, median, standard deviation (report value as a measure of center and measure of spread) What is the distribution? Are there outliers?

10. Statistical inference P-value – probability that the observed result is due to chance The probability that from a randomized controlled experiment, the null hypothesis is correct Relationship between 2 quantitative variables Scatter plot and regression Plot independent variable on x-axis; discuss pattern If linear, calculate correlation coefficient to measure strength and relationship Use least squares regression to determine model of relationship between two variables Conduct t-test If not linear, more advanced methods needed

11. Statistical inference Compare data from two different groups Box plots and t-tests Plot data using side-by-side box plot Determine alternative hypothesis Define level of significance – use two-sample t-test Evaluate practical significance of difference between groups Report the results as inferential analysis When testing against set value (not between groups), use one sample t-test 7. If comparing across more than 2 groups, consider ANOVA

12. Statistical inference Inference for categorical variables Testing results against expected distribution Start with two-way table. Calculate marginal distributions and differences between marginal distributions of experimental and control groups Use bar graph or pie chart to show distributions differences among groups 3. Use chi-square test for goodness of fit. Determine test statistic and p-value If chi-square finds significant results, examine to find largest components

13. Presenting results • State statistical hypothesis along with your scientific hypothesis • Use flowchart to show experimental design • Show how replication, control and randomization are used • Show both exploratory data analysis and inferential analysis • Discuss meaning of graphs and measures • State level of significance in tests (p-value) • State conclusions of tests • State the statistical and practical significance of results • Tell whether null hypothesis was accepted or rejected • Statistics are meaningless if they are misrepresented or misunderstood!

14. 95% CI Confidence intervals Standard Deviation – measure of spread of data How far away from the mean are data points? Variance = Standard deviation = How do we express? According to the current data, 68% of data falls within Average 1 standard deviation P-value of .05 = 95% confidence interval = 2 standard deviations If this experiment were repeated on multiple samples, the calculated confidence interval would encompass the true population of the parameter 95% of the time.

15. Regression analysis Estimates relationship between independent and dependent variables Calculates line of best fit between data points Correlation measures how close line fits the data – correlation coefficient (R2) – closest to 1 (positive relationship) or -1 (negative relationship)

16. Is there a statistical difference? Plant height Control = 15 mL water Treatment A = 30 mL water Treatment B = 45 mL water Does the amount of water affect the height of the plant? On the surface, we might say yes! The averages are higher for treatment A and B. But is this really the case?

17. T-test Determines whether the difference in the averages for two or more treatments is mostly caused by the experimental treatment or whether the difference can be explained by random variation Requirements: Two or more comparison groups (control and one treatment, or two or more treatments). A sample size of 10 or more for each experimental group. Numerically measured data (no categories, even if labeled with numbers). Easy to calculate T-Test http://graphpad.com/quickcalcs/ttest1.cfm Excel Other statistics software

18. Control against treatment a P-value: 0.05 is generally accepted > 0.05, no difference between means P-value of 0.0033 = .33% these results are by chance Watering plants with 30 mL instead of 15 mL will increase height under these same conditions

19. Control against treatment B P-value of 0.0117 = 1.17% these results are by chance Watering plants with 45 mL instead of 15 mL will increase height under these same conditions But what about between 30 mL and 45 mL?

20. Treatment a against treatment B P-value of 0.8513 = 85.13% that difference between treatment averages are by chance No significant difference between watering with 30 mL or 45 mL

21. Chi-Square test Compare observed data with expected data according to a hypothesis Convert to a probability using degrees of freedom (n-1) P-value = probability that null hypothesis (observed and expected are not different) is correct Simple example – flip a coin 100 times. Expected 50 heads, tails. Observed 41 heads, 59 tails. = 3.72 2 groups = 1 Degree of Freedom 7% chance that null hypothesis is correct 93% chance what we saw is different than expected BUT… > 0.05 so we support null hypothesis

22. ANOVA Analysis of Variance test Determine if there is a difference between means of 3 or more independent, unrelated groups Tests the null hypothesis that no difference between the groups exists. Easy to perform analysis with software If you determine there is a difference between the groups, additional testing will be needed Tukey HSD Scheffe post hoc test Games Howell Dunnett’s C post hoc test

23. Final thoughts Proper use and understanding of statistics are often the deciding factor between the top projects Give your students the upper hand by discussing statistics at the start of the project Review data and industry standards to determine the correct statistical analysis based on experimental design Properly quantify statistical results in the context of project If possible, review statistical procedures/results with students prior to science fair by knowledgeable mentor

24. References Online statistics calculator - http://graphpad.com/quickcalcs/ Judge’s perspective - http://www.nsta.org/publications/news/story.aspx?id=53713 Statistics for science projects - https://slvsef.org/documents/teachers/SLVSEF_statistics_for_science_fair_students.pdf Data analysis for science projects - http://www.sciencebuddies.org/science-fair-projects/top_research-project_data-analysis.shtml http://static.nsta.org/files/PB343Xweb.pdf Biological Statistics - http://www.biostathandbook.com/ Engineering Statistics - http://www.itl.nist.gov/div898/handbook/ Contact Kim Unger – kunger@databrains.com