1 / 11

Stat 513 – Day 4

Stat 513 – Day 4. Practical and Statistical Significance (1.2, 1.3a). Last Time. “effect” = Group mean – Overall mean Black cap effect: 42.78 – 84.665 = -41.885 Clear cap effect: 126.548 – 84.664 = 41.885 SSModel = 18(41.885) 2. Partitioning variability. R-squared.

tsmothers
Download Presentation

Stat 513 – Day 4

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Stat 513 – Day 4 Practical and Statistical Significance (1.2, 1.3a)

  2. Last Time “effect” = Group mean – Overall mean Black cap effect: 42.78 – 84.665 = -41.885 Clear cap effect: 126.548 – 84.664 = 41.885 SSModel = 18(41.885)2

  3. Partitioning variability

  4. R-squared • R2 = SSModel / SSTotal x 100% = percentage of variability in the response variable that is explained by the model

  5. Practical Significance • R2 is one way to support that you have a “good model” • Residual standard error also helps you judge the prediction error

  6. Practical Significance • “Effect size”: Difference in means divided by the residual standard error • 9.86 / 6.69 = 1.47 • Cohen has suggested that effect sizes of 0.2, 0.5, and 0.8 represent small, medium, and large effect sizes respectively • With an effect size of 0.50, we would expect about 2/3 of the data in one population to be below the mean of the other population, large enough to be “visible with the naked eye.”

  7. But what about … • Random assignment hopes to create equivalent treatment/control groups • Can random assignment create differences in groups? • Small p-value rules out “back luck” as an explanation for the group differences • P-value: how often would we get a statistic at least this extreme if the null hypothesis was true

  8. Recall? • Null hypothesis: No treatment effect • In the long run, the mean number of letters remembered will be the same for both treatment conditions • Alternative hypothesis: Is a treatment effect • We think those with the “JFK” sequence will tend to remember more letters, on average

  9. Strategy • Assume the null hypothesis is true (like assuming a defendant is innocent) • Generate results that vary due to random assignment alone (simulate the random assignment of the observed scores to the two groups) • See how often we get a statistic at least as extreme as the actual study when the null is true • If not very often when the null is true, then have evidence against the null hypothesis (guilty)

  10. “3S Strategy” • 1. Statistic: Compute a statistic from the observed sample data which measures the comparison of interest (e.g., difference in group means) • 2. Simulate: Identify a “by-chance-alone” explanation for the data (the null hypothesis). Then use a computer to repeatedly simulate values of the statistic, mirroring the randomness of the study design, that could have happened if the chance explanation is true. • 3. Strength of evidence: If the observed statistic is unlikely to have occurred when the chance explanation is true, then we say we have “strong evidence” against the reasonableness of chance alone as an explanation for the study results.

  11. For next time • Submit HW 1 • HW 2 should get posted this weekend

More Related