1 / 42

Synthesis and Review 3/26/12

Synthesis and Review 3/26/12. Multiple Comparisons Review of Concepts Review of Methods - Prezi. Essential Synthesis 3. Professor Kari Lock Morgan Duke University. To Do. Study and prepare for Exam 2 (Wednesday and Thursday). Exam Policies.

gari
Download Presentation

Synthesis and Review 3/26/12

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Synthesis and Review • 3/26/12 • Multiple Comparisons • Review of Concepts • Review of Methods - Prezi • Essential Synthesis 3 • Professor Kari Lock Morgan • Duke University

  2. To Do • Study and prepare for Exam 2(Wednesday and Thursday)

  3. Exam Policies • An exam absence is only excused if a short term illness form is submitted before the exam • In this case, your final exam grade will be substituted • Keep in mind that you will be responsible for a LOT more material on the final exam, and it is already worth 25% of your grade • You can ONLY take the lab exam during your designated section. Set two alarms if needed.

  4. Exam Policies • Any cheating (either on the in-class exam or the lab exam) will result in an automatic 0, and will be treated as a serious case of academic misconduct • This includes, but is not limited to, • Looking at someone else’s exam or computer screen • For the in-class exam, using pages of notes prepared by someone else • Communicating (in any form) with anyone besides myself or your TAs during the exam • Communicating (in any way) with any classmates about the lab exam, or sharing any code or materials related to the lab exam, before 4pm on Thursday, 3/29

  5. Talk • Analytic Approaches to Basketball • Mike Zarren (Boston Celtics) • Tuesday, 3/27, 5pm in 2231 French Family Science • Michael Zarren is the Boston Celtics’ Assistant General Manager and Associate Team Counsel. Mike is widely recognized as one of the leaders in the field of advanced statistical analysis of basketball players and teams, and is an important part of the team’s strategic planning and player personnel evaluation processes. Mike is also the team’s salary cap expert and lead in-house counsel, and is responsible for the development of new technologies for team use, including the team’s statistical database and video archive/delivery system. Read more here: http://goo.gl/l4P3I.

  6. Office Hours before Exam • You have LOTS of opportunities for help! • Monday, 3 – 4 pm (Prof Morgan) • Monday, 4 – 6 pm (Christine) • Tuesday, 3 – 6 pm (Prof Morgan) • Tuesday, 6 – 8 pm (Yue)

  7. Importing from a Google Doc • RStudio no longer supports importing data from a google doc 

  8. Extrasensory Perception • Is there such a thing as ESP? • Let’s find out by conducting our own experiments!

  9. Extrasensory Perception • Get into pairs. • “Randomly” choose A, B, C, or D, and write it down • Try to transmit this information to your partner, without communicating the letter in any way that can be perceived by any of the five senses! • Partner: guess the letter. • Repeat this 10 times each, and keep track of the number of correct guesses. • Once you have n = 20, come to the board and plot your sample proportion • Test whether your experiment provides evidence of ESP

  10. Extrasensory Perception • Did your experiment provide evidence of extrasensory perception, using  = 0.05? • (a) Yes • (b) No

  11. Test for a Proportion • Which of the following ways are appropriate to test whether your sample proportion is significantly different from p = ¼? • Randomization Test (only) • Normal distribution (only) • t-distribution (only) • Either (a) or (b) • Either (a), (b), or (c)

  12. Randomization Distribution • IF there is no such thing as ESP, then you all just created a randomization distribution.

  13. Extrasensory Perception • If there is no such thing as ESP, what percentage of experiments on ESP will get results that are significant, using  = 0.05? • (a) None • (b) All of them • (c) 95% • (d) 5%

  14. www.causeweb.org Author: JB Landers

  15. www.causeweb.org Author: JB Landers

  16. www.causeweb.org Author: JB Landers

  17. Multiple Comparisons • Consider a topic that is being investigated by research teams all over the world •  5% of teams are going to find something significant, even if the null is true

  18. Multiple Comparisons • Consider a research team/company doing many hypothesis tests • 5% of tests are going to be significant, even if the nulls are all true

  19. Multiple Comparisons • Consider an experiment that randomizes units to treatment groups, and then looks at many response variables • 5% of variables are going to be significantly different between the groups, just by random chance

  20. Pairwise Comparisons • Consider a study with many different treatment groups, and so many possible pairwise comparisons • 5% of comparisons are going to be significantly different, even if no differences actually exist (This is the main reason for only testing pairwise comparisons if the overall ANOVA is found to be significant)

  21. Publication Bias • publication bias: usually, only the significant results get published • The one study that turns out significant gets published, and no one knows about all the insignificant results

  22. Jelly Beans Cause Acne! http://xkcd.com/882/

  23. http://xkcd.com/882/

  24. http://xkcd.com/882/

  25. http://xkcd.com/882/

  26. Multiple Comparisons • This is a serious problem • The most important thing is to simply be aware of this issue, and not to trust claims that are obviously one of many tests (unless they specifically mention an adjustment for multiple testing)

  27. REVIEW

  28. Data Collection Was the explanatory variable randomly assigned? Was the sample randomly selected? Yes No Yes No Possible to generalize to the population Should not generalize to the population Can not make conclusions about causality Possible to make conclusions about causality

  29. Confidence Interval • A confidence intervalfor a parameter is an interval computed from sample data by a method that will capture the parameter for a specified proportion of all samples • A 95% confidence interval will contain the true parameter for 95% of all samples

  30. Hypothesis Testing • How unusual would it be to get results as extreme (or more extreme) than those observed, if the null hypothesis is true? • If it would be very unusual, then the null hypothesis is probably not true! • If it would not be very unusual, then there is not evidence against the null hypothesis

  31. p-value • The p-value is the probability of getting a statistic as extreme (or more extreme) as that observed, just by random chance, if the null hypothesis is true • The p-value measures evidence against the null hypothesis

  32. Hypothesis Testing • State Hypotheses • Calculate a test statistic, based on your sample data • Create a distribution of this test statistic, as it would be observed if the null hypothesis were true • Use this distribution to measure how extreme your test statistic is

  33. Distribution of the Sample Statistic Sampling distribution: distribution of the statistic based on many samples from the population Bootstrap Distribution: distribution of the statistic based on many samples with replacement from the original sample Randomization Distribution: distribution of the statistic assuming the null hypothesis is true Normal, t,2, F: Theoretical distributions used to approximate the distribution of the statistic

  34. Sample Size Conditions • For large sample sizes, either simulation methods or theoretical methods work • If sample sizes are too small, only simulation methods can be used

  35. Using Distributions • For confidence intervals, you find the desired percentage in the middle of the distribution, then find the corresponding value on the x-axis • For p-values, you find the value of the observed statistic on the x-axis, then find the area in the tail(s) of the distribution

  36. Confidence Intervals

  37. Confidence Intervals Return to original scale with

  38. Hypothesis Testing

  39. General Formulas • When performing inference for a single parameter (or difference in two parameters), the following formulas are used:

  40. Standard Error • The standard error is the standard deviation of the sample statistic • The formula for the standard error depends on the type of statistic (which depends on the type of variable(s) being analyzed)

  41. Multiple Categories • These formulas do not work for categorical variables with more than two categories, because there are multiple parameters • For one or two categorical variables with multiple categories, use 2 tests • For testing for a difference in means across multiple groups, use ANOVA

  42. Inference Methods http://prezi.com/c1xz1on-p4eb/stat-101/

More Related