1 / 10

Publication/Availability Bias

Publication/Availability Bias. Problems with bad data. Problem of missing studies. Missing at random is okay Nonrandom is a problem Sources of nonrandom samples of studies Publication bias – significant, small N Grad student work; other filedrawer

xavier-booker
Download Presentation

Publication/Availability Bias

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Publication/Availability Bias Problems with bad data

  2. Problem of missing studies • Missing at random is okay • Nonrandom is a problem • Sources of nonrandom samples of studies • Publication bias – significant, small N • Grad student work; other filedrawer • Deliberate misrepresentation for financial reasons

  3. Forest Plot Drift to the right by precision? Source: Borenstein, Hedges, Higgins & Rothstein, 2009, p. 282

  4. Funnel plot Assymetry? Esp lower right. These appear biased and heterogeneous to me. Source: Sutton (2009). In Cooper, Hedges, & Valentine (Eds) Handbook fo Research Synthesis Methods p. 501

  5. Fail-safe N • How many studies do we need to make the result n.s.? (Rosenthal) • How many studies do we need to make the summary effect small enough to ignore? (Orwin) kfs = failsafe studies kobt = studies in meta dobt = summary ES dc = desired lower boutnd dfs= studies with this (eg 0) size needed to lower ES Corwin, R. G. (1983). A fail-safe N for effect size in meta-analysis. Journal of Educational Statistics, 8, 157-159.

  6. Trim & Fill Creates symmetry; Adjusts summary ES Source: Borenstein, Hedges, Higgins & Rothstein, 2009, p. 287

  7. Cumulative Forest Source: Borenstein, Hedges, Higgins & Rothstein, 2009, p. 288

  8. Egger’s Regression Ti = effect size; vi=sampling variance of ES Should be flat (centered) if no bias. This shows small studies have higher values. Source: Sutton (2009). In Cooper, Hedges, & Valentine (Eds) Handbook fo Research Synthesis Methods p. 441

  9. Sensitivity Analysis Outliers. Run twice. Pray. Source: Greenhouse & Iyengar (2009). In Cooper, Hedges, & Valentine (Eds) Handbook fo Research Synthesis Methods p. 422

  10. Sensitivity Analysis Varying levels of tau-squared Source: Louis & Zelterman (1994). In Cooper, Hedges (Eds) Handbook of Research Synthesis p. 418

More Related