1 / 25

Sample Size Considerations for Answering Quantitative Research Questions

Sample Size Considerations for Answering Quantitative Research Questions. Lunch & Learn May 15, 2013 M Boyle. National Children’s Study in the US Proposed Birth Cohort 100,000 to age 21. Planning Costs 2000-2006: $54.7M Implementation Costs 2007-2011: $744.6. Sample Size Justification: ?.

rolf
Download Presentation

Sample Size Considerations for Answering Quantitative Research Questions

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Sample Size Considerations for Answering Quantitative Research Questions Lunch & Learn May 15, 2013 M Boyle

  2. National Children’s Study in the US Proposed Birth Cohort 100,000 to age 21 Planning Costs 2000-2006: $54.7M Implementation Costs 2007-2011: $744.6 Sample Size Justification: ?

  3. What is Statistical Power? • The statistical power of a test is the probability of correctly rejecting H0 when it is false. In other words, power is the likelihood that you will identify a statistically significant effect when one exists

  4. Types of Power Analysis A priori: Used to plan a study (Question: What sample size is needed to obtain a certain level of power)? Post hoc: Used to evaluate a study faced with a constrained sample size (Question: Do you have a large enough sample to detect a meaningful effect)? [Types of constraints: (1) a completed study; (2) a proposed study with limited number of eligible subjects; (3) a proposed study faced with limited resources]

  5. Elements of Power Calculations • Effect size ∆ • Measurement variability SD • Type I error Alpha (α) typically specified at p=0.05, 2-tailed • Type II error Beta (β) typically specified at p=0.20 • Power = 1-β; typically 0.80 • Sample Size

  6. Hypothesized distributions, effect sizes and error rates Effect Size ∆ Type I Type II Measurement Variability +/- 1 SD

  7. Decisions Disease Status Present Absent Medical Diagnosis +ve Test Result -ve correct false +ve false -ve correct Population Status H0 false H0 true[H1 true] Hypothesis Testing correct Type II 1-αβ Accept H0 Decision Reject H0 [Accept H1] Type I correct α 1-β (power)

  8. Example Power Calculation H0: At 2 years of age, the IQs of newborns randomly allocated to the NFP program will be no different than newborns allocated to usual care. H1: At 2 years of age, the IQs of newborns randomly allocated the NFP program will be 5 points higher. Effect size ∆ = SD = Alpha (α) = Beta (β) = Power = Sample Size ?

  9. Example Power Calculation H0: At 2 years of age, the IQs of newborns randomly allocated to the NFP program will be no different than newborns allocated to usual care. H1: At 2 years of age, the IQs of newborns randomly allocated the NFP program will be 5 points higher. Effect size ∆ = 5 SD = 15 Alpha (α) = 0.05 2-tailed Beta (β) = 0.20 Power = 80 Sample Size 146 per group

  10. n=292

  11. FACTORS THAT INFLUENCE SAMPLE SIZE PLANNING AND STATISTICAL POWER

  12. Sample Size Planningand Power 1. Error rates Type I (α) -smaller α requires larger sample sizes -2-sided tests requires larger sample sizes Type II (β) Statistical power: -smaller β (more power) requires larger sample sizes [Use conventional levels & worry about the trade- offs between effect size and sample size]

  13. Sample Size Planningand Power 2. Effect Size ∆ “What is the minimally important effect based on clinical, biological or social implications of the findings?”

  14. Sample Size Planningand Power • Effect size ∆ • What do you know about the nature of the effect – its scale of measurement and its perceived importance to practice, policy, resource allocation (e.g., infant mortality; dollars; self-esteem)? • What do previous empirical studies tell you about achievable effects?

  15. Sample Size Planningand Power 1.Effect Size ∆ • Can you generate a consensus among your investigative team on a minimally important effect? • Is it reasonable to use conventional estimates of small, medium and large? • Are you limited by the dollar amount you can request?

  16. Sample Size Planningand Power • The measurement scale of the dependent variable: discrete, ordinal, interval -interval level measurements require smaller samples • The variability of the dependent variable in the general population (SD, Variance) -lower variability requires smaller sample sizes

  17. Sample Size Planningand Power • The statistical test -simple estimation; differences between groups; correlation and prediction. The test must be appropriate for the question and data. A key element in sample size planning 5. Sample distribution, for example, exposed versus not exposed) -balanced is the most powerful

  18. Sample Size Planningand Power 6. Attrition loss of subjects -higher attrition leads to lower power 7. Measurement reliability -complicated: if true variance is constant and error variance is reduced statistical power will increase

  19. Sample Size Planningand Power 8.Study costs – what the market will bear 9. Analytical complexity – what to do when your models require much more information than you can get?

  20. Adding Complexity • Multilevel Model yij = β0j + β1z0j+(u0j + eij) y H0 The association between neighbourhood affluence measured on resident 4-16 year olds in 1983 and years of education assessed in 2001 will be = 0.00 standard units x H0∆ = β1z0j > 0.20 Neigh Affluence

  21. Estimates • 2-level balanced data, nested model • Significance level = 0.025 (to get 0.05 2-tailed) • Number of simulations per setting = 100 • Response variable = normal • Estimation method = IGLS • Fixed intercept = yes • Random intercept = yes • Number of explanatory variables = 1 • Type of predictor = continuous

  22. Estimates • Mean of the predictor = 0.0 • Variance of the predictor at level 1 = 0.0 • Variance of the predictor at level 2 = 1.0 • Smallest/Largest # units at L1 (increment) • Smallest/Largest # units at L2 (increment) • Estimate β0 = 0 • Estimateβ1= 0.15 • Estimate L2 variance 0.05 • Estimate L1 variance 0.95

  23. Comments • Ask specific, quantifiable research questions • Consult with colleagues about clinical, biological and social importance of your outcomes • Move from simple to complex hypotheses. Complex models – SEM, Multilevel – can require you to provide an enormous number of parameters. • When estimating sample size requirements for complex models, you will inevitably use standardized variables

  24. Comments • Estimating sample size requirements is part game, subject to practical constraints (limited resources and subjects) and convincing reviewers that you know what your doing • Take a ‘reasoned’ approach – most reviewers will have no clue what you are going on about • The hardest part of the process is acquiring the information you need.

More Related