1 / 17

Pilot studies

Pilot studies. Karla Hemming The University of Birmingham. Pilot or feasibility study?. No consensus on the definition Pilot study: Mini version of the full trial Tests protocol Feasibility study: Tests individual components Refines / develops intervention. Why have a pilot study?.

luann
Download Presentation

Pilot studies

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Pilot studies Karla Hemming The University of Birmingham

  2. Pilot or feasibility study? • No consensus on the definition • Pilot study: • Mini version of the full trial • Tests protocol • Feasibility study: • Tests individual components • Refines / develops intervention

  3. Why have a pilot study? • 70% of publically funded trials fail to recruit required number of participants • NIHR increasingly require demonstration of proof of principal that the TRIAL will work (not the intervention)

  4. What pilot studies are not about • Not around estimating effect size or CI of effect size • Why? • Pilot studies are small • Any estimate of effect will be highly uncertain • Mean estimate tells us nothing!

  5. Objectives of a pilot study • Show proof of principal of intervention (?) • Test out data collection methods • Acceptability of questionnaires • Inform outcome • Only insofar as what is important to the patient • Estimate important parameters to inform sample size calculation • (not effect size!)

  6. Proof of principal • Need a well defined surrogate outcome • That surrogate outcome would have to have a very clear link to important clinical outcome • Example – weight management programme • Important clinical outcome BMI • Show adherence improved in a pilot? • Difficulty – to show adherence improved might need as large a sample as for BMI

  7. Important parameters to estimate • Examples include but are not limited to: • Recruitment rate (informs how long the trial will have to last for) • Consent rate (informs how amenable patients are to be randomised) • Retention rate (inform drop out rate) • Continuous outcomes – SD • Binary outcomes – underlying event rate

  8. Sample size justification • Sample size should be justified • Depends on objective

  9. Objective: estimate SD • Browne 1995 • Sample size of 30 is reasonable to give an estimate of the SD • BUT! Don’t just use mean estimate of SD in subsequent power calculations – use upper 95 percentile value

  10. Objective: estimate a process rate • Show this rate is above some value • Or within some range (CI) • Called a precision based approach • )

  11. Example • Want to show retention rate (p) is above some value (c) • Retention should not differ between arms (blinded trial) – so include both arms in calculation • Precision based approach • Show retention rate above c% • Need an estimate of actual retention rate p% • Specify significance or width of CI (precision)

  12. Example cont… • Show retention rate above 60% • Estimate actual retention rate 80% • Recruit SS of 35, number retained 28 (80%) • 95% CI would be: 63% to 92% • Great! Showed retention rate above 60% and can proceed to definitive trial

  13. Example cont… • What if retention rate only 70% • That is, recruit 35 • But, only 24 are retained • 95% CI would be 51% to 83% • Oh no… haven't shown retention rate above 60%

  14. Power of this sample size • Method was motivated around CI or precision based approach (common in pilot studies) • But, the power of a SS of 35 to show that the retention rate is above 60% is only 50% (assuming event rate observed is 80%)

  15. Precision based methods • Precision methods (CI) crucially depend on actual observed rate • Associated power of this approach is much lower than 80% • So, if you use the precision based approach do not incorporate stringent criteria to follow to definitive trial

  16. Recommendations • Application viewed unfavourably if geared around effect sizes and hypothesis testing • Rather, should be focused around testing TRIAL protocol (or components) • Sample size needs to be justified • Don’t have stringent stop go criteria for full trial

  17. Other things… • More than one objective – take the larger SS • Internal pilots • Full trial fundable? • If full trial not likely to be fundable by NIHR, RfPB unlikely to fund

More Related