270 likes | 664 Views
Decision and cost-effectiveness analysis: Understanding sensitivity analysis. Advanced Training in Clinical Research Lecture 5 UCSF Department of Epidemiology and Biostatistics February 17, 2011. Objectives. To understand the purpose of sensitivity analysis.
E N D
Decision and cost-effectiveness analysis: Understanding sensitivity analysis Advanced Training in Clinical Research Lecture 5 UCSF Department of Epidemiology and Biostatistics February 17, 2011
Objectives • To understand the purpose of sensitivity analysis. • To understand techniques used for sensitivity analysis. Health Strategies International, UCSF
Why do Sensitivity Analyses? • All CEAs have substantial uncertainty. • Sensitivity analyses deal with uncertainty systematically. • Convince audience that results are robust.
Four Topics • Types of uncertainty. • Deterministic sensitivity analyses. • One-way, multi-way, scenario. • Probabilistic sensitivity analyses. • Monte Carlo. • Uses of sensitivity analyses. Health Strategies International, Super Models for Global Health
Types of Uncertainty • Truth uncertainty: • What are the correct input values? • Trait uncertainty: • What if population characteristics or other circumstances change? • Methodological uncertainty: • What if the analysis were done differently? Health Strategies International, Super Models for Global Health
Deterministic Sensitivity Analyses • One-way/univariate: • Vary one input at a time. • Multi-way/multivariate: • Vary 2+ inputs at a time. Health Strategies International, Super Models for Global Health
Deterministic Sensitivity Analyses • Scenario (variant of multi-way): • Tests set of relevant conditions. • Threshold analysis (one-way or multi-way): • Input values beyond which cost-effectiveness is achieved (or lost). Health Strategies International, Super Models for Global Health
One-way Sensitivity Analysis Base case est. of annual rupture risk = 0.0005
Univariate Sensitivity Analyses: Base case and range of outcomes for 1,000 FC users
Automating one-way SAs: • Male circumcision for HIV prevention in South Africa
Three-way Sensitivity Analysis Adult male circumcision (Kahn at al, PlosMedicine 2006) Health Strategies International, Super Models for Global Health
Threshold Analysis: NVP for Prevention of Vertical Transmission of HIV in Sub-Saharan Africa Input values needed for $50/DALLYMarseille at a,l Lancet, 1999
Using scenario analysis to quantify effect of unknown parameterMarseille, BMGF White Paper, 2009,. Health Strategies International, Super Models for Global Health
Probabilistic Sensitivity Analysis What is it? What is it good for?
Probabilistic Sensitivity Analysis • Operational definition: • Outputs are calculated based on random assignment of values to inputs drawn from user-selected probability distribution. • Examples: • Monte Carlo, Latin Hypercube Software: @Risk®; Crystal Ball® TreeAge ® Health Strategies International, Super Models for Global Health
The Problem with Deterministic SAs No estimate of the probability of achieving a particular outcome. Probabilistic SAs are the remedy.
Probabilistic Sensitivity Analyses • Value: • Return the likelihood of attaining a particular outcome or outcome range. • Everything known about each input is expressed at once. • Particularly valuable when many inputs are important. Health Strategies International, Super Models for Global Health
Probabilistic Sensitivity Analyses • Drawbacks: • Need to be able to make decent estimates of the underlying probability distribution. • “Black box” Health Strategies International, Super Models for Global Health
Other Uses of SA:(The Inner Teachings) • Planning the analysis. • Debugging the model. • Documenting relationships between inputs and outputs. • Identifying thresholds. • Influencing policy. Health Strategies International, Super Models for Global Health
Planning the Analysis • Program software to permit SAs on likely SA variables. • SA curves provide a check on the integrity of the model. • Identify candidates for more data collection early. Health Strategies International, Super Models for Global Health
Debugging the ModelTricks of the Trade • One-ways are best: simple and intuitive. • Plug in extreme values. • Separate diagnosis of numerator from denominator. • Break outputs down further if necessary • (intervention versus control arms).
Documenting Relationships Between Inputs and Outputs • Distinguish between ‘bugs’ and insights. • Examples of insights: • Slowing disease progression can increase costs. • Higher disease prevalence can mean lower benefits. • Benefits decrease with age.
Identify Thresholds – Influence Policy Preventing HIV vertical transmission in sub-Saharan Africa • Cost of ARVs to prevent vertical transmission. • Universal versus targeted provision of NVP. • Hard-to quantify potential benefits of FC
Cost per DALY of HIVNET 012 NVP regimen as function of HIV seroprevalence and type of counseling/testing regimen
Summary • SA is a set of techniques for the explicit management of uncertainty. • Essential part of establishing key findings. • Indispensable for convincing an audience that results are technically sound and policy-relevant.