Using Evidence to Guide Drug Therapy Decisions

1. Using Evidence to Guide Drug Therapy Decisions Dean Haxby, Pharm.D. Associate Professor of Pharmacy Oregon State University, College of Pharmacy To receive 1.5 AMA PRA Category 1 Credits™, you must review this progam and pass the CME quiz at the end. Release Date: January 2009 Expiration Date: January 2012

2. Attachments • The attachments tab contains documents that supplement the presentation. • The slides are available as an attachment to print out to use as a handout. • Another attachment provides a listing of additional resources

3. Program Funding This work was made possible by a grant from the state Attorney General Consumer and Prescriber Education Program which is funded by the multi-state settlement of consumer fraud claims regarding the marketing of the prescription drug Neurontin.

4. Continuing Education Sponsors Continuing Medical Education for the following activity titled “Using Evidence to Guide Drug Therapy Decisions”,is jointly sponsored by The University of Texas Southwestern Medical Center and the Federation of State Medical Board’s Research and Education Foundation.

5. Program Speaker/Author: Dean Haxby, PharmD, Course Director:Barbara S. Schneidman, MD, MPH Federation of State Medical Boards Research and Education Foundation, Secretary Federation of State Medical Boards , Interim President and Chief Executive Officer Program Directors:David Pass, MD Director, Health Resources Commission, Oregon Office for Health Policy and Research Dean Haxby, PharmD Associate Professor of Pharmacy Practice, Oregon State University College of Pharmacy Daniel Hartung, PharmD, MPH Assistant Professor of Pharmacy Practice, Oregon State University College of Pharmacy Target Audience: This educational activity is intended for health professionals who are involved with medication prescribing, and those that are involved with committees involved with medication use policies. Educational Objectives: Upon completion of this activity, participants should be able to: Describe the advantages and limitations of evidence-based medicine (EBM), and its role in improving prescribing decisions; Describe the steps in the EBM process; Identify strategies to search for evidence-based medical literature, including useful web sites; Review key factors that should be appraised with systematic reviews, clinical practice guidelines, and individual RCTs; Describe the influence different methods of presenting study results can have on decisions; Given the results of a study calculate ARR and NNT; Discuss how EBM principles can be applied to evaluating new drugs. CME Information

6. CME Policies Accreditation: This activity has been planned and implemented in accordance with the Essential Areas & Policies of the Accreditation Council for Continuing Medical Education through the joint sponsorship of The University of Texas Southwestern Medical Center and the Federation of State Medical Boards Research and Education Foundation. The University of Texas Southwestern Medical Center is accredited by the ACCME to provide continuing medical education for physicians. Credit Designation: The University of Texas Southwestern Medical Center designates this educational activity for a maximum of 1.5 AMA PRA Category 1 Credits™. Physicians should only claim credit commensurate with the extent of their participation in the activity. Conflict of Interest: It is the policy of UT Southwestern Medical Center that participants in CME activities should be made aware of any affiliation or financial interest that may affect the authors presentation. Each author has completed and signed a conflict of interest statement. The faculty members’ relationships will be disclosed in the course material. Discussion of Off-Label Use: Because this course is meant to educate physicians with what is currently in use and what may be available in the future, “off-label” use may be discussed. Authors have been requested to inform the audience when off-label use is discussed.

7. DISCLOSURE TO PARTICIPANTS It is the policy of the CME Office at The University of Texas Southwestern Medical Center to ensure balance, independence, objectivity, and scientific rigor in all directly or jointly sponsored educational activities. Program directors and authors have completed and signed a conflict of interest statement disclosing a financial or other relationship with a commercial interest related directly or indirectly to the program. Information and opinion offered by the authors represent their viewpoints. Conclusions drawn by the audience should be derived from careful consideration of all available scientific information. Products may be discussed in treatment outside current approved labeling. FINANCIAL RELATIONSHIP DISCLOSURE FacultyType of Relationship/Name of Commercial Interest(s) David Pass, M.D.None Dean Haxby, Pharm.DEmployment/CareOregon Daniel Hartung, Pharm.D., MPHNone Barbara S. Schneidman, MD, MPHNone

8. Learning Objectives • Describe the advantages and limitations of evidence-based medicine (EBM), and its role in improving prescribing decisions • Describe the steps in the EBM process • Identify strategies to search for evidence-based medical literature, including useful web sites • Review key factors that should be appraised with systematic reviews, clinical practice guidelines, and individual RCTs

9. Learning Objectives • Describe the influence different methods of presenting study results can have on decisions • Given the results of a study, calculate ARR and NNT • Discuss how EBM principles can be applied to evaluating new drugs.

10. Problems With Drug Therapy • Over the past 15 years, drug costs have escalated at a higher rate than any other area of health care • The US has 5% of the worlds population, but has 50% of drug consumption • The US ranks very low in measures of health care quality • Overuse, underuse and inappropriate use of medications are an important quality issue • Many of these problems are preventable

11. Limitations of the FDA Drug Approval Process • FDA review determines if a drug is effective and has acceptable safety. • It does not determine place in therapy. • Value not assessed. • Often there are many unanswered questions at the time a drug is approved. • Limited numbers and types of patients • Limited duration of studies • May not know outcomes of greatest interest • Much of the FDA funding comes from industry

12. What is Evidence-based Medicine? • A process to identify and use the best available scientific evidence to guide decision making • Patient care • Policy • The scientific evidence is integrated with clinical judgment and the patients unique circumstances to provide the best clinical decisions • The result is predictable improvement in patient outcomes

13. EBM Drug Therapy Decisions • When there is evidence of benefit and value, do it. • When there is evidence of no benefit, harm or poor value, don’t do it. • When there is insufficient evidence to know for sure, be conservative. David Eddy. Formulary 2002;37:525-30

14. Decision Making in Clinical Practice • Unfortunately, many decisions are made based on unreliable “evidence” • Personal observation or anecdotal experience • Tends to overestimate efficacy • Reasoning based on pharmacology, pharmacokinetics or theory instead of “patient oriented evidence that matters” • Use of observational studies or case series to draw cause and effect conclusions • Expert opinion plays a heavy role in medical decision making • Opinions vary • Unbiased expert opinion using EBM is very useful • The consensus approach based on uncontrolled clinical experience risks widespread application of useless or even harmful treatment

15. Advantages of Using an EBM Approach • Guides prescribing decisions to get predictable improvements in patient outcomes. • Helps sort through the marketing, opinions, and theory to get an accurate assessment of the benefits and risks of various treatments. • Can help identify and target opportunities to improve drug therapy. • Assists with decisions about use of limited resources. • Provides incentive to conduct useful research.

16. Limitations of EBM • Studies may not answer the question you are trying to answer • Evidence may be lacking or of low quality • Surrogate endpoints • The body of evidence continues to change • It can take a lot of resources to conduct clinical trials or create original high quality evidence reviews • It does not make clinical decisions or value judgments about cost versus benefit • But it can assist you in doing so

17. Steps in the EBM Process • Formulate the question(s) to be answered • Gather Evidence • Decide on type of literature • Conduct search • Critical appraisal to identify the best evidence • Evidence summary • Evaluating the strength of the evidence • Magnitude benefits and risks (ARR,NNT, NNH) • Form recommendations/conclusions and apply the evidence

18. Step 1: Formulate Clinical Question(s) • An important and sometimes difficult step • Well constructed questions guide the process • Helps define what you really want to know

19. Clinical Questions • Ask for information about managing patients • Contain several important components (PICOS) • 1. Population and/or clinical problem/disorder • 2. Intervention • 3. Comparison intervention (placebo, gold standard) • 4. Outcomes • helps define what is most important • 5. Setting

20. Clinical Question Example • For cigarette smokers, does varenicline differ in efficacy for promoting long term smoking cessation than bupropion SR or NRT in the primary care setting? • Population: cigarette smokers • Intervention: varenicline • Comparator: bupropion, NRT, placebo • Outcome: smoking cessation at one year • Setting: Primary Care

21. Step 2: Gathering Evidence • When evaluating drug therapy, several types of literature are typically most useful • Randomized controlled trials • Systematic reviews of RCTs • Evidence-based CPGs • The goal is to locate the best available evidence to answer your questions

22. Evidence Hierarchy Systematic Reviews RCTs Cohort Studies Case Control Studies Case Series Case Reports Editorials and Opinions Lab studies and Animal Research

23. Strengths of Randomized Controlled Trials (RCT) • The foundation for evidence-based evaluation of drug therapy • If high quality, can establish benefits and harms of drug therapy • Effective randomization minimizes risk of unexpected factors influencing results (confounding) • Fortunately, RCTs are required for FDA approval

24. Limitations of RCTs • Experimental design and inclusion-exclusion criteria may not reflect general practice and make it difficult to apply or generalize results (external validity) • Outcomes trials in chronic diseases may take years • Expensive • It takes time and effort to review individual studies once they are published

25. Systematic Reviews (SRs) • Usually a good starting place when gathering evidence about drug therapy • A SR is an evidenced-based review of the medical literature up to a certain time point • Predefined, detailed methods are used to search, screen, critically evaluate, and summarize the medical research to answer specific questions • SRs can be qualitative or quantitative (meta-analyses) • Meta-analyses combine results of individual studies, but not all meta-analyses are SRs • High quality SRs save time and can provide a clearer picture of the body of evidence than individual studies

26. Traditional Reviews • Typically lack the rigor of a systematic review • Can be more of an opinion piece rather than a careful unbiased review of the literature. • Selective use of literature can be used to support the authors point of view. • The medical literature is full of non-systematic reviews.

27. Clinical Practice Guidelines • Usually broader in scope than SR; goal - influence practice • Are especially useful if they are well developed and free from bias • Political, financial or other factors can introduce bias • Over 2500 guidelines are available • Vary in quality (evidence vs. consensus opinion) and recommendations can vary, so need to carefully evaluate • High quality guidelines have two major components • A systematic review of the evidence • Specific recommendations with explicit links to the evidence and graded for strength of evidence

28. Searching for Evidence • This program will focus on a few basic search strategies or useful sources to locate the following: • RCT’s • Systematic reviews • Clinical practice guidelines • Additional sources of information useful for evaluating new drugs

29. Search Criteria • Decide on the type of literature best suited to answer your question • The clinical question components can help guide the search (PICOS) • Select search terms • Screening to make sure the literature is addressing your question

30. Searching for Evidence: PubMed • A very user friendly approach to Medline • Can access online through various medical libraries • Can also go to pubmed.gov • Enter your search terms based on your clinical question • Example: varenicline smoking cessation • Click on limits tab and scroll down to “type of article” • limit to types of literature you want (RCT, Meta-analysis and/or guidelines) • other limits can also help target your search • Can adjust search based on results

31. PubMed: Searching for Systematic Reviews • On the PubMed side bar on the left of the screen, click on “Clinical Queries” • Scroll down to “Find Systematic Reviews” • Enter your search terms based on your clinical question • This search retrieves systematic reviews, meta-analyses and other EBM literature

32. High Quality Systematic Reviews • Drug Effectiveness Review Project (DERP) • Contains systematic reviews of various drug classes • Developed by the OHSU Evidence-based Practice Center • http://www.ohsu.edu/ohsuedu/research/policycenter/DERP/about/final-products.cfm • http://www.oregonrx.gov • Canadian Agency for Drugs & Technologies in Health • www.cadth.ca/ • Therapeutics Initiative • www.ti.ubc.ca

33. High Quality Systematic Reviews • Agency for Health Care Research and Quality Effective Healthcare Program • http://effectivehealthcare.ahrq.gov • Cochrane Collaboration (subscription) • Can include unpublished data • http://www.cochrane.org • Center for Reviews and Dissemination (CRD) • http://www.crd.york.ac.uk/crdweb • Database of abstracts of reviews of effects (DARE)

34. Clinical Practice Guidelines • National Guidelines Clearinghouse • www.guideline.gov • New Zealand Guidelines Group • www.nzgg.org.nz/ • National Institute for Health and Clinical Excellence (NICE) • www.nice.org.uk/ • Veterans Health Administration • www.oqp.med.va.gov/cpg/cpg.htm • www.pbm.va.gov (monographs, drug use criteria)

35. FDA Website: fda.gov • An important place to look for data regarding new drugs, but can be a difficult site to navigate • The advisory panel transcripts can contain a wealth of information including information on studies not published, and analyses by fda staff • Information about unresolved issues and post marketing requirements may be available • Drug approval letter and product labeling

36. Information from Manufacturers • Package Insert • A great starting point for a new drug review • Will indicate how many studies were used in the review and approval of the drug • AMCP Dossier Format • Can be a source of information on unpublished studies • Contains an economic analysis • http://clinicalstudyresults.org

37. Step 3: Critical Appraisal • Once you have identified literature that meets your criteria, it needs to be critically appraised to decide if it is worth using. • Critical appraisal is an important step in the EBM process. • This step helps identify the best evidence to answer our question. • Much of the medical literature is not useful or reliable, and in some cases may even be misleading. • With critical appraisal, we determine confidence in the results (internal validity) and if the results are applicable to our practice (external validity).

38. Additional Programs on Critical Appraisal • In this program, a very brief introduction to critical appraisal will be provided. • For a more in-depth review, please see the following programs that are available from the same website you accessed this program: • “Critical Appraisal: Randomized Controlled Trials for Drug Therapy ” • “Critical Appraisal: Systematic Reviews and Clinical Practice Guidelines for Drug Therapy”

39. Evaluating Individual RCTs • Internal validity reflects the confidence that the results are due to the intervention • Focus on the methods and results sections • Good studies provide a valid estimate of the efficacy of an intervention and are usually easier to review • Form your own conclusions based on the above

40. Adequate sample size with statistical power Randomization methods appropriate Comparable groups at baseline Equal co-treatment Compliance Low dropout rate Adequate length of follow-up Blind assessment Equal assessment Intention-to-treat analysis Post-hoc analysis Items to Evaluate for Internal Validity of RCTs

41. Applicability (External Validity) of RCTs • How relevant is the study? • Were clinically meaningful endpoints studied? • What is the magnitude of the treatment effect and is it clinically significant? • How generalizable are the results? • How comparable were the patients studied to your practice? • Does the study reflect real world practice? • If internal validity unacceptable, the applicability is irrelevant

42. Pharmaceutical Industry Sponsored Studies • Generally have acceptable internal validity • Applicability is more often the issue • Active controls may not be gold standard • Surrogate outcomes often involved • Dosing regimens not comparable • Population may not reflect population of most interest • Publication bias

43. Items to Assess with Systematic Reviews • Evaluate for bias, methods and generalizability • Is there a clear, clinically relevant question that addresses what you are looking for, that was defined before hand? • Were study eligibility criteria defined and appropriate? • Was the search detailed and exhaustive? • Were the studies critically appraised by more than one person? • Were methods for data synthesis described? • Were conclusions clear and reflect evidence? • Was funding disclosed and potential conflict of interest minimized?

44. DARE • Database of Abstracts of Reviews of Effects • A useful resource for readers of SRs • Provides concise summaries of published SRs of healthcare interventions • Assesses strengths and weaknesses • http://www.crd.york.ac.uk/crdweb and click on “DARE” tab

45. Critically Evaluate Clinical Practice Guidelines (CPG) • Studies of guidelines have found that methodological problems, or potential conflicts of interest, are common • Objectives, key questions and applicable population clearly defined • Stakeholders involvement (relevant disciplines, patient views, target user pilot) • Editorial independence from funding, methods to manage conflict of interest, bias

46. CPG Rigor of Methods • Systematic methods used to search for evidence • Criteria for selecting evidence described • Appropriate and clearly defined methods for formulating recommendations • Benefits, risks and costs considered • Explicit link between supporting evidence and recommendations (grades of recommendations) • Guideline externally reviewed by unbiased experts

47. Step 4: Evidence SummaryRate Confidence in Strength of Evidence High: Large, well-designed randomized controlled trials, rigorous systematic reviews Small, or not-so-well designed controlled clinical trials Non-randomized prospective cohort studies Non-randomized case-control studies Low : Case series

48. Evidence Summary: Quantify Results RR: % treatment group % control group RRR: 1-RR X 100% ARR: % difference between control and treatment NNT: 100% ARR RR= Relative RiskRRR = Relative Risk Reduction ARR = Absolute Risk Reduction/absolute response rate NNT = Number Needed to Treat

49. Limitations of Relative Risk • Example 1: risk of death is 1% Tx and 2% P • RRR is a 50% reduction • ARR is 1% • NNT= 100 • Example 2: risk of death: 25% Tx vs 50%P • RRR is a 50% decrease • ARR is 25% • NNT= 4 • Same RRR, but different clinical implications

50. Calculating ARR and NNT • Useful if a statistically significant difference exists • the confidence interval helps determine precision • Important to link to the treatment duration • Can only do this if specific responders are reported • Can’t do calculations if results are presented as mean changes for the population. • Examples: • Mean change in BP vs. % reaching target BP • Mean change in HgA1c vs. % below HgA1c of 7