Clinical Trials in Surgery

1. Clinical Trials in Surgery Mazen S. Zenati, MD, MPH, Ph.D. University of Pittsburgh, Department of Surgery and Epidemiology

2. Modern Clinical Trials in History • Clinical trials began in 1800 onward to proliferate and more attention was paid to study design. • Placebos were first used in 1863 • The idea of randomization was introduced in 1923. • The first trial using properly randomized treatment and control groups also featured double blind assessment was carried out in 1948 by the Medical Research Council, and involved the use of streptomycin to treat pulmonary tuberculosis..

3. The Need for Clinical Trials in Surgery • In surgery many plausible clinical theory are institutionalized into practice before as systematic risk-benefit, cost-effectiveness and long term quality of life have been measured and evaluated • Many surgical procedures implemented for years before we discovered the ineffectiveness to eventually discredited: • Halsted radical mastectomy • Kidney decapsulation (for hypertension) • Uterine suspension • Legation of internal mammary arteries (for coronary insufficiency)

4. The Need for Clinical Trials in Surgery.. cont. • New surgical procedures are not regulated in the same way new drugs or devices are • New procedures can be approved in the hospital based on the surgeon ability to convince the patients and some time upper management of the potential benefit of the new idea or the technique. • No enough cases or controls, • Protocols and IRB in many cases are often not required • No proper documentation of the outcome in systematic manner • No real plan for study or follow up

5. The Unique Nature of Surgical Trials • Blinding of patients or Surgeon is almost infeasible • Placebo-controlled procedures are ethically controversial specially if invasive • The placebo effect is powerful in surgical procedures and should be identify • The bias of the investigator due to the need for that procedure or program to be successful • Patient selection, extra skills in performance, and extra care and attention (efficacious vs. effective)

6. The Unique Nature of Surgical Trials.. cont. • Difficulty in selecting centers to represent the practicing community (VA sites vs. academic centers) • Standardization across the sites is difficult • When we compare a well-established technique with an alternative, the skills and experience in the first might bias the comparison (the need for video tapes and operative reports for QC)

7. Clinical trial in Surgery Necessary for • To test the safety and efficacy of a new technology in the short and long term outcome • To test if the new technology is Cost effective • To test if the new technology is at least as effective and/or safe as existing proven techniques • To test for the feasibility of implementing the new technology in relation to the needed skills/training, equipment, and other factors.

8. Barriers to Clinical Trials in Surgery • Underutilized due to difficulties in design and recruitments • The high cost: hospitalization, operation, personnel, and related special method of data collection and analysis • Funding is difficult to obtain from public and agencies • Peer-reviewers, IRB, and other over criticisms (protecting patients right and safety) delay the process and takes years that the timeliness of introduction may be lost

9. Barriers to Clinical Trials in Surgery.. cont. • Multicenter trials might starts in different point with unbalance recruitments) • Funding fro the industry carries its own risk of design and ownership • Mastering complex procedures related to learning curve • Consent documents very sensitive (the risk is more of an open ended format)

10. Forces Opposing Clinical Trials in Surgery • The magnitude of potential benefits might subside the necessity for wait and for the total prove through a clinical trial • The high risk might hold recruitment for a control clinical trial • The wide range and high number of potential recipients also a factor in determine the need compared to a rare disease that may require special surgical procedure.

11. Exception • When we deal with disease that carry very bad prognosis as in oncology, cardiology, and transplant where the intervention is highly morbid and invasive • The evidence-based medicine appraised by patients themselves and third-party payers might modify a total embracement traditionally done by the surgical community

12. Randomized Control Trial • Can we really have a controls in surgical clinical trials? • Controls is it: • Not to operate on half of the patients? • Mock operation not related to the indication? • Can we really deny half of the patients in need for surgical interventions just for the sake of research? • Are we then doom half of them to their death? • But which half is that? (will be really exposed to a greater eventful hazard?) • The interventions one or the controls one… do we really know?

13. The Basic design of randomized Controlled Clinical Trial Population Eligibility Criteria Ineligible Eligible Decline to Participate Recruitment Agree to Participate, Informed Consent Randomization Controls Current standard No therapy/placebo Intervention New surgical technique Good Outcome Bad Outcome Good Outcome Bad Outcome

14. Barriers to Performing Randomized Trials • Surgeon bias for or against specific procedures • Morbidity associated with surgery • The for granted acceptance of lesser forms of clinical evidence by the surgical community • Difficulties in recruitments • Ethics and legality … consents

15. Threats of the Lack of Randomized Control Group • Internal validity related to the temporal trends • Surgical learning curve effects • Regression to the mean • The frequent lack of equipoise among surgeons • Amplifying the fact that the clinical practice outpaces research evaluation

16. RCT vs. Quasi-experimental Study Design • RCT minimize the measurable confounding that potentially biasing the study's conclusions. • RCT reduces selection bias • RCT however, restricts the generalizablity of the results because of the strict inclusion and exclusion criteria • Some worthy to consider designs in case of unlikely to achieve RCT • The use of properly selected nonrandomized control groups • Using pre and post comparisons

17. Bias and Validity • Bias: Systematic error within the study that results in mistaken estimate of the effect of therapy on disease • Internal validity: The ability of a trial to come to the correct conclusion regarding the question being studied • External validity: The ability of trial to produce results that are generalizable to a larger population of patients with disease

18. Formulating Research Question • RCT to answer as narrow as a possible clinical research question • To answer an important question justifying: • Recruitment of large number of patients, • The high expenses of the trial • The need to compare: the exist of legitimate uncertainty between the effective of at least two therapies of the same disease (clinical equipoise) • In surgery there are many patients avoid surgery • Many surgeons are and perhaps patients reluctant to relinquish the control of choice of treatment of deeply held beliefs of most appropriate treatment • Ideal problems to study by RCT: • Similar morbidity of the procedure using for example similar incisions • The disease has high morbidity, new procedure promise to improve outcome • Disease with current treatment result in potential long term morbidities that will may justify the risk for exploring a new choice

19. Issues in Formulating Research Question • What are the end points • Can these end points be accurately and reliably measured • What potential surrogate end points are available and at what expenses to the internal validity • All the above in consideration to practicality, expenses, and feasibility of RCT • Pragmatic trials vs. explanatory trials

20. Pragmatic vs. Explanatory Pragmatic: attempt to stimulate clinical realities more accurately. The outcomes can be easily generalized and accepted into clinical practice Explanatory: attempts to answer more specific and narrow question (strict inclusion and exclusion criteria, homogeneous population)

21. Eligibility Criteria • Inclusions and exclusions • Important to determine the external validity • The study population should not be too narrow or to broad • It is a balance between statistical advantage and the generalizability of the results • Pragmatic trial tend to be broader in inclusion • Explanatory trials are narrower in focus

22. Outcome Measurement in Surgery • Traditional outcome measures: • Mortality • Measures • Mortality rate, crude mortality rates, specific mortality rates, adjusted mortality rates, standardized • Morbidity • Measures • Prevalence, point prevalence, period prevalence, incidence, cumulative incidence, attack rate, incidence rate • Speed of recovery: ICU LOS, Hospital LOS… • Health related quality of Life • Measuring patients satisfactory

23. Example on Eligibility • A study in vascular surgery to compare between a newly MIT and the conventional approach in treating occlusive peripheral vascular disease of lower extremities • Inclusion criteria patients< 65 yr with clinical and radiographic evidence • Randomly allocated for the above interventions • End points: angiographic resolution of the occlusion • Typical patients may be older than 65 yr, lack of generalizability lower the external validity • Eligibility criteria should be more objectively defined • Severity of lesion, symptoms • Endpoint might not be the best to determinant of therapeutic success: • Patient symptoms, activity levels, and wound healing (more appropriate and more relevant clinically)

24. Example on selecting Endpoint • A RCT to comparing between prothrombin complex concentrate and FFP in emergency need (surgery/ trauma) for reversing the effect of warfarin. The INR has been suggested as an endpoint but the FDA did not accept the proposal and asked for more clinically related outcome. Any suggestion? • A RCT to test the effect of preoperative oral Citrulline on surgical wound healing . The speed of wound healing, strength of the healing or LOS was though of as endpoint outcomes. Anything wrong with these?

25. Sample Size • We might need in surgery to conduct first a pilot nonrandomized observational study on homogenous population to determine the differences anticipated in outcome between the randomized groups in case we do not have enough data • The above will be also a good source to estimate the variability in the outcome data: i.e. standard deviation, or standard error • What is clinically relevant is more important of just statistically significant in power analysis • Statistical consultation in this regard is very essential and should be obtained early in the course of study preparation

26. Controls and Level of Evidence • Controls to increase the level of evidence without controls is just as case series study which is important in surgery to obtain all the descriptive analysis of outcome but will not provide evidence of superiority (biased) • Without control we can not account for the placebo effect • Without control we can not account for confounding variables • Historical controls might be used frequently in surgery when a prospective controls is beyond the reach, specially when performed by the same surgeon and in the same institution • Population and the time change in technology still differ

27. Controls and Level of Evidence • Variables that we usually match with in selecting controls in surgical trials beside all inclusion and exclusion criteria: • Age, sex, BMI, • Comorbidities, preexisting illness • Disease stage, duration, severity, ISS, mechanism of injury, site • Surgeon, procedure • Others: smoking, alcohol use, depression, insurance coverage, etc “Still the superior level of evidence attributed to RCT is contingent on a successful randomization”

28. Randomized Trial in Surgery • Random allocation of the subjects to the trial groups • Breaks the link between any unmeasured confounding variables and treatment status • Assume all differences in effect between treatment groups to be a result of the differences in treatment (absent confounding) • Randomization is the major challenge clinical trials in surgery and causes considerable weakness the design when not satisfactory attained • The Challenge is convincing patients to relinquish control of their care to random process by whether or not they receive surgery or even which kind of surgery they actually received • Randomized trials in surgery can be either Pragmatic or Explanatory

29. Pragmatic vs. Explanatory • It is similar to effectiveness vs. efficacy in drug trials where: • Effectiveness: is the ability of an intervention to accomplish its intended outcome in population under real-life circumstances • Efficacy: Is the ability of an intervention to obtain its intended outcome under ideal situation Most randomized trials in surgery tend to be categorized as pragmatic “effectiveness”, comparing surgical technique and outcome under usual clinical conditions

30. Randomization • Stabilizes the internal validity of the study • Randomization process can be predicted by specific logistics and practical constraints of the study. The strong desire to operate or not “by a resident” my tempt for the selection bias. • The process must be truly unpredictable and concealed from the investigator. Allocate the patient to randomization after obtaining the consent. Using random number table/a computer generated and then enclosed in opaque, serially numbered enveloped. • Investigator selection bias in selecting the intervention to subjects whenever more than one patient is presented at the same time as in a busy trauma center for instance • Use distance randomization by phone or internet “Randomized Trials appear to annoy human nature-if properly conducted, indeed they should”

31. Randomization, cont.. • Block Randomization: to result equal distributed patients between the arms of the study • Stratified Randomization: to assure certain traits are equal among treatment groups

32. Example on Randomization • In a busy trauma center, RCT took place to assess whether abdominal ultrasound or diagnostic peritoneal lavage was more accurate in diagnosing intra-abdominal injury after blunt trauma. In the ER a clinical indicator was used to determine the need for rapid abdominal assessment. If met, a resident would open the next opaque, serially numbered envelope stored in a designated folder in the ED to indicate which assessment would be used. In such busy trauma centers, two or more patients may presented simultaneously. At the end of trial, it was noted that, more often than not, diagnostic peritoneal lavage was performed on thinner patients!

33. Example on Randomization • Two cases or more at the same time might offer the busy resident an opportunity to open more than one envelope at once and subsequently decide which patients receive which assessment. • The process was corrupted by a third part motivated by self interest. Likely/hopefully not understanding the potential effects on the outcome of the trial.

34. Masking…. Blinding • To prevent or reduce observation bias and placebo effect. • Observation bias: is systematic variation between the true outcome and the outcome observed (consciously or unconsciously) • Placebo effect: exaggerate the effects of treatment based on their enthusiasm and expectations • Single blinded:treatment assignment is concealed from subject only • Double-blinded: treatment assignment is concealed from both subject and the investigator who ascertains the study end point • Contrary to studies of pharmacologic therapies for a disease, blinding in surgery is not that easy: • Logistic Challenges: we do not have placebo surgery similar to placebo tablets. We have to use sham surgery • Ethical Challenges: many of the sham-surgery interventions are considered unethical by today’s standards

35. Masking…. Blinding.. Cont. • Using blinded third party (not the operating surgeon) might be necessary • We could verify the effectiveness of blinding by asking all study participants to guess which treatment they had received. • If the patient’s guesses were no better than random chance.. indicated that the blinding remained successful • Oftentimes, it is not possible to perform a randomized trial in blinded fashion: different surgical techniques, therapies, approaches, or incisions

36. Example on Blinding • In 1959, Cobb et al. reported a RCT designed to determine if the internal mammary artery ligation was effective in the treatment of angina. The researchers enrolled 17 patients with classic symptoms and recorded preoperative severity of symptoms as well as stress test electrocardiogram data. The patients then underwent surgery. After the internal mammary arteries were isolated, an envelope was opened and directed to either ligate or simply end the operation. The patients were informed preoperatively that the ligation procedure was experimental, but they unaware of the randomized nature of the study. Post operatively data were collected and found that the patients undergoing the sham procedure had greater symptomatic relief than those undergoing ligation, and one patient in the sham group had reversal of stress test electrocardiogram abnormalities. Overall, patients in both groups showed only modest improvement.

37. Example on Blinding.. cont. • The importance of sham surgery, without it (as case series design) the author could incorrectly concluded that intervention had a merit • The differences might be related to: • Placebo effect caused the improvement in both groups • Something beside the trial intervention caused the improvement • The observed benefit was part of the natural history of the disease

38. Patient Crossover and Intention-To-Treat Analysis • Intention-To-Treat Analysis: Analyze the outcome based on the original randomization assignments, regardless of treatment actually received • The alternative: Analyze the data based on the treatment actually received not by random allocation (as if here we are breaking the randomization introducing selection bias since crossover does not occur by chance) • A high crossover rates questioned the validity of the study • High crossover rates in surgical clinical trials is common • Especially in trials comparing surgical with medical interventions

39. Patient Crossover and Intention-To-Treat Analysis.. cont. • There is a specific need to multiple statistical methods and mixed modeling techniques to deal with crossover • We stick with Intention-To-Treat Analysis approach whenever randomization is preserved and in order to keep the statistical assumptions valid for the analysis approaches • What is the nonrandom factor that caused patients to crossover • These nonrandom factors will biases the study results

40. Example on Crossover • From 1972 through 1974, the Veterans Administration Coronary Artery Bypass Surgery Cooperative Study Group enrolled patients into a randomized study comparing coronary artery bypass surgery with medical therapy in patients with angina and radiographic evidence of CAD. Patients were randomized to receive either surgical or medical therapy. After 14 yr of follow up, 55% of patients assigned to receive medical therapy had crossed over to receive surgical treatment, whereas 6% of the patients assigned to receive surgical treatment decided to not undergo the procedure and were thus managed clinically. Should we analyze the data based on the intention-to-treat (original randomization) or analyze the data based on the treatment actually received (not by random allocation)?

41. Example on Crossover.. Cont. • Should we analyze based on the intention-to-treat with such large cross over. Is not that seem counterintuitive of the randomization process? • Should we accept to break the randomization and allowing the introduction of selection bias by analyzing according to the actual treatment received?

42. Conclusions • Surgeons have fallen behind the rest of medical community partly because of the challenges required to complete a valid randomized trial of surgical therapy • Clinical Trials in surgery is underutilized • Recent era of increase in medical evidence demand from policy maker and caregivers and even patients resulted in an increase in the demand of randomized clinical trials for publications and quality assurance. And partly due to our acceptance and reliance on lesser forms of evidence • Clinical Trials in surgery is urgently needed • Randomized clinical trials in surgery are not easy but still can be accomplished with the appropriate planning and selecting suitable design • Good controls, selection of end point, randomization, blinding are the keys for successful CT • We need to standardized the ascertainment of outcome, attempting to quantify outcome as objectively as possible • We also should minimize the potential observation bias and placebo effect by selecting the most appropriate design

43. Final Thoughts • Only until clinical trails in surgery became the gold slandered to prove safety and effectiveness of the treatment, the current practice will continue to be dominant by opinions or the enthusiasm for a procedure. • Still however the surgeon's Judgment will never be a dispensable • The results still need to be interpreted in light of the study design and clinical reasonableness • The better commitment to evidence-based medical practice and the persistence and the application of novel approaches to overcoming difficult methodological hurdles will be continuously necessary to advance medicine

44. References and recommended Reading: • John Gallin, Principles and Practice of Clinical Research, Academic Press, 2002 • H. Troidl, Surgical Research, Springer, 1987 • David Machin, Text book of Clinical Trials, Wiley, 2006 • John Wei, Clinical Research Methods for Surgeons, Humana Press, 2006 • Friedman LM, Fundamentals of clinical Trials-Third Edition, PSH Inc, 1998 • Ovid MEDLINE: • http://www.hsls.pitt.edu/resources/ovid/ • PubMed: http://www.ncbi.nlm.nih.gov/sites/entrez?holding=upittlib • FDA: http://www.fda.gov/cder/guidance/iche6.htm • Clinical Research Office for Surgery and Trauma (CROST), University of Pittsburgh, Department of Surgery: http://www.surgery.upmc.edu/General/research.htm

45. Clinical Trials in Surgery