Is surgery an anachronism in an evidence based age?

1. Is surgery an anachronism in an evidence based age? Richard Smith Editor, BMJ www.bmj.com/talks

4. What I want to talk about • “The history of surgery: my contribution” • Surgery: historical and literary reflections • The state of evidence in surgery • Improving the evidence base

5. “The history of surgery: my contribution”

8. My role as emotional lightning conductor --hanging onto the second retractor • “Married are you, Richard?” • “No.” • “Good idea. The only point I can see in being married is that it saves you having to find somebody to go on holiday with.” • “Do you have horses?” • “No. I live in a flat.” • “Good chap. Super weather, isn’t it?” • And so it went until he cut something he shouldn’t have. “Bloody hell. Will you pull on that retractor, Smith? You are a bloody fool. Don’t you know anything about surgery? For God’s sake, pull harder.”

10. My vasectomy • Twenty years later I had a similar conversation, only this time I was under the knife. I was having a vasectomy. The surgeon, who did 20 vasectomies a day five days a week, had already cut my right vas when he asked me what I did. • “I’m a sort of journalist.” • “What sort?” • “A medical journalist.” • “Oh. Where do you work?” • “At the BMJ.” • “What exactly do you do there?” • “I’m the editor.” • Suddenly the surgeon began to sweat. It had taken him two minutes to find and cut one vas. He now took 20 minutes to find the second. “Oh God,” he said, “I suppose I’m going to read about this in next week’s journal.”

11. Surgery: historical and literary reflections

13. Hemicorporectomy • Hemicorporectomy or translumbar amputation is probably the most mutilating operation ever to be described in surgical literature. • Treatment for pelvic malignancy • The procedure involves removal of the bony pelvis, both lower limbs, the external genitalia, the bladder, rectum and anus. • First proposed in 1951 and performed in 1960 • Two of the first three patients died within days of oedema; one survived 19 years • A series of 10 cases reported from New York in 1982

15. Nemesis • Nemesis is the goddess of divine justice and vengeance. Her anger is directed toward human transgression of the natural, right order of things and of the arrogance causing it. Nemesis pursues the insolent and the wicked with inflexible vengeance.

17. Sir William Arbuthnot Lane (1853-1946) • Performed total colectomies for the treatment of “auto-intoxication,” something like chronic fatigue syndrome • Operated on many of London’s fashionable set • Operation had a 10% mortality • Dreadful side effects • The placebo effect guarantees that some will be “cured”

18. Ironic quote from Arbuthnot Lane • "If everyone believes a thing it is probably untrue!" • Quoted by W. E. Tanner in Sir W. Arbuthnot Lane, "Genesis"

20. Monologue by Sir Patrick Ridgeon • I tell you, Colly, chloroform has done a lot of mischief. It's enabled every fool to be a surgeon. I know your Cutler Walpoles and their like. They've found out that a man's body is full of bits and scraps of old organs he has no mortal use for. Thanks to chloroform, you can cut half a dozen of them out without leaving him any the worse, except for the illness and the guineas it costs him. I knew the Walpoles fifteen years ago. The father used to snip off the ends of people's uvulas for fifty guineas, and paint throats with caustic every day for a year at two guineas a time. His brother-in-law extirpated tonsils for two hundred guineas until he took up women's cases at double the fees. Cutler himself worked hard at anatomy to find something fresh to operate on; and at last he got hold of something he calls the nuciform sac, which he's made quite the fashion. People pay him five hundred guineas to cut it out. They might as well get their hair cut for all the difference it makes; but I suppose they feel important after it. You can't go out to dinner now without your neighbor bragging to you of some useless operation or other.

21. Cutler Walpole’s Nuciform Sac • Cutler Walpole made his fortune from removing the nuciform sac from the fashionable in London • “And have you had your nuciform sac removed, Mr Walpole.” • “I’m one of the lucky and unusual people who was born without one.”

22. GBS on operations • The large range of operations which consist of amputating limbs and extirpating organs admits of no direct verification of their necessity. There is a fashion in operations as there is in sleeves and skirts. ... There are men and women whom the operating table seems to fascinate: half-alive people who through vanity, or hypochondria, or a craving to be the constant objects of anxious attention or what not, lose such feeble sense as they ever had of the value of their own organs and limbs.

23. GBS on doctors sticking together • The only evidence that can decide a case of malpractice is expert evidence: that is, the evidence of other doctors; and every doctor will allow a colleague to decimate a whole countryside sooner that violate the bond of professional etiquette by giving him away.

24. GBS on EBM • It does happen exceptionally that a practising doctor makes a contribution to science; but it happens much oftener that he draws disastrous conclusions from his clinical experience because he has no conception of scientific method, and believes, like any rustic, that the handling of evidence and statistics needs no expertness.

25. John Rowan Wilson on surgeons • Old Fred had always tended to discount the surgeons. He visualised them as a pack of blood and thunder, cut and thrust, bombastic dunderheads, too lacking in intelligence to constitute a danger to his own system. In his eyes they were the cavalry officers of medicine, dashing and romantic, useful to impress the simple hearted, but totally unimportant when there was any serious business to be done. He had outwitted the surgeons all his life and regarded them with scant respect.

26. The state of evidence in surgery

27. Useless operations • Ligation of the internal mammary artery for angina • Extracranial/intracranial artery connection • Radical mastectomy • Sympathectomy for peripheral vascular disease • Spinal fusion

28. State of the evidence • “Unless assessments of surgical procedures are seen to be unbiased, properly randomised, and with objective assessment of outcomes they will continue to lack credibility.” • A G Johnson and J Michael DixonRemoving bias in surgical trialsBMJ, Mar 1997; 314: 916.

29. Large, randomised trials are needed • Benefits are often small--so large trials are needed so as not to miss small but still important effects. Large trials are also better at detecting harms, which are often rare. • Randomisation is needed to avoid the bias that is all pervasive • The higher the quality of the trial the smaller the beneficial effect

30. The safety and efficacy of new interventional procedures:published evidence and clinical guidance Tom Dent, Sally Wortley, Bruce Campbell

31. NICE review of evidence: methods • 245 procedures • 84 are in the Safety and Efficacy Register of New Interventional Procedures (SERNIP) • Authors randomly selected 25 procedures considered “safe and efficacious enough for routine use” and 25 of uncertain safety and efficacy • Two authors independently reviewed the evidence

32. NICE review of evidence: results • 262 studies (an average of 5.2 per procedure) • 178 (68%) were case series • 43 (16%) non-randomised comparisons • 33 (13%) randomised trials • 8 (3%) systematic reviews

33. NICE review of evidence: results • A fifth of studies supporting “safe and efficacious” procedures by randomised trials--in contrast to 3% of the studies supporting uncertain procedures • Median number of patients in the studies supporting each procedure 583 (range 3 to 2873) • Follow up poorly reported

34. NICE review of evidence: conclusions • “Many interventional procedures enter clinical practice with limited published evaluation of their safety and efficacy” • “The nature of the available studies means that the validity and durability of benefit often cannot be adequately assessed and important risks cannot be excluded”

35. State of the evidence • Few systematic reviews • Few randomised trials • Those that exist are often of poor quality--which is true of all trials • Very few placebo controlled trials • Too many case series, which are ultimately uninterpretable

36. A snapshot of the evidence • RCTs declined from 14% of research articles in the British Journalof Surgery in 1985 to 5% in 1992 • Treatments in generalsurgery are half as likely to be based on RCT evidence as treatmentsin internal medicine • 75% of “surgical trials” are actually of medical treatments in surgical patients • Only a third of surgical trials have adequate blinding • In a study of 10 international journalsfrom 1988 to 1994, Hall et al found that, of the few randomisedcontrolled trials that were published, less than half includedobjective methods for assessing outcome

37. Article in two issues of the British Journal of Surgery • Case series 29(51%) • Animal studies 7 (12%) • Non-systematic reviews 6 (11%) • RCTs 4 (7%) • Audit 3 (5%) • Four other designs 8 (14%)

38. Analysis of general practice studies from the BMJ • Questionnaire survey 6 (27%) • RCT 5 (23%) • Database study 2 (9%) • Case control study 2 (9%) • Five other designs 7 (42%)

39. The evidence patients and doctors need on a new treatment • A large, well conducted RCT comparing the new treatment with established evidence based treatments • To be “evidence based” a treatment will at some time have had to be tested against placebo • An RCT against placebo only if no evidence based treatment exists

40. A placebo controlled trial of athroscopy for osteoarthritis of the knee • 185 patients randomise to arthroscopic débridement,arthroscopic lavage, or placebo surgery • Patients in the placebogroup received skin incisions and underwent a simulated débridementwithout insertion of the arthroscope • Blind assessment of outcome • Followed up for two years • Several outcome measures • 165 patients completed the trial • Moseley JB, O'Malley K, Petersen NJ, Menke TJ, Brody BA, Kuykendall DH, et al. A controlled trial of arthroscopic surgery for osteoarthritis of the knee. N Engl J Med 2002; 347: 81-88[

41. Mean Values (and 95 Percent Confidence Intervals) on the Knee-Specific Pain Scale Moseley, J. B. et al. N Engl J Med 2002;347:81-88

42. Causes of lack of evidence and solutions • Much of what I have to say comes from the following article • Randomised trials in surgery: problems and possible solutions. • Peter McCulloch, Irving Taylor, Mitsuru Sasako, Bryony Lovett, and Damian GriffinBMJ 2002 324: 1448-1451.

43. Causes of lack of evidence and solutions • History: Many operations developed before RCTs were expected • Solution: Comprehensively review surgical evidence. Be clear what we know and don’t know.

44. Causes of lack of evidence and solutions • Commercial competition and personal prestige: many financial drivers in surgery • Hard to test objectively “your operation” • “Surgeons' eagernessto learn the operation [laparascopic cholecystectomy] seemed related more to commercial concernsthan to concern forpatients.”

45. Causes of lack of evidence and solutions • Solution: Move towards constant collection of data on outcomes of surgery and use statistical systems of continuous quality control to accumulate evidence of effectiveness and harms and decide when RCTs are necessary

46. Causes of lack of evidence and solutions • Surgeon’s equipoise: Surgeons are (attractively) decisive and used to acting in uncertain circumstances. They may thus be less comfortable with accepting the uncertainty and recognising that they have the “equipoise” necessary for a trial. • Solution: include parallel, non-randomised, preference arms alongsideRCTs.

47. Causes of lack of evidence and solutions • Lack of funding, infrastructure, and experience of data collection • Solution: change to a culture of cooperationrather than competition. Form large groups to perform specific trials

48. Causes of lack of evidence and solutions • Lack of education in clinical epidemiology. Surgeons' knowledge of clinicalepidemiology remains poor • Solution: Include training in routine surgical training. (Clinical epidemiology is the basic science of clinical research.) More advanced training for some.

49. Causes of lack of evidence and solutions • The learning curve: Should RCTs be done on new treatments or once surgeons have learnt to do them? Obvious dangers in comparing a new with a familiar operation. Solution: statistical techniques can help. Determine whether the top of the curve has been reached before analysing data.

50. Causes of lack of evidence and solutions • The quality of the surgery: It must be similar for all interventions being tested • Solution: quality control must be part of the trial. Video evidence might be collected.