Stop scratching your head: Take two aspirin and call me .

1. Stop scratching your head: Take two aspirin and call me. Paolo Prolo UCLA School of Dentistry; West Los Angeles VAMC; Psychoneuroimmunology Group, Inc., Los Angeles, CA.

2. Fattori di rischio e vantaggio del paziente. Applicazione di metodi analitici socio-economici nella pratica medico-clinica. Obiettivo • Maggiore trasparenza su rischi e flessibilità per il paziente nella scelta dei trattamenti clinici e farmacologici cui sottoporsi e loro conseguenze.   • E’ preferibile un trattamento con rischio morte 50% e qualità di vita 10 per un anno o un trattamento con rischio 20%, ma qualità 2 per 5 anni?

3. Initial Concept

4. Initial Concept cont’d

5. Initial Concept cont’d

6. Initial Concept cont’d

7. Some Practical Views on the Evidence Based Approach Three Questions to ask …three times when you read a paper 1. What What is the Outcome, What are the Findings, What is the Significance? Who is the Sample, Who needs to be treated (NNT), Who does it impact (ie, own Practice)? 2. Who How were Measurementsdone, How was the Design conducted, How were the Data analyzed 3. How

8. So What?

9. The process of applying evidence Estimation & evaluation Stakeholder domain Clinical situation Analyst domain Evaluation (abstracts & papers) PIC/PO 1 search Consensus statement Implementation & evaluation Calculate expected utilities Choose the option with the highest expected utility 1PIC/PO: problem/population – intervention – comparison/prediction – outcome

10. Relying Upon Data • According to the basic principle underlying evidence-based medicine, the recommended treatment for a given condition should be the one that the data say is the best. • The practitioner’s goal is to decide whether a treatment is sufficiently superior to alter practice. This kind of superiority is referred to as clinical significance. Can clinical significance be determined using an algorithm, similar to a statistical procedure?

11. How To Lose Weight • 2001 JCCP study showed that participants in a group weight loss program lost significantly more weight than those getting one-on-one intervention. Researchers then provided a theoretical rationale for the efficacy of the group approach (and it’s cheaper!). • However, the authors acknowledged that the mean 4 lb difference in weight loss was not clinically significant.

12. How Large Is The Difference? • Suppose a study establishes that the new medical treatment extends life by more days than the standard treatment, and the difference between the treatments is statistically significant. • How many days are required before the difference is clinically significant? Is this simply a matter of opinion? Whose opinion?

13. Not Just For Clinicians • I was once involved in a study in which we expected to observe different attitudes toward rape myths in men vs. women. The results showed a significant difference in the expected direction, but the mean difference was “small” (~.5 on a 7-point scale) – much smaller than expected. The authors essentially dismissed the difference, creating a new story that accounted for the similarity across genders.

14. Determining The Best Treatment • For simplicity, let’s consider: • 1) There are two contending treatments for the condition of interest. One can be viewed as the standard treatment, and the other as the new treatment on the block • 2) We have many comparable patients who can be randomly assigned to either treatment • 3) Patient outcomes can be unambiguously scored on a single continuum

15. Just Like In The Textbooks • The situation described is the standard one used in statistics classes to illustrate the value of significance testing. • If one treatment were overwhelmingly superior (i.e., little overlap in the observed scores), then interocular testing would be sufficient. • But if the data were not so clear-cut, then significance testing would be needed to determine the winner.

16. You Think You Know Where I’m Going • Both null hypothesis testing logic and professional inertia favor the standard treatment. • Had the results favored the null hypothesis, we might have gone there. • So let’s suppose that the statistical test tells us the new treatment is significantly better. If it doesn’t, there is little to discuss.

17. Clinical vs. Statistical Significance • Because any difference, however small, can yield a significant difference when the sample size is large enough, those in applied fields have long sought a more intuitively appealing measure of a treatment’s comparative effectiveness.

18. The Formula Tells All Independent Groups: Dependent Groups:

19. Evil Denominator? • 1) Creates dimensionless measure • 2) Increasing probability of statistically significant difference as n increases • But we think in numerator terms. • The numerator has units that have extra-statistical meaning.

20. The Magnitude Of TheTrue Difference • The examples illustrate that what impresses is the size of the effect, measured in units that are meaningful. • It’s not the magnitude of the numbers per se, because changing from days to hours would yield a larger numerical difference but wouldn’t impress us more. • Effect size expressed in dimensionless (variability) units, as standard measures such as  or 2call for, is not clinically meaningful.

21. Neil Jacobson’s Clinically Significant Difference • Developed in the context of psychotherapy. Patients are measured on a continuous scale of functionality. • If a patient is moved from “disturbed” status to “normal functioning”, then the treatment has achieved a clinically significant result. Any other improvement is not clinically significant. We examine how many of the people were clinically significantly changed. • In effect, Jacobson proposes a 2-point scale. His transformation loses sight of the amount of change.

22. Information Is ATerrible Thing To Waste • Jacobson is on a promising track, but the two-point scale has the potential for peculiar results. A therapy might change one patient from terribly dysfunctional to mildly dysfunctional (which would count as “no change”), while another patient might be changed from mildly dysfunctional to normal (“clinically significant change”). The smaller change counts as a greater success.

23. The Heart Of The Matter • What does a patient who faces a choice among treatments want to know? • Am I more likely to be better off with a particular option? (probability) • And • How much better off am I likely to be after each of the possible options? (utility)

24. More Heart • What does a researcher comparing conditions want to know? • Is the difference I observed reliable? (probability) • And • Is the observed difference large enough to incorporate in my theorizing? (utility)

25. We Knew That • Significance testing addresses reliability. • Utility is inherently subjective. So clinical significance is as well. • Jacobson buried the subjectivity in his partitioning of the outcome measure. In his opinion, only outcomes in a particular region had positive utility.

26. Clinical Significance = Expected Multiattribute Utility • Sometimes, we will want to consider more than one aspect of the outcome. • For medical treatment, the clinician’s measure may not coincide with patient satisfaction. And patient satisfaction is surely important to a person considering which treatment to choose. • But patient satisfaction can be manipulated, so objective measures are important too. • We may have to worry about the validity of subjective responses.

27. Comparing Treatments • We need to determine the outcome for each of the aspects we decide is important. • The probability distribution for each aspect of the outcome is estimated from the data. • Weight the aspects by importance (more subjectivity) • Compare expected utility across treatment options.

28. Trust No One? • We could ask Jacobson’s patients to rate their status before and after therapy, and the subjective evaluations would be a component of the utility analysis. If the patients were considered incompetent to rate, then someone else could do the ratings. We would have to decide whether to accept.

29. Trust Me! • And if a knowledgeable researcher says that a small but statistically significant difference is not meaningful, we can recognize that view as subjective – but that doesn’t mean we should reject it automatically.

30. Assessing Utilities Of Potential Outcomes • One might rely upon “experts” to judge utility (gee, how can we tell who is truly expert?). • We must use focus groups of relevant people (say, patients and possibly families or payers). • We have to proceed on a case-by-case basis until we figure out what generalizations are possible.

31. Level of Evidence (EBM Scores) • I Evidence is obtained from meta-analysis of multiple, well-designed, controlled studies. Randomized trials with low false-positive and low false-negative errors (high power). • II Evidence is obtained from at least one well-designed experimental study. Randomized trials with high false-positive and/or negative errors (low power). • III Evidence is obtained from well-designed, quasi-experimental studies such as non-randomized, controlled single-group, pre-post, cohort, time, or matched case-control series. • IV Evidence is from well-designed, non-experimental studies such as comparative and correlational descriptive and case studies V Evidence from case reports and clinical examples www.bmj.com

32. Metanalisi Figura 2: Risultati ottenuti da une di studi di trattamento della malattiua di Alzheimer con Gingko-Biloba e Idebenone. E’ stato considerato come indice di miglioramento sintomatologico la scala di valutazione delle capacità cognitive ADAS. Tre studi sono favorevoli al trattamento con Gingko-Biloba, mentre uno studio con Idebenone non mostra differenze rispetto al placebo. F Chiappelli, AN Navarro, DR Moradi, E Manfrini, P Prolo. Evidence-Based Research in Complementary and Alternative Medicine III: Treatment of Patients with Alzheimer's Disease. Evid Based Complement Alternat Med. 2006;3::411-424.

33. Metanalysis Definition: A statistical technique where all data from all available studies of something are combined, regardless of data quality. The technique is used by researchers to get a maximum of statistical information without worrying about distortion in the results. www.nih.gov

34. Banca dati: Prima fase: Selezione 1 OHG Register 198 159 MEDLINE 620 16 Central 230 8 EMBASE 130 6 PaPaS 39 0 Totale 1217 189 P Prolo, DL Domingo, Z Fedorowicz, TL Outhouse. Interventions for recurrent aphthous stomatitis (mouth ulcers) (Protocol) The Cochrane Database of Systematic Reviews, Issue 3. Art. No.: CD005411. DOI: 10.1002/14651858.CD005411; 2005.

35. E’ quindi fondamentale in un’ottica di controllo dei costi, anche ai fini di una migliore qualità di vita del paziente, studiare un algoritmo che permetta di stabilire con la migliore approssimazione possibile il rischio paziente in termini di effetti inattesi del trattamento riguardanti le aspettative del paziente, piuttosto che nella misura assoluta convenzionale di tali effetti indesiderabili. • Crediamo che questa definizione composita della definizione di rischio e di programma di utilità riguardante le aspettative del paziente possa aumentare sostanzialmente la flessibilità degli strumenti di valutazione. • Tale nuova funzione, chiamata Utilità Paziente (Patient Utility) (PU) diviene parametro-chiave di metodo per derivare EBM Scores. • RS + Metanalisi ►LG ►Processo Decisionale ► PU

36. Modello • Le risposte di un insieme dei pazienti sono analizzate riguardo alle loro aspettative di Patient Utility- PU - espressa sia come speranza di vita che come qualità di vita – riguardo a un determinato trattamento; • per ciascuna delle quattro dimensioni fisiologiche chiave (cardiovascolare, neuropsichiatrica, gastroenterica-epatica e renale), i risultati empirici di effetti di side/adverse effects derivate dalle prove cliniche di un trattamento sono misurati su una scala che varia da 0 a 10, per cui a. 0 significa zero effetti in una dimensione qualunque; b. 1 - 5 si riferisce ai side effects (eventi secondari) in seguito al trattamento secondo una scala di aumento della gravità; c. 6 - 10 si riferisce agli adverse effects (eventi avversi) del trattamento, sempre secondo una scala di aumento della gravità; • I risultati reali sono paragonati alle aspettative del paziente. In questo contesto, il rischio paziente è associato con gli effetti inattesi di un trattamento. Ciò significa che un trattamento può effettivamente provocare effetti molto severi - o persino la morte - per il paziente ed ancora qualificarsi per il rischio basso fino al punto in cui tali effetti sono stati debitamente comunicati in anticipo al paziente ed il paziente ha accettato il trattamento; • Il valore attribuito a uno studio o a una RS in termini di evidenza (EBM score) allora sarà correlato inversamente con il rischio paziente - secondo la definizione relativa di PU che è stata offerta sopra. In pratica, più è alto il numero di effetti inattesi, più è basso il valore di tale studio o RS. In questo senso, l’EBM score è inteso come un valore riassuntivo (summary figure) di quanto sia significativa l’informazione contenuta in un determinato studio o RS riguardo agli effetti secondari o avversi di un certo trattamento.

37. Take-Home Message • Whether an observed difference is clinically significant is just another decision, subject to a standard decision analysis. Statistical significance testing is necessary, but is only (a small) part of that analysis. • One extra day of life may turn out to generate clinically significant differences in utility, even if it doesn’t seem like much to me. I may not be the right one to ask.

38. Conclusioni • Si intende applicare questo modello inizialmente nell’ambito della medicina complementare (MC), dove è senz’altro di primaria importanza, come anche evidenziato da documenti della World Health Organization (WHO) in collaborazione con la Regione Lombardia. • Nell’ambito della MC sono tuttora assenti, se non in misura trascurabile, studi controllati e/o randomizzati, gli unici studi come abbiamo visto considerati dalla EBM, se si seguono le attuali metodiche. • All’interno della MC, si intende usare come primo modello la malattia di Alzheimer. Il motivo è molto semplice, trattandosi di una malattia diffusissima nei Paesi industrializzati. • Il modello una volta sviluppato e testato potrà essere poi trasferito ad altre patologie. • L’introduzione di un EBM Score non fine a se stesso o legato a un uso puramente accademico, ma significativamente correlato alla funzione di Utilità Paziente PU sarà in grado di offrire uno strumento di uso intuitivo e semplice utile sia al medico sia al paziente e in ultima analisi si tradurrà in una riduzione dei costi sanitari e in un insieme di benefici sociali per tutti noi.

39. The End?