Clinical audit on quality indicators in radiotherapy for Specific Tumor Site Treatment (STST)

1. Clinical audit on quality indicators in radiotherapy for Specific Tumor Site Treatment(STST) Antonella Rosi Health and Technology Department, Istituto Superiore di Sanità (ROMA) Tampere 8-10 september 2008

2. QA in Radiotherapy: ISS role The tasks of the Italian National Health Institute (Istituto Superiore di Sanità ISS) include the production of technical and scientific advices to the State and Regions on problems related to the risks of ionising and non ionising radiations in the environment and/or in medical field In this framework the ISS established a multidisciplinary Working Group for Quality Assurance in Radiotherapy : • to develop guidelines on general radiotherapy topics and on specific techniques • to organize and coordinate clinical and dosimetric audits in radiotherapy • to organize training Courses on topics related to Quality Assurance in Radiotherapy Tampere 8-10 september 2008

3. ISS activity in RQA had been addressed : not only to: • Quality Controls of equipments but especially to: • Patient related activities Tampere 8-10 september 2008

4. The first step Development of Guidelines for quality assurance in radiotherapy Evaluation Tampere 8-10 september 2008

5. Quality Indicators • What is a Quality Indicator? • A measurable element for monitoring and evaluating resources, processes or outcomes of care • Where are they mainly applied? • Health Technology Assessment (HTA) • Continuous Quality Improvement Programs Measure what is measurable, what is not…. make it measurable G.Galilei Tampere 8-10 september 2008

6. Indicators to….. Indicators are designed not only to identify structures of excellence, but mainly to assess operative conditions and draw up plans of action to provide a continuous quality improvement. A comprehensive indicator system should: • encompass structural, process and outcome dimensions • produce information useful for decision making • become both a sign and a source of motivation for quality commitment Tampere 8-10 september 2008

7. Two points as matter of concern • Interconnection of structure, process, and patient outcome • Top management and medical staff leadership must be involved in CQI programs • The choice of thresholds and standards • Professional societies, governmental agencies and in general health-care organizations • Sharing experiences among professionals Standard threshold Tampere 8-10 september 2008

8. Indicators in Radiotherapy • General indicators to: • provide an overall evaluation of the Centre Cionini L. et alRadiother Oncol. 2007 82(2):191-200 • Specific Tumor Site Treatment (STST) indicators to : • provide indications on the quality level in the treatment of a specific tumor site (manuscript in preparation) Tampere 8-10 september 2008

9. The Grid Tampere 8-10 september 2008

10. From the grid…. Rationale • Why to develope indicators on a particular topic • What you plan to avoid what to promote • Which the advantages you expect to improve the quality • How relevant is the indicator for the overall quality of your product

11. From the grid….. Standard • From literature data and/or from guidelines of Scientific Associations • Empiric (on the basis of collected data) • Updated relating to technology improvment or to additional resources • Consistent with Centre resources • Complying tominima criteria

12. From the grid… • Conformity • requires a reference value to be reached(standard) • can be expressed as yes or no • requires a score attribution • allows a step by step evaluation • allows a graduated intercomparison among different Centres

13. Even indicators need to be evaluated Are they ? • proper (able to evaluate the phenomenon to be monitored)? • reproducible ? • adoptable ? • applicable? (in terms of time and costs) • understandable ? • able to demonstrate differences (among Centres/among subsequent evaluations)? And ……….. is it possible to exclude confounding elements during data collection? Tampere 8-10 september 2008

14. Indicators in Radiotherapy • Specific Tumor Site Treatment (STST) indicators to : • provide indications on the quality level in the treatment of a specific tumor site Tampere 8-10 september 2008

15. Methodology

16. Working Groups Gynaecological tumors Gianstefano GARDANI UO di Radioterapia, Università di Milano-Bicocca, A.O. S.Gerardo, Monza Luigi BOVATI UO di Radioterapia, AO S.Gerardo, Monza Carlo CAPIRCI Radioterapia Oncologica, Azienda USSL 18, Rovigo Vincenzo CERCIELLO Servizio di fisica sanitaria, Istituto Nazionale Cura Tumori Fond.Pascale, Napoli Luca CIONINI Dipartimento di Oncologia, AO Pisana, Università degli Studi, Pisa Claudio FIORINO Servizio di Fisica Sanitaria IRCCS San Raffaele, Milano Alberto MAJORANA Servizio di Fisica Sanitaria casa di Cura e Sollievo della Sofferenza San Giovanni Rotondo, Foggia Paolo MONTEMAGGI UO Radioterapia Presidio Oncologico M. Ascoli, Palermo Aldo SAINATO UO Radioterapia, AO Pisana, Pisa Francesca TORTORETO UO Radioterapia Ospedale San Giovanni Calibita Fatebenefratelli, Roma Breast Maurizio AMICHETTI UO di Radioterapia oncologica Ospedale A.Businco, Cagliari e ATreP, Agenzia Provinciale per la Protonterapia, Trento Cynthia ARISTEI UO Radioterapia Università Di Perugia ed Ospedale Monteluce, Perugia Luisa BEGNOZZI Servizio fisica sanitaria Ospedale San Giovanni Calibita Fatebenefratelli, Roma Antonella CIABATTONI UO Radioterapia San Filippo Neri, Roma Franca FOPPIANO Servizio di Fisica Medica, Istituto per la ricerca sul Cancro, Genova Marina GUENZI Radioterapia, Istituto per la ricerca sul Cancro, Genova Cristina LEONARDI Divisione di Radioterapia, Istituto Europeo di Oncologia, Milano Laura LOZZA Dipartimento di Radioterapia, Istituto Nazionale per lo Studio e la Cura dei Tumori, Milano Secondo MAGRI Servizio di Fisica Sanitaria, Azienda Istituti Ospitalieri, Cremona Sofia MEREGALLI UO di Radioterapia, AO S.Gerardo, Monza Angelo Filippo MONTI Servizio di Fisica Sanitaria, Ospedale Sant’Anna, Como Giovanni PENDUZZU Divisione di Radioterapia, Ospedale Mauriziano Torino e UOdi Radioterapia, IRCC, Candiolo, Torino Emanuele PIGNOLI Servizio di fisica sanitaria, Istituto Nazionale per lo Studio e la Cura dei Tumori, Milano Francesco SCIUMÈ UO Radioterapia Presidio Oncologico M. Ascoli, Palermo Bone Metastasis Giovanni SILVANO S.C. Radioterapia Oncologica - Azienda Sanitaria Locale TA/1, Taranto Luigi F. CAZZANIGA Divisione di Radioterapia, Azienda Ospedaliera, S. Anna, Como Pietro D’ADDATO S.C. Radioterapia Oncologica - Azienda Sanitaria Locale TA/1, Taranto Gianstefano GARDANI UO di Radioterapia, Università di Milano-Bicocca, AO S. Gerardo, Monza Patrizia OLMI Dipartimento di Radioterapia, Istituto Nazionale per lo Studio e la Cura dei Tumori, Milano Umberto RICARDI Radioterapia Università di Torino, AO San Giovanni Battista di Torino Prostate Riccardo VALDAGNI Direzione Scientifica, Istituto Nazionale per lo Studio e la Cura dei Tumori, Milano Gianfranco BRUSADIN UO Radioterapia Centro di riferimento oncologico, Aviano, Pordenone Rita CONSORTI Servizio di fisica sanitaria San Filippo Neri, Roma Andrea CRESPI Servizio di Fisica Sanitaria, Ospedale S. Gerardo, Monza Claudio FIORINO Servizio di Fisica Sanitaria IRCCS San Raffaele, Milano Pietro GABRIELE Divisione di Radioterapia, Ospedale Mauriziano Torino e UOdi Radioterapia, IRCC, Candiolo, Torino Giovanni MANDOLITI Radioterapia Oncologica, Azienda USSL 18, Rovigo Alessandra MIRRI UO Radioterapia IFO Istituto Regina Elena, Roma Alessio MORGANTI UO Complessa di Radioterapia Univ Cattolica del S. Cuore - Centro di Ricerca Formazione ad Alta Tecnologia nelle Scienze Biomed, Campobasso Francesco SCIUMÈ UO Radioterapia Presidio Oncologico M. Ascoli, Palermo Antonella SORIANI Laboratorio di Fisica Medica IFO Istituto Regina Elena, Roma

17. Working Groups Lung Ermanno EMILIANI Servizio di Radioterapia, Ospedale S. Maria delle Croci, Ravenna Giovanna BALASSO UO Radioterapia Ospedale di Circolo, Varese Claudio FIORINO Servizio di Fisica Medica S.Raffaele Milano Pietro GABRIELE Divisione di Radioterapia, Ospedale Mauriziano Torino e UO di Radioterapia, IRCC, Candiolo, Torino Gianstefano GARDANI UO di Radioterapia, Università di Milano-Bicocca, A.O. S.Gerardo, Monza Giovanni MANDOLITI Radioterapia Oncologica, Azienda USSL 18, Rovigo Maria MORELLI Servizio diFisica Sanitaria, Ospedale S. Maria delle Croci, Ravenna Nicola PERNA Servizio di Fisica Sanitaria - Azienda Sanitaria Locale TA/1, Taranto Emanuele PIGNOLI Servizio di Fisica Sanitaria Istituto Nazionale Tumori, Milano Enzo RAVOUnità di Radioterapia, Ist. Naz.Cura Tumori Fond.Pascale, Napoli Umberto RICARDI Radioterapia Università di Torino - Azienda Ospedaliera San Giovanni Battista di Torino Ruggero RUGGIERI Servizio di fisica sanitaria, Azienda Ospedaliera "Bianchi - Melacrino - Morelli", Reggio Calabria Francesco SCIUMÈ UO di Radioterapia Ospedale “G. Ascoli”, Palermo Giovanni SILVANO S.C. Radioterapia Oncologica - Azienda Sanitaria Locale TA/1, Taranto Antonella SORIANI Laboratorio di Fisica Medica, Istituto Regina Elena, Roma. Rectum Carlo CAPIRCI Radioterapia Oncologica, Azienda USSL 18, Rovigo Vincenzo CERCIELLO Servizio di fisica sanitaria, Ist. Naz.Cura Tumori Fond.Pascale, Napoli Antonella CIABATTONI UO Radioterapia San Filippo Neri, Roma Luca CIONINI Dipartimento di Oncologia, AO Pisana, Università degli Studi, Pisa Brunello MORRICA Unità di Radioterapia, Ist. Naz.Cura Tumori Fond. Pascale, Napoli Luigi RAFFAELE Azienda Policlinico Universitario, Catania Vincenzo VALENTINI UO Radioterapia Università Cattolica Policlinico Gemelli, Roma Head and neck Patrizia OLMI Dipartimento di Radioterapia, Istituto Nazionale per lo Studio e la Cura dei Tumori, Milano Giovanna BALASSO UO Radioterapia Ospedale di Circolo, Varese Filippo Grillo RUGGIERI UO Radioterapia Ospedale Umberto I, Ancona Stefania MAGGI Servizio di Fisica Sanitaria, Ospedale Umberto I, Ancona Giovanni PAVANATO Radioterapia Oncologica, Azienda USSL 18, Rovigo Mara SCISCIOLI Divisione di Radioterapia, Ospedale Mauriziano Torino e UOdi Radioterapia, IRCC, Candiolo, Torino Carlo SOATTI UO Radioterapia, Ospedale A. Manzoni, Lecco Pierluigi ZORAT UO Radioterapia ospedale Ca’ Foncello, Treviso ISS Manuela LUZI Servizio Informatico, Documentazione, Biblioteca ed Attività Editoriali Pierluigi MOROSINI Centro Nazionale di Epidemiologia, Sorveglianza e Promozione della Salute Paolo ROAZZI Servizio Informatico, Documentazione, Biblioteca ed Attività Editoriali Antonella ROSI Dipartimento Tecnologie e Salute Vincenza VITI Dipartimento Tecnologie e Salute

18. 30 Centres have been enruled for clinical audit (18 Centres operating in Hospitals, 8 in Universities and 4 in IRCCS (Research and Therapy Institutes with Scientific Character) 7 Tumor sites 45 indicators Bone Metastasis 6 Breast 6 Gynaecologic tumors 6 Head and neck 7 Lung 7 Prostate 7 Rectum 6 mainly process indicators 1 structure indicator at maximum at least 1 outcome indicator Tampere 8-10 september 2008

19. STST indicators • an indicator has to provide a tool to evaluate and to improve the procedures currently used in a Centre for the treatment of a specific tumor site • indicators have to comply with guidelines based on evidence criteria • quality indicatorsfor the treatment of a specific tumor site are NOT a therapeutic protocol,but they imply the existence of a protocol • each Centre should define its own indicators • indicators must be consistent with the Centre resources Tampere 8-10 september 2008

20. Clinical Indicators

21. Bone Metastasis • Waiting Times This indicator sholud be used in general clinical audits considering the influence of team or machine numeric adequacy on waiting time for radiation treatments beginnings. It could be a method for internal clinical audit only when human and technological resources are appropriate • Diagnostic work up and dose fractionation These indicators can be applied to evaluate the structure (or the organization) of the department (availability of procedures concerning the diagnostic work up and the dose fractionation prescription for patients with bone metastases) or for internal clinical audit (have procedures been respected). Conformity was reached by more than half of Centres to indicate a high quality standard of treatment for palliation • Multidisciplinary approach Conformity was not reached at any Centre. Such an indicator should be analyzed prospectively after a training to sensibly physicians (radiation oncologist and orthopedists) to approach together a patient with critical bone metastasis • Follow up information Appropriate information about radiation treatment results were available by only one Centre to remark how difficult is to collect data about palliative treatments

22. Breast Regarding some critical issues of breast indicators, the following evaluations can be made: • Anatomic data acquisition This indicator should be used in general clinical audits due to the possibility of different data acquisition systems among different Centres • Dose to OAR This indicator revealed some different interpretations and disagreement in the contouring of OAR among different Centres • Multidisciplinary approach Very cross tool to use in radiotherapy as a common method in all pathologies • Patients satisfaction Could be useful to use a validated questionnaire from Scientific Associations or Groups (such as EORTC…) to better compare results from different Countries

23. Gynaecologic tumors The small proportion of participants (27%) is probably due to the not wide diffusion of brachytherapy facilities, inasmuch as the availability of this technique is often mandatory for the pathology. • Multidisciplinary approach The majority of the patients (91%) are planned for radiotherapy after discussion between two specialists (gynecologist and radiotherapist) but only 50% after simultaneous physical examination • Acute toxicity The analysis of the questionnaires revealed that two conformity index must be considered, for patients previously submitted to surgery and for those planned for radical treatment with radio-chemotherapy • Monitoring of anemia Very simple and easy to evaluate: the conformity index as high as 95% is clearly optimistic but the authors consider it correct and stimulating for a necessary improvement • Dose Volume Histograms This indicator should be used not as a method for internal audit but as audit among different Centres; in fact the are only two possibilities: Centres always using DVH e Centres never using them • Overall radiation treatment time This must be considered one of the most important indicators, probably as common audit for many other tumors. For gynaecological cancers, stratification between postoperative radiotherapy and curative radio-chemotherapy is mandatory • Central boost with brachytherapy As internal audit may be used only if the Centres follow specific guidelines. It may be useful for audit among Centres when a stratification for stage of disease is considered

24. Head and neck • Waiting times for radical radiotherapy Most Centres missed these indicators and the answering Centres get a low conformity value. Patients selected for radical radiotherapy are usually affected by low staging tumor. In this case a reduced waiting time is essential to avoid tumor progression • Waiting times for post-operative radiotherapy When radiotherapy follows surgery as exclusive treatment or combined with antitumoral drugs is important to respect a treatment time from surgery to the end of radiotherapy into a maximum of 100 days to obtain better results • Multidisciplinary approach and multidisciplinary protocols The most Centres missed these indicators and even answering Centres get low conformity values indicating the necessity to stress that in clinical audit a shared approach in the clinical study of head and neck tumors is strongly suggested • Break for acute toxicity The treatment interruption due to acute toxicity could reduce the local control probability of tumor. Centres indicated different conformity value depending on stratification. To avoid treatment interruption it is suggested to support patient during treatment also with feeding tube or gastrostomy for a correct nutrition and the maintenance of blood parameters For more details on Head and Neck indicators see poster

25. Lung A general result for lung tumor is that the treatment of this patology well comply the standard as concern instrumental resources but it needs to be improved for accepting and applying multidisciplinary guidelines for staging and sharing of therapeutic decision. • Technical resources This indicators depends on interconnection between top management and medical staff leadership and can be suggested for an intercomparison among Centres • Staging and follow up The low conformity values get by Centres testify the difficulty to share opinion when patients are previously treated by other professional (many patients are referred for diagnosis and staging to Oncologists or Pneumologists) • Volume definition The low conformity values were get by most Centres with respect to ICRU 62. The use of score for each item of this indicator is an important tool to monitor improvment in following audits • Set-up verification The low conformity values get for this indicator are due to difficulties in volume definition which influence the set-up verification. The different conformity values obtained in the cases of technique change (different items in stratification) suggest to separatly evaluate results of these two different items

26. Prostate The results obtained from data analysis indicated that already in 2004 prostate cancer was treated at least with 3D CRT. Indicators: infrastrucure and methodologies, volume definition, execution and complete follow up resulted proper and well understood Regarding some critical issues of two indicators, the following evaluations can be made: • Staging Among the suggested parameters (TNM, PSA, GPS and Comorbidity) the TNM value was partially missed by Centres partially because most patients underwent RT following ormonal therapy that can mask the real T value; partially because the urologist rarely records the initial T value • Set-up errors 30% of audited Centres did not collect data for this indicator due to the need of additional human resources to monitor set-up errors. A strict reccomandation had be given by the working group to the involved Centres to improve in this direction taking into account the high doses used in prostate cancer treatment with 3D CRT

27. Rectum • Multidisciplinary approach This indicator indicated different results for patients coming from the same Centre (where systematic multidisciplinary approach was used), with respect to patients coming from other different Centres • Set-up procedures The analysis of this point indicates that all Centres obtained the minimum score, but large differences in maximum score among different Centres were observed. So this indicator could be use for a inter-Centres comparison • Set-up verification The constraints for the compliance to this indicator were too selective and was not possible to describe the different procedures used. It is necessary to modify it • Dose Volume Histogram This indicator is not suggested for internal audit but as audit among different Centres, since some Centres always used DVH, others Centres never used them • Quality of life evaluation Only 3/14 Centres answered to this indicator. It is a strong signal for the necessity to give more attention to this aspect of treatment implications The AIRO association has realized a multicentric prospective study to monitor the Italian therapeutic approach to gastro-intestinal cancers. For rectal cancer patients all the indicators are included into the request information. In Italy it is the first time that specific organ indicators are tested in a large national study.