MONITORING THE QUALITY OF CARE USING CLINICAL INDICATORS

1. MONITORING THE QUALITY OF CARE USING CLINICAL INDICATORS

2. Jan Mainz, The National Indicator Project and Aarhus University

3. Jan Mainz, The National Indicator Project and Aarhus University DEFINITIONS Indicators should be explicit statements of desirable (or undesirable) structural, process or outcome dimensions. They should be supported by either research that establishes the efficacy or effectiveness of the indicators by a formal process of obtaining experts consensus. The tools for measurement should be tested and evaluated for reliability, validity and feasibility. Results should be repeated in a format that maximizes the likelihood that the information can be interpreted and used in appropriate decision contexts. (RAND, 1998)

4. Jan Mainz, The National Indicator Project and Aarhus University DEFINITIONS Indicators are measures that assess a particular health care process or outcome. (European Medical Associations, 1992) Indicators are measures of the clinical management and outcome of care and are objective measures of either the process or outcome of patient care in quantitative terms. (ACHS, 1993)

5. Jan Mainz, The National Indicator Project and Aarhus University USES OF INDICATORS To document the quality of care To make comparisons Over time Between places (e.g. hospitals) To make judgements and priorities e.g. choosing a hospital or surgery e.g. organising medical care To support accountability To support quality improvement Transparency for society

6. Jan Mainz, The National Indicator Project and Aarhus University SAME MEASURE CAN SERVE MUTIPLE PURPOSES Physician - ”How am I doing?” Patient - ”What are my chances?” - ”Which is the best hospital?” - ”Which is the best doctor?” Society - ”What does it cost?”

7. Jan Mainz, The National Indicator Project and Aarhus University KEY CHALLENGE Everyone wants measurement No one wants to be measured

8. Jan Mainz, The National Indicator Project and Aarhus University INDICATORS Must be precisely defined i.e. Be based on agreed definitions which can be uniformly implemented. Have specificity Be valid and reliable (validation and reliability testing) Have discrimination ability Be risk-adjusted to enable comparison Relate to clearly identifiable events Permit useful comparisons Evaluation and review would incorporate changes over time

9. Jan Mainz, The National Indicator Project and Aarhus University INDICATORS Significance Ownership reflected in the development and use Widely accepted Ease of data extraction Be interpreted in the light of socio-economic and cultural issues. Responsive Will not violate patient confidentiality Be cost effective Be public available

10. Jan Mainz, The National Indicator Project and Aarhus University INDICATORS ARE BASED ON: Best evidence (cochrane, metaanalyses, RCT etc) Consensus among health professionals

11. Jan Mainz, The National Indicator Project and Aarhus University INDICATORS Can be categorised by Type of care Preventive Acute Chronic Function Screening Diagnosis Treatment Follow up Modality History Physical examination Laboratory/radiology study Medication Other interventions Generic or disease specific Rate-based or sentinel

12. Jan Mainz, The National Indicator Project and Aarhus University DEFINITIONS E.G NOSOCOMIAL INFECTIONS Dirty: Operations in which a perforated viscus or pus is found. Contaminated: Operations breaching the gastrointestinal, respiratory and genitourinary tracts, or in which a break in aseptic technique occurs and in traumatic wounds. Clean: All other operations where the criteria set out in ’dirty’ and ’contaminated’ do not apply. Wound infection: Any surgical wound from which purulent material drains or is obtained. Hospital-acquired bacteraemia: A positive blood culture for inpatients who were afebrile on admission (i.e. temperatures less than 37,4°C) on blood collected 48h after admission. Reference: ACHS. J. Qual. Clin Practice 1997

13. Jan Mainz, The National Indicator Project and Aarhus University DATA FORMAT E.G NOSOCOMIAL INFECTIONS Clean and contaminated wound infection A) Numerator: The number of patients who develop wound infection from the fifth post-operative day after (i) clean surgery, (ii) contaminated surgery. B) Denominator: The total number of patients undergoing (i) clean and (ii) contaminated surgery within the time period under study who have a post-operative length of stay of 5 or more days Hospital-acquired bacteraemia A) Numerator: Total number of patients who acquire bacteraemia as defined above. B) Denominator: Total number of patients in hospital during the study period. Reference: ACHS. J. Qual. Clin Practice 1997

14. Jan Mainz, The National Indicator Project and Aarhus University CONCEPTUAL FRAMEWORK Structural quality – assesses health system characteristics that affect the system’s ability to meet the health care needs of individual patients or a community (e.g. the nurse-to-bed ratio in a hospital) Process quality – assesses what the provider did for the patient and how well he or she did it (e.g. proper diagnostic approach to symptoms) Outcome quality – assesses the influence of the health care delivery process on the individual’s health (e.g. morbidity and mortality)

15. Jan Mainz, The National Indicator Project and Aarhus University EXAMPLES OF STRUCTURE INDICATORS Numbers of specialists compared to other doctors Access to specific technologies Availability of specific units (e.g.. stroke units) Clinical guidelines revised every 2nd year Physiotherapists associated to specific units

16. Jan Mainz, The National Indicator Project and Aarhus University EXAMPLES OF PROCESS INDICATORS Patients treated according to clinical guidelines Patients with MI, who received thrombolyses Door to needletime for MI patients Breast cancer patients < 75 years, who got axillary resections Waiting time for doctor contact for patients admitted acute

17. Jan Mainz, The National Indicator Project and Aarhus University OUTCOMES OF DISEASES (THE FIVE D’s) Death A bad outcome if untimely Disease A set of symptoms, physical signs and laboratory abnormalities Discomfort Symptoms such as pain, nausea, dyspnoea etc. Disability Impaired ability connected to usual activities at home, work or in recreation Dissatisfaction Emotional reactions to disease and its care, such as sadness or anger

18. Jan Mainz, The National Indicator Project and Aarhus University EXAMPLES OF INTERMEDIATE OUTCOME INDICATORS HbA1C for diabetics Lipid profile for patients with hyperlipidemia Numbers of lymph nodes removed at breast cancer surgery Blood pressure for hypertensive patients

19. Jan Mainz, The National Indicator Project and Aarhus University OUTCOME INDICATORS Mortality Morbidity Functional status Health measurement status Work status Complications Quality of life Patient satisfaction

20. THE OUTCOME OF CARE The Patient Demographic factors (age, sex, height) Lifestyle factors (smoking, alcohol, weight, diet, physical exercise) Psychosocial factors (social status, education) Compliance + The Illness Severity, prognosis Comorbidity + The Treatment (Prevention, diagnostics, care, rehabilitation, therapy and control) Competence Technical equipment Evidence based clinical practise Efficacy, accuracy + The Organisation Use of clinical guidelines Cooperation Delay = OUTCOME

21. Jan Mainz, The National Indicator Project and Aarhus University PROBLEMS WITH INDICATOR USE Inappropriate definitions Ranking instability Discriminative power Viewed as absolute measure of quality Lack of timely access Data incompleteness Lack of interest Lack of trust Lack of ownership

22. Jan Mainz, The National Indicator Project and Aarhus University DIMENSIONS OF THE QUALITY OF CARE Quality of the technical care in terms of prevention, diagnostics, treatment and rehabilitation Quality of the interpersonal relationship in terms of communication and information Quality of the organisation of care in terms of continuity and coordination

23. Jan Mainz, The National Indicator Project and Aarhus University WHAT DO WE KNOW ABOUT THE QUALITY OF THE TECHNICAL CARE? Lack of documentation about how major illnesses are treated in the health care system Few goals regarding the technical quality Lack of outcome assessment Lack of resource evaluation Persisting variations No formal monitoring systems The principal quality problems and their prevalence and incidence are unknown

24. Jan Mainz, The National Indicator Project and Aarhus University THE NATIONAL INDICATOR PROJECT-a concerted action between: The Ministry of Health The National Board of Health The County Counsellors’ Association The Scientific Societies The Danish Medical Association The Danish Nursing Association The Danish Physiotherapist Association

25. Jan Mainz, The National Indicator Project and Aarhus University THE NATIONAL INDICATOR PROJECT AIMS: Improving prevention, diagnostics, treatment and rehabilitation Documentation for making priorities Information for patients and consumers

26. Jan Mainz, The National Indicator Project and Aarhus University THE NATIONAL INDICATOR PROJECT All major diseases are evaluated Evidence based process and outcome indicators are derived by health professionals on national level Health professionals and clinical epidemiologists are responsible for data-collection, analyses, evaluation and interpretation of results Hospitals are compared at county and national and international levels Audit activities are organised at county and national level Improvements are initiated if necessary

27. Jan Mainz, The National Indicator Project and Aarhus University BASIC PRINCIPLES Health professionals develop evidence based standards and indicators for all major diseases Health professionals assess and interpret results before public release of data

28. Jan Mainz, The National Indicator Project and Aarhus University THE NATIONAL INDICATOR PROJECT Established 2000 Developed 6 sets of indicators covering 96 individual clinical indicators Mandatory participation by all hospitals and relevant clinical departments in Denmark.

29. Jan Mainz, The National Indicator Project and Aarhus University DISEASES Stroke Hip fracture Schizophrenia Acute surgery Heart failure Lung cancer

30. Jan Mainz, The National Indicator Project and Aarhus University INDICATORS Stroke Stroke patients treated at stroke units Medical secondary prophylactic treatment CT/MR scan Patients assessed by physiotherapist Patients assessed by occupational therapist Assessment of nutritional status Mortality at 30 days, 3,6 and 12 months Discharge destination

31. Jan Mainz, The National Indicator Project and Aarhus University

32. Jan Mainz, The National Indicator Project and Aarhus University

33. Jan Mainz, The National Indicator Project and Aarhus University

34. Jan Mainz, The National Indicator Project and Aarhus University

35. Jan Mainz, The National Indicator Project and Aarhus University INDICATORS Schizophrenics Assessment of Side-effects of The Psychotic drugs Family Intervention Psycho - education Pharmacological Treatment Continuity of Contacts to The Health Care System Psychosis – relapse Functional Status

36. Jan Mainz, The National Indicator Project and Aarhus University

37. Jan Mainz, The National Indicator Project and Aarhus University

38. Jan Mainz, The National Indicator Project and Aarhus University

39. Jan Mainz, The National Indicator Project and Aarhus University

40. Jan Mainz, The National Indicator Project and Aarhus University INDICATORS Lung Cancer Survival Delays in Diagnosis and Treatment Staging (Classification) Treatment and Resection (Appropriateness) Admission Time the last 3 months before Death Supporting Ambulatory Psychological Contact (Palliative Care)

41. Jan Mainz, The National Indicator Project and Aarhus University

42. Jan Mainz, The National Indicator Project and Aarhus University

43. Jan Mainz, The National Indicator Project and Aarhus University

44. Jan Mainz, The National Indicator Project and Aarhus University INDICATORS Hip Fracture Risk Assessment of Nutritional Status Pain Intensity at Mobilisation at 5th Post Operative Day Functional Status at 5th Post Operative Day ADL: Functional Recovery Discharge Destination Reoperative by Different Types of Fractures Mortality at 30 Days, 4, 6, 12 months

45. Jan Mainz, The National Indicator Project and Aarhus University

46. Jan Mainz, The National Indicator Project and Aarhus University

47. Jan Mainz, The National Indicator Project and Aarhus University

48. Jan Mainz, The National Indicator Project and Aarhus University PHASE 1: COMMENCEMENT OF THE TASK Selection of group participants Classification of concepts, definitions and limitations Organising the work in the group. Creation of smaller working subgroups and delegation of projects to individual group participants

49. Jan Mainz, The National Indicator Project and Aarhus University PHASE 2: OVERVIEW OF EXCISTING KNOWLEDGE AND PRACTICE Presentation of knowledge and documentation from literature and meta analyses Consensus about existing knowledge, practice and conclusions Determination of the clinical epidemiological task

50. Jan Mainz, The National Indicator Project and Aarhus University PHASE 3: DETERMINATION OF PROFESSIONAL INDICATORS Determination of professional indicators based on existing professional and clinical epidemiological knowledge Decisions about data collection. Inclusion criteria. Exclusion criteria. Determination of interpretation (how to read data, limination and acceptable tolerances)

51. Jan Mainz, The National Indicator Project and Aarhus University PHASE 4: DATA COLLECTION The data collection should be supported by clinical epidemiologists Data from medical records, questionnaires, clinical databases, registers should be used

52. Jan Mainz, The National Indicator Project and Aarhus University CLINICAL DATABASES A register, that contains specific clinical indicators, which can describe the quality of care for a specific patient group.

53. Jan Mainz, The National Indicator Project and Aarhus University PHASE 5: DATA ANALYSES Analyses, evaluation, interpretation Professional discussions of processed data results

54. Jan Mainz, The National Indicator Project and Aarhus University INTERPRETATIONS OF FINDINGS If differences are assessed in a clinical course there are different levels of explanation: Bias due to selection of patients or bias in data collection Confounding (different prognostic factors between groups) Random variations There is a difference

55. Jan Mainz, The National Indicator Project and Aarhus University PHASE 6: REFLECTIONS AND DESCRIPTIONS Reflections about cause-and effects. Connections between these and conclusions based on data and cause analysis Creations of reports to hospital quality committees Feed back to professionals Implementing quality improvement

56. Jan Mainz, The National Indicator Project and Aarhus University PUBLIC ACCESS TO DATA Data are released to the public at: - National level - County level - Hospital level - Clinical unit level Data on individual doctor level will not be released

57. Jan Mainz, The National Indicator Project and Aarhus University COMMUNICABILITY Data must be transformed into meaningful and useful information, reportable in a relevant manner to all stakeholders Feedback required on a regular basis for behavioural change

58. Jan Mainz, The National Indicator Project and Aarhus University PERSPECTIVES Research CME Quality development - Clinical guidelines - Patients pathways - Audit

59. Jan Mainz, The National Indicator Project and Aarhus University INDICATORS Are strategic markers monitoring aspects of the quality of care Measure the extent to which set targets are achieved The surveillance of health care quality is impossible without the use of relevant indicators Should be valid (measure exactly what we want to measure) Should be sensitive (reflect correctly changes occurring given the situation) Should be specific (to avoid the measurement of changes arising from external factors not related to the objectives and targets) Should be evidence based The use of indicators should be followed by professional assessment, evaluation and interpretation

60. Jan Mainz, The National Indicator Project and Aarhus University PERSPECTIVES THE EUROPEAN INDICATOR PROJECT under the flag of The European Union and ESQH

61. Jan Mainz, The National Indicator Project and Aarhus University The European Indicator ProjectPRINCIPLES Health professionals develop evidence based standards and indicators for all major diseases One or two indicators for specific diseases shoul be developed Health professionals assess and interpret results before public release of data

62. Jan Mainz, The National Indicator Project and Aarhus University The European Indicator ProjectAIM Improving prevention, diagnostics, treatment and rehabilitation Documentation for making priorities Information for patients and consumers International benchmarking

63. Jan Mainz, The National Indicator Project and Aarhus University The European Indicator ProjectORGANISATION All interested European countries would be able to participate Representatives from each country establish a steering committee International and national fundings should finance the project Important diseases are identified which would be relevant for international comparisons The project should be conducted within a limited time period (e.g. 3 years) The project should be evaluated in order to decide whether it would be feasible to continue the project

64. Jan Mainz, The National Indicator Project and Aarhus University ”I am called eccentric for saying in public that hospitals, if they wish to be sure of improvement, must find out what their results are. Must analyze their results to find their strong and weak points. Must compare their results with those of other hospitals… Such opinions will not be eccentric a few years hence.” E.A. Codman, MD, 1917.

65. In God we trust –everybody else has to bring data