Measuring Clinical Lab Ordering Quality: Theory and Practice

Measuring Clinical Lab Ordering Quality: Theory and Practice Steven M. Asch MD MPH VA, RAND, UCLA April 29, 2005

INSTITUTE OF MEDICINEDEFINITION OF QUALITY The degree to which health services for individuals and populations * increase the likelihood of desired health outcomes and * are consistent with current professional knowledge

Were results used Has the right test properly to improve care? been ordered? Action Interpretation Ordering Collection Reporting Analysis Identification Transportation Preparation The 9 steps in the performance of any laboratory test. The brain-to-brain turnaround time loop. Lundberg , 1981

WHAT IS POOR QUALITY? • Too little care – underuse • Failure to provide an effective service when it could have produced a good outcome • Too much care – overuse • Providing care when its risks of harm greater than potential benefit • The wrong care – misuse • Avoidable complications of appropriate care

STRUCTURE PROCESS OUTCOMES Community Characteristics Biological Status Mortality • Technical Excellence • • Right choices • • Effective/skillful Health CareOrganization Characteristics Functional Status • Interpersonal Excellence • • Patient-centered • • Responsive Provider Characteristics Population Characteristics Satisfaction CONCEPTUAL FRAMEWORK

EXAMPLES OF STRUCTURAL MEASURES • Health care organization characteristics • Weekend and night hours and convenient locations of laboratories • Volume • Provider characteristics • Number of pathologists • Training of laboratory staff

HTN NEW DIAGNOSIS LABS Asch et. al. BMC CV, 2005

QATOOL SCORES BY MODE McGlynn, Asch et. al. NEJM 2003

WHY MEASURE OUTCOMES? • Allow innovation in process • People care about outcomes directly

SAMPLE SIZE PROBLEMS • For mortality, need huge samples: • CHF patients: 12% vs 16%, need 957 patients at each hospital. • Rarer outcomes • People care, but statistical comparison is impossible.

DOES SICKNESS OR QUALITY DETERMINE CHF MORTALITY? Sickness at Process Admission Poor Medium Good Total Least Sick 1/4 4 7 4 5 Middle 1/2 11 8 8 9 Most Sick 1/4 37 32 26 32 Total 16 14 12 14

ACCOUNTABILITY: IS PROVIDER RESPONSIBLE FOR PROBLEM? • Current treatment must have big impact relative to other factors. • Do not want providers avoiding those who: • have a bigger chance of problems • are less likely to adhere to treatment

CHOOSING MEASURES:PRACTICAL CONSIDERATIONS • Choosing areas to measure • Selecting indicators • Designing specifications • Testing the measure

CHOOSING AREAS:ASSESSING HEALTH IMPORTANCE • Mortality • Morbidity • Utilization • Cost

ConditionWork Loss Days/100 Persons Injuries Influenza Infections and parasitic disease Common cold Digestive system conditions Other upper respiratory Acute ear infections PREVALENCE OF SELECTED ACUTE CONDITIONS AMONG WORKING ADULTS 85.5 53.1 20.6 15.4 12.3 9.3 3.2

CHOOSING AREAS: POTENTIAL FOR IMPROVEMENT • What are the key outcomes of interest? • What processes produce those outcomes? • How well are key elements of care delivered today? • How variable is care delivery?

CHOOSING MEASURES: DEGREE OF PROVIDER CONTROL • How might the measure be affected by characteristics of the enrolled population? • What actions can providers or clinical laboratories take to improve performance?

STRENGTH OF SCIENTIFIC EVIDENCE I: Randomized controlled trial II-1: Nonrandomized controlled trial I-2: Cohort or case control studies II-3: Multiple time series III: Opinions or descriptive studies

COST-EFFECTIVENESS OF PROCESS MMR Immunization $14 saved/$1 spent Cervical cancer screening $21,000 spent/year (ages 20-28) Cervical cancer screening $11,000 spent/year (ages 29-50)

DESIGNING MEASURE SPECIFICATIONS • Define indicator • Identify target population • Define eligible population • Determine need for risk adjustment • Identify data sources • Write data collection instructions • Develop scoring rules

Example measure • Men with a new diagnosis of prostate cancer, who have not had a serum PSA in the prior three months, should have serum PSA checked within one month after diagnosis or prior to any treatment, whichever comes first.

EVALUATING DATA SOURCES DATA SOURCE STRENGTHS WEAKNESSES Medical Record Clinical Detail Expense Missing links Administrative Use of services Clinical detail Patient Surveys General Health Expense Interpersonal Clinical detail

TESTING THE MEASURE • Reliability: The proportion of times that repeated use of measure in same population gives the same result • Validity: The extent to which the measure accurately represents the concept being assessed • Interpretability: Ease with which target audience can understand and use information generated by measure

WHY SHOULD CLINICIANS CARE ABOUT MEASURING QUALITY? • Internal quality improvement • External monitoring and evaluation • Consumer/purchaser decision-making

ADEQUACY OF CASE-MIX CONTROL • Severity of disease • Incidence and prevalence by demographics • age • race • gender

Measuring Clinical Lab Ordering Quality: Theory and Practice