How to develop a novel medical diagnostic for a disease that is inaccurately defined?

1. How to develop a novel medical diagnostic for a disease that is inaccurately defined? Lyncean Seminar, May 4, 2011 by Ken Kupfer, Ph.D.

2. Outline • Medical Diagnostics, The Basics • When the “Gold Standard” is accurate • Example, Acute Coronary Syndrome • When the “Gold Standard” is inaccurate • Example, Acute Kidney Injury • Example, Sepsis • Conclusions

3. Medical Diagnositics, The Basics • The Problem: Diagnostic Test (T) vs. Gold Standard (D): • Patient’s present in some clinical context defined as the “Study Population”, e.g., patients presenting with signs and symptoms of a particular disease. • Gold Standard (D): Physician makes a diagnosis based on the clinical course of the patient over time (blinded to the results of test T). This dichotomizes the patient population: • D+ = patient has the disease of interest. • D- = patient does NOT have the disease interest. • Diagnostic Test (T): Patient undergoes a diagnostic test, e.g., blood sample is taken upon presentation and sent to a central laboratory for analysis. This dichotomizes the patient population: • T+ = test positive. • T- = test negative.

4. The ROC Curve & the Two-by-Two Simulated data for bi-normal ROC with STD=1.0 and Shift = 2.0 Area Under the ROC Curve (AUC) = 0.925 Two-by-Two Table with Specificity and Sensitivity (Marker Value > 1.5)

5. When the Gold Standard is Accurate: Example: Diagnosis of Acute Coronary Syndrome Study Population: Patients presenting to the Emergency Department (ED) with chest pain, or other symptoms consistent with Acute Coronary Syndrome (coronary ischemia with, or without necrosis). Symptom Onset Serial Blood Draws (T=0, 1.5 , 3.0, 6.0 hr) Final Adjudicated Diagnosis -4 hr 0 hr 10 hr 30 Days ED Presentation (T=0 hr) Clinical Course Disposition from ED, Hospital Admission, Objective Testing, Procedures, etc. Follow-up

6. Patients presenting to the Emergency Department (ED) with chest pain, Final Diagnosis Sub-Categories for 858 patients*: Non-MI MI *858 patients with 3 evaluable serial draws at 0, 1.5 hr, and 3.0 hr. **CPNOS includes 10 patients classified as non-ACS without a sub-category assignment. Population is Dichotomized: Myocardial Infarction (MI) vs. non-MI

7. ROC Curve for Peak Troponin I (TnI) based on testing of 3 serial draws over 3 hours (at T=0, 1.5 hr, 3.0 hr) in 858 patients (776 non-MI vs. 82 MI): Test > Cutoff

8. ROC Curve for Peak Troponin I (TnI) based at each of 4 serial draws over 6 hours (at T=0, 1.5 hr, 3.0 hr, 6.0 hr) in 680 patients (608 non-MI vs. 72 MI):

9. Patients presenting to the Emergency Department (ED) with chest pain: Now focus on the non-MI categories, particularly NCCP and UA. *858 patients with 3 evaluable serial draws at 0, 1.5 hr, and 3.0 hr. **CPNOS includes 10 patients classified as non-ACS without a sub-category assignment. Population is Stratified and then Dichotomized: UA vs. NCCP

10. Focus on non-MI patients to distinguish between UA (ischemia) and NCCP (where ischemia has been ruled-out): Troponin is not a good marker. ROC Curve for Peak Troponin I (TnI) based on testing of 3 serial draws over 3 hours (at T=0, 1.5 hr, 3.0 hr) in 515 patients (403 NCCP vs. 112 UA):

11. When the Gold Standard is Inaccurate: Example:Diagnosis of Acute Kidney Injury (AKI) How Kidney Function is Measured in Clinical Practice: Measurement of serum Creatinine over time. One-Compartment Model of Creatinine Kinetics: d(CV)/dt + K(t)C(t) = G C = Plasma Creatinine Concentration V = Volume of Distribution of Creatinine K = Creatinine Clearance G = Generation Rate of Creatinine C(t) is measured over time to infer the value of K(t), assuming G and V are approximately constant. Increases in C(t) are related to decreases in K(t), which represent a decline in kidney function. Typical values: K ~ 60 dL/hr, G ~ 60 mg/hr, V ~ 400 dL, C ~ 1 mg/dL

12. Definition of Chronic Kidney Disease (CKD) KDOQI [Kidney Disease Outcomes Quality Initiative] stages of CKD are based on estimated kidney function (Glomerular Filtration Rate, or GFR) measured by a single serum Creatinine. Progression is tracked over time (months to years). *GFR from MDRD [Levey et al, Annals of Internal Medicine, 2006] in Units of mL/min per 1.73 m^2 GFR = 175 x sCr ^ -1.154 * age ^ -0.203 * 1.212 (if patient is black) * 0.742 (if female)

13. Definition of Acute Kidney Injury (AKI) RIFLE Classification System [Hoste et al., Critical Care 2006] for AKI is based on a loss of kidney function (Glomerular Filtration Rate) measured by an acute increase (within 1-7 days) in serum Creatinine relative to the patient’s baseline. AKI is classified by Severity (RIFLE R, I, or F). Note, AKIN Classification (Stage I, II, or III) is same as R, I, or F, except that Stage I includes either a 1.5x rise, or an absolute rise of 0.3 mg/dl, provided it occurs rapidly (within 48 hr).

14. Simulations of sCr Kinetics assuming an instantaneous loss in renal filtration rate due to kidney injury [Waikar et al., J Am Soc Nephrol 20: 672-679, 2009]: The clinical syndrome of “Acute Kidney Injury” will miss kidney injury if the decrease in renal filtration is not sufficiently large (>33% reduction), or if patient has CKD and creatinine rise is too slow.

15. Etiology of Acute Kidney Injury & Renal Failure The driving force for renal function (glomerular filtration) is the pressure gradient from the glomerulus to the Bowman space, which is primarily dependent on renal blood flow (RBF): Prerenal failure - Defined by conditions with normal tubular and glomerular function; GFR is depressed by compromised renal perfusion. Volume loss from GI, renal, cutaneous (eg, burns), and internal or external hemorrhage, or decreased renal perfusion in patients with heart failure or shock (eg, sepsis, anaphylaxis) can result in this syndrome. Intrinsic renal failure - Includes diseases of the kidney itself, predominantly affecting the glomerulus or tubule, which are associated with release of renal afferent vasoconstrictors; ischemic renal injury is the most common cause of intrinsic renal failure. Postobstructive renal failure - Initially causes an increase in tubular pressure, decreasing the filtration driving force; this pressure gradient soon equalizes, and maintenance of a depressed GFR is then dependent on renal efferent vasoconstriction .

16. Diagnosis of Acute Kidney Injury in the ICU: Study Population: All patients admitted to the ICU (at a single center), excluding patients with end-stage renal disease (ESRD), chronic dialysis (RRT), or Nephrectomy. AKI is from daily creatinine data (up to 7 days after ICU admission) using the RILFE criteria (R, I, and F). Baseline Creatinine from medical record up to 4 weeks prior to ICU Admission. Blood & Urine Collection (T= 0) Record in-hospital outcomes, mortality, need for RRT, etc. -24 hr 0 hr 24 hr 48 hr 72 hr 96 hr 120 hr 144 hr 168 hr ICU Admission Daily Creatinine from (up to) 4 days prior to 7 days post ICU Admission (depending on duration of hospitalization)

17. Patient ICU Diagnosis • Inclusion n= 632 patients *p-value for significance of association with AKI is determined by Chi-Square test.

18. AKI Severity and Timing: Inclusion n= 632 patients “1st Draw” is ICU Admission, i.e., patient’s may satisfy RIFLE criteria at admission.

19. ROCs for AKI at each Severity Threshold: Plasma NGAL Urine NGAL ROC curve analysis for ICU admission NGAL at each AKI severity threshold. RIFLE R includes patients who reached RIFLE R or higher, RIFLE I includes patients who reached RIFLE I or higher. Legend gives AUC +/- SE.

20. “Fool's Gold: Creatinine as the Diagnostic Standard for AKI Biomarker Studies,” Waikar et al., presented at ASN 2010 [SA-PO2051] Sn, sensitivity; Sp, specificity

21. Inclusion criteria: • 18+ years • 2+ SIRS criteria • Confirmed or suspected infection and/or lactate > 2.5 mmol/L Outcome Within 72 h and 30 days Provisional Diagnosis Hospital Arrival Plasma Specimen within 3 hours • Adjudication Final Diagnosis • Severe Sepsis • Early Sepsis / High Risk of Progression • Exclusion Criteria: • Cardiac arrest • Do not resuscitate order • Pregnancy When the Gold Standard is Inaccurate: Early Diagnosis of Sepsis in Patients presenting to the Emergency Department with Suspected Sepsis. SIRS Criteria: Temperature, Hear rate, Respiratory rate, WBC

22. Sepsis Pathophysiology

23. Marker Screening & Data Mining

24. Marker List: “Top 9” after screening

25. Biomarker Score (Multiple Logistic Regression)for Severe Sepsis

26. Biomarker Score increases with Severity of Sepsis The differences in median values (red lines) among all groups are significant (p<0.05).

27. Biomarker Score (Multiple Logistic Regression)for Early Sepsis (Patient’s at High Risk of Progressing)

28. What if Gold Standard for Sepsis was defined by another biomarker? “Toy Model”: Biomarker Score (Multiple Logistic Regression)for Prediction of TNF-R1a Low vs. High

29. Conclusions • Simple blood tests are very helpful in medical decision making. • Physicians define a disease based on the clinical course of the patient. • Companies collect blood samples from patients hoping to develop diagnostics for the disease, i.e., classifiers that discriminate disease from non-disease. • However, in many cases, the clinical definition of disease does not accurately reflect the underlying biology and scientists are faced with the problem of developing a new diagnostic test whose success requires an altered definition of disease.