Clinical Outcome Assessments and epistemic Risk

1. Clinical Outcome Assessments and epistemic Risk Leah McClimans

2. Outcome Measure Ambiguity • What is a health outcome? • There are three major health outcomes: mortality, morbidity and quality of life • What is an outcome measure? • Clinical outcome assessments, e.g. patient-reported outcome measures (PROMs) • Refer to measuring instruments or tools, usually questionnaires, that measure patients’ health status by asking them questions about it • Odds Ratio, number needed to treat, relative risk, etc. • Refer to formal statements that aim to quantify causality of e.g. exposure to an outcome

3. Measuring what is hard to quantify • Patient reported outcome measures • Usually take the form of a questionnaire, answers to questions come from patient without “amendment or interpretation…by a clinician or anyone else” (FDA) • Attempts to quantify “symptoms or unobservable concepts known only to the patient, e.g. pain severity or nausea” or functionings or activities that are observable to others but which the patient’s perspective is considered important (FDA)

4. How Do PROMs quantify their phenomena?

5. Two Measurement Theories Classical test theory Latent trait theory • Most popular measurement theory in psychology • Utilizes a simple model: O=T+E • Unfalsifiable • Ontologically profligate • Popular in psychometrics and educational testing • Utilizes a variety of models that relate observed scores to a latent variable, e.g. ability. • The position on the latent variable “causes” the observed scores • Adequacy of the model is determined by the goodness of fit between the observed scores and the predictions of the model • Falsifiable • Ontologically parsimonious

6. Which Theory is better? • Better how? • Interpretable? • Simple? • Useful? (to whom?) • Ethical/value adequate? • Reliable? • Valid? • Ensure quantitative variable?

7. Making choices, accepting risk • Radical suggestion: No single measurement theory or model is better on all dimensions • Classical test theory • Simple, but difficult to interpret • Latent trait theory • Complex, but interpretable • Both have found uses, yet have validity and ethical/ value adequacy problems • QoL is difficult to quantify because it’s a “Ballung” concept • Operationalizing it, determining what model to use involves emphasizing certain qualities over others • Simplicity, interpretability, reliability, etc. • This emphasis entails accepting a level of epistemic risk

8. Risky Choices or metaphysical commitments? • Classical test theory vs. Latent Trait theory redux • Metaphysics in psychological measurement • A radical suggestion • If Risks are adjudicated in light of values and interests, then values and interests should play a more explicit role than they do in qoL measurement • “Coherent interpretation of many claims about measurement in the psychological sciences depend on philosophically realist commitments regarding psychological attributes” (Andrew Maul, “on the ontology of psychological attributes) • To summarize the realist position: understanding measurements under the umbrella of the realist concept of truth, commits us not just to the logically independent existence of things in space and time, but also to the existence of quantitatively structured properties and relations, and to the existence of real numbers, understood as relations of ratio between specific levels of such attributes. (Joel Michell, “The logic of measurement: a realist overview) • “Latent variable theory adheres to entity realism, because this form of realism is needed to motivate the choice of model in psychological measurement” (Denny Borsboom, measuring the Mind)