The decision problem: What to do to who, when?

2. How (Biomedical) Science Thinks...and How We Can Help it Think BetterJeff ShragerCancer Commons, and Symbolic Systems

3. 1. Precision Oncology: Cancer in the Omic Era2. The obvious thing: Model/Literature-based tools: MMMP Targeted Therapy Advisor (TTA)3. Gtype->Rx Models are not a complete solution4. Enter Molecular Tumor Boards (MTBs)5. Advanced tool concepts based on MTB process:cTRACS - Coding and using MTB reasoningExpert-based Apps (CollabRx’s “Norman”)Coordinating distributed MTB reasoning (“Bayes Community”) Global Cumulative Treatment Analysis (GCTA)

4. The decision problem: What to do to who, when? Molecular Disease Model

5. AI Mag, 2011

6. Knowledge/Data Opportunities

7. Learning Opportunities

8. Planning Opportunities

9. The decision problem: What to do to who, when? Molecular Disease Model

10. Go Cancer Self-Driving Cars Jeopardy (random factoids) Closed World Closed World (wikipedia) Open World Relatively Open World Different Types of Problems Static rules (for thousands of years) Static Knowledge (wikipedia) Highly Dynamic (biology doesn’t change but everything else does) Static rules (for decades) Simulation is trivial Simulation is un-necessary Simulation is essentially impossible Simulation isn’t easy but is possible Experiments are free Experiments are un-necessary Experiments aren’t just extremely expensive, but every experiment kills people or animals in horrible, painful ways! Experiments are relatively in- expensive, but errors can kill! Feedback is instantaneous Feedback is instantaneous Feedback can take years and is very noisy Feedback is instantaneous and near perfect Data is essentially unlimited Data is essentially unlimited Data is essentially non-existent Data is plentiful

11. The decision problem: What to do to who, when? Molecular Disease Model

12. Treatment Validation in the pre-OMIC Era ~100 cells ~1 Million patients/year ~10,000 patients/cell Treatment selection was a statistical diagnosis problem: The diagnosis gave the treatment directly. A handful of treatments (chemo1, chemo2, etc.) A handful of phenotypes (lung, breast, etc.)

13. Pre-Omic Oncology Workflow Biopsy Biopsies Monitor Observations Treatment Treatment Selection Patient TreatmentPlanning Adapted from NY Times

14. Treatment Validation in the OMIC Era 109 cells! (11z) STILL ONLY A MILLION PATIENTS/YEAR! Approx. ZERO patients/cell! Thousands of treatments (New drugs + combinations) Treatment selection and planning are now reasoning problems, based upon huge amount of complex knowledge and data. Millions of omic phenotypes (enormous feature vector!)

15. Precision Oncology Workflow Normal skin cell Scans Original cancer cell Biopsies Serum Markers Panomics Biomarkers Treated cell Modeling and Targeting Combo Therapies Patient TreatmentPlanning Test In vivo In silico In vitro

16. 1. Precision Oncology: Cancer in the Omic Era2. The obvious thing: Model/Literature-based tools: MMMP Targeted Therapy Advisor (TTA)3. Gtype->Rx Models are not a complete solution4. Enter Molecular Tumor Boards (MTBs)5. Advanced tool concepts based on MTB process:cTRACS - Coding and using MTB reasoningExpert-based Apps (CollabRx’s “Norman”)Coordinating distributed MTB reasoning (“Bayes Community”) Global Cumulative Treatment Analysis (GCTA)

17. Precision Oncology Adapted from NY Times Normal skin cell Scans Original cancer cell Biopsy Biopsies Serum Markers Panomics Monitor Sequence Biomarkers Treated cell Combo Therapies Treat Compare Modeling Patient Target TreatmentPlanning Test In vivo In silico In vitro

18. The decision problem: What to do to who, when? Molecular Disease Model

19. Precision Oncology: Target Treatments to Patients Based on Molecular Disease Models (MDMs) Molecular Disease Model Usually Literature Based

22. Manually curated a decade of melanoma research into this table! PLoS ONE, 2010

23. Model system Number of cases 5/37 is an observational study with 37 subjects

26. PLoS ONE, 2010

28. Manually curated a decade of melanoma research into this table! PLoS ONE, 2010

29. Semi-Automated Literature Analysis Goal: Conditional probability of a cure given particular treatments and diseases: p ( cure | treatment, disease characterization) [p(c|t,d)] Meta-analysis extracts p(c|t,d) that is statistically tacit in the literature. Other potentially useful statistically tacit results derivable via Pubmed co-occurrence: Personalized (e.g., genetically guided) treatment hypotheses: p(t|g,d) Drug mechanism hypotheses p(g|t) Potential treatment response predictive biomarkers p(g|t,d) Potential markers of genetic diseases p(g|d) [g = gene; t = treatment (usually drug); d = diagnosis (or disease characterization)] AMIA, 2009

30. Method Unguided co-occurrence is hard to compute: 20,000 genes x ~1000 drugs x ~1000 diseases = 2^22,000 combinations Actual numbers: ~19 million articles x ~200 UMLS CUIs/article (via MGrep; ~85% accurate for diseases) + gene2pubmed assignments ~3 billion assignments in all Guidance: “related” or “positively related” in PharmGKB => 1,730 disease/drug/gene relationships with up to 6 conditional variables Extracted conditional probability tables (cpt's) using co-occurrence statistics over the ~3 billion assignments. => 19,092 conditional probabilities (v. ~2^22,000 for full-joint distribution) From these, any conditional probability of variables/entities expressed in a pharmGKB relationship can be directly computed!

31. Example Results A miniature Bayesian network for each of the pharmGKB relationships Example: p ( antidepressants | affective disorders, GNB3) = 0.33 Interpretation: Out of all the documents that mention “affective disorder” and gene “GNB3”, about one third also mention anti-depressants. Subordinate relationships: p(~an|af,g) =.67, p(an|~af,g) = 0.04, p(~an|~af,g) = 0.96, p(an|af,~g) = 0.11, p(~an|af,~g) = 0.89, p(an|~af, ~g) = 0.0, and p(~an|~af,~g) = 1.0 (Complementary conditional probabilities sum to 1.0) Many of these subordinate relationships may, of course, be Irrelevant. Example: p( azidothymidine | HIV, ABCC4) = 0.6. 60% of the papers where HIV and ABCC4 are mentioned, azidothymidine is also mentioned.

32. Example Results Relationships are not symmetric across the conditional: Example: p(mercaptopurine | azathioprine, thioguanine, TPMT) = 0.84 p(thioguanine | azathioprine, mercaptopurine, TPMT) = 0.73 p(azathioprine | mercaptopurine, thioguanine, TPMT) = 0.89 Subordinate relationships: p(azathioprine | thioguanine, TPMT) = 0.86 p(thioguanine | azathioprine, TPMT) = 0.44 The most clinically important results are the conditional probabilities of different treatments (e.g., drugs), per disease&gene combinations – (Approaching Personalized Medicine) Example: p(salmeterol | Asthma, ADRB2) = 0.07 p(salbutamol | Asthma, ADRB2) = 0.16 How do we interpret these results? At best these are pointers to hypotheses: Stronger biomarker? More reported side effects? Simple recency? Many interpretations are possible!

33. Learning from “N-of-1” Case Experiences Model system Number of cases 5/37 is an observational study with 37 subjects

34. As patients contribute their data, “5/1” rows are added to the TTD, updating the TTA in real time with individual case data 5/1 is a case study! (Ignore the middle column) Number of cases: 1 = case study 5 = Human Pervasive Health Care, 2012

35. 1. Precision Oncology: Cancer in the Omic Era2. The obvious thing: Model/Literature-based tools: MMMP Targeted Therapy Advisor (TTA)3. Gtype->Rx Models are not a complete solution4. Enter Molecular Tumor Boards (MTBs)5. Advanced tool concepts based on MTB process:cTRACS - Coding and using MTB reasoningExpert-based Apps (CollabRx’s “Norman”)Coordinating distributed MTB reasoning (“Bayes Community”) Global Cumulative Treatment Analysis (GCTA)

36. Go Cancer Self-Driving Cars Jeopardy (random factoids) Closed World Closed World (wikipedia) Open World Relatively Open World Static rules (for thousands of years) Static Knowledge (wikipedia) Highly Dynamic (biology doesn’t change but everything else does) Static rules (for decades) Simulation is trivial Simulation is un-necessary Simulation is essentially impossible Simulation isn’t easy but is possible Experiments are free Experiments are un-necessary Experiments aren’t just extremely expensive, but every experiment kills people or animals in horrible, painful ways! Experiments are relatively in- expensive, but errors can kill! Feedback is instantaneous Feedback is instantaneous Feedback can take years and is very noisy Feedback is instantaneous and near perfect Data is essentially unlimited Data is essentially unlimited Data is essentially non-existent Data is plentiful

37. Precision Oncology: Target Treatments to Patients Based on Molecular Disease Models (MDMs) Drug Availabilityand Affordability Clinical History Trial Practicalities Guidelines Dosing and Sequencing Details Disease Model Predictions Shared Decision Making PatientPreferences Patient Education Tumor Availability for Testing Treatment History FinancialConsiderations

38. Simon’s Nth Heuristic: “Follow the Experts!”

39. Normal skin cell Scans Original cancer cell Biopsy Biopsies Serum Markers Panomics Monitor Sequence Biomarkers Treated cell Combo Therapies Treat Modeling Patient Target TreatmentPlanning Test In vivo In silico In vitro

40. 1. Precision Oncology: Cancer in the Omic Era2. The obvious thing: Model/Literature-based tools: MMMP Targeted Therapy Advisor (TTA)3. Models are not a complete solution4. Enter Molecular Tumor Boards (MTBs)5. Advanced tool concepts based on MTB process:cTRACS - Coding and using MTB reasoningExpert-based Apps (CollabRx’s “Norman”)Coordinating distributed MTB reasoning (“Bayes Community”) Global Cumulative Treatment Analysis (GCTA)

41. Molecular Tumor Board Broad-based clinicalexperience: Medical oncologists Surgical oncologist Radiation oncologists Other medical specialists Biostatisticians Radiologists Pathologists Clinical geneticists Pathway experts Pharmacologists Oncological nurses Reimbursement Admins Computational Biologists

42. MTBs are literally engineering treatment plans for dying patients ... with NO TOOLS!! “We gotta find a way to make this, fit into the hole for this, use’nnothin but that.”

43. MTBs are engineering teams building treatment plans for dying patients...with NO TOOLS! Broad-based clinicalexperience: Medical oncologists Surgical oncologist Radiation oncologists Other medical specialists Biostatisticians Radiologists Pathologists Clinical geneticists Pathway experts Pharmacologists Oncological nurses Reimbursement Admins Computational Biologists Clinical Course The Patient Therapeutic Response Tumor Biology Disease Model

44. Typical Clinical Workflow • Treatment HypothesisGeneration& Testing Cycles Molecular Testing MTB Meets Patient Recommended for MTB Process Patient & Care Team Decide • Causal Mech. • Hyp-1 • Hyp-2 • - • Rx • Strategy • Opt-1 • Opt-2 • - • Decision • Line 1 • Line 2 • - • CaseSummary Reimbursement Support Process

45. Precision Oncology: Target Treatments to Patients Based on Molecular Disease Models (MDMs) Drug Availabilityand Affordability Trial Practicalities Guidelines Dosing and Sequencing Details Disease Model Predictions Shared Decision Making PatientPreferences Patient Education Clinical History Tumor Availability for Testing FinancialConsiderations Treatment History

46. Real MTB Report Out Findings were discussed at the ITOMIC tumor board. It was mentioned that complete loss of BRCA1 heterozygous loss of CHEK2 might underlie the patient’s excellent response to Cisplatin, and might predict sensitivity to a Poly-ADP ribose polymerase (PARP) inhibitor (Fong et al., N Engl J Med 2009;361:123-134, Tutt et al., Lancet 2010;376:235-44; Somlo, San Antonio Breast Cancer Conference 2013). Since PARP inhibitors are investigational, it was recommended that the patient consider enrollment into a PARP inhibitor trial. The two FGFR2 missense mutations were also discussed. Patient's tumor has FGFR mis-sense mutations similar to S252W. The COSMIC database lists 44 endometrial cancers bearing the FGFR2(S252W) variant, and 4 endometrial cancers with FGFR2(Y375C). Neither mutation has been described previously in breast cancer. Of note a single COSMIC entry describes an endometrial cancer (1105632) bearing FGFR2(S252W;Y375C). Both variants occur within the ligand binding domain, and S252W expands the repertoire of ligands capable of activating FGFR2 (Greulich and Pollock, Trends Mol Med. 2011;17: doi:10.1016/j.molmed.2011.01.012, DuttProcNatlAcadSci U S A 2008 PMID1855176, Yu et al. Proc Natl Acad Sci U S A. 2000 97:14536-41). ... This example thanks to Tony Blau, University of Washington.Please do not reproduce or pass on without permission.

47. 1. In Molecular Tumor Boards, cancer experts from many fields reason about the most difficult cases.2. The goal of MTBs is explicitly to assemble a model of the disease process inthe patient, based upon thebest current knowledge, and then to reasonfrom that model and everything known about the patient, to rank treatment options.3. MTBs expose this knowledge and reasoning in their inputs, discussions, and reports.4. MTB participants, being experts, use knowledgethat is not available from the literature, nor from anyother easily accessible source.

48. Kinds of knowledge unique to practice: (most conveniently accessible via MTBs)1. What parts of the literature are too early? (too tentative to be useful in practice)2. What parts of the literature are outdated? (no longer believed to be useful in practice)3. Knowledge not available at all in the literature (e.g., ethical, political) 4. What are the importance relationships between the various knowledge elements?5. How knowledge is interpreted in context of use?6. In-context cross-domain translations/relations.7. Assembly and explanation heuristics.(the “how to” of practical model-based reasoning) SCIENTIFIC PRAGMATICS

49. Kinds of knowledge unique to MTBs:1. Arguments pro and con hypotheses. (Published case reports are highly sanitized and are written explicitly for educational purposes.)2. Rationales for rejected hypotheses.(Often these never show up anywhere else!)3. Explicit “insider” (unpublished) knowledge.(e.g., as above: too early, outdated, ethical, political, ...)4. Implicit (unpublishable) knowledge relating to this particular sort of problem solving. 5. Stats on the sorts of cases that typically occur, and on the sorts of difficulties typically arise. IN CONTEXT KNOWLEDGE

50. What MTBs use and produce: Data from actual molecular tumorboard summary materials!