Advancements in Automated Synoptic Reporting

1. Advancements in Automated Synoptic Reporting George Cernile, Manager A.I. Technology Group Artificial Intelligence In Medicine Inc.

2. Agenda

3. Pathology Report Synoptic Report CLINICAL HISTORY/MACROSCOPY Right mastectomy and axillary tissue. A right mastectomy specimen with overlying skin measuring 220mm x 85mm and underlying breast tissue measuring 220mm x 100mm x 70mm. The axillary tail measures 125 x 60mm. The nipple is slightly retracted and located centrally. The superior margin is painted red, the inferior margin painted green and the deep cut margin is painted blue. Cut sections of the underlying breast tissue shows an ill-defined grey white yellow lesion with patchy areas of haemorrhage measuring 35 x 35 x 35mm located immediately below the nipple, 20mm from the inferior margin, 45mm from the deep cut margin, 50mm from the superior margin, 85mm from the medial margin and 100mm from the lateral cut margin. A1 - nipple, B1 - upper outer quadrant, C1 - upper inner quadrant, D1 - lower outer quadrant, E1 - lower inner quadrant, F1, G1 - tumour composite blocks, H1, I1 - tumour composite blocks, J1 - deep cut margin, K1 - superior margin, L1 – inferior margin, M4 - lymph nodes, N4 - lymph nodes, O - 3 serial slices, lymph node, P - 3 lymph nodes. MICROSCOPY This right mastectomy specimen demonstrates an invasive ductal carcinoma with the following pathological features: TUMOUR HISTOLOGY & GRADE The tumour is of an infiltrating poorly differentiated ductal carcinoma of non-otherwise specified type. The tumour is poorly defined and extremely infiltrative, comprising poorly-formed tubules, nests or strands of cuboidal tumour cells displaying high grade nuclei. The tumour cells are set within fibrotic desmoplastic stroma. Many lactiferous ducts are entrapped within the tumour. Frequent tumour mitoses are seen. Microcalcification is seen in some neoplastic tubules. Tumour grade (Modified Bloom-Richardson Scoring System): Tubular formation: 3 Nuclear atypia: 3 Tumour mitoses: 2 Total score: 8 (Grade III) TUMOUR LOCATION, SIZE AND EXTENT The tumour is located 5mm below the nipple and has a macroscopic size of 35mm across. The border of the tumour is poorly circumscribed and infiltrative. INTRA-LYMPHOVASCULAR OR PERINEURAL TUMOUR PERMEATION Focal intralymphatic tumour permeation is noted. No perineural tumour invasion is seen in sections submitted.

4. Pathology Report Synoptic Report CLINICAL HISTORY/MACROSCOPY Right mastectomy and axillary tissue. A right mastectomy specimen with overlying skin measuring 220mm x 85mm and underlying breast tissue measuring 220mm x 100mm x 70mm. The axillary tail measures 125 x 60mm. The nipple is slightly retracted and located centrally. The superior margin is painted red, the inferior margin painted green and the deep cut margin is painted blue. Cut sections of the underlying breast tissue shows an ill-defined grey white yellow lesion with patchy areas of haemorrhage measuring 35 x 35 x 35mm located immediately below the nipple, 20mm from the inferior margin, 45mm from the deep cut margin, 50mm from the superior margin, 85mm from the medial margin and 100mm from the lateral cut margin. A1 - nipple, B1 - upper outer quadrant, C1 - upper inner quadrant, D1 - lower outer quadrant, E1 - lower inner quadrant, F1, G1 - tumour composite blocks, H1, I1 - tumour composite blocks, J1 - deep cut margin, K1 - superior margin, L1 – inferior margin, M4 - lymph nodes, N4 - lymph nodes, O - 3 serial slices, lymph node, P - 3 lymph nodes. MICROSCOPY This right mastectomy specimen demonstrates an invasive ductal carcinoma with the following pathological features: TUMOUR HISTOLOGY & GRADE The tumour is of an infiltrating poorly differentiated ductal carcinoma of non-otherwise specified type. The tumour is poorly defined and extremely infiltrative, comprising poorly-formed tubules, nests or strands of cuboidal tumour cells displaying high grade nuclei. The tumour cells are set within fibrotic desmoplastic stroma. Many lactiferous ducts are entrapped within the tumour. Frequent tumour mitoses are seen. Microcalcification is seen in some neoplastic tubules. Tumour grade (Modified Bloom-Richardson Scoring System): Tubular formation: 3 Nuclear atypia: 3 Tumour mitoses: 2 Total score: 8 (Grade III) TUMOUR LOCATION, SIZE AND EXTENT The tumour is located 5mm below the nipple and has a macroscopic size of 35mm across. The border of the tumour is poorly circumscribed and infiltrative. INTRA-LYMPHOVASCULAR OR PERINEURAL TUMOUR PERMEATION Focal intralymphatic tumour permeation is noted. No perineural tumour invasion is seen in sections submitted.

5. Comparison to manual review • Manual extraction of Synoptic data from text 15-20 min. per report • Manual extraction with Synoptex Assist 2 min. per report • Fully automated extraction 2 – 3 seconds per report

6. How smart is it? Specimen type : Hemicolectomy, right, partial excision of urinary bladder, and salpingo-oophorectomy, right Orientation Clinical information : Colon, right Anatomical landmarks : Ileum = proximal Surgical markings : Absent Pathlogy markings : Ink = margins of urinary bladder wall Ileum, terminal Length : 8 cm Serosa : Unremarkable Wall : Thickness: 0.4 cm Mucosa : Slightly edematous Vermiform appendix : Absent Colon, right Length : 22 cm Serosa : Cecum: Puckering and attached portion of urinary bladder wall and right adnexa (see below) Otherwise: Unremarkable Wall : Thickness: 0.4 cm Mucosa : Tumour: - Location: Ileocecal valve - Configuration: Fungating - Size: - Length: 7 cm - Width: 12 cm - Thickness: 7 cm - Extent of invasion: Urinary bladder - Margins: - Proximal: 8 cm - Distal: 15 cm - Radial: 0.1 cm` Non-neoplastic colon: Unremarkable Data can be found in many formats Data aggregate detection Pattern matching Heuristics for logical consistency Language processing

9. Agenda

10. Measuring Synoptex Performance How often does Synoptex return the expected value? How often does Synoptex return a non-expected value? Is there a tradeoff between completeness and accuracy? How does Synoptex improve with new iterations?

11. How do we measure performance? Sensitivity and Specificity measure the performance of a binary classification system • Present/Not present • True/False In the Synoptex case, this could only be applied to a single value, for example histologic type Need a new method to evaluate Synoptex that is analogous to Sensitivity and Specificity Sensitivity and Specificity? Synoptex has many variables and many checklists

12. Accuracy Model step 1 • C= Correct: • Sum of correctly returned values by Synoptex. • ∑ (all correctly found values) • I= Incorrect: • Sum of incorrectly returned values by Synoptex • ∑ (all incorrect values) • M= Missed: • Sum of values the were missed by Synoptex • ∑ (all missed values) Build a manual reference data set Define three measures from the manually reviewed reference data

13. Accuracy Modelstep 2 • Required • Correct + Missed = C + M • Returned • Correct + Incorrect = C + I Define two new concepts with the variables C = Correct (all correct values) I = Incorrect (all incorrect values) M = Missed (all missed values)

14. Accuracy Model • Completeness • Correct / Required= C / (C + M) • The ability to find the correct data when it is in the path report. • Accuracy • Correct / Returned= C / (C + I) • The ability to find only the correct data when it is in the path report. Accuracy can be seen as a measure of exactness or fidelity. Sensitivity C = Correct I = Incorrect M = Missed Specificity

15. Accuracy results - Melanoma Completeness 81.2% average Accuracy 82.0% average

16. Accuracy results - Breast Completeness 93.4% average Accuracy 93.8% average

17. Agenda

18. The Challenge: How do we expand capabilities of Synoptex system

23. Agenda

24. Synoptex Applications • Standardization of Pathology Reports for Registries • Data Quality Assessments • Audits • Mining of Current and Legacy Data • Export of Indexed Data for Statistical Processing • Annotated Datasets for Biospecimen Repositories • Data for Automated Record Matching • Clinical Trials • Case Control Studies

25. Pathology Data Completeness Report Element Analysis Site: Breast Reports Sample Size: 49776 Date Range: 2004 - 2007 No. Of Cancer Reports: 8422 No. Of Breast Cancers: 1678

26. Example: ER Positivity vs. HER2 Status - ER + 324 (61%) - HER2 + (68%) 148 (68%) (61%) ER Positive %

27. Agenda

28. The challenge: • How frequently are they updated? • How do we manage updates/versions? • How can systems be maintained ? • How can custom additions be kept across versions? CAP Checklists version updates

29. CAP electronic Cancer Checklist (CAP eCC)

30. Checklist Editor for CAP eCC

31. How is this related to Synoptex? Interoperability • Synoptex output same format as Manual checklists • SNOMED codes • CKEYS • Synoptex data can be viewed with STS checklist system

32. Synoptex uses same database as (CAP eCC) Export Synoptic Database Export

33. Automatic Updates Knowledge Base Manager CAP eCC updates Import Export Export Synoptic Database

34. Synoptex Benefits • Renders Pathology Reports Machine Readable • Facilitates Research • Identifies Some Collaborative Stage Elements • Saves Labor in Deriving Synoptic Reports – both manually and fully automated • Verifies Report Content (Audits) • Compare reporting accuracy hospital-to-hospital • Apply Synoptex to other data sources • Surgical reports • Medical records • Radiology reports • Hematology reports

35. The End Thank You