Sequence-based Treatment Care Plan

1. S-TCP Sequence-basedTreatment Care Plan YanirVigler and Eddie Galfrin Under the supervision of Esther Goldbraich, IBM Feb 2014

2. Terminology • RTP – Recommended Treatment Program • ATP – Actual Treatment Program • CPG – Clinical Practice Guide • Program – A collection of sequences • Sequence – An ordered collection of treatments • Treatment - one of following: • Chemotherapy – with its drugs, cycles and type. • Surgery – with its type. • Radiotherapy.

3. Objectives • Modeling ATPs and building the model based on input annotations. • Developing an algorithm which receives the ATPs and relevant RTPs that are in the CPG and: • Detect if the ATP complies with one of the RTPs • Otherwise, find the least different RTP and evaluate the difference with the ATP. • Cost Model – Develop a cost model for ranking the deviations between the ATP and the best matched RTPs.

4. Methodology - Matching • The chosen algorithm for the matching is a variant of the Smith-Waterman (S-W) algorithm. • The original algorithm provides the best local alignment only. • The S-W calculates the differences between an RTP to every permutation of the ATP to support reordering deviations. • We’re grouping all the cycles of each chemotherapy to overcome the cycle comparison problem.

5. Methodology - Comparison • Comparison between an ATP and an RTP is difficult because they are represented by different models. • We’ve extracted the needed data from each data model and represented it in the same way. • These extractors are called ATP Parser and RTP Parser respectively.

6. ATP-RTP Parsers • The RTP parser, which takes data from the database and constructs sequences out of it. • The ATP parser iterates through the lines of the CSV file, converts those lines to treatments \ treatment parts (e.g. surgery type, cycle number of chemotherapy, etc.), from that it builds two sequences - one with treatments a physician prescribed to the patient, and another one of what actually was executed. Because we assume a "partial data world" we take the data from both sequences and combine them together to create one sequence of the treatment given to the patient.

7. Methodology - Cost Model • The cost model is naturally inserted into the algorithm: • +2 when the two items are the same • -1 when they are different, there is an insertion or deletion of an item to\from the sequence respectively. • In our extended algorithm we broaden this concept by defining positive costs to similar treatments with increasing costs (similar treatments are scored higher), and decreasing negative costs to different treatments. For example: • cost(Chemotherapy, Same) = 1 • cost(Permutation Penelty) = -0.15 • cost(Chemotherapy, Extra) = -0.8 • etc…

8. Methodology - Heuristics • If only we could’ve lived in a perfect world… • But, as we saw, it’s not the case, we needed to add several heuristics and assumptions on the data. For example: • In ATP, the order of the treatments is determined by the order of their appearance. • A patient can have only one surgery in a treatment in a row. • If a chemotherapy taken from the ATP has no drugs – we assume the drugs are identical to those in the RTP. • More to come.

9. Methodology - Scalability • IBM hasn’t provided us the entire data and new data is coming every time. • We needed to create our program with the ability to simply add data - treatments, drugs, etc. The main problem was that we have two different sets of data with no ability to automatically match between them. • All the dictionaries we constructed are stored in external CSV files, so when a new procedure will appear, a simple update of those CSV files will be enough to instruct the software how to deal with the new data.

10. For example Treatment 1 Treatment 2 …. …. Treatment 8 Treatment 7

11. Cost Model Examples

12. Achievements • When we started this project, IBM had a working software that solves this problem . • IBM wanted to see whether a better score can be achieved by representing treatments as sequences.

13. So…How did we do? • As agreed after our kick-off presentation there will be 3 parameters to validate and mark our algorithm: • Accuracy of ATP model • Accuracy of the deviation labeling • Accuracy of the comparison algorithm

14. Achievements • Accuracy of ATP model: • Tests the accuracy of the ATP parser and the correctness of the annotations against the ATPs which were manually extracted from plain text by a hired professional. • Recall - The percentage of ATPs that we identified correctly out of all ATPs. • IBM’s recall score: 0.705 • Our recall score: 0.724 • Because we got almost the same score, the problem is probably in the annotations extracted by the NLP tools.

15. Achievements • Accuracy of the deviation labeling: • Tests the matching algorithm. Checks the number of ATPs we marked as deviated against the ATPs which were manuallymarked as deviated from plain text by a hired professional. • Precision – the percentage of ATPs that should be labeled as deviated out of those labeled as deviated by the algorithm. • Recall – the percentage of ATPs that the algorithm marked as deviated out of total number of deviated programs. • Score is • IBM’s score: 0.74 • Our score: 0.68

16. Achievements • Accuracy of the comparison algorithm: • Tests the comparison algorithm. Checks if the algorithm finds all the ATPs’ existing deviations in comparison to the algorithm’s best found matching RTP. • Recall – The percentage of ATPs that the algorithm identified the differences from a specific RTP correctly out of all ATPs. • IBM’s recall score – 0.88 • Our recall score* – 0.9 • This test checks the only the correctness of our data without the correctness of the NLP tools. (*)  We have improved the comparison algorithm, so it handles the remaining 10% of the tests. However, IBM haven't checked yet the new algorithm with the entire data set, so the old score still applies.

17. Extra Achievements • The final results are stored both in the database and in a LyX file and may be previewed as a well formed PDF file for ease of use. • Since the cost model provides us only with “a per treatment” cost, when reviewing the scoring as a whole, we can see that there may be several matching sequences not thought of a-priori.

18. Future work • Improving the result may be done in several ways • Revise the specific costs • Define more unknown factors, such as new drugs and more. • Improve the comparison logic • Improve annotations and NLP analyses. • Try to remove redundancy caused by using multiple NLP tools. • Adding more annotations. • Source code – fix treatment hierarchy.

19. Conclusions • Good planning before coding saves lots of time later on. • In real-life programming there are many edge cases. • This course is a good pick at a real-world programming work. • Sleep deprivation creates the best bugs.