Real-time Text Mining for the Biomedical Literature a collaboration between Discovery Net & myGrid

1. Real-time Text Mining for the Biomedical Literature a collaboration between Discovery Net & myGrid Rob Gaizauskas Department of Computer Science University of Sheffield Moustafa M. Ghanem Department of Computing Imperial College London EPSRC E-Science Meeting, NeSC

2. Outline • Context • Workflows, Services and Text Mining • Discovery Net & myGrid • Aims and Objectives of New Project • Architecture of New System • Integration of Existing Components • Approach to Text Mining • Data Resources & Evaluation • Techniques for Go Tagging • Interface and Results Presentation • Lessons Learnt So far, Conclusions and Broader Applicability of Work EPSRC E-Science Meeting, NeSC

3. Workflows, Web Services and Text Mining for Bioinformatics • Workflows • useful computational models for processes that require repeated execution of a series of complex analytical tasks • e.g. biologist researching genetic basis of a disease repeatedly • maps reactive spot in microarray data to gene sequence • uses a sequence alignment tool to find proteins/DNA of similar structure • mines info about these homologues from remote DBs • annotates unknown gene sequence with this discovered info EPSRC E-Science Meeting, NeSC

4. Workflows, Web Services and Text Mining for Bioinformatics • Web services • Processing resources that are • available via the Internet • use standardised messaging formats, such as XML • enable communication between applications without being tied to a particular operating system/programming language • Useful for bioinformatics where data used in research is • heterogeneous in nature – DB records, numerical results, NL texts • distributed across the internet in research institutions around the world • available on a variety of platforms and via non-uniform interfaces EPSRC E-Science Meeting, NeSC

5. Workflows, Web Services and Text Mining for Bioinformatics • Text mining • any process of revealing information – regularities, patterns or trends – in textual data • includes more established research areas such as information extraction (IE), information retrieval (IR), natural language processing (NLP), knowledge discovery from databases (KDD) and traditional data mining (DM) • relevant to bioinformatics because of • explosive growth of biomedical literature • availability of some information in textual form only, e.g. clinical records EPSRC E-Science Meeting, NeSC

6. Workflows, Web Services and Text Mining for Bioinformatics Web services Workflows Text mining Bioinformatics EPSRC E-Science Meeting, NeSC

7. Discovery Net & myGrid • Discovery Net: An e-Science testbed for High Throughput Informatics • £2.2M EPSRC Pilot Project • Started Oct 01, Ended in March 05 • Service-based infrastructure/workflow model for Life Sciences, Environmental Modelling and Geo-hazard Modelling • Infrastructure for mixed data mining / text mining • Machine learning methods for text mining • myGrid: Directly Supporting the e-Scientist • £3.5M EPSRC Pilot Project • Started Oct 01, Ends June 05 • Service-based infrastructure/workflow model for Life Sciences • Infrastructure for Text Collection Server, Text Services Workflow Server and Interface/Browsing Client • Service-based Terminology Servers EPSRC E-Science Meeting, NeSC

8. myGrid • Overall aim: develop an e-biologist’s workbench – a platform allowing biologists to execute, analyze, repeat multi-stage in silico experiments involving distributed data, code and processing resources • Workflow model for composing/executing processing components • Web services for distribution • Problem: how to integrate text mining into a biological workflow? • Most text mining runs off-line and supports interactive browsing of results • Most workflows run end to end with no user intervention • What are the inputs to text mining to be? • Solution: tap off result of a workflow step and treat as implicit query EPSRC E-Science Meeting, NeSC

9. A myGrid example studying the Genetic Basis of Disease • Graves’ Disease • an autoimmune condition affecting tissues in the thyroid and orbit • being investigated using the micro-array methods • micro-array shows which genes are differentially expressed in normal patients vs patients with the disease = candidate genes • sequence alignment search (e.g. BLAST) finds genes/proteins with similar structure • function of these “homologues” may suggest function of candidate gene • key step for text mining follows BLAST search • for homologous proteins BLAST report contains references to proteins in SWISSPROT protein database • Swissprot records contain ids of abstracts describing the protein in Medline abstract database • abstracts can be mined directly or used as ``seed'' documents to assemble a set of related abstracts EPSRC E-Science Meeting, NeSC

10. myGrid Text Services Architecture Workflow definition + parameters User Client Workflow Server Clustered PubMed Ids + titles Initial Workflow Cluster Abstracts Workflow Enactment Swissprot/Blast record Extract PubMed Id Get Related Abstracts Term-annotated Medline abstracts Get Medline Abstract Medline Server Medline Abstracts PubMed Ids Medline: pre-processed offline to extract biomedical terms + indexed PubMed Ids EPSRC E-Science Meeting, NeSC

11. myGrid Text Services Architecture • 3-way division of labour sensible way to deliver distributed text mining services • Providers of e-archives, such as Medline, will make archives available via web-services interface • Cannot offer tailored sevices for every application • Will provide core, common services • Specialist workflow designers will add value to basic services from archive to meet their organization’s needs • Users will prefer to execute predefined workflows via standard light clients such as a browser • Architecture appropriate for many research areas, not just bioinformatics EPSRC E-Science Meeting, NeSC

12. Abstract Titles MeSH Tree Abstract body Search scope restrictors Linked terms Get Related Abstracts Free text search myGrid Interface/Browsing Client EPSRC E-Science Meeting, NeSC

13. Discovery Net: Adding text mining to e-Science workflows Gene Expression Analysis Find Relevant Genes from Online Databases Find Associations between Frequent Terms • DNet Workflow server executes DPML workflow and uses Discovery Net’s InfoGrid data access and integration wrappers and web services EPSRC E-Science Meeting, NeSC

14. Text Mining in e-Science workflows • Problem: how to develop new distributed text mining applications using a workflow? • Most text mining applications require the integration of a mixture of components (Services) for text processing tasks (e.g. parsing and cleaning), natural language processing (e.g. named entity recognition), statistics and data mining (e.g. classification, clustering, etc). • There are many design alternatives and end users may want to prototype and compare alternative implementations. • Once application developed, most workflows run end to end with no user intervention • Solution: Extend service infrastructure to allow composition of text mining services. EPSRC E-Science Meeting, NeSC

15. Data Mining Classification, Clustering, Association, Statistical Analysis, Visual Analysis, etc … Feature Extraction Statistical: Word Counts, Pattern Extraction & Counts, etc Domain-specific Gene Name counts, etc NLP-specific Phrase counts, etc Text Processing Stemming, Stop-word filters, Pattern filters, Lexicon matching, Ontologies, NLP parsing etc, .. Retrieval/ Storage Indexing Access Drivers Storage Numerical Feature Vectors Text docs Text docs Text documents Building text mining applications from workflows Using workflow technologies to build text mining applications and services using finer grain components/services Text Mining Pipelines Features are summarized into vector forms which are suitable for data mining Results can be document characterization or hidden relationship extraction Pre-process documents to enhance the ease of feature extraction Retrieve and organize relevant documents EPSRC E-Science Meeting, NeSC

16. Simplified Document Classification Workflow Examples of Extracted Patterns GENE_NAME protein GENE_NAME express express GENE_NAME GENE_NAME mutant GENE_NAME activity activity GENE_NAME GENE_NAME drosophila Examples of Pattern Definitions delet\s([a-z]*(\s)+)*genenam+\s depend\s([a-z]*(\s)+)*genenam+\s describ\s([a-z]*(\s)+)*genenam+\s detect\s([a-z]*(\s)+)*genenam+\s determin\s([a-z]*(\s)+)*genenam+\s differ\s([a-z]*(\s)+)*genenam+\s disc\s([a-z]*(\s)+)*genenam+\s dna\s([a-z]*(\s)+)*genenam+\s Predictive Accuracy of Relevance prediction, using Support Vector Machine classification Overall accuracy: 84.5% Precision 78.11% Recall 73.40% EPSRC E-Science Meeting, NeSC

17. Text Meta Data Model Build Classifier training phase using workflow co-ordinating distributed services Build Prediction phase using workflow co-ordinating distributed services Metadata Model: Service Interfaces only tell you how to invoke remote service but it is up to you to decide what information flows between services ! EPSRC E-Science Meeting, NeSC

18. Aims & Objectives of New Project • Aim: to develop a unified real-time e-Science text-mining infrastructure that leverages the technologies and methods developed by both Discovery Net and myGrid • Software engineering challenge: integrate complementary service-based text mining capabilities with different metadata models into a single framework • Application challenge: annotate biomedical abstracts with semantic categories from the Gene Ontology • Deliverables: • D1: A GO Annotation Service • D2: A Generic Shared Infrastructure for Grid-enabled Biomedical Document Categorization • D3: Infrastructure for Semantic Document Annotation • D4: A Detailed Case Study (analysing/evaluating the GO annotator) • D5: Developing a common framework for representing + exchanging information about: 1. Data: biomedical documents/doc collections + metadata, biomedical dictionaries 2. Intermediate data: Document indexes and Document feature vectors 3. Text Analysis Results EPSRC E-Science Meeting, NeSC

19. Go TAG: A Novel Application • The GO TAG Application: Automatic Assignment of GO (Gene Ontology) Codes to Medline Documents EPSRC E-Science Meeting, NeSC

20. A Machine Learning Approach Overview of Training Phase EPSRC E-Science Meeting, NeSC

21. Run-time System Overview of Run-time System EPSRC E-Science Meeting, NeSC

22. GO Annotator – Version 1 • Version 1a: • Direct search for GO Annotation descriptions and synonyms in document text • If description is found, document is labelled with this GO Annotation • Description is also marked-up in document • Version 1b: • 1a + search for gene names extracted from yeast genome DB • If gene name found, document labelled with GO annotation(s) associated with gene in DB • Gene name also marked up in document • Termino web-service, hosted at Sheffield, provides lookup capability • This is wrapped in a DiscoveryNet workflow to include PubMed query, results visualization and performance calculations • Workflow is deployed as a web application for end users which includes applet to interactively browse results EPSRC E-Science Meeting, NeSC

23. GO Annotator – Version 1Underlying Discovery Net Workflow EPSRC E-Science Meeting, NeSC

24. GO Annotator – Version 1Underlying Discovery Net Workflow Enter query and retrieve abstracts from PubMed. EPSRC E-Science Meeting, NeSC

25. GO Annotator – Version 1Underlying Discovery Net Workflow Use Termino to mark-up abstracts with GO Annotations when match for GO Annotation description is found. EPSRC E-Science Meeting, NeSC

26. GO Annotator – Version 1Underlying Discovery Net Workflow Tabulate GO Annotations by PMID. EPSRC E-Science Meeting, NeSC

27. GO Annotator – Version 1Underlying Discovery Net Workflow Join PMIDs and matching GO Annotations with abstracts and titles. EPSRC E-Science Meeting, NeSC

28. Workflow Deployment EPSRC E-Science Meeting, NeSC

29. GO Annotator – Version 2 • Use Saccharomyces (Yeast) Genome Database as source of papers expertly curated with GO Annotations • Train classifier using these papers • Hierarchical classification • Training data sufficient to classify over 2000 GO Annotations • Classifier is then applied to assign unseen papers with GO Annotations • Main Issues: • Choice of features to be extracted from the training documents • Choice of feature reduction methods to produce accurate classification • Choice of classification algorithm to be used? EPSRC E-Science Meeting, NeSC

30. GO Annotator – Version 2Underlying Discovery Net Workflow EPSRC E-Science Meeting, NeSC

31. GO Annotator – Version 2Underlying DiscoveryNet Workflow Papers expertly curated with GO Annotations from SGD database. EPSRC E-Science Meeting, NeSC

32. GO Annotator – Version 2Underlying Discovery Net Workflow Generate vector of features (frequent phrases) for each paper. This is used to train classifier. EPSRC E-Science Meeting, NeSC

33. GO Annotator – Version 2Underlying Discovery Net Workflow Generate a Naïve Bayesian classification model. EPSRC E-Science Meeting, NeSC

34. GO Annotator – Version 2Underlying Discovery Net Workflow Generate vector of features (frequent phrases) for each paper in test data set. This is used to test the classifier. EPSRC E-Science Meeting, NeSC

35. GO Annotator – Version 2Underlying Discovery Net Workflow Apply classification model to test data to evaluate classification accuracy. EPSRC E-Science Meeting, NeSC

36. Interface + Results Presentation Abstract Titles GO Hierarchy Abstract Bodies Go Labels/ Gene Names EPSRC E-Science Meeting, NeSC

37. Achievements to date • Infrastructure Interoperability • More than just remote web service invocation: interoperable metadata models • Mark 1 System Implemented • Annotation based on terminology lookups • 15% Recall & 5% Precision (Exact matches for 18,000 GO terms) • Measures inadequate due to incompleteness of gold standard • In process of Finalising Training Data Sets and Evaluation Metrics • 4,922 papers referencing 2,455 GO Terms • Mark 2 Systems in Progress • Naïve Bayesian Approach • 41% Recall and 27% Precision • User Interfaces • Mark 3, 4, … Systems and Evaluation EPSRC E-Science Meeting, NeSC

38. Implementation Options • Feature Vector Options • Bag of words • Frequent Phrases • Key Phrases (Gene Names, Protein Names, MeSH terms, etc). • Classifier Options • Bayesian Classifiers • Support Vector Machines • Drag Push (a novel centroid based method) EPSRC E-Science Meeting, NeSC

39. Lessons Learnt and Challenges to Face • Infrastructure • Interoperability Issues • Performance Issues: • Communication vs Persistence of remote server • Off-line vs on-line feature extraction • Text Mining • Usability Issues • Evaluation Issues EPSRC E-Science Meeting, NeSC