1 / 38

Database Issues in Nutritional Genomics

Tony Travis & Peter Gray Rowett Research Institute & University of Aberdeen Jan 2005. Database Issues in Nutritional Genomics. Un Oslo. Rowett. Un. Ulster. Un Newcastle. Un Lund. Trinity. DiFE. IFR. Un Cork. EBI. Rivm. Rikilt. TNO. Un Reading. Un Wageningen. Un Maastricht.

merry
Download Presentation

Database Issues in Nutritional Genomics

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Tony Travis & Peter Gray Rowett Research Institute & University of Aberdeen Jan 2005 Database Issues in Nutritional Genomics

  2. Un Oslo Rowett Un. Ulster Un Newcastle Un Lund Trinity DiFE IFR Un Cork EBI Rivm Rikilt TNO Un Reading Un Wageningen Un Maastricht Un Krakow Nu GO Un Munich Un Florence Inserm Marseille Un Balearic Illes

  3. Utopian view • Share data freely • Everyone benefits • Ideas develop • Science prospers

  4. Big pharma disagree! • Sell data commercially • Big pharma benefits • Ideas are exploited • Science is a business

  5. Scientists are confused… • Intellectual freedom? • Curiosity driven science • Poor funding • Intellectual property? • Commercially driven science • Good funding

  6. Autonomy Scientist or institution control who their data is disclosed to Control data sharing by collaborators who share their IP Needs federated solution Security Prevent unauthorised access to data Prevent unauthorised use of data Maintain integrity and provenance of data Preserving intellectual property

  7. Typical NutriGenomics Use Case • Example of pragmatic solution • DNA microarray work at RRI • Autonomy • Data held locally on PC spreadsheets • Completely under control of investigator • Collaborators • Each create spreadsheet of local results • All collaborators exchange spreadsheets

  8. Spreadsheet microarray data

  9. Distribution of one spreadsheet A B D C

  10. Exchange of all spreadsheets A B D C

  11. Advantages Simple peer-to-peerdata transfer via email Each collaborator has entire database locally Local analysis tools are readily available Complete control of IP within collaboration Disadvantages N(N-1) solution Does not scale well Each collaborator must merge data into local database replica No control over data integrity or provenance Manual replication of database

  12. Spreadsheet Replicated Data Model • Distributed • Data originates at each collaborator’s site • Replicated • Copy of the entire database at each site • Manually updated • Data and corrections are pushed from each collaborator to all others via email of Excel spreadsheets containing expression data which is merged into a single spreadsheet

  13. Local analysis tools: maxd • Microarray Bioinformatics Group University of Manchester (UK) • Java-based • maxdView • Visualise and analyse gene expression data. • maxdLoad2 • Store and curate gene expression data to MIAME standards • Export in MAGE/ML format for submission to ArrayExpress.

  14. Import spreadsheet data into maxd

  15. Analyse expression profiles • 10,000 genes • Four experiments byone collaborator • Normalised • Clustered • Comparison of gene expression profiles between experiments

  16. Upgrade spreadsheet solution • MaxdLoad2 • Replace spreadsheets • Use MIAME standard • JDBC compliant interface • SQL92 (MySQL, Postgres)

  17. Candidate Mediator middleware • Maxd • Designed for use with single database • P/FDM • Integration of heterogeneous data sources • Federated union/join of relations • Biomart • MartShell scripting language • Federate database instances

  18. Example federated DB

  19. MartShell • Command line (text mode) user Interface to BioMart that can be used by programs • Mart Query Language (MQL) • Queries can be executed in ‘batch’ mode using stored procedures in MQL scripts

  20. BioArray Software Environment • BASE is a comprehensive database server to manage massive amounts of data generated by microarray analysis • Lund University +Oklahoma University • Data can be analysed using a web-based GUI to server-side PHP scripts or data can be extracted from the BASE database by applications such as Genespring

  21. Querying a Federated DB There are two kinds of distributed query that you can send out to the federation: • Federated Join - like adding extra columns with cross-referenced information on the same object or related objects. • Federated Union – like adding extra rows with the same column headings – the same kinds of experiments but done at different sites.

  22. Comparing expression profiles(e.g.looking for co-regulation)

  23. Conditions for making a Federated DB work Needs Common Ontologyfor data of same type. BEWARE measurements made in different units,or using a very different exptl. procedure,or qualitative measurements such as "large".."medium"

  24. Conditions for making a Federated DB work Need Common Unique Identifiers :if no property allows you to tell that one entity instance is the same as another then integration is UNSAFE! (Note - it might be OK for say 95 percent of identifiers...)

  25. Conditions for making a Federated DB work Mechanisation of Value mapping : • if data values can only be compared or made compatible with others using the judgement of an experienced scientist, then one must use a Warehouse (as in early PDB), otherwise • if you can mechanise it using rules or equations then it can be done by a view, • or by a mediator accessing the Federation

  26. Conditions for making a Federated DB work Need Standard Interchange Formats : • Formats such as MMCIF helped reduce human intervention in PDB. The widely used MIAME format may do the same for MicroArray Data. • However such data is much harder to integrate as it may be measured under different conditions with different technology.

  27. Difficulties of Federated Approach • Reliability - Sites must be availablecontinuously, and not crash too often; • Support costs - must be proof against Virus attacks, etc., and have people able to bring them back up again promptly

  28. Difficulties of Federated Approach • Compatibility - must provide a common interface - may be able to share development of some downloadable server software (like Java WebStart), responding to SOAP protocol messages and commands, config-urable through web forms that keeps logs of errors.

  29. Difficulties of Federated Approach • Performance: Warehouses will provide better performance for data mining programs and others programs with a high hit rate. • Federated systems compete well on more focused queries which allow the use of indexes in remote systems.

  30. Having it Both Ways: • A Federated Solution can include some sites that are adopting Warehouse technology to collect and vet large volumes of data of a particular kind. • The NUGO data model and ontologies are bound to change a lot in ways we cannot forsee. Thus it makes sense to be flexible to start, allowing site autonomy, and to delay committing to large warehouses until we understand more about the data model and IPR issues.

  31. Discovering the Model Birney & Clamp (2004) say – "the true biological interpretation of data stored in a database will change over time, and discovering new relationships between aspects of the data is an important part of the motivation for storing it..”

  32. Conclusion (1) - Spreadsheets • Spreadsheets are easy and popular • Integrating Spreadsheets manually is time wasting and can easily lead to errors and wrong conclusions • Scientists need the discipline of a shared Data Model and the automation of data transfer and conversion, usually provided by a Mediator

  33. Conclusion (2) – Shared Data Model • Agreement on a shared Ontology is mainly a problem of agreeing Standards for names, units, and specialised types. • Agreeing a shared Data Model is more subtle. It may need experimentation in advance of a standard. • The Data Model, based on Entity-Relationship Model with SubTypes, must be able to evolve - not fixed in stone, coping with the unforseen.

  34. Conclusion (2) – Shared Data Model • The Data Model must be at Conceptual Level - independent of Storage Technique - arrays, ASN-1, XML, tables etc... Otherwise agreeing a Shared Model becomes too hard! • The Data Model must provide External Views both to restrict access and to provide a consistent API to External Applications; these may be Spreadsheets or Statistical Packages or MaxD or Genespring etc...

  35. Conclusion (3) – Federating Microarray Data • Usually, a federation is based on a federated Join, through common identifiers, because irrelevant joins can be left out, to speed up the query. • Federated Joins suit integrating other types of data with Microarray data, e.g. physiological, epidemiological data • This is easily done, on the fly; it allows us to evolve the data model and experiment with it without making changes to a centralised warehouse. Once the data model is more stable, parts of it can be stored in warehouse.

  36. Conclusion (3) – Federating Microarray Data • Queries that want to compare Gene Expression Profiles across many Experiments need a federated Union of data from different experimenters. • Comparing one profile against those from many experimental sites could be done in parallel. Trusted methods could work with an encrypted profile to keep it confidential.

  37. Conclusion (4) – IPR and Federation • Scientists want to retain their autonomy and right to recognised authorshipof the data, otherwise they may not share it! • If Database Right (EU proposal) becomes established, scientists may wish to keep data in their own DB in order to take advantage of it. Thus we may need to make more use of federated techniques to bring such data together. • Revenue-Raising Potential may become important (iTunes for example).

More Related