Overview of the Science Environment for Ecological Knowledge (SEEK)

1. Overview of theScience Environment for Ecological Knowledge (SEEK) http://seek.ecoinformatics.org http://kepler-project.org Ricardo Scachetti Pereira (with many, many slides from Matt Jones, Bertram Ludäscher, Ilkay Altintas, Chad Berkeley and others) University of Kansas, USA June 30, 2005

2. Outline • Introduction to SEEK • Introduction to Kepler • Kepler capabilities and sample workflows • Current and future developments

3. What is SEEK? Science Environment for Ecological Knowledge Multidisciplinary project to create: Scientific-workflow system (Kepler) • Design, document, reuse, and execute scientific analyses Distributed data network (EcoGrid) • Environmental, ecological, and systematics data Knowledge Representation & Semantic Mediation • Discover, integrate, and compose hard-to-relate data and services via ontologies Taxonomic, Biology, and Education subcomponents Collaborators (the SEEK team) • NCEAS, UNM, SDSC/UCSD, U Kansas, UC Davis • Vermont, Napier, ASU, UNC

4. Scientific Workflows • Model the way scientists work with their data now • Mentally coordinate export and import of data among software systems • Capture data in the field • Digitize it into Excel spreadsheets • Export as CSV files • Import into statistical package • Perform analysis • Export results, tables and graphics • Write and publish article Archive output to EcoGrid with workflow metadata Query EcoGrid to find data

5. Scientific Workflows • Scientific workflows are: • Not linear • Involve multiple data sets • Involve multiple analytical steps

6. Metadata driven data ingestion • Key information needed to read and machine process a data file is in the metadata • File descriptors (CSV, Excel, RDBMS, etc.) • Entity (table) and Attribute (column) descriptions • Name • Type (integer, float, string, etc.) • Codes (missing values, nulls, etc.) • In the future, this will include semantic typing

7. Metadata driven data ingestion • Metadata is revised following any transformation • Versioning of metadata and data is very important • This process results in a lineage of the data file as it has been transformed

8. Data integration • Integration of heterogeneous data requires much more advanced metadata and processing • Attributes must be semantically typed • Collection protocols must be known • Units and measurement scale must be known • Measurement mechanics must be known (i.e. that Density=Count/Area) • This is an advanced research topic within the SEEK project

9. Semantic typing • Label data with semantic types • Label inputs and outputs of analytical components with semantic types • Use SMS to generate transformation steps • Beware analytical constraints • Use SMS to discover relevant components • Ontology = specification of a conceptualization (a knowledge map) Data Ontology Workflow Components

10. SEEK Components Revisited

11. SEEK EcoGrid • Goal: allow diverse environmental data systems to interoperate • Hides complexity of underlying systems using lightweight interfaces • Integrate diverse data networks from ecology, biodiversity, and environmental sciences • Data systems • Any system can implement these interfaces • Prototyping using: • Metacat, SRB, DiGIR, Xanthoria, etc. • Supports multiple metadata standards • EML, Darwin Core as foci • Implemented as OGSA Grid Services • Query() • Get() • Put() • Login() • … • Tiered-implementation critical to adoption

12. Kepler: Scientific Workflows • Implements the workflow system in SEEK • Open, collaborative effort of: • SEEK, SciDAC/SDM, GEON, Ptolemy Project • Ecology, biodiversity, molecular bio, geology, engineering • Based on Ptolemy II system • Kepler aims to extend the Ptolemy system with: • Web and grid service access • Data integration support • Semantic reasoning • Kepler actors are written in Java but can wrap other applications (such as MATLAB, GRASS) • Actors can call arbitrary Web (or Grid) Services • Ptolemy already has a very large inventory of actors

13. Actor Search and Browse • Actors Panel • Large number of actors • Organized hirarchically • Search makes it easy to find right actor • Ontology-based • Plan to support multiple views

14. EcoGrid: EML Data Access

15. EcoGrid: Queries

16. EcoGrid: Queries

17. EML Metadata Display

18. EcoGrid: DarwinCore Access

19. Kepler: database access

20. Kepler: web service example

21. Kepler: grid services access

22. Kepler: ecological modeling

23. New ENM Workflow

24. Data Analysis: Biodiversity Indices

25. ‘R’ in Kepler Source: Dan Higgins, Kepler/SEEK

26. ORB

27. Kepler today • Supports scientific workflows • Ecology, molecular bio, geology, … • Variety of analytical components (including spatial data transformations) • Support for R scripts and Matlab scripts • EcoGrid access to heterogeneous data • EML Data support • Experimental data, survey data, spatial raster and vector data, etc. • DarwinCore Data support • Museum collections • EcoGrid registry to discover data sources • Ontology-based browsing for analytical components • Exploit semantics to improve the user experience • Demonstration workflows • Ecology: Ecological Niche Modeling • Genomics: Promoter Identification Workflow • Geology: Geologic Map Information Integration • Oceanography: Real-time Revelle example of data access

28. Kepler this year • Usability engineering • Full evaluation and user-oriented customization of all UI components • Distributed computing/grid computing • Large jobs, lots of machines • Detached execution • Component repository / downloadable components • “Smart” data and component discovery • Support annotating data sources • Automated data and service integration and transformation using ontologies • Complete EcoGrid access • Full EML support • Support for “large” data and 3rd-party transfer • More data sources and types of data sources (e.g., JDBC, GEON data) • Provenance and metadata propagation

29. Acknowledgements This material is based upon work supported by: The National Science Foundation under Grant Numbers 9980154, 9904777, 0131178, 9905838, 0129792, and 0225676. Collaborators: NCEAS (UC Santa Barbara), University of New Mexico (Long Term Ecological Research Network Office), San Diego Supercomputer Center, University of Kansas (Center for Biodiversity Research), University of Vermont, University of North Carolina, Napier University, Arizona State University, UC Davis The National Center for Ecological Analysis and Synthesis, a Center funded by NSF (Grant Number 0072909), the University of California, and the UC Santa Barbara campus. The Andrew W. Mellon Foundation. Kepler contributors: SEEK, Ptolemy II, SDM/SciDAC, GEON