1 / 14

Scientific Workflows & GEON

CYBERINFRASTRUCTURE FOR THE GEOSCIENCES. UC DAVIS Department of Computer Science. San Diego Supercomputer Center. Scientific Workflows & GEON. Efrat Jaeger – SDSC Bertram Ludäscher – UC DAVIS Krishna Sinha – Virginia Tech Ashraf Memon – SDSC Ghulam Memon – SDSC

kovit
Download Presentation

Scientific Workflows & GEON

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CYBERINFRASTRUCTURE FOR THE GEOSCIENCES UC DAVIS Department of Computer Science San Diego Supercomputer Center Scientific Workflows & GEON Efrat Jaeger – SDSC Bertram Ludäscher – UC DAVIS Krishna Sinha – Virginia Tech Ashraf Memon – SDSC Ghulam Memon – SDSC Ilkay Altintas – SDSC Kai Lin – SDSC & many others esp. KEPLER community

  2. Scientific Workflows Pre-Cyberinfrastructure • Data Federation & Grid “Plumbing”: • access, move, replicate, query … data (Data-Grid) • authenticate … SRB Sget/Sput … OPeNDAP, … Antelope/ORBs • schedule, launch, monitor jobs (Compute-Grid) • Globus, Condor, Nimrod, APST, … • Data Integration: • Conceptual querying & integration, structure & semantics, e.g. mediation w/ SQL, XQuery + OWL (Semantics-enabled Mediator) • Data Analysis, Mining, Knowledge Discovery: • manual/textbook (e.g. ternary diagrams), Excel, R, simulations, … • Visualization: • 3-D (volume), 4-D (spatio-temporal), n-D (conceptual views) … • one-of-a-kind custom apps., detached (island) solutions • workflows are hard to reproduce, maintain • no/little workflow design, automation, reuse, documentation • need for an integrated scientific workflow environment

  3. Analysis Workflow in KEPLER • Scientific Workflow (SWF) design • SWF automation • Exploration & discovery mode (change parameters, data sets, etc. and rerun) • SWF reuse, documentation, reproducibility

  4. Some KEPLER Components (Actors)

  5. KEPLER Team Work: GEON Dataset Generation & Registration % Makefile $> ant run SQL database access (JDBC) Matt,Chad, Dan et al. (SEEK) Efrat (GEON) Ilkay (SDM) Yang (Ptolemy) Xiaowen (SDM) Edward et al.(Ptolemy)

  6. Ilkay Altintas SDM, Resurgence, NLADR,… Kim Baldridge Resurgence, NMI Chad Berkley SEEK Shawn Bowers SEEK Terence Critchlow SDM Tobin Fricke ROADNet Jeffrey Grethe BIRN Christopher H. Brooks Ptolemy II Zhengang Cheng SDM Dan Higgins SEEK Efrat Jaeger GEON Matt Jones SEEK Werner Krebs, EOL Edward A. Lee Ptolemy II Kai Lin GEON Bertram Ludaescher GEON, SDM, SEEK, BIRN,ROADNet Mark Miller EOL Steve Mock NMI Steve Neuendorffer Ptolemy II Jing Tao SEEK Mladen Vouk SDM Xiaowen Xin SDM Yang Zhao Ptolemy II Bing Zhu SEEK ••• KEPLER: an open source, cross-project collaboration Ptolemy II www.kepler-project.org Your Logos & Names HERE!!!

  7. Demonstration by Efrat Jaeger

  8. Q & A

  9. KEPLER: An Open Collaboration • Initiated by members from NSF/ITR SEEK and DOE SDM/SPA; now several other projects (GEON, Ptolemy II, EOL, Resurgence/NMI, …) • Open Source (BSD-style license) • Intensive Communications: • Web-archived mailing lists • IRC (!) • Meetings, Hackathons • Co-development: • via shared CVS repository • joining as a new co-developer (currently): • get a CVS account (read-only) • local development + contribution via existing KEPLER member • be voted “in” as a member/co-developer

  10. Scientific Workflow (SWF) Design • Support SWF design & reuse, via: • Structural data types • Semantic types • Associations (=constraints) between them • Type checking, inference, propagation Separation of concerns: • structure, semantics, WF orchestration, etc.

  11. Related Publications Scientific Workflows • Scientific Workflow Management and the Kepler System, B. Ludäscher, I. Altintas, C. Berkley, D. Higgins, E. Jaeger-Frank, M. Jones, E. Lee, J. Tao, Y. Zhao, Concurrency and Computation: Practice & Experience, Special Issue on Scientific Workflows, to appear, 2005. • A Framework for the Design and Reuse of Grid Workflows, Ilkay Altintas, Adam Birnbaum, Kim Baldridge, Wibke Sudholt, Mark Miller, Celine Amoreira, Yohann Potier, and Bertram Ludaescher, Intl. Workshop on Scientific Applications on Grid Computing (SAG'04), LNCS 3458, Springer, 2005 • Kepler: An Extensible System for Design and Execution of Scientific Workflows, I. Altintas, C. Berkley, E. Jaeger, M. Jones, B. Ludäscher, S. Mock, 16th International Conference on Scientific and Statistical Database Management (SSDBM'04), 21-23 June 2004, Santorini Island, Greece. • Kepler: Towards a Grid-Enabled System for Scientific Workflows, Ilkay Altintas, Chad Berkley, Efrat Jaeger, Matthew Jones, Bertram Ludäscher, Steve Mock, Workflow in Grid Systems (GGF10), Berlin, March 9th, 2004. • An Ontology-Driven Framework for Data Transformation in Scientific Workflows, S. Bowers and B. Ludäscher, Intl. Workshop on Data Integration in the Life Sciences (DILS'04), March 25-26, 2004 Leipzig, Germany, LNCS 2994. • A Web Service Composition and Deployment Framework for Scientific Workflows, I. Altintas, E. Jaeger, K. Lin, B. Ludaescher, A. Memon, In the 2nd Intl. Conference on Web Services (ICWS), San Diego, California, July 2004.

  12. Data Integration Eco Grid Knowledge Representation Data Federation Process Integration (Scientific Workflows) Source: B. Ludaescher, UC DAVIS ECS-289 Scientific Data Management WQ’05

More Related