1 / 19

Workflow Management in GridMiner

Workflow Management in GridMiner. Günter Kickinger , Jürgen Hofer, Peter Brezany, A Min Tjoa Institute for Software Science University of Vienna. The 3rd Cracow Grid Workshop. Outline. Overview The Knowledge Discovery Process GridMiner Architecture Collaboration of Services Workflows

field
Download Presentation

Workflow Management in GridMiner

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Workflow Management in GridMiner Günter Kickinger, Jürgen Hofer, Peter Brezany, A Min Tjoa Institute for Software Science University of Vienna The 3rd Cracow Grid Workshop

  2. Outline • Overview • The Knowledge Discovery Process • GridMiner Architecture • Collaboration of Services • Workflows • Dynamic Service Composition

  3. Overview • GridMiner • Service-oriented grid-aware data mining system • cope with • very large data sets • high dimensional data sets • geographically distributed data sets • different types of data sets • implemented on top of Globus Toolkit 3.0

  4. The Knowledge Discovery Process Knowledge Evaluation and Presentation Data Mining Selection and Transformation DWH Cleaning and Integration

  5. GridMiner Architecture GridMiner Workflow GM DSCE Dynamic Service Control GridMiner Core GMDIS Integration GMPPS Pre Processing GMDMS Data Mining GMOMS OLAM GMPRS Presentation GridMiner Base GMMS Mediation GMIS Information GMRB Resource Broker GMCMS OLAP / Cubes Grid Core Grid Core Services Security File and Database Access Service Replica Management Fabric Grid Resources Data Source

  6. Collaboration of GM-Services Simple Scenario: GMDIS Integration GMPPS Pre Processing GMDMS Data Mining GMPRS Presentation Intermediate Result 1 Intermediate Result 2 (e.g. “flat table”) Intermediate Result 3 (e.g. PMML) Final Result Data Sources

  7. Collaboration (2) Complex Scenarios: GMPPS GMDIS GMPPS GMDMS GMPRS GMPPS GMDMS GMPRS GMPPS GMPPS GMDMS GMPRS GMDMS GMPRS GMPPS GMPPS GMDIS GMCMS GMOMS GMPRS GMPPS

  8. Workflow Management • Motivation • high complex and dynamic process • order of service execution • selection of services • sequential and parallel execution • long running process • termination of client would terminate the workflow => Additional workflow layer needed !

  9. Workflow Models Static workflows Dynamic workflows

  10. Dynamic Workflows • Dynamic Service Control Language (DSCL) • based on XML • easy to use • Dynamic Service Control Engine (DSCE) • processes workflow according to DSCL DSCL DSCE Service A Service B Service D Service C

  11. Dynamic Service Control Language • Features • Control flow • parallel execution of activities • sequential execution of activities • Activities • creation of new Grid Service Instances • invoking operations on Grid Service Instances • Querying SDEs of Grid Service Instances • assigning and copying variables

  12. DSCL - Example dscl variables composition qreate Service invoke query SDE qreate Service invoke query SDE qreate Service invoke

  13. Dynamic Service Control Engine • Features • processing of a DSCL document • parallelism • hiding complexity • delivery of intermediate results • status of executed service • Caching mechanism included

  14. Dynamic Service Control Engine • Implementation • transient stateful OGSA Grid Service • Operations • updateDSCL() • start() • stop() • resume() • SDE • activities • results, failures, states for each activity

  15. DSCE - Architecture Service Interface Factory Interface DSC Engine DGS Invocation Dynamic Invoker Globus 3.0 Axis 1.1

  16. Current and Future Work • This is work in progress • Additional Features • Notification Model • Exception Handling

  17. Related Work • BPEL4WS: Business Process Execution Language (BEA, IBM, Microsoft, SAP, Siebel) • GSFL: Grid Services Flow Language (Krishnan, Wagstrom, Laszewski) • Data mining. Concepts and Techniques (Han) • Anatomy of the Grid (Foster, Kesselman, Tuecke) • Physiology of the Grid (Foster, Kesselman, Nick, Tuecke) • Open grid service infrastructure (Tuecke, Czaijkowski, Foster)

  18. Conclusions • Dynamic Service Control is an approach allowing the service consumer specify a workflow • General approach – not only restricted to GridMiner

More Related