1 / 35

Triana Generations

Triana Generations. Ian Taylor School of Computer Science, Cardiff & Center for Computation and Technology, CCT, LSU, USA 5th December, 2006. Broad Triana Generations (Reverse Order). Distributed Versions (1998 onwards) Moved to Java (1994) But !

carson
Download Presentation

Triana Generations

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Triana Generations Ian Taylor School of Computer Science, Cardiff & Center for Computation and Technology, CCT, LSU, USA 5th December, 2006

  2. Broad Triana Generations(Reverse Order) • Distributed Versions (1998 onwards) • Moved to Java (1994) • But ! • First version designed and written 16 years ago • In C++ • The first version was ironically, called Grid …

  3. The alternate history of the Grid … 70’s Bernard’s Schutz’s General Relativity Group in Cardiff Gravitational wave data analysis software tools 1990 - Grid software began - Justin Shuttleworth 1995 - name change of Grid software to Triana (August) 1995 - I-WAY Distributed Computing Experiment (November) 1997 - Grid term coined for new distributed computing paradigm - analogy with Electric power Grid …

  4. The Original Retro-looking 1991 Grid .. • C++ Linux Application/Interviews GUI • This is Grid 0.1 alpha • Stopped here - we move to Java

  5. Grid became Triana

  6. New GUI: Building Complex Data-flows

  7. Triana Focus • Two core underlying focuses: • Interactive graphical programming of the distributed tasks - complex editing • Intuitive drag/drop flexible editing - copy/paste services, wizards for creating tools/toolboxes, user interfaces, adding nodes and multi-level grouping. • Has been used as a “graphical editor” for other languages, e.g. DAG, VDLx (DAX in progress). • Heterogeneous workflows - Bridge the gap between different distributed environments • Use cross-environment interfaces • led to integration with GAT (pre SAGA), GAP

  8. Types of Uses • For fine-grained operations, specifying dataflow for local operations • Or course-grained composition of a distributed workflow • Or Both - can connect heterogeneous tools (e.g. Web services, Java units, Jxta services) on one workflow Has been used as a dataflow system, a distributed-workflow environment, workflow-management system, an automated scripting tool, workflow editor.

  9. Current Capabilities • Local Java Units • 600 units in signal, image, audio, text processing, complete math/stats toolboxes etc • Common units - flexible importers/exporters, graphing, duplicators • Data types - strong data types for a number of domains - includes run-time checking • Distributed Integration • GAT - Java GAT implementation - graphical representation of GAT primitives - supports GRAM, GridFTP, etc • GAP - SOA publish, find, bind triad of operations • Bindings: Jxta, P2PS, Web Services, WS-RF • Group unit deployment • Legacy Applications • Can incorporate legacy applications easy (using local GAT adaptor) - standard file in/out interface

  10. Some Examples of Domains • Gravitational wave data analysis (GridOneD) • Radio astronomy (with Manchester) • Astrophysical simulations (Cactus) • Data mining (DIPSO, Data mining Grid) • Biodiversity Problems Bdworld • Galaxy visualization • Audio processing and distributed music information retrieval (MIR) • Distributed peer-to-peer simulations (NRL and AgentJ) • Grid-enabled medical simulations GEMSS) • Environmental science (INFERNO) • E-Health (Contact-Net)

  11. Upperware Middleware GAP GAT & GAP GAP Distributed Work-flow Distributing Triana Units or Groups (Java) Triana Service & Engine Integrating Legacy applications into Workflow Remote Legacy Applications Workflow Commands Workflow, e.g. BPEL4WS Integrating Web Services or P2P Services Distributed services Triana Engine

  12. Triana, the GAT and the GAP Service Based Computing: Grid Computing: Deployment, discovery and communication with distributed services e.g. P2P and (GSI) Web services Job Submission, File services A Graphical Grid Computing Environment or Portal GAP Interface GAT Interface P2PS JXTA Web Services Condor Unicore GridFTP GRMS WSRF Globus RLS PBS .NET GridLab P2PS Discovery UDDI JXTA Discovery SOAP P2PS Pipes SSH SGE LDR Other.. JXTA Pipes

  13. Examples • Serial Tasks • Simple audio example - grouping • Distributed Task • Graphical GAT • Final Gridlab Demo - Visualising Cactus Simualtions • Web Services • Choreographing Web Services • WS-RF example • Graphically representing state in Web services

  14. Audio Processing (Groups)

  15. Group Units

  16. GAT Interface • Main deliverable of Gridlab • Application-level interface • With a set of adapters • That adapt the interface to an underlying capability • Versions in C++ and Java • Pre-cursor to SAGA - Simple API for Grid Applications

  17. Copy File(Machine A, Machine B) Grid FTP Adapter Jxta File Adapter Jxta Pipe Grid FTP Connection Grid Environment P2P Environment GAT Adapters: Example GAT API Resource Management Streaming/ Comms File Management Job Management Collection Management Monitoring GAT Engine

  18. Monitoring Cactus Simulations • GAT Example • The final Gridlab Demo - showing integration of every workpackage • We build a tool to monitor an application • detects files remotely and passes back to a client using Web Services • Scientist can view output of a simulation whilst it is running

  19. WSPeer User Interface Web Service Visualising Cactus Simulations within Triana Workflows Resource Broker (e.g. GRMS) Grid Side submit run User Side Triana GUI cactus cactus http://www.xyx.org:4802/axis/cactus monitor phi_xy.jpg phi_yz.jpg phi_xy.jpg phi_yz.jpg rundir http://www.xyz.org/axis/cactus invoke using SOAP

  20. GridLab GAT File CPI Resource Broker CPI HTTP Adaptor GridFTP Adaptor Data Movement Local Adaptor GRMS Adaptor GRAM Adaptor Local Adaptor File Input (HTTP) Triana WaveToy.par www. gridlab. org

  21. File Staging (GridFTP) Triana GridLab GAT File CPI Resource Broker CPI HTTP Adaptor GridFTP Adaptor Data Movement Local Adaptor GRMS Adaptor GRAM Adaptor Local Adaptor WaveToy.par www. gridlab. org Testbed

  22. Job Submission 1 Triana GridLab GAT File CPI Resource Broker CPI HTTP Adaptor GridFTP Adaptor Data Movement Local Adaptor GRMS Adaptor GRAM Adaptor Local Adaptor GAS GRMS Cactus www. gridlab. org Testbed Data Mvment Testbed WaveToy.par Delphoi Mercury iGrid

  23. Job Submission 2 Triana GridLab GAT File CPI Resource Broker CPI Instant Mess HTTP Adaptor GridFTP Adaptor Data Movement Local Adaptor GRMS Adaptor GRAM Adaptor Local Adaptor Portal GAS GRMS Cactus www. gridlab. org Testbed Data Mvment Testbed Delphoi Mercury iGrid

  24. GAP Interface P2PS JXTA Web Services P2PS Discovery UDDI JXTA Discovery SOAP P2PS Pipes JXTA Pipes GAP Interface • Motivation by GAT • A Simple Service based API, for • Service Deployment, • Service Discovery • Pipe Based Communication • Static application interface with multiple middleware bindings • P2PS (name…?) • JXTA • Web services

  25. Service Discovery Dynamic? Decentralized? Communication Message Format SOAP? Transport Protocol TCP? UDP? Service Oriented Comms en_fr hello network bonjour BabelFish GAP babelfish. altavista. com

  26. Triana Scripting for Dynamic on-the-fly Distribution • This script searches for new services and adds them to the distributed network on-the-fly • In this case, data is streamed between the services

  27. Deploying and Connecting To Remote Services • Running services are automatically discovered via the GAP Interface, and appear in the tool tree • User can drag remote services onto the workspace and connect cables to them like standard tools (except the cables represent actual JXTA/P2PS pipes) Remote Services

  28. Web Service Discovery • Triana allows users to query UDDI repositories • Alternatively, users can import services directly from WSDL

  29. Web Service Discovery • Discovered/Imported Web Services are converted into Triana tools (service name = tool name) (input message parts = in nodes) (output message parts = out nodes) etc… • Web Service tools are displayed in the user’s Tool Tree (alongside local tools)

  30. Complex Data Types • Users can build their own interface for creating/mediating between complex types • Alternatively, Triana can dynamically generate an interface from the WSDL2Java generated bean class

  31. Converting the Bible into French • Simple but powerful example: • read_bible - extracts verses from the bible • BabelFish - translates between English and French (and other languages) • Result = The Bible translated into French ! Red Boxes – Web Services

  32. WS-RF • Double click WS-RF units to select which resource you would like to use for that operation

  33. Music Information Retrieval • P2P Workflow Example • Work in progress • P2P framework, allowing different roles designed, implementation by early 2007 • Digital Audio Retrieval using Triana (DART) • Triana is the workflow manager • Developer creates a workflow to analyse audio (MP3 files) • The Workflow is uploaded to all particpants in the network • Potentially thousands of MP3 files on users’ machines • Files analysed locally on clients machines • Metadata returned e.g. collaborative filtering, audio analysis • Application: music recommendation system - i.e. if you like “Band A”, then try “Band B”

  34. User DART Manager Song Suggestions Consumer Worker Audio Analyser/Provider New Packages Result Advertisements Package Repositories P2PS Network Super Peers Workers Workers Super Peers Package Repositories

  35. Thanks ! • Links: • http://www.trianacode.org/ • http://www.gridoned.org • http://www.gridlab.org/ • http://www.wspeer.org/ New Workflow Book (in print, due around Xmas): Workflows for eScience: I. Taylor, E. Deelman, D. Gannon and M. Shields Type “Workflows for eScience” into Google - 1st hit currently

More Related