1 / 32

Programming the Grid with Components

Explore the concept of dynamic component composition in grid programming through XCAT and CCA specifications, emphasizing efficient collaboration and service communication. Learn about component architecture, software components, industry standards, academic research, and the Common Component Architecture initiative.

gussiel
Download Presentation

Programming the Grid with Components

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Programming the Grid with Components Madhu Govindaraju Aleksander Slominski Dennis Gannon Sriram Krishnan Extreme! Computing Lab, Dept. of Computer Science, Indiana University

  2. Outline • Software Components • XCAT • Component and Services model • Web Services • Programming model • OGSI Extreme! Computing Lab, Dept. of Computer Science, Indiana University

  3. Software Components • Analogy with hardware • Standard design paradigm for app development • Goal: to build applications by composition of well tested and well behaved subsystems (components). • simplify application design process • promote code reusability • plug and play Extreme! Computing Lab, Dept. of Computer Science, Indiana University

  4. Component Architecture • A Component Architecture consists of two parts: • Components • software objects that implement a set of required behaviors • Frameworks • A runtime environment • A set of services used by components Extreme! Computing Lab, Dept. of Computer Science, Indiana University

  5. Industry Standards • COM/DCOM • interoperability for Microsoft applications • Java Beans, Enterprise Java Beans (EJB) • Java based desktop and enterprise applications • OMG CORBA Component Model (CCM) • language neutral superset of EJB Extreme! Computing Lab, Dept. of Computer Science, Indiana University

  6. Academic Research • SciRun from Utah • scalable parallel applications and viz • Webflow from Syracuse • graphical composition palette • CCAT and XCAT from Indiana University • framework for Grid based applications Extreme! Computing Lab, Dept. of Computer Science, Indiana University

  7. CCA: Common Component Architecture • CCA: Initiative to define minimal specification for scientific components • Department of Energy (National Labs) • Few universities • Aim: build components for high performance computing • Draws ideas from CCM and other models as a baseline • Targeting • Parallel • Distributed Extreme! Computing Lab, Dept. of Computer Science, Indiana University

  8. The Common Component Architecture • Key Idea: dynamic composition by linking “ports” • Provides ports: interface of “services” provided • Uses ports: a call-site for a service to be provided by another component. Uses port- Provides Port Extreme! Computing Lab, Dept. of Computer Science, Indiana University

  9. What is new about CCA? • Minimal specification • Envision connections as dynamic • Can add, remove and connect ports at run-time • Tailored to build Problem Solving Environments • End user manipulates connections • Not specific to any underlying distributed object model • Notion of collective ports • logically one connection • implementation: multiple network connections Extreme! Computing Lab, Dept. of Computer Science, Indiana University

  10. Components on the Grid: Issues • Discovering static information about components • Launching components • Discovering references for running instances • Communication protocols • Messages and Events • Authentication and Authorization • Encapsulating legacy applications • Efficient scheduling • Run-time environment Extreme! Computing Lab, Dept. of Computer Science, Indiana University

  11. Grid Programming Model: Needs • Need a set of APIs, protocols, libraries and tools that allow access to Grid resources • Examples: Globus, Condor • provide infrastructure • do not provide programming model • What is missing? • How can we make it easier to “program” the Grid? Extreme! Computing Lab, Dept. of Computer Science, Indiana University

  12. XCAT • Implementation of the CCA specification • Designed for distributed applications • Allows creation of Grid application from components • Wraps Grid Services as components • Test-bed for CCA • Is there an alternative to OMG IDL? Extreme! Computing Lab, Dept. of Computer Science, Indiana University

  13. Component Communication • How do components communicate? • Use Remote Procedure Call (RPC) Mechanism • XCAT uses SOAP 1.1 • XCAT ports can serve as web services • Events/Messages • Objects encoded as XML documents Extreme! Computing Lab, Dept. of Computer Science, Indiana University

  14. XCAT Services Architecture • Default services for all components • XCAT services • Directory • locate components based on port types and other attributes • Registry • locate running instances of components • Creation • create running instance of a component • Connection • connect ports of two running instances • Events • publish/subscribe framework for messages Extreme! Computing Lab, Dept. of Computer Science, Indiana University

  15. IDS: Information Directory Service • Store Component descriptions in XML • Used to search databases and remote repositories MySQL Database Request component info Information Directory Service Component RDBS File system Return XML specs. LDAP Extreme! Computing Lab, Dept. of Computer Science, Indiana University

  16. Creation Service • Creates a running instance of another component • Encapsulates authentication issues • Supports GRAM, SSH and Local-Exec Globus resource Y Launch an instance of component X on resource Y Creation Service Component X Returns: remote reference to new component instance Extreme! Computing Lab, Dept. of Computer Science, Indiana University

  17. Registry Service • Stores instance information (runtime configurations) • Used to advertise and search other instances • Tradeoff: distributed vs. centralized Find a running instance of component X Registry Service Component Returns: remote reference to component instance Extreme! Computing Lab, Dept. of Computer Science, Indiana University

  18. Connection Service • A component that can be used to connect a “uses” port of one component to the “provides” port of another • Can export ports of other components as one’s own X Connection Service Component A Connect port A of component X to port B of component Y B Y Extreme! Computing Lab, Dept. of Computer Science, Indiana University

  19. x x Event Service • Channel for publish-subscribe event model • Creation and connection events • Application users can use it for asynchronous information transfer C Subscribe to listen for events of type X Event Channel Service Component x Notification of publication of events of type X A x B Extreme! Computing Lab, Dept. of Computer Science, Indiana University

  20. Features of Events • Event/Notification Services are an essential part of all distributed systems • Application Events • “I am done.” “I just wrote to a file,” “here is a result.” • Event Publishers & Listeners • Some Listeners subscribe, others poll • Must have persistent event channels • record events for later analysis • application history logs • allow for retrieval in chunks Extreme! Computing Lab, Dept. of Computer Science, Indiana University

  21. XCAT: Other Features • Security: • Every remote method call is intercepted • port, method and parameter level • authentication based on SSL • authorization based on simple ACL • Component Handle: • in XML and can be converted to WSDL • GSH /GSR • Exceptions • Supports throw and catch across components Extreme! Computing Lab, Dept. of Computer Science, Indiana University

  22. XCAT Processes Framework Impl. Creation Service Creation Service Framework Impl. Scripts, Front-end, Tools Connection Service Connection Service Scripts, Front-end Tools Application Code Other Services Other Services Application Code Port Wrapper Code Extreme! Computing Lab, Dept. of Computer Science, Indiana University

  23. XCAT-Web Services Stack Extreme! Computing Lab, Dept. of Computer Science, Indiana University

  24. XCAT Programming: Examples • Builder: A tool used to select and link components together • Builder Options • Swing based GUI • Jython scripting • Portal • XCAT • Jython scripting Extreme! Computing Lab, Dept. of Computer Science, Indiana University

  25. CCAT Gui image Extreme! Computing Lab, Dept. of Computer Science, Indiana University

  26. Scripting XCAT Applications import xcat stringDump = xcat.createComponent(‘StringDump’) printer = xcat.createComponent(‘Printer’) xcat.setCreationMechanism(stringDump, ‘gram’) xcat.setCreationMechanism(printer, ‘ssh’) xcat.setMachineName(stringDump, “exodus”) xcat.setMachineName(printer, “rainier”) Extreme! Computing Lab, Dept. of Computer Science, Indiana University

  27. Scripting XCAT Applications: contd printer.put(“exec-fqn”, “samples.printer.Printer”) stringDump.put(“exec-fqn”, “samples.stringDump.StringDump) xcat.createInstance(printer) xcat.createInstance(stringDump) xcat.connectPorts(stringDump, ‘outputString’, printer, ‘inputString’) Extreme! Computing Lab, Dept. of Computer Science, Indiana University

  28. Encapsulating Legacy Apps • Common Case • Legacy App that reads files and writes files • Use a “scripted component” • Component runs a python script • The App Script • Stages files • Launches and monitors application • Writes output files • publishes event streams Control Event Stream XCAT comp App Script application output files input files Extreme! Computing Lab, Dept. of Computer Science, Indiana University

  29. Scripting Grid Computations • Grid Scripts are executable documents that • Orchestrate Computations • Select resources • Supply application parameters and launch remote jobs • Subscribe to event streams • Create components • Connect components Extreme! Computing Lab, Dept. of Computer Science, Indiana University

  30. Composition Using Components • Two ways of composing applications • Composition in space: • one component/service directly invokes the services of another • Composition in time: • A workflow engine schedules tasks that involve accessing remote services and responding to events Extreme! Computing Lab, Dept. of Computer Science, Indiana University

  31. GS GS Component A Component B XCAT, OGSI and Web Services Grid Service Client Standard Grid Service Extreme! Computing Lab, Dept. of Computer Science, Indiana University

  32. XCAT and OGSI compliance • Add a GridService port to each component • Merge component handle with GSH/GSR • OGSI messaging is non-reliable push model • XCAT has a reliable, persistent model • OGSI has factory portType for instantiating new services • XCAT has an extended distributed factory model Extreme! Computing Lab, Dept. of Computer Science, Indiana University

More Related