1 / 23

Grid Architecture and Some Core Services

Grid Architecture and Some Core Services. US-Australia Workshop on High-Performance Grids and Applications Sydney June 8 2004 Geoffrey Fox Community Grids Lab Indiana University gcf@indiana.edu. WSGA Web Service Grid Architecture.

richardstevens
Download Presentation

Grid Architecture and Some Core Services

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Grid Architecture andSome Core Services US-Australia Workshop on High-Performance Grids and Applications Sydney June 8 2004 Geoffrey Fox Community Grids Lab Indiana University gcf@indiana.edu

  2. WSGA Web Service Grid Architecture • With UK e-Science program, developing list of best practices in Grids based on a pure Web Service infrastructure • Use WS-I Web Service Interoperability Profiles • WSDL, SOAP, UDDI, WS-Security (No Risk Grid Profile) • Some risk for those where IBM and Microsoft agree but not a broader consensus • WS-Addressing, WS-MessageDelivery • WS-ReliableMessaging and WS-Reliability • Workflow: BPEL and WSCL,WSCI (No streams in BPEL but expected to win) • Higher risk where IBM and Microsoft do not agree • WSRF (No Microsoft equivalent; unclear relation to metadata services) • WS-Notification, WS-Eventing (Can be federated to each other and JMS) • Use Service not Distributed Object Paradigm • Use portlet-based portals with user interface fragments for each service • Build Grids of Grids of Simple Services • Identify and build key services (in UK OMII Open Middleware Infrastructure Institute) Can federate different Standards here

  3. Typical Science GridService such as Research Database or simulation Science Grids Campus orEnterprise Administrative Grid Transformed by Grid Filterto form suitable for education Education Grid Publisher Grid Learning Management or LMS Grid Student/Parent … P2P (e.g. AOL) Grid Digital Library Grid Informal Education (Museum) Grid Inservice Teachers Preservice Teachers School of Education Teacher Educator Grids Community Grids Education as a Grid of Grids

  4. What is a Simple Service? • Take any system – it has multiple functionalities • We can implement each functionality as an independent distributed service • Or we can bundle multiple functionalities in a single service • Whether functionality is an independent service or one of many method calls into a “glob of software”, we can always make them as Web services by converting interface to WSDL • Simple services are gotten by taking functionalities and making as small as possible subject to “rule of millisecond” • Distributed services incur messaging overhead of one (local) to 100’s (far apart) of milliseconds to use message rather than method call • Use scripting or compiled integration of functionalities ONLY when require <1 millisecond interaction latency • Apache web site has many projects that are multiple functionalities presented as (Java) globs and NOT (Java) Simple Services • Makes it hard to integrate sharing common security, user profile, file access .. services

  5. NaradaBrokering Audio/Video Conferencing Client Computer Modem Server Peers NaradaBrokering Broker Network Minicomputer Firewall Laptop computer Workstation Peers Audio/Video Conferencing Client PDA Web Service B Queues Stream P2P Messaging Server-enhanced Messaging NB supports messages and streams 1-100 ms. v 1-10 µs NB role for Grid is Similar to MPI role for MPP

  6. Current NaradaBrokering Features

  7. NaradaBrokering Service Integration S1 P2 S2 P1 S2 S1 S? Any Transport Service P? Proxy NB Transport Standard SOAP Transport S1 S2 Proxy Messaging Handler Messaging Notification Internal to Service: SOAP Handlers/Extensions/Plug-ins Java (JAX-RPC) .NET Indigo and special cases: PDA's  gSOAP, Axis C++

  8. Streams and Workflow • NaradaBrokering can manage streams from • Audio/Video conferences • Sensors • Inter-service communication in workflow • http://www.hpsearch.org/demo/ describes scripting managementinterface to NaradaBrokering Grids involve streams as well as compute and data nodes Workflow and dataflow like BPEL imply streams

  9. Future/Ongoing NB Enhancements • Server/Broker-free P2P version to support “immediate deployment” of NB-based Community Grids • Direct integration of P2P and Grids • Rich handler and proxy support for WS-ReliableMessaging and WS-Reliability with federation between standards • Federated support of WS-Eventing, WS-Notification and JMS • Other Mediation/Federation to heterogeneous Grids • Replicated Subscriber (Fault-tolerance/Performance) Services • NaradaBrokering will choose between several subscribing replicated services • VPG Virtual Private Grid: Build standard VPN transport (PPTP, L2TP, IPSec) into NB • Could offer new Grid Security options and/or more choice on firewall/NAT tunneling

  10. Web Services and M-MVC • Web Services are naturally M-MVC – Message based Model View Controller with • Model is Web Service • Controller is Portal and Messages (NaradaBrokering) • View is rendering • OGCE builds M-MVC As Controller

  11. NSF NMI Project for Making Portal Components Robust • University of Chicago • Gregor von Laszewski • Indiana University • Marlon Pierce, Dennis Gannon, Geoffrey Fox, and Beth Plale • University of Michigan • Charles Severance, Joseph Hardin • NCSA/UIUC • Jay Alameda, Joe Futrelle • Texas Advanced Computing Center • Mary Thomas Builds on NCSA Alliance Portal Project

  12. Individual portlet for the Proxy Manager Use tabs or screen real estate to navigate through interfaces to different services

  13. LEAD Portal

  14. Aggregation Portal Architecture Clients (Pure HTML, Java Applet ..) Aggregation and Rendering Portlet Class:WebForm Gateway (IU) Web/Gridservice Computing • Supply component model for user interfaces to match component model for middeware Remoteor ProxyPortlets Portlet Class:IFramePortlet Web/Gridservice Data Stores Portlet Class:JspPortlet GridPort etc. Web/Gridservice Instruments Portlet Class:VelocityPortlet (Java) COG Kit Hierarchical arrangement Jetspeed Internal Services LocalPortlets Clients Portal Portlets Libraries Services Resources (Jetspeed)

  15. OGCE Grid Portal Components • Provides Portlets for • Management of user proxy certificates • Remote file Management via Grid FTP • News/Message systems • for collaborations • Grid Event/Logging service • Access to OGSA/WSGA Grid services • Access to directory services • Specialized Application Factory access • Distributed applications • Workflow • Access to Metadata Index tools • User searchable index • Collaboration Supports distributed modular construction of Services – each with their own Interface

  16. Portlet Component and Container Technologies • Jakarta Jetspeed • Open source Java portlet project • Jetspeed is both a framework and reference implementation • Defines portlets, portal service APIs (login, authorization, customization, etc.) • CHEF from University of Michigan • Uses Jetspeed as a framework • Reimplements many of the core classes • Basis for UM CourseTools • NEESGrid portal • CMCS Portal

  17. Access Grid and Related Portlets

  18. Desktop and Web Services with MMVC • OGCE uses message-based MVC as is obvious for Web Services • Most desktop applications are in fact roughly MVC with controller formed by “system interrupts” with View and Model communicating by “post an event” and define a “listener” programming mode • We propose to integrate desktop and Web Service approach by systematic use of MMVC and NaradaBrokering • Allows easier porting to diverse clients and automatic collaboration • Attractive for next generation of Linux desktop clients • We have demonstrated for SVG Browser (Scalable Vector Graphics), OpenOffice and PowerPoint • “Glob” programming style makes hard

  19. Global-MMCS Community Grid • We are building an open source protocol independent Web Service “MCU” which will scale to an arbitrary number of users and provide collaboration support for thousands of simultaneous users • The function of A/V media server is distributed using NaradaBrokering architecture. • Media Servers mix and convert A/V streams • Open XGSP MCU based on the following open source projects • openh323 is basis of H323 Gateway • NIST SIP stack is basis of SIP Gateway • NaradaBrokering is open source messaging • Java Media Framework basis of Media Servers • Helix Communityhttp://www.helixcommunity.org for Real Media • http://www.globalmmcs.org open source “non advertised” release

  20. Session Server XGSP-based Control Media Servers Filters NaradaBrokering All Messaging Admire SIP H323 Access Grid Native XGSP XGSP Web Service MCU Architecture Use Multiple Media servers to scale to many codecs and many versions of audio/video mixing WebServices High Performance (RTP)and XML/SOAP and .. NB Scales asdistributed Gateways convert to uniform XGSP Messaging NaradaBrokering

  21. Break up into Web Services • Monolithic MCU becomes many different “Simple Services” • Session Control • Thumbnail “image” grabber • Audio Mixer • Video Mixer • Codec Conversion • Helix Real Streaming • PDA Conversion • H323/SIP Gateways • As independent can replicate particular services as needed • Codec conversion might require 20 services for 20 streams spread over 5 machines • 1000 simultaneous users could require: • 1 session controller, 1 audio mixer, 10 video mixers, 20 codec converters, 2 PDA converters and 20 NaradaBrokers • Support with a stream optimized Grid Farm in the sky • Future billion way “Video over IP” serving 3G Phones and home media centers/TV’s could require a lot of computing

  22. GlobalMMCS and NaradaBrokering • All communication – both control and “binary” codecs are handled by NaradaBrokering • Control uses SOAP and codecs use RTP transport • Each stream is regarded as a “topic” for NB • Each RTP packet from this stream is regarded as an “event” for this topic • Can use replay and persistency support in NB to support archiving and late clients • Can build customized stream management to administer replay, and who gets what stream in what codec • NaradaBrokering supports unicast and multicast • Use firewall penetration and network monitoring services in NB to improve Q0S

  23. Some resources • Web Service Grid Architecture and best practice http://grids.ucs.indiana.edu/ptliupages/publications/ • Grid Messaginghttp://www.naradabrokering.org • Data Stream Managementhttp://www.hpsearch.org/demo/crisisGrid/index.html • NSF NMI Grid Portals • http://www.collab-ogce.org/nmi/index.jsp • Web Service Colalboration Services and portlets http://www.globalmmcs.org/ • OMII UK Open Middleware Infrastructure Institute http://www.omii.ac.uk

More Related