1 / 38

Reusable Grid Portal Components

Reusable Grid Portal Components. Mary Thomas. Outline. The Portal Challenge The Solution: The Open Grid Computing Environments Consortium (OGCE) The OGCE Portal Architecture Application Portal Examples NMI OGCE Plans. The Portal Challenge.

dianea
Download Presentation

Reusable Grid Portal Components

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Reusable Grid Portal Components Mary Thomas

  2. Outline • The Portal Challenge • The Solution: • The Open Grid Computing Environments Consortium (OGCE) • The OGCE Portal Architecture • Application Portal Examples • NMI OGCE Plans

  3. The Portal Challenge

  4. Three-tiered architecture is accepted standard for accessing Grid and other services Three-Tiered Architecture Grid and Web Protocols JDBC, Local, or Remote Connection Portal Client Stub Database Service Database Portal User Interface Portal Client Stub Grid Resource Broker Service HPC or Compute Cluster Portal Client Stub Information and Data Services Grid Information Services, SRB

  5. Portal are Gateways to Grid • Early Grid Portals were great success: • Applications oriented, • Provided scientists with some useful tools • Encapsulated some complexities. • But they were: • All “stovepipe” solutions. • Very little reusable code and infrastructure • Nothing could be shared between portals • Many ignored emerging architecture of the Grid • A framework based on distributed ubiquitous services

  6. Impact of Distributed Grid and Web Services • Application Interface Workflow service Grid Portals App Instance App Instance App Instance Open Grid Service Architecture Layer Data Management Service Registries and Name binding Security Policy Logging Accounting Service Administration & Monitoring Reservations And Scheduling Grid Orchestration Event/Mesg Service Resource layer 1000s of PCs ->massive supercomputers Launch, configure And control Open Grid Service Infrastructure (web service component model) Online instruments (slide by Dennis Gannon)

  7. Advantage of Portlet Architecture • Portlet/component approach very compatible with Web/Grid services model • Each Grid service can be associated with a unique portlet • Very easy to add new services • Many different Groups can contribute portlets which can be plugged into a portal. • Currently Indiana, Argonne, Michigan, NCSA,Texas + GridSphere portal group following this model. Most likely others as well. • Jetspeed is Apache standard. Basis for IBM WebSphere portal. Also Oracle, Sun and others. • Now a Java Standard (waiting for JSR 168) • User can select and configure portlets he/she wishes to use • Selection becomes part of persistent context

  8. The Next Generation Grid Portal Event and logging Services The User Application Factory Services Messaging and group collaboration Portal Server Directory & index Services User's Persistent Context MyProxy Server Metadata Directory Service(s) • Building on Standard Technologies • Portlet Design (JSR-168) IBM, Oracle, Sun, BEA, Apache • Grid standards: Java CoG, Web/Grid Services • Web server: JetSpeed (open source) • User configurable, Service Oriented • Philosophy: The Portal is a gateway to distributed Grid and Web Services • With common API, portlets can be exchanged, interoperate

  9. User Requirements Have Not Changed • Users & Clients are: • Portal clients/users • Portal developers • Security • Single sign-on, authentication to all grid resources • Certificate based / uses MyProxy • VO management • File Management • Grid FTP • File staging • Metadata management • Remote Job Management • Job submission, scheduling, and tracking. Log and output mgmt. • Workflow management tools

  10. User Requirements • Information Services • Grid resource and status information directories • Data and metadata catalogs • Personal and VO metadata and service index/directories • Database Interfaces • Application Interfaces • Configure and launch workflow from domain specific interface • Access to Collaboration Tools • VO collaboration. Synchronous and asynchronous • Access Grid venues • Developers need common API

  11. The Solution:The Open Grid Computing Environment (OGCE) Consortium

  12. Open Grid Computing Environments (OGCE) Consortium • Motivation: • Emerging technologies impacting portal developers: • Grid/Web services; ubiquitous database use, XMl • Lessons learned from GCE-RG and other projects (GridPort/HotPage, Alliance Portal, DOE Components, NEESGrid, Gateway, GRaDS, others) • Multiple projects repeating similar tasks, work, development; • OGCE established in Fall 2003 to foster collaborations and sharable components with portal developers worldwide.

  13. OGCE • Funded by: • NSF NMI: software releases will be part of NMI and synchronized with GRIDS releases • Also: NCSA Alliance, NPACI, DOE, NASA, others • Release Schedule: • First Release: 11-15-03 • Next Release: April, 2004 • Website: http://www.ogce.org

  14. OGCE Tasks/Goals • Development of interoperable Grid portal architecture • Define standards for common API’s between portlet service and remote services • Establishment of a Grid Portal Collaboratory • Repository of portlet and portal service components • An online forum for developers of Grid Portals • The building of reusable portal components: • Can be integrated in a common portal container system • Contribute to standards organizations: • NSF NMI program  software releases • GGF if appropriate or applicable

  15. The OGCE Portal Architecture

  16. OGCE Initial Architecture Grid Java Protocols Java Local CoG COG Grid Services Portlets API Kit GRAM, MDS - LDAD MyProxy Grid Services CoG Proxy Interfaces Remote Portlets HTTP Stubs Other Services Portal SOAP CHEF Service Teamlets API Services Jetspeed Internal Services Initial architecture aggregates multiple services into a single portal using portlet containers

  17. OGCE Initial Arch: Features & Capabilities • OGCE approach encompasses multiple portal development systems: • Preserving development investments • Central portal system (Jetspeed) acts as a central portal container: • Portal responsible for managing the lifecycle of pluggable portal components, or portlets. • Grid/computational portlets must provide services on the behalf of the user: • Not just a collection of links/static pages • User view (from the left) is of a single portal composed of portlets.

  18. Supports Aggregation of Diverse Tasks • Portlets provide a uniform management interface to the portal (on the left) connect to heterogeneous set of APIs on the right. • First horizontal thread • Portlets interact with web interfaces written in other languages such as Perl, Python, PHP, ColdFusion, etc. • Architecture for bridged support is depicted in the second horizontal thread • Examples: • Locally running Myproxy Grid portlet uses Java CoG kit to interact with remote Grid services • Job Scheduler portlet uses GridPort Toolkit to submit job via Community Scheduling Frameworks (CSF) service • Teamlets (sharable portlets) share collaboration and Grid services, such as calendars, message boards, file services.

  19. OGCE - Revised Architecture Portal Grid Protocols Portlets and Teamlets Service API Grid Service Stubs Grid Services Java CoG Kit Local Portal Services Remote Content Servers Remote Content Services HTTP Jetspeed Internal Services • Revised portal architecture provides a unified interface for portal services. • Supports aggregation of services in backend

  20. Features • All services will employ a single (generic) service API that allows them to be deployed and accessed in a uniform fashion. • Developer then chooses (during portal configuration) the implementations of particular services. • Services may be local, may access remote sites via Grid protocols, or may access remote content, such as web interfaces written in Perl. • OGCE Twist: • Entire portal will auto-install and include all required software components

  21. Key Technologies???? • Core Portal Framework: • CHEF/JetSpeed (U. Mich) • Core Grid Technologies: • JavaCoG (provides Globus interface) • GridPort Toolkit (aggregates JavaCoG, and other grid services)

  22. Portal/Application Gallery

  23. TeraGrid Portal • Access Point for TeraGrid Services • Version 0: Collecting Initial Services • Public Information about Resources • Private Information for the developers. • Version 1: User centered portal • HotPage/GridPort style access to user accounts, credentials, job submission & management. • Version 2: Portals for Science Collaborations (Q3 2004) • Shared spaces, whiteboards, AG access, group authorization, shared application services

  24. TeraGrid Integrated Architecture Portal Grid Services Portlets and Teamlets Service API Grid Service Stubs Java CoG Kit GridPort Toolkit Web Services Local Portal Services Remote Content Servers Remote Content Services HTTP Jetspeed Internal Services Diagram demonstrates how existing software projects (such as GridPort) can be adapted to support NMI Portals software system

  25. Alliance scientific portal GridFTP MetaCatalog Service(MCS) NEESgrid Meta- Catalog service Storage Resource Broker (SRB) Globus Toolkit 3.0 Teragrid Execute application Generate and store derived products Gather data files, resources Create job description Store and record results Retrieve customized run description template from data management services Broker Service Monitoring results captured in event services persistent channel. myMEAD records address of service and channel Need to store ‘y’ and ‘z’ Need ‘x’ put to ‘l’ by time y:00 monitoring Teragrid data service Teragrid Data Service-single point of contact to server-side data servers Provides context to resolve ‘x’, ‘y’, ’z’ myMEAD personal information space Delayed invocation of GridFTP

  26. DOE Fusion Grid Portal • Portal Development on the DOE SciDAC Grid • Initially, Fusion Grid • Multi-site collaborative project: • TACC, Indiana, SDSC, General Atomics • Developing interoperable portal and Web services • Portal architecture based on NMI Portal software

  27. LEAD Portal

  28. CHEF - NEESGrid • NEES Tools • Data Viewers (Live/Stored) • Telepresense • Video as Data • Electronic Notebook • Data/Metadata Repository • Grid Technologies • Grid sign on using myproxy • Grid computational portal • GridFTP • ..Many more

  29. OGCE Plans

  30. OGCE Portal System OGSA Globus COG NMI Dist System Integration CHEF Toolkit Gridport Alliance Portal Testing Workflow Slide courtesy of Chuck Severance

  31. NMI Features List (April 04 Release) • Content Management: • Resources: Users and groups can upload and share documents and URLs. • News: Several news feeds are available, including XMLHack and BBC News. • Grid Context: Store arbitrary web objects (movies, web pages, audio files) in a customizable, annotated directory tree. • Collaborative: • Chat: Engage other members of your group in online discussion. • Discussion: Participate in discussions with other members of your group • Calendar Program:schedule appointments and attach associated documents and URLs.  Calendars can be used to schedule both user and group activities.  • Anabas Impromptu: system for real time shared display, audio, and chats.

  32. NMI Features List (April 04 Release) • GT 3.0/Interactive: • Proxy Manager and Job Submit: Obtain and manage one or more Grid credentials that can be used to access Grid resources through your browser. • GridFTP: Use Grid credential to browse remote directories and to upload and download files between your desktop and a remote host. • Application Manager • Application Factory Manager • Remote Job Management • GridAnt for workflow • Job submission portal • Community Scheduling Framework (CSF): submit jobs to scheduler • Information Services: • LDAP Browser: • Navigate the LDAP server of your choice. • GridPort Information Repository Portlets (GPIR): Portlets are available for browsing Machine Summaries,  System Status, and Total Grid Capabilities. • Application Portlets: • Specialized portlets for projects

  33. Prerequisite Software Java 2 Standard Edition version 1.4.x or higher.  Apache Ant version 1.5.x or higher.  Preparing the Installation Optionallly, create a "portal" account on your web server. Download the portal bundle Put the download bundle into the home directory of "portal" account. Untar or unzip the portal bundle.  Globus: installation process will create the directories .globus and .globus/certificates in the Software automatically installed includes: Globus/Java CoG CHEF Install the Portal Move into the portal/chef-1.0.7 directory. Edit file "build.properites" Type "ant deploy" to build in and install the portal. Start the Tomcat web server.  Point your browser at http://<your host name>:10081/ogce and create an account. OS/Machines Tested: MAC OSX PC/Windows XP Linux Cygwin Key Feature – Download a Portal & Install

  34. Current Status • SC03 Demonstration portal and software download available • Beta1 • Updated Release available now • Bug fixes • Now supporting GT3.x with J2SDK 1.4.2 • Working on NMI April 2004 release • Including new features to support science portal applications • Open GCE Runtime Engine (OGRE) workflow services and portlets • Sample template portlets for simple science applications • Based on aggregation of existing projects • OGSA/OGSI/GT3.x compatible

  35. Future Directions • Moving towards new portlet standard • JSR 168 • Should allow compatibility with • GridSphere • Academic education and business portals Oncourse, OneStart, CHEF (of course), and uPortal • Major commercial products offerings from Sun, IBM, Oracle, etc. • Incorporate new features: • Grid Services (CSF, OGSA-DAI portlets, Job Sequencer) • Access Grid, Polycom and other AV Portlets • Collaborations and Contributions • DOE SciDAC projects (such as CMCS) • Expand online repository • Develop process for third party contributions

  36. Future Directions • Moving towards new portlet standard • JSR 168 • Should allow compatibility with • GridSphere • Academic education and business portals Oncourse, OneStart, CHEF (of course), and uPortal • Major commercial products offerings from Sun, IBM, Oracle, etc. • Incorporate new features: • Grid Services (CSF, OGSA-DAI portlets, Job Sequencer) • Access Grid, Polycom and other AV Portlets • Collaborations and Contributions • DOE SciDAC projects (such as CMCS) • Expand online repository • Develop process for third party contributions

  37. Participants the globus alliance University of Texas at Austin JavaCoG Argonne National Labs Indiana University University of Michigan National Center for Supercomputing Applications

  38. References • OGCE Website • http://www.ogce.org • NSF Middleware Initiative • http://www.nsf-middleware.org

More Related