1 / 21

Grid Portals: Bridging the gap between scientists and the Grid

Grid Portals: Bridging the gap between scientists and the Grid. Michael Russell, Jason Novotny, Gabrielle Allen Max-Planck-Institute fuer Gravitationphysik Golm, Deutschland. The promises of Grid computing are grand. Uniform access to heterogenous resources

elisa
Download Presentation

Grid Portals: Bridging the gap between scientists and the Grid

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Grid Portals: Bridging the gap between scientists and the Grid Michael Russell, Jason Novotny, Gabrielle Allen Max-Planck-Institute fuer Gravitationphysik Golm, Deutschland

  2. The promises of Grid computing are grand • Uniform access to heterogenous resources • The ability to pool distributed resources together on demand • Resources are either transparently available to users or they simply don’t have to worry about them • Support virtual organizations of distributed researchers collaborationg across institutoional, geographic, and political boundaries… • Support applications with enormous computing and/or data management requirements.

  3. Grid computing is quickly evolving • In the 2 years I’ve been here Globus 1.3 went 2.0 • Grid Portals became “the next big thing” • Luckily, portlets have come to the rescue • Meanwhile, Globus went pro to bring in IBM and other heavyweights • Now OGSA is stepping up to bat • Global Grid Forum is already a hit • Can’t wait for the video games!

  4. So where is the Grid? • How can we use it? Or at least that’s so-called users are probably asking. • The Grid is a work in progress, one of the biggest undertakings in the history of humankind. Most of you are in Scotland to do your part in building the Grid. • And most of you have reasons why you need the Grid, you want to use it right? Or perhaps you want to help others to use it. • The point is, a large gap exists between the Grid and its would-be users.

  5. Grid Portals • So you want to build a Grid portal to bridge that gap. But what does it take to build a Grid portal? Well, it’s going to take a lot… • It takes a solid understanding of the state-of-the-art in Web portal development. • It takes a solid understanding of the state-of-the-art of in Grid computing. • And much more…

  6. The Astrophysics Simulation Collaboratory • The ASC seeks to: • Enable astrophysicists to develop better and more powerful simulations with the Cactus Computational Toolkit. • Enable astrophysicists to run and analyze simulations on Grids. • Build a Grid portal to support these activities and all the rest that comes with Web portals.

  7. The ASC and Cactus • So we worked with the Cactus Project to develop support for working with Cactus from the ASC Portal. • We developed application Web components to: • Install Cactus software from multiple cvs repositories onto target resources. • Build executables with those installations using the autoconf and make capabilities in Cactus. • Upload and edit parameter files. • Run simulations on target resources, as well as for connecting to simulations and monitoring their progress. • Launching the appropriate visualization applications on Cactus data. • In most cases, the Cactus Project developed extensions to Cactus software to support these components.

  8. The ASC and Globus • We worked with the Globus Project to develop support for working with Globus from the ASC Portal. • We developed Grid Web components to: • Enable logging on with one or more Grid proxy certificates stored in MyProxy. • Submit and monitor jobs with Globus Gatekeeper and for maintaing job history. • Manipulate files (listing, copying, moving, deleting files) with GSI-FTP. • We tried to use MDS, but at that time MDS did not meet our needs, so we developed our own components for storing static information about resources and polling for whatever dynamic information could be reliably retrieved from MDS. • We asked Globus to build GSI-CVS and now we’re building Web components and services to use and extend GSI-CVS. • We’ve added support for GSI-authentication in the MindTerm Java SSH implementation.

  9. The ASC and ASC • We realized we needed to support the people within ASC directly from our portal, so we built administrative Web components to: • Manage user accounts. • Assign security roles to users and to which pages users have access. • Manage the proxy certificates users require to authenticate to other services and determine which certificates authenticate to which services. • Manage the resources and services to which we provide access from the ASC Portal.

  10. Problems we faced • The ASC is a Virtual Organization in every sense of the word with users, developers, administrators, software, and resources distributed across the U.S. and Europe. • With our developer(s) in the University of Chicago and our users in Washington University, St. Louis and Max Planck Institute in Golm, Germany, there wasn’t nearly enough direct contact between our users and developers. This made it difficult to meet their needs. • It was easy enough to prototype Web components, but we needed to build a Grid portal framework that would support future development and sustain a production-quality Grid portal, and that took several months to develop. • We were closely associated with Globus, but this made us too reliant upon Globus as our interface to resources… • For while we had identified the resources our users required, the Grid as we knew it then and today just wasn’t enough.

  11. It takes more than just a portal You need to build a virtual organization or otherwise join a virtual organization and plug into their work!

  12. Other lessons we learned • Put the needs of scientists at the very top of your list. Be familiar with their research and the day-to-day problems they face in using computing technology to conduct that research. • Next, consider what it is you really need in the form of: • Enhancements to the applications scientists are developing. For new applications, consider how you can build-in support for Grid operations. For legacy applications, consider how you can provide better support for their applications with external services. • Enhancements to the Grid infrastructure with respect to your applications. These enhancements should build off other… • High-level services that coordinate the use of resources. We’re beginning to see Globus as the “system” layer with respect to Grids.

  13. Don’t forget your infrastructure • Create a vialable testbed that includes both the resources your users need and use in their everyday reserch as well as resources with which you can experiment. • In a VO like the ASC, you will not have administrative control over these resources. In fact, this is a very difficult problem to overcome, it takes a lot of effort to see changes applied where and when you need them. • You need tools for testing things out and you’re going to need to keep track of all your resources and providers, the change requests you make, the problems your users experience, and so forth…

  14. Get into production mode • In order to build a viable Grid portal, realize you need to build a production system, something your users can rely upon to work every time. For instance, what happens to your production database when your software and data model change? • What makes a production system? Solid engineering practices and attention to user requirements, security issues, persistence management, quality assurance, release management, performance issues… • Project management at the VO level is complex, make sure you understand Grid-level management issues before you start writing those cool Grid portal proposals!

  15. Build a team • It’s important to communicate with partners in the Global Grid Forum, but try to cover all the bases within your project. If you don’t have the funds to build a large team, then allocate funds towards developing explicit links between yours and other projects. • Because you need application experts, Grid portal experts, Grid service experts, Grid testbed administrators, and so on… division of labor is a cornerstone of project management.

  16. GridLab • Well, so we’re taking the lessons we learned in the ASC and elsewhere (GPDK, for example!) through the collective experience of everyone involved in GridLab and we’re applying them towards building a… • Production Grid across Europe (and the U.S.) • Grid Application Toolkit for developing applications with built-in support for Grid operations, like migrateApplication() • Grid Portal to support GridLab

  17. Key points • We’re going to work as closely as possible with scientists and their applications throughout the project. • We’ve created a testbed of resources that our scientists really want to use. • We’re working to build higher-level services to better coordinate the use of those resources, and the requirements for these services will come either directly from our application groups or indirectly through using a Grid portal. • We’re developing application frameworks to enable scientists to make use of these higher-level services as basic function/method calls within the applications. • And we’re coordinating these activities through the development and use of the GridLab Portal.

  18. The GridLab Portal • We’re using the ASC Portal software to allow us to focus on the needs of scientists from the very beginning of our project development. • We’re simultaneously building a new framework that takes the best of current practices in Web and Grid computing, and we’re documenting just about every bit of its requirements, design, and development. • As we develop this framework, we’ll be preparing the Web interfaces we develop with the ASC Portal for migration to the new framework.

  19. Bridging the gap • Before you begin your Grid portal efforts, look at what’s already out there. For example: • GPDK - JSP/Servlets, Great way for getting an introduction to Grid Portal development • GridPort - Perl-CGI, well-managed project and they have real application groups using NPACI resources. • Bear in mind that Portlets are where it’s all heading, but there is no Portlet API quite yet. A lot of people looking at building Grid portals with the JetSpeed codebase (but not everyone!). • Or, you should consider working with us…

  20. A possible collaboration with GridLab Grid CVS • Cactus • Triana GridLab Application Toolkits GridLab Application Manager Grid Make GridLab Portal GridLab Resource Manager Compute Resources Help us design better Grid technologies. Plug in your own application and supporting services.Then develop your own Web pages with our Grid portal… Run my application! GridLab Information Service

  21. Some online references • Astrophysics Simulation Collaboratory: http://www.ascportal.org • Cactus Project: http://www.cactuscode.org • Globus Project: http://ww.globus.org • GridLab: http://www.gridlab.org • GridPort: http://www.gridport.org • Grid Portal Development Toolkit: http://www.doesciencegrid.org/Projects/GPDK • Jakarta JetSpeed: jakarta.apache.org/jetspeed • OGSA: http://www.globus.org/ogsa • Portlet Specification: http://www.jcp.org/jsr/detail/168.jsp

More Related