Building Web Portals as Science Gateways

1. Building Web Portals as Science Gateways Marlon Pierce mpierce@cs.indiana.edu with slides from Dennis Gannon, Ahmet Sayar, Galip Aydin Indiana University

2. Outline of Talk • Overview of computing portals as gateways to cyberinfrastructure. • With several example projects • Architectures and standards of computing portals. • Review of portal architectures • Review of JSR168 • Building science portals. • With a motivating example • New technologies and techniques • JSF • AJAX

3. Computing for the Greater Good • Recent devastating natural disasters have demonstrated the need for large scale, distributed scientific computing. • Katrina, Rita, … • Christmas 2004 tsunamis • Earthquakes in Pakistan and future major earthquakes in California, Japan, … • 1906 was 99 years ago… • Research in large scale distributed computing can make a difference in people’s lives. • But many challenges lie ahead.

4. Cyberinfrastructure (CI) Overview • Cyberinfrasture combines • Grid computing: • Globus toolkit, Storage Resource Broker, Condor, UK OMII, OGSA-DAI, etc. • Provides the software for tying together distributed computing resources • High speed networking • Hardware: supercomputers, clusters, workstations • Applications: modeling, simulation, data analysis codes in many fields. • Data: raw, cooked, shared, derived, cataloged, etc. • Information/Internet Technology: web services, portals, databases, XML, semantic web/grid, audio/video collaboration, shared Web spaces, etc. • People • Cyberinfrastructure is supposed to enable remote collaborations and distributed teams. • This is the reality of a lot of science (and science funding)

5. Storms Forming Forecast Model Streaming Observations Data Mining On-Demand Storm predictions Scientific Challenges • The current and future generations of scientific problems are: • Data Oriented (CI challenge) • Increasingly stream based. • Often need petabyte archives • But linking data to applications across groups is a general problem. • In need of on-demand computing resources (CI challenge) • Conducted by geographically distributed teams of specialists • Who don’t want to become experts in grid computing.

6. Science Web Portals The goals of a Science Web Portal are • To give a community of researchers easy access to • Computational applications and tools • Primary data, shared data, derived data • Computational resources • All are needed to solve today’s scientific and engineering problems. • To do this in a discipline specific language that is common to the target community. • Science Portals are “gateways” to underlying Grid tools • Grids provide access to computing resources • Gateways manage the user’s interaction with the grid.

7. What is a Science Portal? • A standards-based web portal • Provides each user with • a secure way to login and authenticate identity • A view of a private workspace for accessing and managing data and tools • Tutorials and community news • Announcements of new services • Collaboration tools • Chat, calendaring, group messaging • Core Science Functions …

8. Portal Science Capabilities • Data Access is the most important • Allow the user community access to important shared data resources • Visualize it, publish it, download it, curate it. • Data Discovery • Searchable metadata directories. • Web access to important tools • Web-form interfaces to allow users to run important community codes • Webstart access to common java-based tools • Workflow tools • Allow users to combine community codes into workflows managed by the portal back-end resources.

9. Let’s look at a few real examples (about a dozen … many more exist!)

10. ServoGrid Portal

11. NEESGrid Realtime access to earthquake Shake table experiments at remote sites.

12. Geological Information Grid Portal

13. Mesoscale Meteorology NSF LEAD project - making the tools that are needed to make accurate predictions of tornados and hurricanes. - Data exploration and Grid workflow

14. Renci Bio Portal • Providing access to biotechnology tools running on a back-end Grid. • - leverage state-wide • investment in • bioinformatics • undergraduate & • graduate education, • faculty research • another portal • soon: national evolutionary synthesis center

15. Set Run Dates Submit Job To GRID Katrina Storm Surge Prediction (Leuttich/Blanton) Current ADCIRC grid Decomposition SCOOP ADCIRC Interface • SURA SCOOP • Southeastern Coastal Ocean Observing Program • Advanced Circulation Model (ADCIRC) • Finite Element Hydraulic Model for Coastal Areas, Inlets and Rivers • Rick Luettich (UNC), Joannes Westerink (Notre Dame) • Randall Kolar (Oklahoma), Clint Dawson (Texas), Corps of Engineers • Collaborators: • Leuttich/Blanton (UNC Marine Sciences) • Ramakrishnan/Lander (RENCI)

16. X-Ray Crystallography

17. Flood Modeling • University of Texas • TACC • Center for Research in Water Resources • ORNL • Purdue Large-scale flooding along Brays Bayou in central Houston triggered by heavy rainfall during Tropical Storm Allison (June 9, 2001) caused more than $2 billion of damage. Gordon Wells, UT; David Maidment, UT; Budhu Bhaduri, ORNL, Gilbert Rochon, Purdue

18. Teragrid Gateway Portals • TeraGrid • The largest “public” super-computing grid in the US. • 50 TFlops • Massive Data storage capability • Over 1 Petabyte on-line storage • TeraGrid Gateways • Access points for communities. • Two types: • A gateway between 2 grids • Open Science Grid gate way is one example. • Science Community Portal is the other type • Includes LEAD portal, Nanohub, Renci Bioportal, NVO • An additional set of new science portals just being developed.

19. The Portal as a Service Framework • Many important science portals are designed to access and orchestrate stand-alone web services and databases • Others are service frameworks that sit on top of an existing Grid, such as TeraGrid. • In both cases: • The architecture is similar • Common portal services are often shared • A Common architecture is emerging

20. The Architecture of Gateway Services The Users Desktop. Grid Portal Server Gateway Services Proxy Certificate Server / vault User Metadata Catalog Application Workflow Application Deployment Application Events Resource Broker App. Resource catalogs Replica Mgmt Core Grid Services Security Services Information Services Self Management Resource Management Execution Management Data Services OGSA-like Layer Physical Resource Layer

21. Next Steps • Many components of this architecture are now shared between different portal groups. • We now look at some of the technology behind these portals.

22. A Portal Architecture Review

23. A Famous Web Portal

24. A Famous Portal, After Login

25. What to Notice • After logging in, my colors, layouts, and content all changed. • I get my stock list, my Bloomington weather, my news stories, etc. • I got rid of “Garfield” • As we will see later, each of these content fragments (encircled) is managed by a thing called a portlet… • … I don’t guarantee that this is true for Yahoo’s web site but it is true for a large class of enterprise Java portal systems. • Portlets are the key to portal software reuse.

26. Let 10,000 Flowers Bloom • Many portal projects have been launched since late ’90s. • First Generation: HotPage from SDSC, NCSA efforts, DOD, DOE Portals, NASA IPG • 2002 Special Issue of Concurrency and Computation: Practice and Experience. • The field continues to be active • Global Grid Forum 14 Science Gateway workshop in June 2005. • About 15 gateways will be described in upcoming issue of Concurrency. • GCE2005 workshop at Supercomputing 05. • How do we share and reuse all of this work?

27. Three-tiered architecture is accepted standard for accessing Grid and other services Three-Tiered Architecture Grid and Web Protocols JDBC, Local, or Remote Connection Portal Client Stub Database Service Database Portal User Interface Portal Client Stub Grid Resource Broker Service HPC or Compute Cluster Portal Client Stub Information and Data Services Grid Information Services, SRB

28. JSR 168 Overview Java Portlet Standard

29. What Is a Portlet? • A portlet is a piece of Java code that manages the content of one section of a web portal’s HTML. • It can do anything else that a Java web application can do. • You can connect a portlet to a database, invoke a web service, download an RSS feed, etc. • It lives in a portlet container, which creates, manages, and destroys all the portlets of the portal. • Portlet containers are part of portals. • Portals must do other things like manage login, users, groups, layouts, etc. • JSR 168 standardizes two main things: • How the portlet container manages portlet lifecycles • How the portlets are programmed.

30. What is JSR 168? • From the portlet development point of view, it is really very simple: • You write a java class that extends GenericPortlet. • You override/implement several methods inherited from GenericPortlet. • You use some supporting classes/interfaces • Many are analogous to their servlet equivalents • Some (portletsession) actually seem to be trivial wrappers around servlet equivalents in Pluto. • A complete example is available from • http://www.servogrid.org/slide/iSERVO/NMI/OGCETutorialMaterial

31. Some Open Source JSR 168 Containers • GridSphere • http://www.gridsphere.org • uPortal • http://www.uportal.org • LifeRay • http://sourceforge.net/projects/lportal • eXo platform • http://www.exoplatform.com • StringBeans • http://www.nabh.com/projects/sbportal • Jetspeed2 • http://portals.apache.org/jetspeed-2/

32. Some GenericPortlet.java Methods

33. Some Supporting Classes/Interfaces

34. The Big Picture • As a portlet developer, the previous set of classes are all you normally touch. • The portlet container is responsible for running your portlets. • Init, invoke methods, destroy. • Portlets have a very limited way of interacting with the container. • The API is basically one-way.

35. In Action: Get started. public class JunkPortlet extends GenericPortlet { public void init(){ //Do any initialization. } //Rest of the methods on following slides go here. … }

36. Override doView() protected void doView( RenderRequest req, RenderResponse res) throws PortletException, IOException { //Include the desired JSP or HTML page. //We could also use out to write directly to the response. WindowState state=req.getWindowState(); if(!state.equals(WindowState.MINIMIZED)) { res.setContentType("text/html"); PortletRequestDispatcher rd= getPortletContext().getRequestDispatcher(“MyJSP.jsp”); rd.include(req,res); } }

37. The JSP Page <portlet:defineObjects/> <% PortletSession portletSession=renderRequest.getPortletSession(); portletSession.setAttribute("localattname","localattval"); PortletURL url=renderResponse.createActionURL(); String theActionString=url.toString(); %> HTML Content is here. A form is below. <form method=post action="<%=theActionString%>"> <input type=…> </form>

38. Some Notes • Include the <%portlet:definedObjects/%> tag, which will instantiate renderRequest, renderResponse, and portletConfig objects. • You can then just use them, as with request, response, and other JSP implicit objects. • The renderRequest gives you access to the PortletSession, if you want to store session variables. • One of the trouble points. • The renderResponse gives you access to the PortletURL object. • Use the PortletURL to generate a URL for the <form action> • So that it points to portlet container and gets handled by the processAction() method, rather than going of into space. • Handle href URLs similarly. • This is one of the sticking points.

39. When you invoke the form on the previous JSP, the portlet container will pass the action handling to the processAction method. The ActionRequest can be used to get any of the <input> parameters in a way similar to the usual HttpServletRequest. When the processAction method returns, the container then invokes the appropriate do method (usually doView). If you need to pass <form> parameters on to doView, add them to the ActionResponse. This will give them to the RenderRequest. The example shows how to add ALL parameters. public void processAction (ActionRequest request, ActionResponse actionResponse) throws PortletException, java.io.IOException { //Process request parameters … //Add other params // to the renderRequest actionResponse.setRenderParameters(request.getParameterMap()); } Lastly, Override processAction()

40. A Comment on Portlet Coding • JSR 168 seems to make some important and dubious assumptions • Developers will gladly ignore other development methodologies/frameworks like Velocity, Struts, and Java Server Faces. • Developers instead want to write a GenericPortlet extension for every single portlet they develop. • And write really complicated processAction() and doView() methods. • Developers will like the specific JSR 168 portlet-style Model-View-Controller that it forces on them. • Fortunately, these other development environments can be mapped to portlet actions. • In the OGCE project, we have developed support for Velocity portlets. • We are transitioning to Java Server Faces

41. A Grid Portlet Scenario Developing a Simple Grid Portlet Application

42. A Quantum Chemistry Code Submission Form • You have been asked to develop a submission form for the Democritos group’s Quantum Espresso (QE) package. • These forms should help users set up and run QE applications on the TeraGrid and other Grid installations. • Mix of GT 2, GT 4, Condor, etc., for submission • You are told to initially support the Plane Wave Self Consistent Field (PWSCF) code. • Other QE applications may follow. • These may be coupled with PWSCF into simple workflows, but this is a later problem.

43. Your Deployment Architecture It acts as a common Gateway to different grid toolkit installations and resources Your portal server runs at FSU. GT 4 @TG TeraGrid LSF Portal Server @FSU GT 2 @UMN MSI PBS

44. Some Issues • You decide the JSR 168 style portlets are the way to go… • But of course the PWSCF portlet doesn’t exist yet. • You will need to also support other Quantum Espresso codes. • Would like to reuse as much code as possible. • But your PWSCF portlet isn’t reusable at that level. • You also would like to simplify saving user input data and session archiving.

45. PWSCF Web Forms for Submission

46. Your Choices • JSR 168 will allow you to share your portlet code with other collaborators. • The Java COG Kit will help hide the differences between Grid toolkits for common tasks. • Vanilla JSF will help simplify your portlet development in several ways. • JSF decouples your backing code from the Servlet API • You can write your backing code as pure Java Beans/Plain Old Java Objects. • You don’t have to adopt, maintain HTTP parameter name conventions. • Your form input backing beans can be serialized/deserialized with Castor. • Coupling JSF and COG will allow you to compose your Grid actions using simple JSF taglibs. • You can reuse your Grid taglibs in other Quantum Espresso portlets. • You can compose composite actions

47. The Java CoG Kit Gregor von Laszewski Argonne National Laboratory University of Chicago gregor@mcs.anl.gov http://www.cogkit.org (as interpreted by MEP)

48. CoG Kits • CoG Kits make Grid programming simple and new technologies are easy to integrate • We focus on a CoG Kit for Java • Python also available (K. Jackson, LBNL) • Availability: Java CoG Kit since 1997 • The CoG provides two important things • A higher level client programming environment than stubs. • A shield against different versions of the Globus toolkit • Same high level API works with GT 2.4, GT 3.0.2, GT 3.2.0, GT 3.2.1, GT 4.0.0

49. CoG Gridfaces Layer CoG Gridfaces Layer CoG Data and Task Management Layer CoG Data and Task Management Layer CoG GridIDE CoG GridIDE CoG Abstraction Layer CoG Abstraction Layer CoG CoG CoG CoG CoG CoG CoG CoG CoG CoG CoG CoG CoG CoG CoG Abstraction Layers Development Support Nano materials Bio- Informatics Disaster Management Portals Applications GT2 GT3 OGSI classic GT4 WS-RF Condor Unicore SSH Others Avaki SETI