2 nd Generation HPCPortal v3 for the NGS

1. 2nd Generation HPCPortal v3 for the NGS Rob Allan and Dharmesh Chohan e-Science Centre CCLRC, Daresbury Laboratory

2. 1st Generation Portal Interfaces HPCPortal DataPortal

3. InfoPortal, MDS Browser and Active Map Operational monitoring: resource-centric and site-centric views of Grid resource and network status http://www.grids.ac.uk/InfoPortal

4. What are the Restrictions? 1st generation Grid portals have been focused on providing basic task-oriented services, such as user authentication, job submission, monitoring, data transfer. However, they are typically tightly coupled with Grid middleware tools such as Globus. The main limitations of first generation portals can be summarized as follows. • Lack of Customization: Portal developers instead of portal users normally build portals because the knowledge and expertise required to use the portal toolkits. When end users access the Grid via a portal, it is almost impossible for them to customize the portal to meet their specific needs, e.g., to add or remove some portal services. • Restricted Grid Services: First generation Grid portals are tightly coupled with specific Grid middleware technologies such as Globus, which results in restricted portal services. It is hard to integrate Grid services provided by different Grid middleware technologies via a portal of this generation. • Static Grid Services : A Grid environment is dynamic in nature with more and more Grid services are being developed. However, first generation portals can only provide static Grid services in that they lack a facility to easily expose newly created Grid services to users.

5. 2nd Generation OGCE Portal

6. NMI Portal and OGCE With funding from the US National Science Foundation Middleware Initiative (NMI), the Open Grid Computing Environments (OGCE) project was established in autumn 2003 to foster collaborations and sharable components with portal developers worldwide. Tasks include the establishment of a Grid Portal Collaboratory, a repository of portlets and portal service components, an online forum for developers of Grid Portals, and the building of reusable portal components that can be integrated in a common portal container system. The development of OGCE is based on the following projects: • Java CoG Kit sponsored by SciDAC and NSF Alliance; • The CHEF Project of the University of Michigan; • The Grid Portals Information Repository (GPIR) and GridPort of the Texas Advanced Computing Center; • The Alliance Portal Expedition project, including NCSA, Indiana University's Extreme Labs, and the Community Grids Lab at Indiana University.

7. Alliance Grid Portal Services The Alliance Portal is an ongoing project that focuses on building second-generation Grid portals. It is based on Jetspeed and targeted at the construction of Grid portals using Grid Portlets. Currently the Alliance Portal can provide the following Grid Portlets that are leveraged from existing Grid services. • A Proxy Manager: portlet utility that allows users to load GSI proxy credentials into their account via MyProxy. • An LDAP Browser: portlet interface to access the contents of the LDAP servers. • A GridFTP Client: portlet provides the basic client functions of Grid FTP with a user-friendly interface.

8. More… • Gram Job Launcher: portlet allows a user to submit jobs to a Grid environment using the Globus GRAM protocol. For this the user must have a valid GSI Proxy Certificate which can be loaded through the Proxy Manager portlet. • Grid Utilities: include the "Grid Ping" utility and the "Grid Job Submission" utility. The GridPing portlet can ping a resource in a Grid environment and determine if a user has the access to it or not. The Grid Job Submission portlet is similar to the Gram Job Launcher portlet except that this portlet will not return till the job has completed. The output and the error of the job is displayed by the portlet. • OGSA Browser: portlet alows users to query a Grid service for its Service Data Elements (SDE). Users can query a SDE by using the name of the SDE as the query string. Users can also obtain a list of SDEs by using "serviceDataName" as the query string. Once users get a list of all the SDEs, they can click on each SDE to query it.

9. Collaboration Tools Collaboration and content management tools come as default: • Schedule - calendar for work site • If you are a member of multiple work sites your personal calendar will be updated with all appointments • Resources - resource list and uploaded documents and links • News - news service using RSS feed • We have created a prototype news feed and web service • Discussion - threaded discussion tool • Chat - online chat room for group • Web – link to ordinary Web site Uses CHEF content-management capabilities ported to PostgreSQL We previously tried using PHPNuke to provide similar functionality in HPCPortal and InfoPortal v2. This is still being used by the e-Minerals project. Aim is to enhance community participation by providing suitable tools.

10. Content Resource Tool Example of Sakai tool from ReDRESS project.

11. Other tools from SDSC/ UTexas/ Indiana • Anabas - join and participate in Anabas Impromptu collaborative session via applets (requires a license) • Newsgroup (Read/ post) - topic-based threaded newsgroup tool using Indiana XML schema • Newsgroup (Request) - request a new topic • Newsgroup (Admin) - for admins only! • Bibtex (Read/ edit) - add/ edit entries in Bibtex topics for shared authorship of reference lists. Uses Indiana XML schema • Bibtex (Request) - request new Bibtex topic • Bibtex (Admin) - for admins only! • Bibtex (Super Admin) - for super admins only! • GridPing - test a remote Grid service • GridContext - browse and manage DNs in Xdirectory server - provides context services for managing metadata • GPIR Browser - browse Grid resources via Grid Port Information Repository giving information on HPC status, jobs, etc. • Historical GPIR - use GPIR to view stored information • CSF Job Submission - submit job to a Community Scheduling Framework service • Sequencer - ditto via GridPort sequencing service • OGRELauncher - Open Grid Computing Runtime Engine: launch simulation event via OGRE from NCSA, workflow based on ANT or BPEL • OGREEventView - monitor OGRE events • COG-Workflow - use workflow enactment via the Java CoG kit • App Manager - register, de-register and edit information about a Grid application • Condor - submit jobs via Condor or DAGMan and monitor them

12. Work sites for groups/ projects User and group management • Login/ Logout • as it says on the tin • Set up new work site • Only if enabled by admin • Membership • join/ unjoin from work site (admin can control permissions) • Customize • chose which tools you want in the work site from set provided Portal My workspace Work sites Tools Portlet views Some projects could have their own work site to customise and use the collaboration tools. Need to decide how we manage this capability.

13. Why not simply deploy OGCE? We have started with CHEF and will bring in the tools we need • OGCE is a large Java stack, some tools not appropriate • Need to import UK-specific tools from existing projects • Learn to write portlets and do customisation • Link into Web services • Work with new projects, e.g. VRE developers • In future: adopt JSR-168 and WSRP standards

14. Some Project Requirements • e-HTPX • Customising OGCE tools and will move into NGS framework • Integrative Biology • Have a VRE project funded to use OGCE. Will require their own portal instance as they support “experiments” as work sites • MyGrid • Currently developing portlet interfaces to their tools • RealityGrid and GeneGrid • Previously used GridSphere – will move portlets into new framework? • e-Minerals • Continue to use bespoke solution for time being What will others do? What other tools do users want? InfoPortal tools? VRE tools will be coming…

15. Hosting the Portals • NGS, e-HTPX and e-Minerals portals hosted on IBM HS20 BladeCenter at Daresbury • NGS portal aliases: • http://ganges.dl.ac.uk:8080 • http://ngsportal.ac.uk • http://www.grids.ac.uk/ngsportal • Development system for testing on 3rd blade • CVS into dedicated server, RAID and Tivoli tape store at Daresbury • Backup facilites to ATLAS tape store at RAL • Mirroring PostgreSQL DB is planned

16. Future Technology We are currently exploring the “best of breed” 2nd generation portal –Sakai http://www.sakaiproject.org • Originated from U. Michigan (the CHEF project) with partners at U. Indiana, MIT, Stanford. • Mellon Foundation Funding with OKI standard interfaces • Rich set of Collaboration and Educational tools with templates for tool creation • We have proposed using this as a VRE “front end” by extending the framework to a Service Oriented Architecture • Evaluation exercise funded by JISC http://tardis.dl.ac.k/ReDRESS/sakai_doc.pdf • Using in the ReDRESS project http://redress.lancs.ac.uk We will be: • Using technology: JSR-168 to create plug-in portlets • Using technology: WSRP to link to remote portlets and content • Using Grid and Web services for SOA including UDDI registry

17. Virtual Research Environments JISC are funding a number of portal-based VRE projects – most based on Sakai. • Need to engage this community • Has links into education and digital information • Will be developing additional tools which can be used for NGS • e.g. a Wiki portlet • Can we prioritise other tools? • We have proposed a “Sakai Alliance” as a means of keeping track of developments, helping projects communicate with each other, and feed back to US developers • First meeting next week in Oxford

18. See it in action… Demos will be given over the tea break this afternoon. Some of the people involved are: Responsible for NGS portal: • Dharmesh Chohan – NGS Portal and Grid Information Services Other development work by: • Dave Meredith – e-HTPX Portal and Hub • RikTyer – e-Minerals Mini-Grid and Portal • Xiao Dong Wang – OGSA and OGSA-DAI Grid Services and portlets • Xiaobo Yang – recently started work at DL • Rob Allan – ReDRESS portal for Awareness and Training, Sakai VRE We will try to answer your questions!

19. Proxy Login

20. Proxy List

21. LDAP Browser

22. Job Launcher

23. GridFTP

24. Web Content in MyWorkspace