PTLIU Laboratory for Community Grids Geoffrey Fox Computer Science, Informatics, Physics

1. Overview of Grid Computing EnvironmentsProposed GGF Information DocumentG.Fox, D. Gannon, M. Pierce, M. Thomas PTLIU Laboratory for Community Grids Geoffrey Fox Computer Science, Informatics, Physics Indiana University, Bloomington IN 47404 http://grids.ucs.indiana.edu/ptliupages

2. Source of Report • Last Half of 2001: Call from GCE RG for Papers for Journal special issues – 28 papers submitted and reviewed • Published in Concurrency and Computation: Practice and Experience Vol. 14, Grid Computing Environments Special Issue 13-15, Fall 2002 • http://aspen.ucs.indiana.edu/gce/gce2001index.html • Augmented by chapters (about 14) in Fran Berman, Geoffrey Fox and Tony Hey, ‘Grid Computing: Making the Global Infrastructure a Reality’, ISBN 0-470-85319-0, John Wiley & Sons Ltd, Chichester, 2003. See http://www.grid2002.org • Other key papers which we knew about

3. Database The fuzzy definition of Grid Computing Environments UserServices Aggregation Portal GridComputingEnvironments Portal Services PortalServices Application Service Application Service OGSA (OGSI) Interfaces “Core”Grid SystemServices SystemServices SystemServices Raw (HPC) Resources

4. Two Major Areas • “Programming the User Side of the Grid” which involves discussing computing model for Grid • Controlling user interaction – rendering any output and allowing user input in some (web) page. • This includes aggregation of multiple data sources in a single portal page (Jetspeed). • Has natural overlaps with information (commercial) portals

5. Classification of GCE Papers in Programming the User Side of the Grid • Technology for building GCE systems -Interface with backend Infrastructure e.g. Community Grid Kits, GPDK • Problem Solving Environments • Domain specific collection of tools and user interface. E.g. XCAT, Polder, SCIRun, Astrophysics Collaboratory • GCE Tools • Support parameter sweep, visualization, job status, files, security, workflow .. • GCE Shell Portals providing a general interface to many Grid capabilities • Analogous (not usually command line) to role of UNIX shell providing access to UNIX tools and user programs, files … • Note UNIX has core system and higher level tools accessed by Shell • E.g. Unicore, Hotpage, Mississippi Portal • PSE’s often built on top of GCEShell portals

6. Aspects of Programming Model I • See review of programming model by GGF APM RG • Chapter 21 of Wiley Book and web page • Handling of the basic components of a distributed computing system – files, computing and data resources, programs, and accounts. • The GCE will typically interface with an environment like Globus or a batch scheduler like PBS to actually handle the back-end resources. • However the GCE will present the user interfaces to handle these resources. • We can follow the lead of UNIX (and Legion) and define a basic GCEShell providing access to the core distributed computing functions. • JXTA also builds some Grid-like capabilities with a modification of UNIX shell model. • GCEShell can have a command line or more visually appealing graphical user interface.

7. Implications of 3-Tier Model • The 3-tier model implies that any given capability (say run a matrix inversion program) can appear at multiple levels. • Maybe there is a backend parallel computer running an MPI job; this is front-ended perhaps as a service by some middle-tier component running on a totally different computer, which could even be in a different security domain. • One can “interact” with this service at either level; a high performance I/O transfer at the parallel computing level and/or by a slower middle-tier protocol like SOAP at the service level. • These two (or more) calls (component interactions) can represent different functions or the middle tier call can be coupled with a high performance mirror. • Typically the middle tier provides control and the back end “raw data transfer”.

8. Application Metadata Actual Application Database The fuzzy definition of Grid Computing Environments UserServices Aggregation Portal GridComputingEnvironments Portal Services PortalServices Application Service Application Service OGSA (OGSI) Interfaces “Core”Grid SystemServices SystemServices SystemServices Raw (HPC) Resources

9. Technology for building GCE Systems • CoG Kits for Java Perl Python CORBA …. • GPDK extending CoG kits with JavaBean and JSP middleware • Grid Portal Toolkit • Other such interfaces with also C and XML tools • Event Service • Simulations

10. GCEShell Portals and PSE’s • One can divide GCE capabilities into generic (copy file) or domain specific (generate the multigrid solver suitable to simulate Middle Earth) • Correspondingly one finds portal frameworks or GCEShell Portals presenting general capabilities • Grid or web-based Problem Solving Environments optimized for a particular domain • PSE’s are often (and perhaps should be) built on top of GCFShell Portals

11. Application Services Core Middleware Services Interface Middleware withbackend resources Globus GT2 ServicesManage backend resources Typical Layered Architecture User Interface-Client Aggregate component InterfacesJetspeed Problem SolvingEnvironments GCEShell Portals

12. GCE Tools • Data Management • File manipulation in all tiers including client, middleware and back end • Security • In all tiers and providing interfaces to core Grid Security • Workflow or “Programming the Grid” • Grid versions of MPI • Higher-level tools include visualization or support of parameter sweep applications • UNIX Shell has mix of lower level and higher level tools

13. More on Programming Models • Basic 2-level programming model is assumed by most projects • First you use classic (parallel) programming to produce “simulation” nuggets (wrapped as application web services) • JDBC / OGSA-DAI to package data resources as objects or services • Then you need to compose (orchestrate) the control and dataflow between nuggets • Many different models for how this is to be done and can call this workflow – Next thrust of GCE RG? • Some work on optimization between levels as in GrADS project

14. Research in “2-level” Programming Models • Basic user interface to middle tier proxies controlling backend (software) resources • Component Models like ICENI (Imperial College) or DoE Common Component Architecture • Commercial workflow engines as in BPEL4WS • Scripting front-ends as in Matlab or Python • Network server model as in NetSolve or Ninf • Computational Economies • Semantic Grid technologies (ontology based integration of resources) as in MyGrid or DiscoveryNet • Agents?

15. Programming Infrastructure (Hosting Environment) • Different approachs assume different run-time (implementation) and user programming “ether” • BPEL4WS thinks about specifying interactions between components; Matlab interface invokes components from a script • Java Interface to OGSI WG emphasizes that we don’t have an established implementation even if you pick a language • Jini • JXTA • Servlet • Enterprise JavaBeans EJB • Or perhaps some future SJB (Scientific JavaBean) supporting high volume dataflow better than EJB (short transactions)

16. WSDL W S Application orContent source R P Web Service Web Services as a Portlet Application as a WSGeneral Application PortsInterface with other WebServices • EachWeb Service naturally has a user interface specified as “just another port” • Customizable for universal access • This gives each Web Service a Portlet view specified (in XML as always) by WSRP (Web services for Remote Portals) • So component model for resources “automatically” gives a component model for user interfaces • When you build your application, you define portletat same time User Face ofWeb ServiceWSRP Ports define WS as a Portlet Web Services have other ports (Grid Service) to be OGSI compliant

17. Online Knowledge Center built from Portlets A set of UIComponents • Web Services provide a component model for the middleware (see large “common component architecture” effort in Dept. of Energy) • Should match each WSDL component with a corresponding user interface component • Thus one “must use” a component model for the portal with again an XML specification (portalML) of portal component

18. Other WS Resource orService-facingPorts User-facingPorts PortalAggregateWS-User Facing Fragments WSDL R U F F Content Provider I I O O Web Service Other WSUser FacingPorts Render User Facing Ports for Web Service

19. SelectionView WSDL R U Customized View Application orContent source F F Filter I I O Control Channel O Selector Web Service Portal(Aggregator) Customized View UserProfile Control Channel Render CustomizedUser-FacingPorts As used in Universal Access

21. HTML Jetspeed Architecture Turbine Servlet JSP template ECS Root to HTML Screen Manager PSML ECS PortletController PortletController ECS ECS ECS PortletControl ECS ECS ECS ECS ECS Portlet Portlet Portlet Portlet Portlet Portlets HTML Local files JSP or VM Local templates WebPage Remote HTML Portlets User implemented using Portal API XML RSS, OCS, or other Local or remote Data

22. Portlets and Portal Stacks • User interfaces to Portal services (Code Submission, Job Monitoring, File Management for Host X) are all managed asportlets. • Users, administrators can customize their portal interfaces to just precisely the services they want. Aggregation Portals (Jetspeed) User facing Web Service Ports Message Security, Information Services Application Grid Web Services Core Grid Services

23. 4 available portletslinking to Web ServicesI choose two Jetspeed Computing Portal: Choose Portlets

24. Choose Portlet Layout Choose 1-column Layout Original 2-column Layout

25. File management Tabs indicate available portlet interfaces. Lists user files on selected host, noahsark. File operations include Upload, download, Copy, rename, crossload

26. Sample page with several portlets: proxy credential manager, submission, monitoring

27. Administer Grid Portal Provide information about application and host parameters Select application to edit