Web Services for Grid-Enabled Problem-Solving Environments

1. Web Services for Grid-Enabled Problem-Solving Environments David W. Walker Department of Computer Science Cardiff University http://www.cs.cf.ac.uk/User/David.W.Walker/

2. Some History • PSE project involving Cardiff (Walker,Rana), Southampton (Hey, PAC) and BAE Systems looked at (Oct 1998 – Jan 2002): • Component-based application composition • XML-based component models • Scheduling components in a distributed environment (using Intrepid) • Componentization of “legacy code” • This work is now being re-cast in terms of Grid Services, specifically OGSA as it develops. david@cs.cf.ac.uk

3. The Grid “Grid” is not an acronym, so please don’t write “GRID” • Global Resources Inherently Distributed • Grant Reviews Invariably Disappointing • Generally Results In Disaster • Great Research Involving David david@cs.cf.ac.uk

4. Recent Trends • The Grid. Use distributed hardware and software infrastructure to provide reliable, pervasive, and inexpensive access to computational resources irrespective of their physical location or access point. • Application Service Provider. Provide or sell computational services via web interface. • Virtual Organization. A group of people or institutions with some common purpose or interest that need to share (computer) resources to further their objectives. david@cs.cf.ac.uk

5. Recent Software Technologies • Jini. Java-based infrastructure for distributed service provision. • XML. Standard way of storing structure data in a text file. • XSLT. High level data transformation language, often used to transform XML files. • Semantic Web. Will give information a well-defined meaning to better enable computers to understand the content of documents, and thereby allow people, agents, and services to work together. david@cs.cf.ac.uk

6. The Grid • Treat CPU cycles and software like commodities. • Enable the coordinated use of geographically distributed resources – in the absence of central control and existing trust relationships. • Computing power is produced much like utilities such as power and water are produced for consumers. • Users will have access to “power” on demand • “When the Network is as fast as the computer’s internal links, the machine disintegrates across the Net into a set of special purpose appliances” • Gilder Technology Report June 2000 This slide is courtesy of Professor Jack Dongarra david@cs.cf.ac.uk

7. The Transparent Grid • Transparency: The user is not aware (and doesn’t care) what computing resources are used to solve their problem • Similarly, in an electrical grid we ignore the source of the power • Heterogeneity • Resource discovery • Scheduling Distributed computing issues david@cs.cf.ac.uk

8. The Pervasive Grid • Pervasive: The Grid can be accessed from any networked device, eg, laptop, mobile phone, PDA, etc. • In electrical analogy, any appliance can access power through a standard interface, eg, a wall socket. • Standard interfaces • Protocols • Legacy software david@cs.cf.ac.uk

9. Other Grid Issues • Security: When resources are shared across organisation boundaries security is an important issue. • Dependability: The Grid must be robust and resilient to failure. • Efficiency: Resources should not be wasted, good load balancing needed. • Cost: For broad impact The Grid should be inexpensive. • Portability: Grid applications should be able to run on a wide range of hardware. david@cs.cf.ac.uk

10. Views of the Grid • Virtual Organisations form “mini-Grids” for specific purposes. Membership of VO is constrained, and resources in VO are controlled and managed. • The Consumer Grid. Services and resources can be anywhere. Important issues of dynamic resource discovery, trust, and digital reputation. david@cs.cf.ac.uk

11. Application Service Providers • ASPs provide remote services such as compute cycles, specific applications, or storage. • Selective outsourcing: certain functions are performed remotely. • Application hosting: remote sites act as application servers. • Browser-based computing: online applications accessible through web site. david@cs.cf.ac.uk

12. ASP User Benefits • Ability to choose and combine services available online • No installation worries • No compatibility issues • No support overhead • Reduced downtime david@cs.cf.ac.uk

13. Service Providers and Brokers • NetSolve is an example of an ASP providing numerical software. • Trend is towards network-based computing paradigm. • Nodes offer different sets of computing services with known advertised interfaces. • Software is increasingly seen as a “pay-as-you-go” service rather than a product that you buy once computational economies. • Web services important in architecture of the Grid Open Grid Services Architecture david@cs.cf.ac.uk

14. Web and Grid Servcies • A web service (WS) is a network-enabled entity that provides some capability through the exchange of messages • WS is a platform and implementation independent software component • WS being standardised through W3C • WS is described using WSDL (operations and bindings) • WS offers a set of operations via its ports • Grid Services extend WS for e-Science david@cs.cf.ac.uk

15. Web Service Characteristics A Web Service can be: • Described using WSDL • Published to a registry of services (UDDI) • Discovered through standard mechanisms • Invoked (remotely) through a declared API • Composed with other services david@cs.cf.ac.uk

16. PSE Definition Gallopoulos et al. (1991): A PSE is “a computer system that provides all the computational facilities necessary to solve a target class of problems.” • Note that this definition: • Pre-dates the Web • Ties a particular PSE to a particular class of problems david@cs.cf.ac.uk

17. Vision for PSEs • PSEs herald a new era in scientific computing, both in power and how resources are accessed and used • PSEs will become the main gateway for scientists to access terascale computing resources. • PSEs will allow users to access these resources from any web connection. • PSE’s support for collaborative computational science will change the pervading research culture, making it more open and accountable. david@cs.cf.ac.uk

18. Aspects of PSEs • Distributed • Collaborative because interesting problems are often complex and draw on many types of expertise. • Heterogeneity pervades network environments • Transparency is usually desirable • Intelligence is needed to deal with various types of complexity – especially complexity arising from the collaborative and distributed use of PSEs in heterogeneous dynamic environments. david@cs.cf.ac.uk

19. Problem-Solving Environments • We can view a PSE as an environment through which end-users of a Virtual Organization exploit Grid resources. • PSEs are application specific. • PSEs accessed through the Web/Grid are the same as “Application Portals.” david@cs.cf.ac.uk

20. Intelligence in PSEs • Collaborative and distributed computing adds new types of complexity to the software environment. • PSEs need to be intelligent to minimize impact of this complexity on users. david@cs.cf.ac.uk

21. Who is Involved in PSEs? • Application end users (scientists,engineers,etc.) • Solve a particular problem which is domain specific • Undertake “what if” investigations • Developers (programmers) • Create components and place them in Component Repository • Make new algorithms and techniques available to the end users. • Software infrastructure builders • Create services and interfaces • Provide abstractions • Develop standards david@cs.cf.ac.uk

22. PSEs and the Grid • PSEs share many characteristics of the Grid. • When Cardiff research into distributed PSEs started there was no Grid infrastructure. • Aim now is to build PSEs on top of generic Grid services (e.g. OGSA) • Many companies interested in web services: Sun (Jini, JXTA), IBM (Web Sphere), Microsoft (.NET), etc. david@cs.cf.ac.uk

23. Service-Based PSEs PSEs • Domain-specific intelligence • Problem specification tools • Recommender systems Applications and supporting tools (toolkits, Grid-enabled libraries,etc) Application development and execution support services (CORBA, Globus, Jini, etc) Generic web services (monitoring, security, brokering, fault management, etc) Distributed resources (computers, storage, instruments, sensors, visualisation, etc) david@cs.cf.ac.uk

24. PSEs: Problem Specification and Solution • Use either • Visual programming environment to link software components • High-level language specification • Recommender systems can be used to help user choose best way to solve problem and locate software. david@cs.cf.ac.uk

25. Software Components • Components are specified in terms of their input and output interfaces. • User doesn’t need to know about the internal details of components. • Different components can be scheduled on different resources. • Components can be plugged together within a visual editor. david@cs.cf.ac.uk

26. Collaborative Code Development Environment • The collaborative code development environment uses a visual programming tool for seamlessly integrating code from multiple sources. • Applications are created by plugging together software components. • Different developers can place their components in a shared repository. • Legacy codes in any major scientific programming language can be handled. david@cs.cf.ac.uk

27. Plug and Play Components • Can link the output of one component to the input of another. • Store components in a repository. See Triana for example http://triana.co.uk/ david@cs.cf.ac.uk

28. Two Views of Components • A component is an executable that runs on a certain specified machine. • A component can be viewed as a contract. It says: “If you give me these inputs then I’ll give you these outputs.” In the second case the component is not tied to any particular executable. Problem specification is separate from service provision. david@cs.cf.ac.uk

29. Binding Service Requests to Resources • In a fully transparent system the scheduler would decide where components execute based on: • Availability and performance of resources • Cost and time constraints • This is a hard problem. • Possible solution is to supply “hints” about where it can run in a component’s XML specification. david@cs.cf.ac.uk

30. Legacy Code • Want to automatically componentize legacy software. • Especially large libraries of scientific and numerical software. • Typical approach is to “wrap” each piece of code for use as a component. david@cs.cf.ac.uk

31. JACAW • JAva-CAutomatic Wrapper • JACAW automatically generates the JNI interface for making C native method calls from Java. • Uses the C header files. • Complete source code is not needed. david@cs.cf.ac.uk

32. Legacy Issues Addressed • Expect better performance through use of C (compared with pure Java). • But expect some overhead associated with the JNI interface due to copying of arguments. • Makes use of existing validated code. • JACAW can be applied automatically to large libraries of C software. david@cs.cf.ac.uk

33. JACAW Example • Suppose the header file util.h contains: signed int rand (); void bubblesort (int, int[]); • JACAW builds class files for the two routines and the JNI interface code for them. david@cs.cf.ac.uk

34. What JACAW Does utilJ_bubblesortM.java Create Java files utilJ_randM.java utilJ.java utilJ_bubblesortM.class Compile Java files utilJ_randM.class util.h utilJ.class Create the header file utilJ.h Create the C file utilJCall.c Build the library libutil.so david@cs.cf.ac.uk

35. Software Technologies for PSEs Wherever possible use accepted software standards and component-based software engineering. • XML is used for interface specification and defining the component model. • Java is used for platform-independent programming. • Transparent interaction between distributed resources is provided by technologies such as CORBA, Globus, and OGSA (in the future) • Agents for user support, resource monitoring and discovery. david@cs.cf.ac.uk

36. Final Remarks • PSEs provide an integrated computational environment for solving problems in a particular domain. • PSEs provide problem solving power without the end user needing to be an expert in the technologies used to effect the solution. • PSEs will be implemented on top of Grid services • How will Grid services be implemented? david@cs.cf.ac.uk

37. More Final Remarks • Collaboration, intelligence, and visualization are important aspects of PSEs. • Components are a good way of encapsulating software services. • XML is an important part of the software infrastructure of PSEs. • Resource/service discovery and scheduling in Grid environments are critical areas for further research. • Semantic Web may provide basis for future PSEs. david@cs.cf.ac.uk