Grid Federation JXTA Jini etc.

1. GGF7 Tokyo March 5 2003 Grid FederationJXTA Jini etc. PTLIU Laboratory for Community Grids Geoffrey Fox Computer Science, Informatics, Physics Indiana University, Bloomington IN 47404http://www.naradabrokering.org http://grids.ucs.indiana.edu/ptliupages gcf@indiana.edu uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

2. AWS AWS AWS AWS Application/User Framework supporting development and deployment of OGSI compliant AWS (Application Web Services) Database Users Portal such as “Jetspeed” Hosting Environment Hosting Environment GridComputing or ProgrammingEnvironments Generic Application Services Web Services “Core”Grid “Core” System Services Resource Grid Services e.g. DAI compliantdatabase Resources uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

3. Standards Compliant InterGrids Federation Environment JXTA AWS1 Platform AWS3 Service1 GT3 Grid Avaki Grid IBM Grid Jini Grid AWS2 Resource1 Resource2 Service2 Federation/Interoperability Problem? • Have a collection of Web Services running in Grids defined by different suppliers? • Interoperability – “particular application Web Service of supplier X” can utilize “core service of supplier Y” • Federation– “core service of supplier X” can be integrated with “core service of supplier Y” to provide a integration/amalgam that is also a realization of core service. Need mediation to link different Grid Islands uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

4. OGSI Interface Implications • OGSI defines the features that all Web Services must have to be called Grid Services • Using Web Services is equivalent roughly (in old-style computing lingo) to agreeing on a common (object-oriented) language • It is equivalent to choosing C++ or Java or more precisely agreeing to use CORBA IDL or Java to define external interfaces (method calls) without prejudice as to implementation language • For example, having a “Grid Information Port” is VERY roughly analogous to agreeing that all programs have a standard input and a standard output familiar from UNIX • Agreeing that “Grid Information Ports” should communicate XML records (SDE Service Data Element) is roughly equivalent to saying standard output will use Fortan namelist syntax • So this is not yet a complete interoperability or federation specification uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

5. InterGrids • Develop Grid software including both the packaging of Grid Islands and federation between Grids • Key underlying principles: • Support small Grids (university departments, Private Grids) with easy deployment (e.g. JXTA Jini …) • Support composability (P2P) and federation with itself and other Grids • Use existing (Avaki, Globus JXTA ICENI ..) bits and pieces if possible – encourage such projects to produce modules useable in other Grid systems like InterGrids • Good test/benchmark/tutorial material and good support uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

6. Features of InterGrids I • At outer levels of Grid, just need interoperable services but these services would need mediation as they access resources in a different grid • Interoperable Services have interfaces allowing them to access appropriate other services in their own grid and through mediation the analogous services in other grids • At inner levels of Grid, need to federate services • Federated services produce results which need to be merged with other services of the same type when Grids are joined together • In case of Jini based Grid, we keep Jini services as Jini and don’t wrap them • Rather the OGSA wrapping of Jini occurs in the mediator • Issues if multiple Grids overlap • Can multiple brokers manage same resources (schedulers on computers) • What happens if one shares resources between virtual organizations (Grids) uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

7. Features of InterGrids II • Mediation service integrates • VPN with adaptable port • Firewalls and security negotiation between domains • Mediation • Difference between PURL/URI and URL • Mediation Service is distributed i.e. is not a single proxy server • Each logical message (this could be several physical messages) in system is examined – destination URI is “just” mapped to URL if internal • If external routed to “best mediator” in same way NaradaBrokering optimally routes messages • Logical messages are self contained i.e. have enough information to fully specify any transformations needed in mediator i.e. they are “all” the Schema instance uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

8. Features of InterGrids III • When message arrives at mediator service, its source and destination Grids are examined • Note Grids must register themselves in mediation service and in that registration (or a later update) give specification for service and any needed transformations) • This registration involves introduction of concept of Grid Gridtype with specification attached to a particular Gridtype • If they have same specification for this service, then message is routed on unchanged • If specification different, the mediator looks to see if any special translations available for given source or destination • If no special actions, the source mapper is used to convert message to FGSA (Federated Grid Service Architecture -- hopefully same as OGSA) and the destination inverse mapper is used to map FGSA to destination format • This can generate an error returned to calling service and logged for both Grids uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

9. Features of InterGrids IV • This mediation is sufficient for an “interoperable” service • Some services such as “search” or “look-up” are registered as federated • i.e. requests to them are multi-cast to multiple grids and results merged • Rules must be defined for federated services defining nature of result • Is more than one answer allowed • If >1 answer possible, then rules for merging results of services in each Grid must be specified • Another complication is if a single service in one Grid corresponds to composition of multiple services in another Grid • One sees this for look-up which could involve different levels of meta-data in different Grids • This seems complicated but relatively clear what one has to do in composing dynamically services uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

10. Features of InterGrids V • There are two problems mentioned earlier • Shared Resource Reference Services – this must be a standard issue in federation. This occurs when federation of say look up services leads to duplicate results. The look-up may not be quite the same and one would want to remove or combine duplicate responses • Shared Resource Access Services – these are services in different Grids that access and affect the same backend resource. This issue comes up even inside a single Grid • It is possible that mediator could be used to resolve this problem but an heuristic needs to be developed for this uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

11. Resource Resource Resource Resource Resource Resource R Grid Instance Grid Instance Grid Instance Grid Instance Service Service Service Service Service Service Service Service M Resource Resource Service Service Service Service M R M R M R M R M R Resource Resource M M Resource Resource R Resource Resource Resource Resource Federation Architecture Routing Node Mediation Node uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

12. NaradaBrokering Publish/Subscribe Distributed Event/Message System • http://www.naradabrokering.org • “MQSeries/JMS” P/S applied to Collaboration, Grid, P2P(JXTA) • Supports UDP, TCP/IP, Firewalls (actual transport  user call) • Used in other projects: Collaboration, Portal and Handheld uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

13. NaradaBrokering • Based on a network of cooperating broker nodes • Cluster based architecture allows system to scale to arbitrary size • Originally designed to provide uniform software multicast to support real-time collaboration linked to publish-subscribe for asynchronous systems. • Now has five major core functions • Message transport (based on performance measurement) in heterogeneous multi-link fashion • General publish-subscribe including JMS & JXTA and support for RTP-based audio/video conferencing • Distributed XML data-base using P/S XPATH metaphor • Filtering for heterogeneous clients • Federation of multiple instances of Grid services as illustrated by JXTA peer-group linkage uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

14. uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

15. uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

16. Narada JXTA Event Request/Response Present if targeted atParticular peer NaradaBrokering and JXTA Narada-JXTA provides JXTA guaranteed long distance delivery uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

17. uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

18. uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

19. uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

20. NBHub Federation of Services • If you have a Service – Notification (as with Grid or JXTA advertisements), Search, Scheduling, Audio-Video conferencing …. • With a standard for client and server components • Then can build a “server” that services all clients • Alternatively can hierarchically consider collection of existing Server-client domains • IBM or Globus OGSA islands • Sun Grid Engine Schedulers • Polycom/Access Grid A/V sessions • Enterprise Search Engines • Federation links servers for each island together • JXTA Search has XML specified federation approach – will try to include and generalize as a NaradaBrokering federation framework • DoD High Level Architecture HLA does this for simulation uri="http://www.naradabrokering.org" email="gcf@indiana.edu"