A Scalable Virtual Registry Service for jGMA

1. A Scalable Virtual Registry Service for jGMA Matthew Grove CCGRID 2005 - WIPMay 2005

2. Outline • Introduction and motivation, • The jGMA architecture, • The Virtual Registry (VR), • An implementation of the VR using IRC, • Future work, • Summary.

3. Introduction • jGMA is a pure Java reference implementation of the GGF's Grid Monitoring Architecture, initially designed to be a “standards-based” messaging layer to bind together the global layer of a project called GridRM, • jGMA requirements: • Compliant to the GMA specification, • Supports both non-blocking and blocking events, • Work locally over a LAN or a wide-area, such as the Internet, • Fast, and have a minimal impact on its hosts, • Scalable, • An integrated registry service, • Able to work through firewalls.

4. Design Philosophy • Reuse existing software components, if possible, rather than reinvent functionality, • Small and well defined API, • Minimal number of other installation dependencies, • Simple to install, configure and use, • Provide a pluggable infrastructure with a choice to extend core functionality with a further more-sophisticated component.

5. jGMA Implementation • jGMA has the following components: • A producer and consumer (shared code-base), • A Mediator which provides WAN access to producers and consumers and hooks for a distributed Virtual Registry, and debugging/monitoring. • jGMA features: • Written in Java, so takes advantage of usual features and libraries, • Small API (<15 calls), • Released as single “jar” of currently 28Kbytes, • Limited dependencies: • Java.

6. jGMA Architecture

7. The Mediator The Mediator replaces the PC and Registry servlets.

8. The jGMA Library and API A choice local registries, two have been implemented: Java data structure and MySQL.

9. Virtual Registry Requirements • To complete the implementation jGMA requires a registry component which must: • Be scalable, • Store sufficient information to be GMA compliant, • Be secure, and prevent unauthorised access to the data, • Need minimal configuration, • Ideally have no single point of failure, • Be robust and tolerant to failures, • Efficient query routing between VRs.

10. Virtual Registry Services • Core Services: • Boot-strapping: Joining the jGMA network with minimal hardwiring, • Communications: Efficient routing of queries between VR peers, • Caching: Keeping a temporary local copy of some information to reduce the amount of communications between peers. • The API is designed to allow different implementations of these core services to be mixed and matched.

11. IRC VR Service • The IRC VR service is an attempt at meeting the criteria of the jGMA VR by building a service which uses the mature Internet Relay Chat protocol, • IRC networks have servers connected via a graph topology (attempting to efficiently route messages and provide fault tolerance), some networks manage thousands of users. • Fault tolerant boot-strapping via DNS, • Various degrees of security can be provided by combinations of: • Leveraging functionality built into the IRC protocol (locking the channel, private messages), • Using features of the IRC daemons such as encrypted routing for IRC messages, • Running your own IRC daemons rather than using a public network.

12. IRC VR Service Continued • Each Mediator has an “IRC Bot” which connects to a chosen IRC network: • The Bot joins a channel and announces itself to other Bots, • The Bot publishes search queries to the IRC channel, • Other Bots receive the queries and pass them onto the Registry backend, • The Bots reply to the queries via a private message over IRC. • All messages are routed by the IRC network.

13. IRC VR Schematic

14. IRC VR Issues / Questions • Currently IRC channels are used to partition the VR by project, • There needs to be other ways to reduce the impact of a ‘flood query’ on the network: • What are the limits on individual channels, is there a need to have multiple channels and route messages between them? • Using IRC requires an extra port be accessible to the mediator (out going port 6667).

15. Current Status • LAN (socket) and WAN (HTTP) layers complete, • The initial implementation of the IRC VR is complete and functional, • Demo applications and benchmarking has thoroughly tested the jGMA software, • Permanent on-line demo: http://dsg.port.ac.uk/projects/jGMA/demos/web/

16. Webcam Browser Demo http://dsg.port.ac.uk/projects/jGMA/demos/web/

17. Future Work • Short term: • Implement a generic caching service for the VR, • Write a simple P2P VR, which can be used in place of the IRC VR, • Add hooks for debugging, • Benchmark jGMA against Naradabroker. • Longer term: • Provide a updated binary release of jGMA, • Develop an application or library that uses jGMA.

18. Summary • jGMA is functional despite a few missing features, • Current work is focusing on the implementation of the VR, • The design of the jGMA architecture is complete.

19. Acknowledgements • University of Portsmouth, • Members of the DSG: • Mark Baker, • Hong Ong, • Garry Smith, • Mr Boe-akis, • Aamir Shafi, • Helen Xaing.

20. Links • Project Web page: • http://dsg.port.ac.uk/projects/jGMA/ • The DSG Web page: • http://dsg.port.ac.uk/