1 / 27

JuxMem: An Adaptive Supportive Platform for Data Sharing on the Grid

JuxMem: An Adaptive Supportive Platform for Data Sharing on the Grid. Gabriel Antoniu, Luc Bougé, Mathieu Jan IRISA / INRIA & ENS Cachan , France. Grid Data Service (GDS) meeting, Rennes, September 22 th 2003. Context: Data Management on the Grid.

jolene
Download Presentation

JuxMem: An Adaptive Supportive Platform for Data Sharing on the Grid

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. JuxMem: An Adaptive Supportive Platform for Data Sharing on the Grid Gabriel Antoniu, Luc Bougé, Mathieu Jan IRISA / INRIA & ENS Cachan, France Grid Data Service (GDS) meeting, Rennes, September 22th 2003

  2. Context: Data Management on the Grid • Distributed numerical simulations (code coupling) • Needs data sharing • No many functional systems Designing a satellite Solid mechanics Optics Dynamics Thermodynamics

  3. Existing Data Management Systems • Non-transparent large scale data management • GridFTP (Globus) and MPI-IO • Internet Backplane Protocol (IBP) • Explicit transfert • No consistency ensured • Transparent small-scale data management • Distributed shared memory (DSM) • Consistency models and protocols • Transparent access • Small-scale, static and homogeneous architecture TreadMarksTM

  4. Another approache: peer-to-peer systems • Peer-to-peer systems (P2P) • Distributed (large-scale) • Volatile peers • Peers have the same capacities and responsabilities • Sharing of immutable data • Centralized (Napster) • Flooding (Gnutella, KaZaA) • Distributed hash table (CFS, PAST) • Sharing of mutable data • One writer per data + static architecture (OceanStore) • Conflicts have to be manually resolved (Ivy)

  5. Design principles • Proposition: data sharing service for the grid • DSM systems: consistency and transparent access • P2P systems: scalability and high dynamicity

  6. A Data Sharing Service for the Grid Persistent storage Transparency of localization Internet ? Data transfert

  7. A Data Sharing Service for the Grid Optimization of access Data consistency Internet Internet Data transfert

  8. A Data Sharing Service for the Grid Internet Internet Scalability of the architecture

  9. A Data Sharing Service for the Grid Internet Internet Handling volatility

  10. JXTA: a framework for P2P • Open-source platform for programming P2P applications • Specify a set of protocols • A peer • Uniquely identified (ID) • Address independent of the physical location • Several network access point (TCP, HTTP, etc) Peer ID Peer ID Peer ID Peer ID Peer ID Peer ID Peer ID Peer ID Peer Peer Peer TCP/IP Peer Peer Peer Peer Peer Peer Firewall Firewall Peer Peer Peer Peer Peer HTTP

  11. JXTA: peer groups • Set of peers that share a common set of interests • Scope of communications • Specific policy of management • Peer group services NetPeerGroup PeerGroupA Peer ID Peer ID Peer ID Peer ID Peer ID Peer ID Peer ID Peer ID PeerGroupB

  12. JXTA: Advertisements • Every resource is described by an advertisement • Peers • Peer groups • Communication channels • Services • … PeerGroup Advertisement: <?xml version="1.0"?> <!DOCTYPE jxta:PGA> <jxta:PGA> <GID> urn:jxta: uuid- BCBCDEABDBBBABEABBBABA000000 </GID> <MSID> urn:jxta:uuid- BFEFDEDFBABAFRUDBACE00000001 </MSID> <Name> My Group </Name> <Desc> This group is to be used for my own testing </Desc> </jxta:PGA>

  13. JuxMem: a prototype Peer group juxmem Peer group data Peer group cluster A Peer group cluster C Peer group cluster B Logical architecture Physical architecture

  14. API of JuxMem • Alloc (size, attribs) • Map (id, attribs) • Put (id, value) • Get (id) • Lock (id) • Unlock (id)

  15. Managing Memory Resources • Advertisement of type provider: peer group cluster • Advertisement of type cluster: peer group juxmem Peer group cluster Memory provided Peer group juxmem Size 10 Size 10

  16. Managing Shared Data Blocks • Allocation of a memory area = creation of a peer group data • Data blocks identified by the ID of the peer group • Advertisement published in the peer group juxmem • Shared access for clients by knowing the ID • Consistency • Data blocks are replicated on providers • Updated simultaneously (logical multicast) • Clients are not notified of updates • Synchronization • Lock for each data block

  17. Handling the Volatility of Peers • A manager by peer group (cluster and data) • Dynamic monitoring of available peers • Data blocks are automatically replicated (data) • Updates advertisement of type cluster (cluster) • Volatility of managers • Periodic exchange of hearbeats • Dynamic replication of managers if needed on other peers

  18. Implementation of JuxMem • JuxMem • JXTA service • + 5000 lines • Graphical tool

  19. Preliminary Evaluations • Cluster • PentiumII: 450 Mhz and 256 Mb of RAM • Network used: FastEthernet 100 Mb/s • Number of nodes used: 20 • Experiments • Overhead memory consumption • Low: @ 6 % with respect to the underlying JXTA peers used • Study of the volatility of providers

  20. Study of the Volatility of Providers (1) Data of one byte Degree of redundancy = 3 Data manager is not killed 1 client: 100 iterations lock-put-unlock 16 providers Peer group juxmem Peer group data Peer group cluster

  21. Study of the Volatility of Providers (1) Data of one byte Degree of redundancy = 3 Data manager is not killed 1 client: 100 iterations lock-put-unlock 16 providers Peer group juxmem Peer group data Peer group cluster

  22. Study of the Volatility of Providers (2) • Internal lock when replicating • Ensure the consistency of the new replica • Client is blocked Peer group juxmem Peer group data Peer group cluster

  23. Study of the Volatility of Providers (3) • JXTA and Java • Time-out detection adapted for the WAN • Independent of the data size Reconfiguration time @ 11 seconds Targeted volatility is weaker ( >> 80 seconds)

  24. Conclusion • Defines an hierarchical architecture for a data sharing service for the grid • Specific policy for each cluster • Scalability • Storage and transparent access to data blocks • Application level: data block ID • Persistency • Shared consistency • Handling volatility of peers • Actively taken into account

  25. Future Work • Transparent mobility of data blocks • Unavaibility of nodes or even clusters • Affinity data – computations • Affinity data – data • Parameterizable consistency • Specific for each client • Hierarchical synchronization

  26. Allocation Request 6 4 3b 1 2 5 3a 3b 3a

More Related