1 / 20

Latest news on JXTA and JuxMem-C/DIET

Latest news on JXTA and JuxMem-C/DIET. Mathieu Jan. GDS meeting, Rennes, 11 march 2005. Outline. Performance evaluation of JXTA in a grid context Overview of JuxMem-C Overview of initial use of JuxMem by DIET Roadmap. Enabling JXTA for High Performance Grid Computing.

katoka
Download Presentation

Latest news on JXTA and JuxMem-C/DIET

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Latest news on JXTA and JuxMem-C/DIET Mathieu Jan GDS meeting, Rennes, 11 march 2005

  2. Outline • Performance evaluation of JXTA in a grid context • Overview of JuxMem-C • Overview of initial use of JuxMem by DIET • Roadmap

  3. Enabling JXTA for High Performance Grid Computing • Context: convergence of P2P and grid computing • Grid over P2P • New hypothesis for P2P systems • Efficiently use SANs and WANs • Performances of JXTA are not clear • Papers by Emir Halepovic • No in-depth explanations about the results • No precise details about the experimental setup (LAN, WAN) • Complex topologies (relays, etc): what is benchmarked? • Inconsistent results (Emir Halepovic, P3 project)

  4. Description of the Experimental Setup • Bidirectional bandwidth benchmark • JXTA • JXTA-J2SE 2.2.1 and 2.3.2 • JXTA-C (HEAD, 18 january) • Configured with TCP, direct communications • SANs benchmarks • Myrinet & Giga Ethernet (parasol & paraci) • WANs benchmarks • Grid 5000 (Rennes & Toulouse) • Delay = 11.2 ms / Bandwidth = 1Gb/s • Tuned TCP buffer sizes • Direct communications: unusual for P2P systems

  5. Endpoint service Pipe service JXTA Socket Endpoint service Pipe service JXTA Socket JXTA communications layers • - Data-stream interface • Reliability - Dynamic point-to-point communications TCP, HTTP, etc - Static point-to-point communications - Independant from underlaying network topology - Unreliable

  6. Bandwidth of JXTA-J2SE 2.3.2 over SANs Bandwidh of JXTA-J2SE 2.2.1: 145MB/s (endpoint service)

  7. Latency of JXTA over SANs

  8. Bandwidth of JXTA-C over SANs

  9. Fully exploiting SANs capacities: PadicoTM • Myrinet: OS-bypass mode • 2 Gb/s and 7μs • PadicoTM • High-performance framework for multithreading and networking • Virtual sockets • JXTA-C on top of PadicoTM • Modifications inside Apache Portable Layer (thread library) • Measurement bandwidth for the endpoint layer: 140 MB/s • Improvements by only 32 MB/s • Memory copies inside JXTA-C

  10. Bandwidth of JXTA-J2SE 2.3.2 over WANs Similar results for JXTA-J2SE 2.2.1

  11. Bandwidth of JXTA-C over WANs

  12. Discussion • JXTA-J2SE 2.2.1 nearly saturates SANs • Endpoint and pipe layer • Improved results for JXTA Sockets can be obtained by tuning output buffer size • Bandwidth degradation between JXTA-J2SE 2.2.1 and 2.3.2 • Latency improvements with JXTA-J2SE 2.3.2 • Memory copies inside JXTA-C • Prevents an efficient use of PadicoTM • Transparent use of features available in PadicoTM • Parallel streams • On-the-fly compression techniques

  13. New design of JuxMem • Goal: hide the complexity of JXTA to upper layers • Upper layers are consitency protocols • New layers: communication, search, memory and failure detector • Improved semantic for search layer • Based of JXTA discovery mechanism • JuxMem comm layer hides the JXTA comm layer used • Currently based on JXTA pipe service • Other implementations in progress • Implemented by JuxMem-J2SE and JuxMem-C • JXTA-J2SE 2.3.2 • JXTA-C HEAD + small patch

  14. Current status of JuxMem-C • JXTA service • Private deployment using own groups • Own net peer group (juxmem group) • Communication layer is working • Between JuxMem-C peers • Between JuxMem-C peers and JuxMem-J2SE peers • Search layer working only at a cluster level • JuxMem client is working!

  15. Roadmap of JuxMem-C • Search layer for alloc at the juxmem group level • Memory layer • Other types of JuxMem’s peer • Provider • Cluster manager (JXTA-C rdv peers are available!) • Data types in JuxMem • Wrapping file API access • etc • Consistency protocols :-)

  16. Current status of JuxMem-C/DIET • Simple targeted example: dmat_manips • A DIET SeD launches a JuxMem-C client (-ljuxmem) • The JuxMem-C client connects to JuxMem-J2SE cluster manager • The DIET SeD gets a reference on the JuxMem service • That’s all!

  17. Modifications inside DIET • C++ wrapper of JuxMem-C API (JuxMemImpl.[cc|hh]) • JuxMem API (JuxMemRead, JuxMemWrite, etc) • run() • Launches JXTA-C • Gets a reference on the JuxMem service • Modifications inside DIET_server.cc • Creates a JuxMemImpl object • Calls the run method • Calls the linkToJuxMem method • Modifications inside SeDImpl.[cc|hh] • JuxMemImpl attribute • linkToJuxMem • Update the reference on JuxMemImpl

  18. When DIET requests data to JuxMem • When a solve request is received (SeDImpl.cc) • Before the computation • After the computation • Currently for matrices only • diet_is_persistent_juxmem (macro) • JuxMemRead before the computation • Read the data blocks • Profile.parameters[i].desc.id != NULL • JuxMemMap after the computation • Allocate a data block • Profile.parameters[i].desc.id == NULL • JuxMemWrite after the computation • Always

  19. Possible questions for DIET experts • Enable trace levels • RTFM not yet done on this topic :-) • Cleaner configuration of DIET for JuxMem • For now --enable-JuxMem • Hardcoded paths • Any other methods needed by DIET? • Estimate data transfer time? • Other needed modifications inside DIET? • Other?

  20. Roadmap of JuxMem-C/DIET • dmat_manip example with JuxMem-C • Basic configuration • Demo in one month • Test JuxMem-C/DIET with consistency protocols • More complex JuxMem configurations • Cleanup in DIET for commit • JuxMem’s license • Release of JuxMem

More Related