1 / 19

GridTorrent.Framework: High-performance Data Transfer and Sharing for Scientific Computing

GridTorrent.Framework is a high-performance framework for efficient data transfer and sharing in scientific computing. It utilizes P2P (Bittorrent) network structure to make collaboration and data management easy. Tests conducted in LAN and WAN networks show improved bandwidth utilization compared to regular TCP. Overhead due to parallel connections is present but is manageable.

matte
Download Presentation

GridTorrent.Framework: High-performance Data Transfer and Sharing for Scientific Computing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. GridTorrent Framework: A High-performance Data Transfer and Data Sharing Framework for Scientific Computing

  2. Outline • Introduction • Motivation • PTCP • GridTorrent Framework • Test Results • LAN Test Results • WAN Test Results • Overhead • Questions

  3. Introduction • Today’s computational science is data-intensive • Large Hadron Collider (LHC) experiment at CERN generates petabytes of data • Accessibility, replication and creation of the data are made very easy by • Internet • Computational Grid

  4. Motivation • The data is geographically distributed • Users are dispersed • Collaboration environments are required • Resources should be used in efficient and effective way • CPU • Storage • Network

  5. PTCP • TCP has a performance problem over WANs • It was developed to solve the above problem by using striping technique

  6. GridTorrent Framework • It is aimed • to provide collaboration environment for dispersed users • to make data transfer, management, and sharing easy via content manager • to use systems resources efficiently and effectively by harnessing P2P (Bittorrent) network structure

  7. Collaboration & Content Manager • The Content Manager allows users to publish or share their files with selected access control rights • The Collaboration Manager permits users to build a virtual sharing environment by managing working groups or friend list • ACL enforce access control rights for a given content • Task Manager handles the users’ task list

  8. GridTorrent Client • It is responsible for • initiating actual data publishing • data sharing with other GTF clients • ensuring secure environment for the above activities

  9. WS-Tracker • WS-Tracker is a WS enabled server • It assists in the communication between peers (GridTorrent clients) • It delivers task lists which is generated by users to GridTorrent clients • It supplies ACL of each shared file to GridTorrent clients

  10. Experimental Results • PTCP and GridTorrent Framework tests cases were conducted both in LAN and WAN type of computer networks • Server and client machines’ specification and location table • File size is 300 MB

  11. LAN Test Setup Server is located at Bloomington, IN Client is at Indianapolis, IN The number of parallel TCP streams between server and client has increased from 1 to 16 (PTCP) The number of seeders increased from 1 to 16 (GTF) Client and server configuration for PTCP GridTorrent test case configuration for LAN test. Regular Java sockets are used for data transfer.

  12. LAN Test Result • There is no significant improvement in bandwidth for both PTCP and GTF • Experimental data transfer (80-100 Mbps) rate is much lower the theoretical (1000Mbps) and measured data transfer rate (857Mbps)

  13. WAN Test-I Setup Server is located at Bloomington, IN Client is at Tallahassee, FL The number of parallel TCP streams between server and client has increased from 1 to 16 (PTCP) The number of seeders increased from 1 to 16 (GTF) Client and server layout for PTCP test case GridTorrent test case configuration for wide area network test. Regular Java sockets are used for data transfer.

  14. WAN Test-I Result • Bandwidth usage is vastly improved in both GTF and PTCP • PTCP’s bandwidth utilization rate has risen steadily until fifteen streams • its peak value is 118 Mbps • GTF’s bandwidth utilization rate has risen steadily until thirteen streams • GridTorrent is performing better than PTCP when parallel streams number is less than five

  15. WAN Test-II Setup • Server is located at Bloomington, IN • Client is at Tallahassee, FL • The number of parallel TCP streams between server and client has increased from 1 to 16 (PTCP) • Besides Java socket, other data transfer protocols can be exploited in GridTorrent client • The number of seeders increased from 1 to 16 (GTF) • Four parallel TCP sockets were used between peer and seeders Client and server layout for PTCP test case GridTorrent test case configuration for wide area network test. GridTorrent client uses four parallel TCP sockets in each connection for every source

  16. WAN Test-II Result • Using parallel TCP with Bittorrent algorithm demonstrates much better bandwidth usage than standalone GridTorrent and PTCP • The maximum attained bandwidth is around 145 Mbps which is %23 higher than PTCP’s result

  17. Overhead • Both parallel TCP and GridTorrent have overhead due to nature of multiple parallel connections • PTCP’s communication channel overhead time can be compared to GridTorrent WS-Tracker client’s overhead time varying between 300 and 600 milliseconds • Another overhead of GridTorrent is that control messages exchanged between peers to ensure strictly enforced to all participating peers • The total size of overhead messages is between 148KB to 169 KB

  18. Questions

  19. Thanks to All.

More Related