1 / 10

New Development Efforts in GridFTP

New Development Efforts in GridFTP. Raj Kettimuthu Math & Computer Science Division, Argonne National Laboratory, Argonne, IL 60439, U.S.A. New Features. Pipelining the file transfer Better throughput for lots of small files transfer Alternate security mechanisms GridFTP over SSH

nash-mclaughlin
Download Presentation

New Development Efforts in GridFTP

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. New Development Efforts in GridFTP Raj Kettimuthu Math & Computer Science Division, Argonne National Laboratory, Argonne, IL 60439, U.S.A.

  2. New Features • Pipelining the file transfer • Better throughput for lots of small files transfer • Alternate security mechanisms • GridFTP over SSH • Add functionality to globus-url-copy and globus-gridftp-server allowing the control channel connection to be formed with an ssh session. • No proxies, certs, or CAs are needed for a secure control channel. • If you can ssh to the host, you can establish a secure GridFTP session.

  3. Pipelining

  4. Pipelining

  5. Disk Special Device Configurable Protocol Stack Network Protocol Network Protocol Driver Network Protocol Globus XIO Driver Application Driver

  6. GridFTP over UDT • Easy way to write XIO Drivers • Create from third party libraries. • Blocking API • Thread pooling/event call-backs to morph async to sync • Recommend threaded builds • Created an UDT driver

  7. GridFTP over Lightpaths • To minimize changes to the application, this can be implemented as an XIO driver • Users might want to use more aggressive transport protocols like RBUDP, UDT etc • XIO stack used by GridFTP is configurable OSCARS CHEETAH DRAGON GSI CTCP RBUDP UDT

  8. Other New Features • Working with the DRAGON group to create an XIO driver • Data Storage Interface • Several can be available and loaded dynamically via the ERET/ESTO commands • Anything that can implement the interface can be accessed via the GridFTP protocol • We have implemented • POSIX file (used for performance testing) • HPSS (tape system; IBM) • Storage Resource Broker (SRB; SDSC) • NeST (disk space reservation; UWis/Condor)

  9. Data sourceor sink Data Storage Interface Data Processing Module Data Channel Protocol Module Data channel Data Transfer Process

  10. Data Storage Interface • This is a very powerful abstraction • Several can be available and loaded dynamically via the ERET/ESTO commands • Anything that can implement the interface can be accessed via the GridFTP protocol • We have implemented • POSIX file (used for performance testing) • HPSS (tape system; IBM) • Storage Resource Broker (SRB; SDSC) • NeST (disk space reservation; UWis/Condor)

More Related