1 / 18

An alternative to ftp : bbftp

Gilles Farrache In2p3 Computing Center farrache@cc.in2p3.fr. An alternative to ftp : bbftp. History. In 1999, the members of the French Babar collaboration decided to import from SLAC to Lyon a big part of the SLAC objectivity database First approach : via tapes Needs a lot of manpower

mili
Download Presentation

An alternative to ftp : bbftp

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Gilles Farrache In2p3 Computing Center farrache@cc.in2p3.fr An alternative to ftp : bbftp

  2. History • In 1999, the members of the French Babar collaboration decided to import from SLAC to Lyon a big part of the SLAC objectivity database • First approach : via tapes • Needs a lot of manpower • Problems with cartridge compatibility • Delay between data production at SLAC and their availability in Lyon quite long

  3. History • Second approach : via the network using standard ftp • Security problems : • Username and password flying unencrypted • Username and password present in .netrc file • Performances problems (Maximum bandwidth available at this time between SLAC and Lyon was 6 Mbps) : • Throughput performance on one ftp session : 80 Kbytes/s • ~Seven hours for one two-gigabytes file

  4. History • First idea to improve ftp performances : • Start n concurrent ftp • Total throughput reaches 3.5 Mbps • But you still need no network outage for seven hours • Second idea to improve ftp performances : • Cut the file in n parts • Start n concurrent ftp (one for each part) • Rebuild the file • But you double the disk space usage at each endpoint

  5. Goals • Improve performances on single file transfer • Optimize disk space usage • Improve security

  6. Performance improvements • Use several TCP streams to transfer a file • Use large TCP windows (RFC 1323) • Compress the data on the fly (using gzip algorithm)

  7. Performance improvements • The client opens a control connection to the server • Exchange control information • Each side starts as many processes as requested, each process will send a slice of the file.

  8. 0 0 10 10 20 20 30 30 40 40 50 50 60 60 70 70 80 80 90 90 100 100 Performance improvements bbftp server process bbftp client process compress uncompress 100 MB file 100 MB file

  9. Performance improvements • Compression plus parallel streams lead to throughput between 7 and 20 Mbps on the 6 Mbps link (often 12 Mbps with a standard Babar file) • But with higher speed links (34 Mbps), compression often reduces performance due to lack of CPU (perhaps we have to use another algorithm)

  10. Security improvements • Standard bbftp connection method • Use RSA encryption methods to transfer username and password • SSH bbftp connection method (Tim Adye’s idea) • Use ssh to remotely start the server

  11. Security improvements • Standard bbftp connection method server client connection RSA key pair generation send public key Crypt user name and password using the public key send encrypted data Decrypt user name and password using the private key OK or BAD

  12. Security improvements • Standard bbftp connection method without password • Start a bbftpcd client which will keep the username and password in memory • Start a bbftpc client which will talk with bbftpcd and will give it the command to execute • Limitation : bbftpcd and bbftpc have to run on the same computer

  13. Security improvements • SSH bbftp connection method client SSH client SSH daemon bbftpd process Start a ssh client like ssh -x -l username “bbftpd -s” host SSHD start the bbftpd process Use the encrypted connection as control connection

  14. Security improvements • SSH bbftp connection method without password • If the sshd server side allows the RSA authentication, a connection without any password is possible

  15. BBFTP state of the art • Current version 2.0.0 patch03 • Possibility to play with : • TCP window size • Read (Write) buffer size • Compression • Automatic directory creation • File mode • File access time • Number of streams • Remote and local umask • .bbftprc file • RFIO

  16. BBFTP state of the art • Current commands : • cd • get • put • lcd • mkdir • mget • mput

  17. BBFTP state of the art • Control file example : • 1 setnbstream 20 • 2 setremoteumask 022 • 3 setoption nocreatedir • 4 put /home/babar/jon/f1 /home/babar/bbrdist/newfiles/f1 • 5 setoption createdir • 6 put /home/babar/jon/f1 /home/babar/bbrdist/newfiles/f1 • 7 setnbstream 5 • 8 setrecvwinsize 1024 • 9 setoption gzip • 10 put /home/babar/jon/f2 /home/babar/bbrdist/newfiles/f2

  18. BBFTP state of the art • BBFTP home page : http://ccweb.in2p3.fr/bbftp/ • BBFTP discussion list : bbftp-l@in2p3.fr • Archives on : http://ccmail.in2p3.fr/archives/bbftp-l.html • Current experiments or sites using it : PHENIX ALICE(project)

More Related