Globally federated SRB zones

1. Globally federated SRB zones Yoshimi Iida SRB workshop at SDSC February 2-3, 2006

2. Why SRB? • Easy to install and start over • Even works with the password authentication • Light clients including command line, GUI and the web • Server side is also light • Only one SQL server is necessary except SRB servers • Federated zone has independent operation among sites • Every thing is done locally to access local files • Only when accessing remote files does remote server access happen SRB workshop

3. History of KEK SRB system • 2003 Jun. 06 Start operation of the test system 1 with PostgreSQL • HPSS interface • 2003 Oct. 22 test system 2 with DB2 • PostgreSQL looks better than DB2 • 2004 Apr. 02 SRB in BELLE Computer system • Interface to SONY Peta Site via NFS • Test with Melbourne • 2004 Dec. 06 HEP Data Grid workshop 2004 • Together with Michel Wan, SDSC, SRB federation has been established among 5 Belle collaboration institutes and KEK • 2005 Jul. 28 Nagoya Joined the federation SRB workshop

4. The location of the federated zones IHEP, CN Krakow, PL SDSC, US KEK, JP KNU, KR ASCC, TW ANU, AU The latency was measured in December 2004 SRB workshop

5. The testbed and the federation • Install SRB server, a storage resource and MCAT at each site • SRB version 3.2.1p • Open the KEK firewall from the fixed IP address of the other sites • After the installation, the functionality of the federation was tested and confirmed to be working SRB workshop

6. Performance measurement between ANU and KEK • A detailed study of the data transfer performance was previously conducted between the KEK and ANU zones • We have high latency for network communications • Initial tests showed a transfer rate about 120KB/s for a single thread transfer • In order to improve the data transfer performance the TCP window size was increased to 4MB on the ANU server and 8MB on the KEK server • For high latency network, this was expected to improve the performance and the result shows that the data transfer rate had increased to about 260KB/s SRB workshop

7. Parallel transfer between ANU and KEK SRB workshop

8. Transfer measurement from KEK • Copy files from KEK to the other sites • 10 files from 0.5 to 100MB in size • ‘Scp’ (SRB copy) use parallel I/O by default • Configure to allocate one thread per 2MB of the file size up to a maximum of 16 threads (i.e. 32MB) SRB workshop

9. Transfer rates from KEK Measured by ‘Scp’ SRB workshop

10. Transfer tests among 6 sites Results of 100MB data file transfer in MB/s SRB workshop

11. Transfer rate versus the latency • Although ANU has higher band width than other sites, the performance rate was not good enough because of higher latency SRB workshop

12. Transfer at high latency • The setting of TCP window size means that the specified value was requested to use but actual size is determined by best effort • Once a packet loss happens, the size is reset to the default one, and the TCP has the logic for increasing the window size after packet loss to recover the specified window size very slowly • For high latency networks, this process takes a time for slow negotiations and transfer rates goes down • If quality of networks is not good, packet loss happens often and this becomes the reason for less transfer rate than nominal band width • We confirmed this behavior by measuring performance under WAN emulation using the NIST Net SRB workshop

13. Transfer performance on WAN (preliminary) KNU WAN Emulation (NIST Net) ASCC ANU Krakow IHEP Measured in December 2004 SRB workshop

14. Summary • A federation of SRB servers in world-wide was demonstrated successfully among 7 sites • The latency and quality of network is the key to obtain better performance and the higher network bandwidth alone is not sufficient SRB workshop

15. Thank you