1 / 22

SRB and iRODS @ CC-IN2P3

SRB and iRODS @ CC-IN2P3. Jean-Yves Nief Pascal Calvat. Overview. SRB: Status. A couple of examples. Prospects. iRODS: Developments. Test beds. Project in production: digital library. Moving from SRB to iRODS ?

melinda-rowe
Download Presentation

SRB and iRODS @ CC-IN2P3

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. SRB and iRODS @ CC-IN2P3 Jean-Yves Nief Pascal Calvat

  2. Overview • SRB: • Status. • A couple of examples. • Prospects. • iRODS: • Developments. • Test beds. • Project in production: digital library. • Moving from SRB to iRODS ? • JUX (Java Universal eXplorer): GUI in Java for many protocols like iRODS, SRB. SRB and iRODS @ CC-IN2P3

  3. SRB in Lyon SRB and iRODS @ CC-IN2P3

  4. SRB and iRODS hardware and software @ CC-IN2P3 • 14 SRB servers (130 TB of disk space): • Sun v480, v20z, v440 (Sparc III) and Thumpers x4500 (AMD Opteron). • OS: Solaris 9 and Solaris 10. • Almost all the instances on SRB v3.5.0. • Interfaced with our MSS: HPSS v5: • Scripts developed to handle automatic migration/purge of compound resources (disk cache/tape archive). • MCATs: using Oracle 10g. • Going to add RHEL4 machines (Linux) for MCATs enabled servers. SRB and iRODS @ CC-IN2P3

  5. Example in HEP: BaBar • Data import from SLAC to Lyon. • SRB being used since 2004 in production. • Fully automated: • New files created are registered in the SLAC catalog database. • Client application in Lyon: detection of files missing in the Lyon catalog database + transfer of these files. • Automated error recovery. • Up to 5 TB / day (max. rate observed). • Usual rate: 2-3 TB / day (during production periods) • 700 TB imported so far (since 2004), 700,000 files. • Until end 2008, volume x2. SRB and iRODS @ CC-IN2P3

  6. (2) (1) (3) HPSS/Lyon HPSS/SLAC SRB MCAT SRB MCAT SRB SRB SRB Example in HEP: BaBar Import agents SRB Import agents SRB SLAC zone CC-IN2P3 zone SRB and iRODS @ CC-IN2P3

  7. Examples in astrophysics and astroparticles • Underwater: Antares • in the pampa: Pierre Auger Observatory • At the top of the mountain: SuperNovae Factory in Hawaii SRB and iRODS @ CC-IN2P3

  8. Example in biology: BioEmergence • European projects involving 5 countries. • Embryogenesis: zebra fish. • 2 microscopes now (several in the future): amount of data could be huged (PB scale). • Data pushed from the microscopes into the SRB. • SRB integrated within their workflow. • CC-IN2P3: core of the system. SRB and iRODS @ CC-IN2P3

  9. LIGO CC-IN2P3 Cascina Bologna Virgo Visualize data on the WAN through SRB • Interferometer for gravitational waves detection (in production: 60 TB / y). • Need for a reliable data distribution system. • Distribute Ligo data (same experiment in the US) to the european sites: CC-IN2P3 and Bologna. • Have been using bbftp so far. • SRB has replaced bbftp: • Bookkeeping system. • Interface with HPSS. • Handling of Ligo small files. Pull Virgo + Ligo data from Lyon Merge Ligo small files during data import on the CC-IN2P3 server side Push raw data to CC-IN2P3 SRB and iRODS @ CC-IN2P3

  10. SRB @ CC-IN2P3 • MCATs performance enhancement: • Reindexing made automatically on a weekly basis. • Issues with Oracle performances: • Some oddity in the way Oracle optimized requests. • Request analyzis done on all the MCATs on a daily basis.  Not completly resolved. • SRB: big consumer of Oracle resources. • Need to separate Oracle instance for the MCATs. SRB and iRODS @ CC-IN2P3

  11. SRB prospects @ CC-IN2P3 • More than 10 projects using it: • SRB: critical part of their computing system. • Clients on Linux, Mac OSX, Windows, Solaris, AIX (Blue Gene) from Europe, USA. • Daily traffic can be quite big: • Hundreds of thousands of connections per day. • Some projects with more than 200,000 connections per day at peak rate. • Bandwidth peak rate: several Gbits/s. • Will reach 1 PB of data referenced and handled by SRB in 2008. • Need to strengthen the service (add extra MCAT servers). SRB and iRODS @ CC-IN2P3

  12. iRODS developments @ CC-IN2P3 • Tests scripts (JY + Thomas): • icommand test script: extensive test of the binary commands in order to track bugs (both client and server side problems).  To do: include all the new options and new icommands. • Loading test of the system: launch n test scripts in // on the same machine. • To do: ability to launch process on other servers ? (but too specific to a given site: passwrodless ssh etc…). SRB and iRODS @ CC-IN2P3

  13. iRODS developments @ CC-IN2P3 • Host based access control micro-service: • User id, group id. • Hostname or range of IP adresses. • Improved firewall implemented at the iRODS level. SRB and iRODS @ CC-IN2P3

  14. iRODS developments @ CC-IN2P3 • Load balancing micro-services (JY + Jean Aoustet): • Load balancing system: • At a given time, choose the least loaded server to put/get data or do any other operation. • Have to be used wisely: • It only makes sense for identical servers on the same site! • Gather metrics on each server (CPU load, network activity, memory usage, swap, disk occupancy etc…). • Store into iCAT. • Can also be used for monitoring purposes. SRB and iRODS @ CC-IN2P3

  15. iRODS data server Perf script Perf script Perf script Perf script iRODS data server iRODS data server DB iRODS data server Load balancing system • Ask each server for its metrics: rule engine cron task (msi). 2. Performance script launched on each server. iRODS iCAT server 3. Results sent back to the iCAT. 4. Store metrics into iCAT. 5. Compute a «quality factor» for each server stored in an other table: r.e. cron task (msi). SRB and iRODS @ CC-IN2P3

  16. Future developments • HPSS driver: assessment by Thomas Kachelhoffer (HPSS v6)  in progress. • Handling MSS resources with the protocol of your choice: • RFIO, pftp etc… • Easy way to plugin a MSS without writing a driver. • Kind of universal driver. • Micro-services for: • Handling accentuated letters in files and directory names registered into iRODS (eg: Windows world). • Anonymization of DICOM files in biology. SRB and iRODS @ CC-IN2P3

  17. NCSA CC-IN2P3 SDSC iRODS test beds • With KEK (see Yoshimi talk): data transfer. • LSST: • Telescope in Chile: first light in 2014. • Big challenge for data processing. • USA + Chile for now. France starting to get involved. • Computing: data management studies (DC 2 and 3). (now also Indiana and Chile) SRB and iRODS @ CC-IN2P3

  18. iRODS first production instance • Just starting. • Digital preservation area: • Scan old documents and put them into iRODS. • 4-5 users at the moment: transparent access to the documents from various platforms. • ~ 1-2 TB expected for this year. • Needs: • Propagation of ACLs to newly created collections. • Windows support for the icommands. • GUI interface (standalone or Web). • iRODS fits their needs. SRB and iRODS @ CC-IN2P3

  19. Moving from SRB to iRODS ? • No plans yet. • Will slowly moved the existing SRB projects to the new system starting in 2010 or a bit earlier. • Needs to be confident that all the bits and pieces are there before the switch: • MSS integration: one of the key thing. • Probably brand new projects could start directly onto iRODS in 2008 such as Digilib. SRB and iRODS @ CC-IN2P3

  20. Future plans for iRODS • Continue developments: • Focus on MSS integration. • Load balancing. • CC-IN2P3 starting to be involved with Arts and Humanities projects: • Fedora and iRODS interface to be studied. • New projects to start directly on iRODS: • 1 more candidate this year. SRB and iRODS @ CC-IN2P3

  21. JUX: a Java explorer • Java Universal eXplorer: developed by Pascal Calvat (CC-IN2P3). • Based on JSaga APIs: • Saga (Simple Apis for Grid Application): OGF specs. Common set of APIs to interact with middleware (storage, job tasks). • JSaga: implementation of Saga by Sylvain Reynaud, CC-IN2P3. • Plugins developped for various protocols like SRB and iRODS (using Jargon). • JUX can connect to various protocols providing the plugin is available. • Still under development: first version released soon. SRB and iRODS @ CC-IN2P3

  22. Acknowledgement • Thanks to: • Pascal Calvat (JUX), Jean Aoustet (Load Balancing), Thomas Kachelhoffer (tests + HPSS).. • Wilko Kroeger (SLAC): BaBar. • Yoshimi Iida (transfer tests + iCAT stress tests). • Links: • JSaga (Sylvain Reynaud): http://grid.in2p3.fr/jsaga SRB and iRODS @ CC-IN2P3

More Related