1 / 16

Glite I/O Storm Testing in EDG-LCG Framework

Glite I/O Storm Testing in EDG-LCG Framework. Elena Slabospitskaya , Vadim Petukhov , (IHEP, Russia). Gilbert Grosdidier , (CNRC, France). NEC'2005, Sept 16. Outline. Framework’ Contributed Authors TSTG Suite Dedication Design of TSTG Framework S/W

page
Download Presentation

Glite I/O Storm Testing in EDG-LCG Framework

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Glite I/O Storm Testing in EDG-LCG Framework Elena Slabospitskaya, Vadim Petukhov, (IHEP, Russia) Gilbert Grosdidier, (CNRC, France) NEC'2005, Sept 16

  2. Outline • Framework’ Contributed Authors • TSTG Suite Dedication • Design of TSTG Framework S/W • Up to gLite- port TSTG framework to gLite environment • Example of gLite plug-in - I/O Stress Test Suite NEC’05 Elena Slabospitskaya (IHEP) et al.

  3. Framework’Contributed Authors The mainly contributed authors are • Miguel Barcelo • Frederique Choilett • Gilbert Grosdidier • Andrey Kiryanov • Charles Loomis • Gonzalo Merrino • Rene Metery • Danila Oleynik • Andrea Sciaba • Elena Slabospitskaya • Many other people contributed to the design and ideas leading to the current suites, among them the INFN Testing Team NEC’05 Elena Slabospitskaya (IHEP) et al.

  4. New accent of MW testing The coordination of this task will be done in close collaboration with EGEE-II SA3 to address the shortcomings of available tests for gLite middleware. This involves agreement on schedule, provision of access to development test-bed(s), support on issues hindering the development process (e.g. missing or failing test-bed functionality). "The development of full featured test suites that for the certification of the new MW components" "The development of testing procedures focused on special details of various software features" "Porting the exiting test suites and procedures to the TSTG framework" NEC’05 Elena Slabospitskaya (IHEP) et al.

  5. Levels of testing required • Installation and configuration testing • Targets each machine/service individually • 2. Unit testing • Targets each basic functionality of a given service independently from the rest of the TB • 3. Functionality testing • For the all TB exercises full functionalities • 4. Stress testing • Huge number of jobs launched one by one or in parallel NEC’05 Elena Slabospitskaya (IHEP) et al.

  6. TSTG suite dedication • Site certification • Initial check of the install • Check than changes or misconfiguration occur • Regular daily or weekly re-checking (under cron) • 2. TB Certification • Basic functional tests of components (services) • Basic grid functionalities with individual tests • Full grid functionalities on a full TB including remote sites with global/group tests • 3. MW validation (Functionalities, Robustness, Stress Testing) NEC’05 Elena Slabospitskaya (IHEP) et al.

  7. Framework S/W design Configure Step Test Loop Plug-ins Definition Files Presenter Step It is split into 2 main parts, dedicated to 3 tasks -the top level driver, written in Perl -the plug-ins, one of each specific test (Perl, bash, C++) The top level driver is responsible for these 3 tasks ----Configure (build environment) ----Test Loop (run one of several selected tests) ----Presenter : merges the results and builds the web enabled summary page NEC’05 Elena Slabospitskaya (IHEP) et al.

  8. Examples of LCG-2 Plug-ins Job Storm Copy Storm Replica Storm Data Storm GFAL Storm Matchmaking Strom RLS Ses MDS Checksum Storm NEC’05 Elena Slabospitskaya (IHEP) et al.

  9. Structure of CTB SL3 NEC’05 Elena Slabospitskaya (IHEP) et al.

  10. LCG-2 Testing under TSTG Framework

  11. The gLite-I/O server allows posix-like access to files stored in the SRM

  12. gLite Data Access via I/O server Storage Element 1 User Interface Storage Element 2 Input/Output Server Worker Nodes Storage Element 3 Storage Element 4 Worker Nodes NEC’05 Elena Slabospitskaya (IHEP) et al.

  13. gLite I/O plug-in User launches multiple (parallel or one by one) jobs from User Interface to Worker Node.Each job on Worker Node launches a script which compiles and linkages C++ program and creates a jdl file (this program is a part of job in input sandbox) The C++ program via API creates files (size is optional) and transfers (read/write/seek) these files between WN and I/O server (there and back). The result returns to user (to User Interface). Sources and results of tests are arranged at web. NEC’05 Elena Slabospitskaya (IHEP) et al.

  14. Results of I/O Stress Tests NEC’05 Elena Slabospitskaya (IHEP) et al.

  15. I/O test. XML output

  16. Conclusions • 1.Reuseas much as possible existing suites/frameworks/ can be easily incorporated from the LCG-2 certification to gLite • Minimize duplication of effort when creating new tests • Make sure new tests can be easily incorporated into the existing framework • 2. The next most urgent is to write new tests or the functionality being released and /or different from LCG-2 • Most must one from EGEE JRA1 team • But not exclusively, LCG team will contribute

More Related