1 / 16

FAX FDR results

FAX FDR results. 28 th January 2013. For reference. Indico page: https:// indico.cern.ch / conferenceDisplay.py?confId =229966 Twiki : https:// twiki.cern.ch / twiki /bin/ viewauth /Atlas/ JanuaryFDR ML based monitor at SLAC: http://atl-prod07.slac.stanford.edu:8080/ display

jace
Download Presentation

FAX FDR results

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. FAX FDR results 28th January 2013

  2. For reference • Indico page: https://indico.cern.ch/conferenceDisplay.py?confId=229966 • Twiki: https://twiki.cern.ch/twiki/bin/viewauth/Atlas/JanuaryFDR • ML based monitor at SLAC: http://atl-prod07.slac.stanford.edu:8080/display • CERN dashboard: http://dashb-atlas-xrootd-transfers.cern.ch/ui/# • WAN HC based tests:http://ivukotic.web.cern.ch/ivukotic/WAN/index.asp • FDR dedicated test submission page: http://ivukotic.web.cern.ch/ivukotic/FDR/index.asp Ilija Vukotic ivukotic@uchicago.edu

  3. Endpoints • From AGIS • Two more sites added during the week: Lancaster and Liverpool • RALPP and Cambridge joined but still testing against them • PIC should join shortly • All the endpoints showed very high availability during the week. Ilija Vukotic ivukotic@uchicago.edu

  4. Links • WAN HC tests are continually testing full mesh of links. • That’s used for SSB cost matrix. Ilija Vukotic ivukotic@uchicago.edu

  5. Links • But not all of the links worked due to authorization issue • Plot shows situation ten days ago. Ilija Vukotic ivukotic@uchicago.edu

  6. Links • During the week • Not all servers shown Ilija Vukotic ivukotic@uchicago.edu

  7. Input data • Week started without any input data that could be used and confusion concerning dataset and file naming. • Federica kindly provided both a list of datasets to be used (SUSY and SMWZ) and a script to replicate them automatically. • dq2-put/dq2-get combination proved to be inadequate for task at hand. • We decided to start with just the first dataset of SUSY sample: 19 files of 68.5 GB. • Hiro made transfer requests to all of the US sites and than manually re-registered them. • Simone did equivalent thing for CERN and 3 Italian sites, than continued with the DE + RU + UK cloud • Wahid used get/put method for all of the UK sites but only the first dataset. Ilija Vukotic ivukotic@uchicago.edu

  8. Input data • Current distribution status of the first SUSY dataset Ilija Vukotic ivukotic@uchicago.edu

  9. Week prior to FDR Blue and light blue are MWT2 mostly tier3 users using tier2 data through FAX Stream of ~100-150MB/s of testing data. Peaks every 30 min. On 11th Jan just sent a bit more tests. Ilija Vukotic ivukotic@uchicago.edu

  10. During FDR week University of Chicago users were using FAX from both Tier3 and grid jobs at Tier2. Will remove it from further plots. Ilija Vukotic ivukotic@uchicago.edu

  11. During the week • Submitting jobs through the web site. • All of the details down to PandaID’s can be found there. • In short: HC jobs that were normally used for local site tests where changed so they contact Oracle DB at CERN and from there get info: which FAX endpoint to use, which files to use, what to do with them copy/read, how many jobs to do etc. Jobs upload back time, MB/s, ev/s. • System is very easy to use, still some space for improvement: • It lets you try to use link which (currently) does not work. • To get more than two simultaneous jobs you have to make change in HC test. • Even when you request 10 simultaneous jobs you easily end up with much less, depending on how fast are your jobs and how long is the client sites ANALY_* queue. • Testing pattern: • reading from same site (10 files from 10 jobs ) • from site in same cloud (10 files from 10 jobs ) • from main regional site (CERN, BNL) (10 files from 10 jobs ) • from across the pond (10 jobs 1 file) Ilija Vukotic ivukotic@uchicago.edu

  12. During the week Bandwidth measure • Started low: copy and read from Roma1 • Real performance measure • To Napoli penalty 30% • To CERN 50% • Worst case scenario Ilija Vukotic ivukotic@uchicago.edu

  13. During the week • Started low: copy and read from Oxford Ilija Vukotic ivukotic@uchicago.edu

  14. During the week • Started low: copy and read from Oxford Ilija Vukotic ivukotic@uchicago.edu

  15. During the week • QMUL LRZ-LMU Ilija Vukotic ivukotic@uchicago.edu

  16. conclusion • A huge progress in FAX readiness just before and during the week. THANK YOU ALL ! • Sites were testing was possible showed no problems in delivering data. • Performance was fluctuating a lot, depending on other transfers to/from site. Fine spatial granularity cost matrix will be essential for some applications. • We gained a lot of experience and surely next FDR will be much better. Could be made as soon as: • Authentication issue properly solved • Proper implementation of both copy-to-scratch and direct-access modes • The test data will be available from the first day Ilija Vukotic ivukotic@uchicago.edu

More Related