1 / 21

BaBar UK Grid Work

BaBar UK Grid Work. Tim Adye Rutherford Appleton Laboratory BaBar Collaboration Meeting SLAC 11 th December 2002. Talk Plan. RAL Tier A BaBar Job Submission [Janusz Martyniak] Metadata [Alessandra Forti]

lavada
Download Presentation

BaBar UK Grid Work

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. BaBar UK Grid Work Tim Adye Rutherford Appleton Laboratory BaBar Collaboration Meeting SLAC 11th December 2002 Tim Adye

  2. Talk Plan • RAL Tier A • BaBar Job Submission [Janusz Martyniak] • Metadata [Alessandra Forti] • BaBar UK Grid Facilities [Marc Kelly] Tim Adye

  3. BaBar Batch CPU Use at RAL Tim Adye

  4. BaBar Batch Users at RAL(running at least one non-trivial job each week) Tim Adye

  5. BaBar Job Submission Janusz Martyniak Imperial College Tim Adye

  6. Submitter Requirements • A job should be submitted only to a site which holds data required to run the job • The job should run in the standard BaBar environment (srtpath, PARENT link etc) • A user should be allowed to pass his private environment variables with the job • Output should be send back and/or stored on a SE and registered with the RC Tim Adye

  7. Job Submitter Steps • Data preparation (skimData) • User TCL file expansion (dump) • JDL file(s) creation based on skimData delivered index and user options • Submission to the GRID Tim Adye

  8. Data Preparation • A modified version of the skimData program has been developed by Dave Smith and Alessandra Forti • It returns data matching given criteria and creates a file, index • The index groups the files into buckets. Each bucket is defined by a list of sites which hold the data Tim Adye

  9. Index Example • Filenames are named: indexname.number.tcl • The example above would result in 4 JDL files and 4 jobs submitted sites: BABAR-RAL,BABAR-IC 00001 00002 00003 sites: BABAR-MAN 00004 Tim Adye

  10. User TCL File Expansion • Requires release 12.2.2 [Asoka De Silva] • Reads user TCL file (eg. kangaFilterMicro), sourcing all files referenced by it • The result is a single TCL file • The data file list should not be sourced Tim Adye

  11. JDL File Creation • The JDL file creation process analyses the index and user supplied options and creates a set of JDL files to be submitted to the GRID • The options include: • The index filename • User top TCL file (or expanded TCL file) • The executable • Environment variables to be passed to the GRID Tim Adye

  12. Submission to GRID • The index file defines a data file to be used by a job. The data filename is inserted into the expanded TCL file and sourced • A wrapper around the user defined executable is created and sent to GRID • The wrapper defines the environment on the GRID but relies on the existing BaBar setup • The std output and std error are put into the output sandbox as well as any additional user created files (eg. n-tuples) Tim Adye

  13. To Do • A manual is (almost) ready. • The job submitter requires Linux 2.4 in order to run the tcl expansion. Since there is no official EDG UI under 2.4 the expansion and actual submission are done in 2 steps now, but it is planned to combine it later. • Which form the should the code (pure Python) be released? • rpm or Phython distribution tools [or CVS]? Tim Adye

  14. Conclusions • The first ‘dumb’ version of the submitter exists • The command line scripts create automatically a set of JDL files based on the skimData created index • This ensures that job will be submitted only to a site which actually holds the data required • The load balancing is done by the Resource Broker • The environment variables may be transferred to the GRID but they are the same for all JDL files • The python modules will be distributed as rpm or tar files and installed as third party modules (python Distutils) Tim Adye

  15. Metadata etc Alessandra Forti Manchester Tim Adye

  16. RLS – Replica Location Service • RLS is the EDG/Globus replica location service • Will replace the LDAP based replica catalog • distributed system based on MySQL (relational database) • It consists of two levels of information: • LRC (Local Replica Catalog) with local replica or PFN (Physical File Name) information • RLI (Replica Location Index) contains pointers to different LRC for each LFN (Logical File Name) • The RLS doesn’t contain any metadata • Can we use this with skimData? • instead of remote lookups at each potential site Tim Adye

  17. Other Work • Alessandra is maintaining the LDAP-based Replica Catalogue (RC) • Used from INFN, IN2P3 and SLAC for testing • Works in EDG 1.2 • will be upgraded today or tomorrow for EDG 1.3 • Can now submit jobs using Globus, gsiklog, and AFS • works since yesterday! • Also plan to install R-GMA in Manchester for testing outside the testbed Tim Adye

  18. UK Grid Facilities Marc Kelly Bristol Tim Adye

  19. Central Facilities • The "Testbed" resoruce broker is being upgraded to 1.4 at Imperial College • UK Testbench at RAL is also being upgraded to 1.4 • RAL Tier A front-end will follow Tim Adye

  20. UK Farms • Still have a 1.2 Resource Broker for BaBar usage. This machine claims to have the following registered. • gf18.hep.man.ac.uk • gm04.hep.ph.ic.ac.uk • gw32.hep.ph.ic.ac.uk • bfa.hep.ph.ic.ac.uk • farm020.hep.phy.cam.ac.uk • ce.hep.phy.cam.ac.uk • bbr-gate01.slac.stanford.edu • bbr-gate02.slac.stanford.edu • bbr-gate03.slac.stanford.edu • What about the other BaBar UK Farms? Tim Adye

  21. UK Farms (cont) • Many of the UK farms are or are planning to upgrade, but have been following a moving target • 1.3.3, then 1.3.4, now 1.4.0 • Need to decide which version we move to and stick to that. Tim Adye

More Related