1 / 14

BaBarGrid

BaBarGrid. Alessandra Forti 30 January 2003, Gridpp meeting. This Talk. Job submission with EDG and BaBar replica catalog EDG testing BBGUtils (globus clients package) BdbServer++ (data distribution system). Submitter Requirements.

clayton-ayala
Download Presentation

BaBarGrid

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. BaBarGrid Alessandra Forti 30 January 2003, Gridpp meeting

  2. This Talk • Job submission with EDG and BaBar replica catalog • EDG testing • BBGUtils (globus clients package) • BdbServer++ (data distribution system)

  3. Submitter Requirements A job should be submitted only to a site which holds data required to run the job • The job should run in the standard BaBar environment • A user should be allowed to pass his private environment variables with the job • Output should be sent back and/or stored on a SE

  4. Job Submitter Steps • Data preparation (skimData) • skimData is the UI to the metadata catalog • User TCL file expansion (dump) • JDL file(s) creation based on skimData delivered index and user options • Submission to the GRID

  5. Data Preparation • A modified skimData has been developed • It returns data matching given criteria and creates a file, index • The index groups the files into buckets. Each bucket is defined by a list of sites which hold the data

  6. Index Example sites: BABAR-RAL,BABAR-IC 00001 00002 00003 sites: BABAR-MAN 00004 • Filenames are named: indexname.number.tcl • The example above would result in 4 JDL files and 4 jobs submitted

  7. JDL File Creation • The JDL file creation process analyses the index and user supplied options and creates a set of JDL files to be submitted to the GRID • The options include: • The index filename • User top TCL file (or expanded TCL file) • The executable • Environment variables to be passed to the GRID

  8. Submission to GRID • The index file defines a data file to be used by a job. The data filename is inserted into the expanded TCL file and sourced • A wrapper around the user defined executable is created and sent to GRID • The wrapper defines the environment on the GRID but relies on the existing BaBar setup • The std output and std error are put into the output sandbox as well as any additional user created files (eg. n-tuples)

  9. To Do • The job submitter requires Linux 2.4 in order to run the tcl expansion. Since there is no official EDG UI under 2.4 the expansion and actual submission are done in 2 steps now, but it is planned to combine it later. • Job monitoring using BOSS

  10. EDG testing • status of the testing machines at slac • the CE upgraded to edg-1.3.4 under RH6.2. • the UI upgraded to edg-1.4.0. It works fine from both RH6.2 or RH7.2 • the CE was registered with the edg-1.4.0 RB in Imperial College (gm03 machine) • the SE is edg-1.4.0 under RH6.2 • Farms in UK need upgrade. Prolem with the linux version required by babar and the one used by EDG.

  11. EDG testing • Submit a simple analysis job at SLAC, RAL or IN2P3 • OK for both Root files and Objy • Test of the edg Replica Manager / Replica Catalog (LDAP) • Test of registering and replicating a file • Not used for data location in job submission • Stress tests independent of Analysis jobs • Job storms have been launched, and got rather successfully executed (200 jobs for each storm, not less than 25 sec between each individual request, no more than 35 requests running simultaneously)

  12. BBGUtils • Installs Globus client packages in the user space. It is intended to allow Tier C sites to use globus without having to do a full gatekeeper installation • No need to deal with configuration files. • It is all command line driven • has options for multiple installs, logging, user defined paths, • optional packages such as GridFTP can be added • It can install source or binaries. • It is planned also to do the same with the EDG WP1 Resource Broker Client. But there are some difficulties due to software dependencies. It might change when EDG will pass to RH7.*

  13. BdbServer++ • Data Distribution system. • Originally built on top of SRB for Objectivity distribution • It has been extended to use Globus and EDG clients. • Data Location is done through BaBar RC • The UI is a WEB browser

  14. BdbServer++ SLAC other... EDG RB Globus / EDG / SRB Client TCP/IP MySQL Database Web Server ssh World WideWeb Https Authentication & request Web Browser Data

More Related