Plan of the talk: Computational issues Computing model Introduction to grid computing

1. All-sky search for continuous gravitational waves: tests in a grid environment Cristiano Palomba INFN Roma1 • Plan of the talk: • Computational issues • Computing model • Introduction to grid computing • All-sky search • Tests of the all-sky Hough Transform on the grid • Conclusions and next steps Mathematics of Gravitation II Warsaw, International Banach Center, 1st-10th September 2003

2. Computational issues The full-sky search for periodic sources of gravitational waves is computationally very demanding and needs large (distributed) computational resources. We want to explore a portion of the parameter space as large as possible. For a full-sky hierarchical search (i.e. source position completely unknown), with signal frequency up to 2kHz and a minimum source decay time t~10^4 years a computing power of the order, at least, of 1Tflop is required (see Sergio Frasca’s talk). Low granularity:the whole data analysis problem can be divided in several smaller and independent tasks, for instance dividing the frequency band we want to explore in small subsets.

3. Computing model The analysis fits well in a distributed computing environment. Computing environment detector data events We are following two approaches: Standard master/slaves farm: based on PBS, managed by a Supervisor Grid computing: geographically distributed farms, job distribution managed by a Resource Broker

4. Introduction to grid computing • A grid is a collaborative set of computing, data storage and network resources, belonging to different administrative domains. • It enables the coordinated and coherent use of largely distributed resources in a complete transparent way for the user. … and the new one….. The classical view…… mainframe Microcomputer Mini Computer Cluster (by Christophe Jacquet)

5. Introduction to grid computing: some of the major projects European DataGrid (EDG): 2001-2003 - www.edg.org LHC Computing Grid (LCG): 2002-2008-… - cern.ch/lcg CrossGrid: 2002-2005 – www.crossgrid.org DataTAG: 2002-2003 – www.datatag.org GriPhyN – www.griphyn.org PPDG – www.ppdg.net iVDGL – www.ivdgl.org TERAGRID (NSF) – www.teragrid.org

6. Introduction to grid computing: EDG (I) Purpose: to build on the emerging grid technologies to develop a sustainable computing model for effective share of computing resources and data Main partners: CERN - International (Switzerland/France) CNRS – France ESA/ESRIN – International (Italy) INFN – Italy NIKHEF – The Netherlands PPARC - UK

7. Introduction to grid computing: EDG (II) Assistant partners Industrial Partners Datamat (Italy) IBM-UK (UK) CS-SI (France) Research and Academic Institutes CESNET (Czech Republic) Commissariat à l'énergie atomique (CEA) – France Computer and Automation Research Institute,  Hungarian Academy of Sciences (MTA SZTAKI) Consiglio Nazionale delle Ricerche (Italy) Helsinki Institute of Physics – Finland Institut de Fisica d'Altes Energies (IFAE) - Spain Istituto Trentino di Cultura (IRST) – Italy Konrad-Zuse-Zentrum für Informationstechnik Berlin - Germany Royal Netherlands Meteorological Institute (KNMI) Ruprecht-Karls-Universität Heidelberg - Germany Stichting Academisch Rekencentrum Amsterdam (SARA) – Netherlands Swedish Research Council - Sweden

8. Introduction to grid computing: EDG basics (I) Request Request Result Data Client (User Interface) Application Server (Computing Element: Gatekeeper+ Worker Nodes) Data Server (Storage Element) Basic Services: Resource Broker, Information Service, Replica Catalog

9. Introduction to grid computing: EDG basics (III) Resource Broker: matchmaking between job requirements and available resources. Job submission system: wrapper for Condor-G; uses JDl scripts. Information Index: collects information concerning grid resources; read by the RB. Logging & Bookkeeping: stores information about the status and history of submitted jobs Replica Manager, GDMP: tools for creation and management of file replicas Virgo Data Analysis Software Based on Globus 2.0 Authentication, authorization: based on X.509 public key certificates. gridFTP: tool for secure and efficient file transfer. Replica Catalog: resolution of logical file names into physical file names. MDS: publish information about grid resources. Condor-G: job scheduler Fabric management: farm installation, configuration, management, monitoring, ‘gridification’,… Applications High level Grid Middleware Basic Services OS & Net services

10. ResourceBroker I I Introduction to grid computing: Job submission mechanism User Interface Gatekeeper IS Worker Node 1 PBS OS Worker Node 2 IS Worker Node 3 OS Storage Element L&B Gatekeeper Gatekeeper Worker Node 1 Worker Node 1 Worker Node 1 Worker Node 1

11. All-sky search: all-sky Hough Transform • It is the first incoherent step in the hierarchical procedure for the search of periodic signals. • It is the ‘heaviest’ step, because it works on the whole parameter space to be explored. • It has been implemented using look-up tables (LUT). This has shown to be the most efficient way. A LUT is a C array containing the coordinates of all the points of all the circles that can be drawn for a given source frequency. Several simmetries can be exploited when building a LUT. In particular, a LUT can be used for a range of frequencies of the order, at least, of the Doppler band for the initial one. The time needed to build the LUT is negligible respect to the time needed for building the Hough Maps.

12. All-sky search: all-sky Hough Transform (II)

13. All-sky search: implementation on the grid • The f-t peak-map, containing the list of peaks for each time slice, is produced from the SFDB; 2. The peak-map is split in several frequency sub-bands, depending on the number of available nodes; each sub-band will be processed by one Worker Node; E.g.: assuming we have 100 nodes, the band 500-2000Hz could be divided in 1000 sub-bands, 1.5Hz each  each processor will process 5 of these 3. These input files are distributed among the Storage Elements • Several ‘packets’ of jobs are submitted from the User Interface: each packet will be executed on a Worker Node using one of the above input files; Each packet will consist of about 1500 jobs 4a. Each job extracts from the input file a Doppler band around the current reference frequency and calculates the HT; Each job will last about 30 min (in case of crash only the last short job will be restarted); The LUT will be re-calculated about 8 times for each input file

14. All-sky search: implementation on the grid (II) 5. The candidates (position, frequency and spin-down values) are stored in a file; the files are replicated among Storage Elements; About 10^9 candidates will be selected after the first incoherent step 6. Steps 1-5 are repeated for the second observation period All subsequent steps do not need a distributed computing environment

15. All-sky search: Test of the Hough Transform on the Grid Three sites involved. 26 processors used. Virgo Virtual Organization BOLOGNA ROMA NAPOLI

16. All-sky search: Test of the Hough Transform on the Grid - description Each job calculates the Hough Transform for a single initial source frequency. • test-1: intensive job submission Hundreds of jobs are submitted sequentially on the grid Targets: - measure the performances (time needed to complete the jobs) - measure the reliability (fraction of dead jobs) - measure the overhead time due to the RB activity - check the homogeneity in the distribution of jobs • test-2: submission of packets of jobs Several ‘packets of jobs’ are submitted on the grid Targets: - measure the performances (expected lower overhead?) - measure the reliability (expected lower fraction of dead jobs)

17. All-sky search: Test of the Hough Transform on the Grid - results Death rate ~ 3/1000 Death rate < 1/1000

18. Conclusions • Emerging grid technologies appear to be very effective in data analysis for the search of periodic gravitational signals. • Grid software is more and more robust and reliable. • Our data analysis method can be very easily adapted to work on a computing grid. • Other kinds of search for gravitational signals can benefit from grid (e.g. coalescing binaries). • Soon we will pass from tests to production.

19. The European DataGrid project (EDG) is a EU funded project which has been started mainly to answer the computing needs of future LHC experiments. It has produced since now several software releases, which enable the ‘gridification” of a local cluster of computers. DIFFERENZE RISPETTO, PER ES., A CONDOR, SETI ECC.

20. Introduction to grid computing: EDG basics (II) Different kind of services: • Information Service: gives information about the availability of services on the grid (e.g. X cpu and Y GB of disk space are available at a given site) • Authentication & Authorization: allow a user to log on the grid and use its resources • Job submission service: responsible for the job submission according to the user needs/requests and the available resources • Replica Management: responsible for the management of file replicas • Logging & Bookkeeping: