Applications and the Grid

1. Applications and the Grid The European DataGrid Project Team http://www.eu-datagrid.org

2. Overview • An applications view of the the Grid – Use Cases • High Energy Physics • Why we need to use GRIDs in HEP ? • Brief mention of the Monarc Model and its evolutions towards LCG • HEP data analysis and management and HEP requirements. Processing patterns • Testbeds 1 and 2 validation : what has already been done on the testbed ? • Current GRID-based distributed comp.model of the HEP experiments • Earth Observation • Mission and plans • What do typical Earth Obs. applications do ? • Biology • dgBLAST

3. What all applications want from the Grid • A homogeneous way of looking at a ‘virtual computing lab’ made up of heterogeneous resources as part of a VO(Virtual Organisation) which manages the allocation of resources to authenticated and authorised users • A uniform way of ‘logging on’ to the Grid • Basic functions for job submission, data management and monitoring • Ability to obtain resources (services) satisfying user requirements for data, CPU, software, turnaround……

4. Common Applications Issues • Applications are the end-users of the GRID : they are the ones finally making the difference • All applications started modeling their usage of the GRID through USE CASES : a standard technique for gathering requirements in software development methodologies • Use Cases are narrative documents that describe the sequence of events of an actor using a system [...] to complete processes • What Use Cases are NOT: • the description of an architecture • the representation of an implementation

5. Applications domain ALICE ATLAS CMS LHCb OtherHEP Other Apps Domains interface HEP Common Application Layer … DataGRID middlewarePPDG, GriPhyn, EU-DataGRID Bag of Services (GLOBUS, Codor-G ,…) Computer Scientist domain Why Use Cases ? OS & Net services

6. Use Cases (HEPCAL) http://lhcgrid.web.cern.ch/LHCgrid/SC2/RTAG4/finalreport.doc Job catalogue update Job catalogue query Job submission Job Output Access or Retrieval Error Recovery for Aborted or Failing Production Jobs Job Control Steer job submission Job resource estimation Job environment modification Job splitting Production job Analysis 1 Data set transformation Job monitoring Simulation Job Experiment software development for the Grid VO wide resource reservation VO wide resource allocation to users Condition publishing Software publishing Obtain Grid authorisation Ask for revocation of Grid authorisation Grid login Browse Grid resources DS metadata update DS metadata access Dataset registration to the Grid Virtual dataset declaration Virtual dataset materialization Dataset upload User-defined catalogue creation Data set access Dataset transfer to non-Grid storage Dataset replica upload to the Grid Data set access cost evaluation Data set replication Physical data set instance deletion Data set deletion (complete) User defined catalogue deletion (complete) Data retrieval from remote Datasets Data set verification Data set browsing Browse condition database

7. High Energy Physics applications

8. The LHC challenge • HEP is carried out by a community of more than 10,000 users spread all over the world • The CERN Large Hadron Collider at CERN is the most challenging goal for the whole HEP community in the next years • Test the standard model and the models beyond it (SUSY,GUTs) at an energy scale ( 7+7 TeV p-p) corresponding to the very first instants of the universe after the big bang (<10-13 s) , allowing the study of quark-gluon plasma • LHC experiments will produce an unprecedented amount of data to be acquired, stored, analysed : • 1010 collision events / year (+ same from simulation) • This corresponds to 3 - 4 PB data / year / experiment (ALICE, ATLAS, CMS, LHCb) • Data rate ( input to data storage center ): up to 1.2 GB/s per experiment • Collision event records are large: up to 25 MB (real data) and 2 GB (simulation)

9. The LHC detectors CMS ATLAS ~6-8 PetaBytes / year ~1010 events/year ~103 batch and interactive users LHCb

10. 40 MHz (40 TB/sec) level 1 - special hardware 75 KHz (75 GB/sec) level 2 - embedded processors 5 KHz (5 GB/sec) level 3 - PCs 100 Hz (100 MB/sec) data recording & offline analysis online system multi-level trigger filter out background reduce data volume les.robertson@cern.ch

11. Data Handling and Computation for Physics Analysis reconstruction event filter (selection & reconstruction) detector analysis processed data event summary data raw data batch physics analysis event reprocessing simulation analysis objects (extracted by physics topic) event simulation interactive physics analysis les.robertson@cern.ch

12. Lab m Uni x grid for a regional group CERN Tier 1 Uni a UK USA Lab a France Tier 1 Tier3 physics department Uni n Tier2 Japan Italy CERN Tier 0 Desktop Lab b Germany .. .. .. Lab c  Uni y Uni b grid for a physics study group   Deploying the LHCGlobal Grid Service The LHC Computing Centre les.robertson@cern.ch

13. HEP Data Analysis and Datasets • Raw data(RAW) ~ 1 MByte • hits, pulse heights • Reconstructed data (ESD) ~ 100 kByte • tracks, clusters… • Analysis Objects (AOD) ~ 10 kByte • Physics Objects • Summarized • Organized by physics topic • Reduced AODs(TAGs) ~1 kByte • histograms, statistical data on collections of events

14. HEP Data Analysis –processing patterns • Processing fundamentally independent (Embarassing parallel) due to independent nature of ‘events’ • So have concepts of splitting and merging • Processing organised into ‘jobs’ which process N events • (e.g. simulation job organised in groups of ~500 events which takes ~ day to complete on one node) • A processing for 10**6 events would then involve 2,000 jobs merging into total set of 2 Tbyte • Production processing is planned by experiment and physics group data managers(this will vary from expt to expt) • Reconstruction processing (1-3 times a year of 10**9 events) • Physics group processing (? 1/month). Produce ~10**7 AOD+TAG • This may be distributed in several centres

15. Processing Patterns (2) • Individual physics analysis - by definition ‘chaotic’ (according to work patterns of individuals) • Hundreds of physicists distributed in expt may each want to access central AOD+TAG and run their own selections . Will need very selective access to ESD+RAW data (for tuning algorithms, checking occasional events) • Will need replication of AOD+TAG in experiment, and selective replication of RAW+ESD • This will be a function of processing and physics group organisation in the experiment

16. EDG RB EDG CE AliEn CE EDG SE EDG UI AliEn SE WNs Data Catalogue Alice: AliEn-EDG integration Server • EDG UI Installation • JDL translation • Certificates • Alice SE on EDG nodes • Alice Data Catalogue access by EDG nodes (Cerello, Barbera,Buncic Saiz,et al.)

17. What have HEP experiments already done on the EDG testbeds 1.0 and 2.0 • The EDG User Community has actively contributed to the validation of the first and second EDG testbeds (feb 2002 – feb 2003) • All four LHC experiments have ran their software (firstly in some preliminary version) to perform the basics operations supported by the testbed 1 features provided by the EDG middleware • Validation included job submission (JDL), output retrieval, job status query, basic data management operations ( file replication, register into replica catalogs ), check of possible s/w dependencies or incompatibility (e.g. missing libs, rpms) problems • ATLAS, CMS, Alice have run intense production data challenges and stress tests during 2002 and 2003

18. CMS MonteCarlo production using BOSS and Impala tools. Originally designed for submitting and monitoring jobs on a ‘local’ farm (eg. PBS) Modified to treat Grid as ‘local farm’ December 2002 to January 2003 250,000 events generated by job submission at 4 separate UI’s 2,147 event files produced 500Gb data transferred using automated grid tools during production, including transfer to and from mass storage systems at CERN and Lyon Efficiency of 83% for (small) CMKIN jobs, 70% for (large) CMSIM jobs The CMS Stress Test

19. Disk Space Site CE Number of CPUs SE (GB) lxshare0393 100 lxshare0227 122 CERN * lxshare0384 1000(=100*10) + * testbed008 40 grid007g 1000 CNAF gppce05 16 gppse05 360 RAL tbn09 22 tbn03 35 NIKHEF ccgridli03 120 ccgridli07 2 00 LYON ccgridli08 400 cmsgrid001 50 cmsgrid002 513(+513) Legnaro grid001 12 grid005 680 Padova polgrid1 4 polgrid2 220 Ecole Polytechnique gw39 16 fb00 450 Imperial College The CMS Stress Test

20. CMS EDG CE CE CE CE parameters CMS software CMS software CMS software CMS software Job output filtering Runtime monitoring SE JDL data registration Push data or info WN Pull info CMS Stress Test : Architecture of the system SE RefDB BOSS DB Workload Management System SE UI IMPALA/BOSS input data location SE CE Replica Manager SE

21. Nb. of evts time Main results and observations from CMS work • RESULTS • Could distribute and run CMS s/w in EDG environment • Generated ~250K events for physics with ~10,000 jobs in 3 week period • OBSERVATIONS • Were able to quickly add new sites to provide extra resources • Fast turnaround in bug fixing and installing new software • Test was labour intensive (since software was developing and the overall system was initially fragile) • New release EDG 2.0 should fix the major problems providing a system suitable for full integration in distributed production

22. EarthObservation applications (WP9) • Global Ozone (GOME) Satellite Data Processing and Validation by KNMI, IPSL and ESA • The DataGrid testbed provides a collaborative processing environment for 3 geographically distributed EO sites (Holland, France, Italy) 1/7/2020 DataGrid is a project funded by the European Union 22

23. Earth Observation • ESA missions: • about 100 Gbytes of data per day (ERS 1/2) • 500 Gbytes, for the next ENVISAT mission (2002). • DataGrid contribute to EO: • enhance the ability to access high level products • allow reprocessing of large historical archives • improve Earth science complex applications (data fusion, data mining, modelling …) Source: L. Fusco, June 2001 Federico.Carminati , EU review presentation, 1 March 2002

24. ENVISAT • 3500 MEuro programme cost • Launched on February 28, 2002 • 10 instruments on board • 200 Mbps data rate to ground • 400 Tbytes data archived/year • ~100 “standard” products • 10+ dedicated facilities in Europe • ~700 approved science user projects

25. Earth Observation • Two different GOMEprocessing techniques will be investigated • OPERA (Holland) - Tightly coupled - using MPI • NOPREGO (Italy) - Loosely coupled - using Neural Networks • The results are checked by VALIDATION (France). Satellite Observations are compared against ground-based LIDAR measurements coincident in area and time.

26. GOME OZONE Data Processing Model • Level-1 data (raw satellite measurements) are analysed to retrieve actual physical quantities : Level-2 data • Level-2 data provides measurements of OZONE within a vertical column of atmosphere at a given lat/lon location above the Earth’s surface • Coincident data consists of Level-2 data co-registered with LIDAR data (ground-based observations) and compared using statistical methods

27. Raw satellite data from the GOME instrument The EO Data challenge: Processing and validation of 1 year of GOME data Level 1 ESA – KNMI Processing of raw GOME data to ozone profiles With OPERA and NNO LIDAR data IPSL Validate GOME ozone profiles With Ground Based measur. Level 2 DataGrid Visualization

28. Submit job User Interface Replica Manager Replica Catalog Replicate Mydata MetaData Data input data input data Site H input data Site G input data CE Site F SE CE SE Site E CE Site D SE CE SE Site C CE SE Site B CE SE CE SE GOME Processing Steps (1-2) Step 1:Transfer Level1 data to the Grid Storage Element Step 2:Register Level1 data with the ReplicaManager Replicate to other SEs if necessary User

29. Information Index Replica Catalog Certificate Authorities LFN PFN :: LFN PFN :: LFN PFN Check certificate :: Submit job User Interface Resource Broker Search Search Request status Myjob Retrieve result JDL script Executable Site H input data Site G input data CE Site F SE input data CE SE Site E LFN CE Site D SE LFN Logical filename LFN CE SE LFN Site C CE SE Physical filename PFN Site B CE SE CE SE GOME Processing Steps (3-4) Step 3:Submit jobs to process Level1 data, produce Level2 data Step 4:Transfer Level2 data products to the Storage Element User

30. GOME Processing Steps (5-6) Step 5:Produce Level-2 / LIDAR Coincident data Step 6:Visualize Results perform VALIDATION Level 2 LIDAR COINCIDENT DATA Validation

31. Biomedical Applications Genomics, post-genomics, and proteomics Medical images analysis Explore strategies that facilitate the sharing of genomic databases and test grid-aware algorithms for comparative genomics Process the huge amount of data produced by digital imagers in hospitals. Federico.Carminati , EU review presentation, 1 March 2002

32. Biology and Bio-informatics applications • The international community of Biologists has a keen interest in using of bio-informatic algorithms to perform research on the mapping of the human genomic code • Biologist make use of large, geographically distributed databases with already mapped, identified sequences of proteins belonging to sequences of human genetic code (DNA sequencies) • Typical goal of these algorithms is to analyse different databases, related to different samplings, to identify similarities or common parts • dgBLAST (Basic Local Alignment Search Tool) is an example of such an application seeking particular sequences of proteins or DNA in the genomic code

33. Grid technology opens the perspective of large computational power and easy access to heterogeneous data sources. • A grid for health would provide a framework for sharing disk and computing resources, for promoting standards and fostering synergy between bio-informatics and medical informatics • A first biomedical grid is being deployed by the DataGrid IST project http://dbs.cordis.lu/fep-cgi/srchidadb?ACTION=D&SESSION=221592002-10- 18&DOC=27&TBL=EN_PROJ&RCN=EP_RCN_A:63345&CALLER=PROJ_IST

34. Biomedical requirements • High priority jobs • privileged users • Interactivity • communication between user interface and computation • Parallelization • MPI site-wide / grid-wide • Thousands of images • Operated on by 10’s of algorithms • Pipeline processing • pipeline description language / scheduling • Large user community(thousands of users) • anonymous/group login • Data management • data updates and data versioning • Large volume management (a hospital can accumulate TBs of images in a year) • Security • disk / network encryption • Limited response time • fast queues

35. Diverse Users… • Patient • has free access to own medical data • Physician • has complete read access to patients data. Few persons have read/write access. • Researchers • may obtain read access to anonymous medical data for research purposes. Nominative data should be blanked before transmission to these users • Biologist • has free access to public databases. Use web portal to access biology server services. • Chemical/Pharmacological manufacturer • owns private data. Need to control the possible targets for data storage.

36. …and data • Biological Data • Public and private databases • Very fast growth (doubles every 8-12 months) • Frequent updates (versionning) • Heterogenous formats • Medical data • Strong semantic • Distributed over imaging sites • Images and metadata

37. Web portals for biologists • Biologist enters sequences through web interface • Pipelined execution of bio-informatics algorithms • Genomics comparative analysis (thousands of files of ~Gbyte) • Genome comparison takes days of CPU (~n**2) • Phylogenetics • 2D, 3D molecular structure of proteins… • The algorithms are currently executed on a local cluster • Big labs have big clusters … • But growing pressure on resources – Grid will help • More and more biologists • compare larger and larger sequences (whole genomes)… • to more and more genomes… • with fancier and fancier algorithms !!

38. Example GRID application for Biology: dgBLAST • dgBLAST requires as input a given sequence (protein or DNA) to be searched and a pointer to the set of databases to be queried. • Designed for high speed (trade off vs sensitivity) • A score is assigned to every candidate sequence found and the results graphically presented • uses an heuristic algorithm • detects relationships among sequences which share only isolated regions of similarity. • Blastn: compares a nucleotide query sequence against a nucleotide sequence database • Blastp: compares an amino acids query sequence against a protein sequence

39. The Visual DataGrid Blast, a first genomics application on DataGrid • A graphical interface to enter query sequences and select the reference database • A script to execute the BLAST algorithm on the grid • A graphical interface to analyze result • Accessible from the web portal genius.ct.infn.it

41. Other Medical Applications • Complex modelling of anatomical structures • Anatomical and functional models, parallelizatoin • Surgery simulation • Realistic models, real-time constraints • Simulation of MRIs • MRI modelling, artifacts modeling, parallel simulation • Mammographies analysis • Automatic pathologies detection • Shared and distributed data management • Data hierarchy, dynamic indices, optimization, caching

42. Summary ( 1 / 2 ) • HEP, EO and Biology users have deep interest in the deployment and the actual availability of the GRID, boosting their computer power and data storage capacities in an unprecedented way. • Currently evaluating the basic functionality of the tools and their integration into data processing schemes. Will move onto areas of interactive analysis, and more detailed interfacing via APIs • Hopefully experiments will do common work in interfacing applications to GRID under the umbrella of LCG • HEPCAL (Common Use Cases for aHEP Common Application Layer) work will be used as a basis for the integration of Grid tools into the LHC prototype http://lcg.web.cern.ch/LCG/SC2/RTAG4 • There are many grid projects in the world and we must work together with them • e.g. in HEP we have DataTag,Crossgrid,Nordugrid + US Projects(GryPhyn,PPDG,iVDGL)

43. Summary ( 2 / 2 ) • Many challanging issues are facing us : • strengthen effective massive productions on the EDG testbed • keep up the pace with next generation grid computing evolutions, implementing or interfacing them to EDG • further develop middleware components for all EDG workpackages to address growing user’s demands. EDG 2.0 will implement many new functionality.

44. Acknowlegements and references • Thanks to the following who provided material and advice • J Linford(WP9),V Breton(WP10),J Montagnat(WP10),F Carminati(Alice),JJ Blaising(Atlas),C Grandi(CMS),M Frank(LHCb),L Robertson(LCG),D Duellmann(LCG/POOL) ,T Doyle(UK GridPP),O.Maroney (RAL) • F Harris(WP8), I Augustin(WP8) N Brook(LHCb), P Hobson (CMS), J Montagnat (WP10) • Some interesting WEB sites and documents - LHC Review http://lhc-computing-review-public.web.cern.ch/lhc-computing-review-public/Public/Report_final.PDF (LHC Computing Review) • LCG http://lcg.web.cern.ch/LCG http://lcg.web.cern.ch/LCG/SC2/RTAG6 (model for regional centres) http://lcg.web.cern.ch/LCG/SC2/RTAG4 (HEPCAL Grid use cases) • GEANThttp://www.dante.net/geant/ (European Research Networks) • POOLhttp://lcgapp.cern.ch/project/persist/ • WP8 http://datagrid-wp8.web.cern.ch/DataGrid-WP8/ http://edmsoraweb.cern.ch:8001/cedar/doc.info?document_id=332409 ( Requirements) • WP9 http://styx.srin.esa.it/grid http://edmsoraweb.cern.ch:8001/cedar/doc.info?document_id=332411 (Reqts) • WP10 http://marianne.in2p3.fr/datagrid/wp10/ http://www.healthgrid.org http://www.creatis.insa-lyon.fr/MEDIGRID/ http://edmsoraweb.cern.ch:8001/cedar/doc.info?document_id=332412 (Reqts)