230 likes | 375 Views
ATLAS Data Challenges. US ATLAS Physics & Computing ANL October 30th 2001 Gilbert Poulard CERN EP-ATC. Outline. ATLAS Data Challenges & “LHC Computing Grid Project” Goals Scenarios Organization. From CERN Computing Review. CERN Computing Review (December 1999 - February 2001)
E N D
ATLAS Data Challenges US ATLAS Physics & Computing ANL October 30th 2001 Gilbert Poulard CERN EP-ATC
Outline • ATLAS Data Challenges & “LHC Computing Grid Project” • Goals • Scenarios • Organization ATLAS Plenary-18 October 2001
From CERN Computing Review CERN Computing Review (December 1999 - February 2001) Recommendations: • organize the computing for the LHC era • LHC Grid project • Phase 1: Development & prototyping (2001-2004) • Phase 2: Installation of the 1st production system (2005-2007) • Software & Computing Committee (SC2) • Project accepted by the CERN council (20 September) • Ask the experiments to validate their Computing model by iterating on a set of Data Challenges of increasing complexity • However DC’s were in our plans ATLAS Plenary-18 October 2001
LHC Computing GRID projectPhase1: • Prototype construction • develop Grid middleware • acquire experience with high-speed wide-area network • develop model for distributed analysis • adapt LHC applications • deploy a prototype (CERN+Tier1+Tier2) • Software • complete the development of the 1st version of the physics application and enable them for the distributed grid model • develop & support common libraries, tools & frameworks • including simulation, analysis, data management, ... • in parallel LHC collaborations must develop and deploy the first version of their core software ATLAS Plenary-18 October 2001
ATLAS Data challenges • Goal • understand and validate: • our computing model, our data model and our software • our technology choices • How? • In iterating on a set of DCs of increasing complexity • Start with data which looks like real data • Run the filtering and reconstruction chain • Store the output data into our database • Run the analysis • Produce physics results • To study • Performances issues, database technologies, analysis scenarios, ... • To identify • weaknesses, bottle necks, etc… (but also good points) ATLAS Plenary-18 October 2001
ATLAS Data challenges • But: • Today we don’t have ‘real data’ • Needs to produce ‘simulated data’ first • so: • Physics Event generation • Simulation • Pile-up • Detector response • Plus reconstruction and analysis will be part of the first Data Challenges • we need also to “satisfy” the ATLAS communities • HLT, Physics groups, ... ATLAS Plenary-18 October 2001
ATLAS Data Challenges: DC0 • DC0 November-December 2001 • 'continuity' test through the software chain • aim is primarily to check the state of readiness for DC1 • We plan ~100k Z+jet events, or similar • To validate the software: • issues to be checked include • G3 simulation running with the ‘latest’ version of the geometry • reconstruction running • Re-analyze part of the Physics TDR data “reading from & writing to Objectivity” • Would test the “Objy database infrastructure” • Complementary to the “continuity test” ATLAS Plenary-18 October 2001
ATLAS Data Challenges: DC1 • DC1 February-July 2002 • reconstruction & analysis on a large scale • learn about data model; I/O performances; identify bottle necks … • use of GRID as and when possible and appropriate • data management • Use (evaluate) more than one database technology (Objectivity and ROOT I/O) • Learn about distributed analysis • should involve CERN & outside-CERN sites • site planning is going on, an incomplete list already includes sites from Canada, France, Italy, Japan, UK, US, Russia • scale 107 events in 10-20 days, O(1000) PC’s • data needed by HLT & Physics groups (others?) • Study performance of Athena and algorithms for use in HLT • simulation & pile-up will play an important role • checking of Geant4 versus Geant3 ATLAS Plenary-18 October 2001
ATLAS Data Challenges: DC1 • DC1 will have two distinct phases • First, production of events for HLT TDR, where the primary concern is delivery of events to HLT community; • Second, testing of software (G4, dBases, detector description,etc.) with delivery of events for physics studies • Software will change between these two phases • Simulation & pile-up will be of great importance • strategy to be defined (I/O rate, number of “event” servers?) • As we want to do it ‘world-wide’ we will ‘port’ our software to the GRID environment and use as much as possible the GRID middleware (ATLAS kit to be prepared) ATLAS Plenary-18 October 2001
ATLAS Data Challenges: DC2 • DC2 Spring-Autumn 2003 • Scope will depend on what has and has not been achieved in DC0 & DC1 • At this stage the goal includes: • Use of ‘TestBed’ which will be built in the context of the Phase 1 of the “LHC Computing Grid Project” • Scale at a sample of 108 events • System at a complexity ~50% of 2006-2007 system • Extensive use of the GRID middleware • Geant4 should play a major role • Physics samples could(should) have ‘hidden’ new physics • Calibration and alignment procedures should be tested • May be to be synchronized with “Grid” developments ATLAS Plenary-18 October 2001
DC scenario • Production Chain: • Event generation • Detector Simulation • Pile-up • Detectors responses • Reconstruction • Analysis These steps should be as independent as possible ATLAS Plenary-18 October 2001
Production stream “OO-db” is used for “OO database”, it could be Objectivity, ROOT/IO, … ATLAS Plenary-18 October 2001
DC1 Ntuple ATLFAST OO Obj.,Root Pythia, Isajet, Herwig, MyGeneratorModule Ntuple- like HepMC Obj.,Root Ntuple Comb. Ntuple GENZ ATHENA reconstruction Comb. Ntuple Obj., Root RD event ? OO-DB ? G3/DICE Obj. ZEBRA Missing: -- filter, trigger -- Detector description -- HepMC in Root -- Digitisation -- ATLFAST output in Root (TObjects) -- Pile-up -- Link MC truth - ATLFAST -- Reconstruction output in Obj., Root -- EDM (e.g. G3/DICE , G4 input to ATHENA) G4 ATLAS Plenary-18 October 2001
Detector Simulation • Geant3 and Geant4 • For HLT & physics studies we will use Geant3 • Continuity with past studies • ATLAS validation of Geant4 is proceeding well but not completed • Detector simulation in Atlsim (Zebra output) • Some production with Geant4 too • Goals to be defined with G4 and Physics groups • It is important to get experience with ‘large production’ as part of G4 validation • it is important to use the same geometry input • In the early stage we could decide to use only part of the detector • it would also be good to use the same sample of generated events • Detector simulation we propose to use the FADS/Goofy framework • Output will be ‘Hits collections’ in OO-db • Detector responses (& pileup) has to be worked on in new framework ATLAS Plenary-18 October 2001
Reconstruction • Reconstruction • we want to use the ‘new reconstruction’ code being run in Athena framework • Input should be from OO-db • Output in OO-db: • ESD (event summary data) • AOD (analysis object data) • TAG (event tag) ATLAS Plenary-18 October 2001
Analysis • We are just starting to work on this but • Analysis tools evaluation should be part of the DC • It will be a good test of the Event Data Model • Performance issues should be evaluated • Analysis scenario • It is important to know the number of analysis groups, the number of physicists per group, the number of people who want to access the data at the same time • It is of ‘first’ importance to ‘design’ the analysis environment • to measure the response time • to identify the bottle necks • for that users’ input is needed ATLAS Plenary-18 October 2001
Data management • It is a key issue • Evaluation of more than one technology is part of DC1 • Infrastructure has to be put in place: • For Objectivity & ROOT I/O • Software, hardware, tools to manage the data • creation, replication, distribution • discussed in database workshop • Tools are needed to run the production • “bookkeeping” , “cataloguing” … • Run number, random number allocation, … • Working group now set-up • Job submission • Close collaboration with ATLAS Grid (validation of Release 1) ATLAS Plenary-18 October 2001
DC1-HLT - CPU ATLAS Plenary-18 October 2001
DC1-HLT - data ATLAS Plenary-18 October 2001
DC1-HLT data with pile-up • Inaddition to ‘simulated’ data, assuming ‘filtering’ after simulation (~14% of the events kept). • (1) keeping only ‘digits’ • (2) keeping ‘digits’ and ‘hits’ ATLAS Plenary-18 October 2001
Ramp-up scenario @ CERN Week in 2002 ATLAS Plenary-18 October 2001
What next • This week: • Have an updated list of goals & requirements prepared with • HLT, Physics communities • simulation, reconstruction, database communities • people working on ‘infrastructure’ activities • bookkeeping, cataloguing, ... • Have a list of tasks • Some Physics oriented • But also like testing code, running production, … • with ‘established’ responsibilities and priorities • And working groups in place • You can join ATLAS Plenary-18 October 2001
What next • In parallel • Define ATLAS validation plan for • EU-DataGrid Release 1 • ATLAS software for DC0 and DC1 • Understand the involvement of Tier centers • Insure that we have the necessary resources • @ CERN and outside CERN • We have already some input and a ‘table’ is being prepared • “And turn the key” ATLAS Plenary-18 October 2001