1 / 13

ATLAS-Specific Activity in GridPP

ATLAS-Specific Activity in GridPP. EDG Integration LCG Integration Metadata. Personalia. Alvin Tan (Birmingham) (3.1.4,3.1.7) Fully funded by GridPP to work on MC and production Clear link with GANGA activity, to which he contributes. Frederic Brochu: (3.1.7, 3.1.9)

ninon
Download Presentation

ATLAS-Specific Activity in GridPP

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ATLAS-Specific Activity in GridPP EDG Integration LCG Integration Metadata

  2. Personalia • Alvin Tan (Birmingham) (3.1.4,3.1.7) • Fully funded by GridPP to work on MC and production • Clear link with GANGA activity, to which he contributes • Frederic Brochu: (3.1.7, 3.1.9) • Half-funded by GridPP • Integration of EDG and LCG into ATLAS Framework • Data Challenges GridPP8 meeting, Bristol

  3. Mike Gardner (-2003) • Fully funded post at RHUL (3.1.9) • Installation and packaging tools • Analysis architectures • As alluded to in last report, Mike was ill and off work from the end of Q2 • Despite encouraging signs, Mike died of liver cancer on Sunday 14th September • Last contribution a very nice paper for All Hands 03; we intend to publish a fuller version, with a dedication • We will discuss the continuation of the post over the next week or so GridPP8 meeting, Bristol

  4. EDG Integration (3.1.4, 3.1.9) • UK integrating validating EDG grid middleware for ATLAS • EDG1.2 on the core sites revealed many problems with Resource Broker saturation and the use of a single Replica Catalogue (solved with RLS) • The job success rate was only 70%. • Recent UK mini production used • EDG 1.4 • input data stored on RAL’s tape server, • the requirements in JDL • the IC Resource Broker • a boxed set executables. The test took only 1 week, 1 operator and 3000 SpecInt95 days. The success rate was higher than 90 %. Not suitable for production but encouraging step towards a brokered production system. GridPP8 meeting, Bristol

  5. LCG-1 Integration & Validation (3.1.4) • Partially functional LCG-1 release in July: • Full functionality by October • Pool integration underway, but bugs being fixed • LCG-1 is only just ready for integration and validation: • Deliverable will be missed because of late delivery (3 months) • ATLAS-LCG team (FB a key member) now becomes ATLAS-EDG team • EDG-2 is also just ready for integration and validation: • We will also integrate this • We will have another mini-production GridPP8 meeting, Bristol

  6. Installation & Packaging (3.1.9) • Despite the loss of Mike, I&P activity continues: • Incorporation into ATLAS standard release (internal deliverable by Dec) • Extension to analysis code deployment • SCRAM/DAR: • ATLAS has committed to remaining with CMT, pacman, so this remains an active line of development • Deliverables: • 1.2.9.3.1 production of the development kit (started) expected end 12/03 • 1.2.9.3.1.1 deployment of the development kit on test site start: 08/03 end : 10/03 • 1.2.9.3.1.2 integration with Grid infrastructure start: 09/03 end : 12/03 • 1.2.9.3.1.3 systematic deployment of the development kit start: 10/03 end : 12/03 • 1.2.9.3.2 production of the full source kit start: 09/03 end : 12/03 • Looking at ways to make use of pacman from Ganga • Concentration on deployment for analysis GridPP8 meeting, Bristol

  7. Data Challenges (3.1.9) • Data Challenge 2 is the key test of the Grid Computing Model • Tools must be ready and tested for intensive tests in April-June (1 week bursts) • Unlike DC1, data produced is not the main aim, and the computing is the client • Frederic Brochu directs the DC in the UK • DC Tools Task Force now created • GANGA acknowledged as key input tool to the DC • Alvin Tan is on the DCTF • Integrating AMI, MAGDA etc • Defining required bookkeeping and Metadata services • Job analysis skeleton exists (SG, MG); this needs a lot more work for a production version http://www.pp.rhul.ac.uk/~gardner/sub.html GridPP8 meeting, Bristol

  8. Test Bench –Data Challenges • DC 1 Jul 2002-May 2003 • Showed the many resources available (hardware, willing people) • Made clear the need for integrated system • Some tests of Grid software • Mainly driven by HLT and Physics Workshop needs • One external driver is sustainable, two is not! • DC2 April-July 04 • Real test of computing model for computing TDR • Must use Grid systems • Analysis and calibration + reconstruction and simulation • Pre-production period (Nov03…) then 1-week intensive tests • DC3 05/06 • Physics readiness TDR. • Big increase in scale GridPP8 meeting, Bristol

  9. DC2: April – July 2004 • At this stage the goal includes: • Full use of Geant4; POOL; LCG applications • Pile-up and digitization in Athena • Deployment of the complete Event Data Model and the Detector Description • Simulation of full ATLAS and 2004 combined Testbeam • Test the calibration and alignment procedures • Use widely the GRID middleware and tools • Large scale physics analysis • Computing model studies (document end 2004) • Run as much as possible the production on LCG-1 • Combined Test beam operation foreseen as concurrent with DC2 and using same tools GridPP8 meeting, Bristol

  10. DC2:Scenario & Time scale Put in place, understand & validate: Geant4; POOL; LCG applications Event Data Model Digitization; pile-up; byte-stream  Conversion of DC1 data to POOL; large scale persistency tests and reconstruction Testing and validation Run test-production Start final validation Start simulation; Pile-up & digitization Event mixing Transfer data to CERN Intensive Reconstruction on “Tier0” Distribution of ESD & AOD Calibration; alignment Start Physics analysis Reprocessing September 03: Release7 Mid-November 03: pre-production release February 27th 04: Release 8 (production) April 1st 04: June 1st 04: “DC2” July 15th Test beam runs in parallel

  11. Metadata (3.1.9, GridPP2) • Metadata definition and structure is a crucial issue for ATLAS DC2 • Metadata must not only describe (collections of) files but (collections of events) and even at a sub-event level • POOL provides one layer of metadata – how does this interact with RC etc? • We require metadata services (middleware issue) but also an instantiation and design for our own • Metadata workshop in Oxford, 22-24th July 2003 (RWLJ, AS, AT in attendance) • Initial designs evolving • GANGA must allow metadata queries, dynamic collection definition • Metadata service integration, query services, metadata design will be key elements of the ATLAS GridPP2 activity. GridPP8 meeting, Bristol

  12. Papers/Presentations • General ATLAS-UK Grid poster at Lepton-Photon 03 • Installation & packaging paper and talk at All Hands • GANGA paper & talk at all hands • MC production system paper & poster at All Hands 03 • Integration and validation paper & poster at All Hands 03 GridPP8 meeting, Bristol

  13. Conclusions • The GridPP effort for ATLAS has rightly focussed on GANGA • The associated effort tries to realise the services that are needed for ATLAS production and analysis • Installation and Packaging • EDG and LCG integration and validation • The DCs continue to test and validate the Grid tools • Good progress, but Mike Gardner is a sad loss to the project • The metadata services/schema/query tools/bookkeeping etc are moving onto the critical path and will be a major focus in GridPP2 GridPP8 meeting, Bristol

More Related