1 / 16

COMA Status Report

COMA Status Report. Elizabeth Gallas (Oxford) Database Session during ATLAS Software and Computing Week June 4, 2014. Outline:. Some progress since last SC Week (Feb 2014) COMA Conditions DB Metadata – new in production ! Folder Classifications in COMA

ferrol
Download Presentation

COMA Status Report

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. COMA Status Report Elizabeth Gallas (Oxford) Database Session during ATLAS Software and Computing Week June 4, 2014

  2. Outline: • Some progress since last SC Week (Feb 2014) • COMA Conditions DB Metadata – new in production ! • Folder Classifications in COMA • Discussed in last SC Week Conditions DB meetings … • New COMA Folder Tag Report • Show evolution of folder tags with global tags • Requested by COOL Tag Coordination • COMA Run-wise metadata – development final stage • GRL (Good Run List) XML Metadata • LS1 plans • Integration possibilities with Event Index • Improvements in Conditions DB metadata content • Getting ready for Run 2 operations COMA

  3. Folder Classification: Motivation, Status • Discussed in many meetings since the last SC Week • Why classify ? • Scale of the problem: 926 folders in COMP200 • Know which folders are obsolete … which should be in BK Global Tag(s) … Understand conditions evolution and progress for Run 2 … Anticipate folders and volume for Run 2 Conditions DB … Use to develop central tools • March 2014: implemented for real data • All folders to be classified were written to AFS files • Subsystem experts: classify each folder as described in https://twiki.cern.ch/twiki/bin/viewauth/AtlasComputing/LS1ConditionsDB • Classifications are uploaded daily to COMA • Available in COMA Browser and Reports • Available to central tools (Andrea Formica talk today) • Used in folder creation and population into test instances • System designed to apply also to MC if desired COMA

  4. Folder Classification Completeness For reference: • Twiki: https://twiki.cern.ch/twiki/bin/viewauth/AtlasComputing/LS1ConditionsDB • AFS files (one file per subsystem) in /afs/cern.ch/atlas/conditions/poolcond/buffer/COOLFoldersInLS1/ • COMA interface: • https://atlas-tagservices.cern.ch/RBR/rBR_CB_Report.php % of Folders Classified (per subsystem): COMA

  5. Classification keywords (as in TWiki): • General Folder Classifications (replace ‘Unknown’ with): • Global Tag Folder Classifications (repl ‘Unknown_BK’ with): E.Gallas

  6. Browser Menu • Choose your system • (Choose any other criteria) • Choose one of the reports •  Short Folder Report(simple) E.Gallas

  7. ShortReport: SCT • Example input: • System: SCT • Current Global Tag • Click on Short Folder Report • Report: short summary of SCT folders in this GTag E.Gallas

  8. New: COMA Folder Tag Report • Purpose: Show Folders/Tags over 2/more Global Tags • Usage: • Choose any System, Folder … criteria • Select one/more subsystems, folder names, paths etc. • Choose: 2/more Global Tag criteria in many ways: • Most obvious: • By Global Tag name • Input list ‘,’, range ‘:’, wildcards: (‘*’) • Under Expert Menu Inputs: • By Global Tag creation dates • By Global Tag State designations • Note: input Global Tags can cross instances • See which folders are used in MC and real data GTags • Eventually: which folders in COMP200, CONDBR2 Gtags Results order by: System, FullPath, Global Tag, Online/Offline COMA

  9. Use cases: COMA Folder Tag Report • Use cases: • Which folders/tags changed in Current, Next Tag • GTag criteria (comma separated list): • COMCOND-BLKPA-RUN1-05, COMCOND-BLKPA-RUN1-06 • Which folders/tags are used in Current, CurrentHLT • GTag criteria (comma separated list): • COMCOND-BLKPA-RUN1-05, COMCOND-HLTP-004-03 • What is evolution of folders/tags over C*run1* GTags: • GTag criteria (using wildcards): • COMCOND*RUN1* • What is evolution of folders/tags from C*run1*03 to 06: • GTag criteria (using a range): • COMCOND-BLKPA-RUN1-03:COMCOND-BLKPA-RUN1-06 • Input Global Tags can cross instances ! • GTag criteria (an MC and data Global Tag): • OFLCOND-RUN12-SDR-01, COMCOND-BLKPA-RUN1-05 COMA

  10. Revised: COMA Folder Browser Menu • What’s new: • Folder and Global Tag • Not only lists, wildcards • Now also ranges (using ‘:’) • New Folder Tag Report Button COMA

  11. Example: Folder Tag Report COMA

  12. Example: Folder Tag Report COMA

  13. GRL Metadata in COMA • New Oxford student this year: • Lydia Beresford • Project: adding GRL Metadata to COMA • GRLs are a lot like “Data Periods”: they are a “set of runs” • So GRL criteria can be used as any other ‘temporal’ criteria as additional input to customize existing COMA Reports • The implementation: • Enhance COMA schema: added 7 new tables • Storing: File references, their Runs and LB ranges and associated luminosity, criteria used to generate the XML from runQuery, active Streams and event counts during good Run/LB ranges • Develop loading program to load the new tables • Input: GRL XML files on AFS • Excluding the “archive” directory … unsure if this is needed • Develop interfaces (GRL Browser and GRL Reports) • When it goes to production status: • Loading program will run daily (or on demand): • Inserts,Updates,Deletes GRL metadata in sync w/evolving AFS files COMA

  14. Revised COMA Portal • General Reports: •  GRL criteria added to selection criteria • GRL Browser: • New interface in development https://atlas-tagservices.cern.ch/tagservices/RunBrowser/index.html COMA

  15. Status: GRL Metadata in COMA • Meet with DQ and Data Prep Coordinators later this week • Current Status: • Loading program nearing production status • Improving COMA in the area of “Active Streams” • For all Runs in COMA (beyond GRLs) including debug streams • Interfaces: • Preliminary GRL Browser • Works a lot like the Conditions DB Folder Browser • Preliminary GRL Reports • Show GRL metadata … Luminosity and event counts per stream • LB wise for physics, express streams • Include links to related COMA reports imposing chosen GRL • Multi-run report: all runs in the GRL • Project Trigger report: active triggers with luminosity in chosen GRL • Prescale Reports: report Run-wise trigger activation during GRL • Goal: preliminary system in production June 24 (3 weeks) • Show the new system in the upcoming Software Tutorial • Note: This system does not generate GRL. • The runQuery utility remains the recommended GRL generator COMA

  16. COMA Current/LS1 plans • Put GRL metadata system into production • Still some work in loading and interface code • Integration possibilities with Event Index • Now using trigger menus from COMA for real data • Look into MC menus and event count cross checks • Conditions DB metadata content/loading refinements • Continual effort to improve synchronization with COOL • COMA currently contains only leaf and global tags • Adding branch tags ? Needed by CTB • Getting ready for Run 2 operations • Adjusting COMA to handle changes in data sources • Special thanks to Andrea Formica ! • For advice and code for improving COMA content and loading improvements. COMA

More Related