1 / 21

CWG9 Data Quality Monitoring, Quality Assurance and Visualization

CWG9 Data Quality Monitoring, Quality Assurance and Visualization. B. von Haller. 11.03.2014. CERN. CWG 9 in a nutshell. Started in May 2013 along with O 2 Group working on the Data Quality Monitoring the Quality Assurance the Visualization For Run 2 and Run 3. CWG9 Members.

caryn-acosta
Download Presentation

CWG9 Data Quality Monitoring, Quality Assurance and Visualization

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CWG9 Data Quality Monitoring, Quality Assurance and Visualization B. von Haller 11.03.2014 CERN

  2. CWG 9 in a nutshell • Started in May 2013 along with O2 • Group working on • the Data Quality Monitoring • the Quality Assurance • the Visualization • For Run 2 and Run 3 B. von Haller | CWG9 DQM-QA-Visualization | 11.03.2014

  3. CWG9 Members • Renu Bala • Francesca Bellini • Mihaela Gheata • Lukasz Kamil Graczykowski • Malgorzata Anna Janik • Andreas Morsch • Mihai Niculescu • Jeremi Niedziela • Ankita Sharma • Maciej Pawel Szymanski • Barthélémy Von Haller B. von Haller | CWG9 DQM-QA-Visualization | 11.03.2014

  4. Definitions (1) Data Quality Monitoring • Onlinefeedback on the quality of data • Make sure to recordhigh quality data • Identify and solve problem(s) early • Data Quality Monitoring (DQM) involves • Online gathering of data • Analysis by user-defined algorithm • Production of monitoring objects such as histograms • Assessment of the quality of the data based on the objects • Storage of monitoring data • Visualization (+ human assessment of quality) B. von Haller | CWG9 DQM-QA-Visualization | 11.03.2014

  5. Definitions (2) Quality Assurance • Offline evaluation of the quality of data • During and after reconstruction and calibration • Make sure to analyze high quality data • Identify high quality runs • QA involves • Analysis by user-defined algorithm • Production of monitoring objects such as histograms • Assessment of the quality of the data based on the objects • Storage of monitoring data • Visualization (+ human assessment of quality) B. von Haller | CWG9 DQM-QA-Visualization | 11.03.2014

  6. Definitions (3) Visualization • Visualization of DQM/QA results • Visualization of data • Event Display • More details in the comingpresentations ! B. von Haller | CWG9 DQM-QA-Visualization | 11.03.2014

  7. Detailed description of CWG9 • Group focusing on the data Quality Assurance, online and offline, and the visualization of its results and the data itself • Run 3 • Study how to monitor data efficiently and in plenty without interfering with the data taking • Discuss QA output and results, incremental QA and procedures to formalize if the results are acceptable or not • Determine the needs, and design the software, to access, visualize and interpret the results • In addition, define and develop the software to visualize data, raw and reconstructed • Run 2 • Production data taking period -> maintain and improve existing software • Opportunity to test concepts and software for Run 3 Storage Dataflow Monitoring Object Generation Automatic QualityAssessment Visualization B. von Haller | CWG9 DQM-QA-Visualization | 11.03.2014

  8. Past and Current tasks • Bring everyone aboard [done] • Make people aware of others’ work in the field • Give a picture of the current situation to everyone • Run 2 • Event Display review and meetings, Warsaw involvement • Core refactoring • New features • Knowledge transfer  Gain stability and support for Run 2 • DQM/QA review and preparation • Proposal for the online reconstruction and calibration • Run 3 • System requirements and system functionalities document [done] • Detectors needs survey • Definition of the future architecture and design • Prototypes and feasibility tests • Technical Design Report redaction B. von Haller | CWG9 DQM-QA-Visualization | 11.03.2014

  9. TDR and feasibility tests • April 2014: First draft • August 2014: Submission • CWG9 participates to • Chapter 4 System architecture : Quality control and assessment • Table 1: detectors needs • Explain DQM/QA architecture and the choices made • Figure 1: DQM/QA architecture • Explain Event Display architecture and the choices made • Figure 2: Event Display architecture • Chapter 5 Technology survey, evaluations and prototypes • Mergers architecture and feasibility tests with 0MQ • Results of storage tests (e.g. DB technologies) • [Web gui architecture (ROOT JS lib + DABC)] • Event display design as tested (cf Run 2) • Chapter 6 System Design B. von Haller | CWG9 DQM-QA-Visualization | 11.03.2014

  10. Longer term work • Run 2 • Maintenance and support • DQM/QA • Event Display  Organisation ? • Run 3 • Prototypes • Implementation • Interaction withusers • Bring inventive new ideas or approaches • E.g. Investigation of complexautomaticchecks • Room for more people joining ! B. von Haller | CWG9 DQM-QA-Visualization | 11.03.2014

  11. Conclusion • CWG9 is an important and dynamic working group in the O2 project • WUT is a key player in CWG9 • In terms of people and responsibility • Crucial for the future of Visualization in ALICE • We are grateful for your involvement so far and for the increasing responsibilities you are taking in the project ! B. von Haller | CWG9 DQM-QA-Visualization | 11.03.2014

  12. Backups B. von Haller | CWG9 DQM-QA-Visualization | 11.03.2014

  13. O2 Technical Design Report Schedule • October ‘13: • Define table of content • Establish editorial board • December ‘13: • System Requirement Document • High-level dataflow model • Computing platforms benchmarks • Networking benchmark • June ‘14 • Software framework architecture • Sep ‘14 • TDR B. von Haller | CWG9 DQM-QA-Visualization | 11.03.2014

  14. CWG9 TDR Timeline Proposal • January 14 • Define list of tables and figures • Draft of the architecture of the system • Launch subsystems exhaustive survey  Submit list of tables and figures to TDR EC • February 14 • Draft tables and figures • Skeleton of 4.2.4 and 5.6  Submit skeleton to TDR EC • March 14 • Finalize tables and figures, including subsystems input • Iterate on text using input of TDR EC  Submit text and final tables and figures to TDR EC • April 14 • Finalize text • Submit final text to TDR EC • May 14 • Iterate over our sections using CWGs input • Review work of other CWGs (especially what concerns us!) B. von Haller | CWG9 DQM-QA-Visualization | 11.03.2014

  15. TDR – skeleton, tables & figures Chapter 4 System architecture • 4.2 Data processing and computing model • 4.2.4 DQM and QA • « Quality control and assessment » • Table 1: detectors needs • Explain architecture • Figure 1: architecture • Explain the choices • Figure 2: Event display arch. B. von Haller | CWG9 DQM-QA-Visualization | 11.03.2014

  16. TDR – skeleton, tables & figures Chapter 5 Technology survey, evaluations and prototypes • 5.6 DQM and QA • « Quality control and assessment » • Technologies and design choices available concerning key points of our system • Storage • Access to results worldwide • Event display • Feasability tests & prototypes • Table 1: results of storage tests (e.g. DB technologies) • Figure 1: Web gui architecture (ROOT JS lib + DABC) • Figure 2: Event display design as tested (cf Run 2) B. von Haller | CWG9 DQM-QA-Visualization | 11.03.2014

  17. Subsystems survey • What “tasks” (name it agents or algorithms if you prefer) will your subsystem need in Run 3 ? • For each of these task or group of tasks, tell us • Whether it already exist today and if so what is its performance. • What is the expected performance of such a task in Run 3. • How many plots are expected to be produced (for the shifter and for the experts). • Percentage of events needed to carry out the task online (minimum, optimal). • What is the input ? i.e. at which stage will it run ? • How fast the response has to be taken into account in the data flow ? • Whether the DQM/QA results have to become persistent and for how long ? • What does “Calibration QA” mean to you ? B. von Haller | CWG9 DQM-QA-Visualization | 11.03.2014

  18. Survey status B. von Haller | CWG9 DQM-QA-Visualization | 11.03.2014

  19. Current and short term work • Run 2 : Prepare a proposal for the QA in relation with the online reco, calibration and monitoring (QA tasks, validation, bridge to DQM, …) • Online Calibration • Mainly for TPC • Many open questions on • Requirements • General architecture • Implementation • CDB • Working on a proposal to meet requirements while minimizing work • Use analysis QA train within HLT for reco monitoring • Use analysis QA train within HLT for calib monitoring • Use AMORE for raw data monitoring • Use AMORE infrastructure for storage and visualization • Run 2 – Review and preparation • Detectors « interviews » • (DATE Monitoring update) • Run 3 : Prepare requirements of the future system following CWG1 input • For the TDR (2014) • Define requirements and general architecture and features of the QA-DQM-Viz for Run 3 • Write it • Event Display • Decentralized model under implementation • Better stability • Split GUI and reconstruction • Possibility to switch between offline and HLT reco • Bookmarks (for users and for PR) • Involvement of the Warsaw group • Implemented by the end of 2013 B. von Haller | CWG9 DQM-QA-Visualization | 11.03.2014

  20. HCDB OCDB DCS, GRP QC (sort of) Reco Selection/filters Calib Calibparam Raw ESDs QC QC Data on Castor Reading Writing Producing DBsdata procedures PHYSICS Run B. von Haller | CWG9 DQM-QA-Visualization | 11.03.2014

More Related