1 / 22

CWG9 and Event Display

CWG9 and Event Display. B. von Haller. 17.07.2014. CERN. CWG 9 in a nutshell. Started in May 2013 along with O 2 Group working on the Data Quality Monitoring the Quality Assurance the Visualization For Run 2 and Run 3 . CWG9 Members. Renu Bala Francesca Bellini Mihaela Gheata

marv
Download Presentation

CWG9 and Event Display

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CWG9 and Event Display B. von Haller 17.07.2014 CERN

  2. CWG 9 in a nutshell • Started in May 2013 along with O2 • Group working on • the Data Quality Monitoring • the Quality Assurance • the Visualization • For Run 2 and Run 3 B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014

  3. CWG9 Members • RenuBala • Francesca Bellini • MihaelaGheata • Lukasz Graczykowski • MalgorzataJanik • Andreas Morsch • MihaiNiculescu • JeremiNiedziela • Ankita Sharma • Maciej Szymanski • Barthélémy Von Haller B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014

  4. Definitions of DQM and QA Data Quality Monitoring & Quality Assurance • Feedback on the quality of data • Online (DQM) • Make sure to recordhigh quality data • Identify and solve problem(s) early • Offline (QA) • Make sure to analyze high quality data • Identify high quality runs • Involves • [Online gathering of data] • Analysis by user-defined algorithm • Production of monitoring objects such as histograms • Assessment of the quality of the data based on the objects • Storage of monitoring data • Visualization (+ human assessment of quality) B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014

  5. CWG9 Tasks • Group focusing on the data Quality Assurance, online and offline, and the visualization of its results and the data itself • Run 3 • Study how to monitor data efficiently and in plenty without interfering with the data taking • Discuss QA output and results, incremental QA and procedures to formalize if the results are acceptable or not • Determine the needs, and design the software, to access, visualize and interpret the results • Define and develop the software to visualize data, raw and reconstructed • Participate to the writing of the Technical Design Report and to the possible prototyping • Run 2 • Production data taking period -> coordinate maintenance and improvements of software • Opportunity to test concepts and software for Run 3 Storage Dataflow Monitoring Object Generation Automatic QualityAssessment Visualization B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014

  6. Event Display People currentlyworkingdirectly on ED Maintenance and developmenttaken over by Warsaw group • JeremiNiedziela – PhDstudentat CERN • MaciejSzymański – Service Task on ED • Jakub Sala, Jakub Abelski, Adam Felis – Summerstudents • Warsaw group @ WUT for support • Mihai – Offline, former ED developer B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014

  7. Event Display Offline recoEvent Display Off.Reco Off. Reco ED ED Run 2 tasks • Run 2 requirements • Split reconstruction and online ED • Allow multiple sources and unifyexistingEDs • Bookmarks • Under development by J. Niedziela and M. Szymanski • Workwill continue over the summer • Readyfor the commissioning and cosmics HLT HLT HLT HLTED HLTED B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014

  8. Future work • Event Display • Alternativeplatforms (mostly web and/or mobile) • SummerstudentsfromFaculty of Mathematics and Computer Science startedworking on it • Longer term collaboration on thistopicis possible in order to implement a full fledgeevent display • Possible use of earlyalfa components • Prototypingneededhere • DQM/QA • Prototype of merging QA objectsusingmap-reducewithZeroMQ • Web based ROOT objects browser B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014

  9. Future work – CWG10 Control & Configuration • Control and Configuration of hierarchical distributed systems • Development of a zeroMQ-based prototype for configuration distribution and process control • Evaluation of performance based on number of nodes, hierarchical levels, processes and configuration size • Process Management • Evaluation of technologies for process execution in sandbox/container (Docker, etc.) • Evaluation of performance based on number of nodes, hierarchical levels, processes • Evaluation of ZooKeeper for possible usage in O2 • Configuration management • Synchronization • Process control B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014

  10. Conclusion • CWG9 is an important and dynamic working group in the O2 project • WUT is a key player in CWG9 • In terms of people and responsibility • Crucial for the future of Visualization in ALICE • The ED is already profiting greatly from its involvement • There are many more tasks and opportunities for collaboration in CWGs B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014

  11. B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014

  12. Backups B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014

  13. TDR and feasibility tests • April 2014: First draft • August 2014: Submission • CWG9 participates to • Chapter 4 System architecture : Quality control and assessment • Table 1: detectors needs • Explain DQM/QA architecture and the choices made • Figure 1: DQM/QA architecture • Explain Event Display architecture and the choices made • Figure 2: Event Display architecture • Chapter 5 Technology survey, evaluations and prototypes • Mergers architecture and feasibility tests with 0MQ • Results of storage tests (e.g. DB technologies) • [Web gui architecture (ROOT JS lib + DABC)] • Event display design as tested (cf Run 2) • Chapter 6 System Design B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014

  14. Past and Current tasks • Bring everyone aboard [done] • Make people aware of others’ work in the field • Give a picture of the current situation to everyone • Run 2 • Event Display review and meetings, Warsaw involvement • Core refactoring • New features • Knowledge transfer  Gain stability and support for Run 2 • DQM/QA review and preparation • Proposal for the online reconstruction and calibration • Run 3 • System requirements and system functionalities document [done] • Detectors needs survey • Definition of the future architecture and design • Prototypes and feasibility tests • Technical Design Report redaction B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014

  15. O2 Technical Design Report Schedule • October ‘13: • Define table of content • Establish editorial board • December ‘13: • System Requirement Document • High-level dataflow model • Computing platforms benchmarks • Networking benchmark • June ‘14 • Software framework architecture • Sep ‘14 • TDR B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014

  16. CWG9 TDR Timeline Proposal • January 14 • Define list of tables and figures • Draft of the architecture of the system • Launch subsystems exhaustive survey  Submit list of tables and figures to TDR EC • February 14 • Draft tables and figures • Skeleton of 4.2.4 and 5.6  Submit skeleton to TDR EC • March 14 • Finalize tables and figures, including subsystems input • Iterate on text using input of TDR EC  Submit text and final tables and figures to TDR EC • April 14 • Finalize text • Submit final text to TDR EC • May 14 • Iterate over our sections using CWGs input • Review work of other CWGs (especially what concerns us!) B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014

  17. TDR – skeleton, tables & figures Chapter 4 System architecture • 4.2 Data processing and computing model • 4.2.4 DQM and QA • « Quality control and assessment » • Table 1: detectors needs • Explain architecture • Figure 1: architecture • Explain the choices • Figure 2: Event display arch. B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014

  18. TDR – skeleton, tables & figures Chapter 5 Technology survey, evaluations and prototypes • 5.6 DQM and QA • « Quality control and assessment » • Technologies and design choices available concerning key points of our system • Storage • Access to results worldwide • Event display • Feasability tests & prototypes • Table 1: results of storage tests (e.g. DB technologies) • Figure 1: Web gui architecture (ROOT JS lib + DABC) • Figure 2: Event display design as tested (cf Run 2) B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014

  19. Subsystems survey • What “tasks” (name it agents or algorithms if you prefer) will your subsystem need in Run 3 ? • For each of these task or group of tasks, tell us • Whether it already exist today and if so what is its performance. • What is the expected performance of such a task in Run 3. • How many plots are expected to be produced (for the shifter and for the experts). • Percentage of events needed to carry out the task online (minimum, optimal). • What is the input ? i.e. at which stage will it run ? • How fast the response has to be taken into account in the data flow ? • Whether the DQM/QA results have to become persistent and for how long ? • What does “Calibration QA” mean to you ? B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014

  20. Survey status B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014

  21. Current and short term work • Run 2 : Prepare a proposal for the QA in relation with the online reco, calibration and monitoring (QA tasks, validation, bridge to DQM, …) • Online Calibration • Mainly for TPC • Many open questions on • Requirements • General architecture • Implementation • CDB • Working on a proposal to meet requirements while minimizing work • Use analysis QA train within HLT for reco monitoring • Use analysis QA train within HLT for calib monitoring • Use AMORE for raw data monitoring • Use AMORE infrastructure for storage and visualization • Run 2 – Review and preparation • Detectors « interviews » • (DATE Monitoring update) • Run 3 : Prepare requirements of the future system following CWG1 input • For the TDR (2014) • Define requirements and general architecture and features of the QA-DQM-Viz for Run 3 • Write it • Event Display • Decentralized model under implementation • Better stability • Split GUI and reconstruction • Possibility to switch between offline and HLT reco • Bookmarks (for users and for PR) • Involvement of the Warsaw group • Implemented by the end of 2013 B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014

  22. HCDB OCDB DCS, GRP QC (sort of) Reco Selection/filters Calib Calibparam Raw ESDs QC QC Data on Castor Reading Writing Producing DBsdata procedures PHYSICS Run B. von Haller | CWG9 DQM-QA-Visualization | 17.07.2014

More Related