1 / 52

CHEP 2000 Session F: Data Analysis:Technology and Presentation

CHEP 2000 Session F: Data Analysis:Technology and Presentation. Convenors: Francois Etienne - IN2P3, Atlas, Ruth Pordes, FNAL Thank You to all the Speakers for Providing me with their Transparencies to use. 29 Oral presentations; 1 Poster. Outline. Quick Review of Topics/Titles

vic
Download Presentation

CHEP 2000 Session F: Data Analysis:Technology and Presentation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CHEP 2000 Session F: Data Analysis:Technology and Presentation Convenors: Francois Etienne - IN2P3, Atlas, Ruth Pordes, FNAL Thank You to all the Speakers for Providing me with their Transparencies to use.29 Oral presentations; 1 Poster

  2. Outline • Quick Review of Topics/Titles • More Detail for Each Category • Summary

  3. Numerical Library Software - 2 papers • Development of Numerical Library Software in Java – KEK – Initial Work • Scientific Toolkit for Java – Phenix – Really an Analysis Framework • (poster paper not in this session on COLT - Cern/IT) • All in Java

  4. Electronic Document Management systems - 4 papers from 2 systems • Java-based User Interfaces for CERN's Engineering Data Management System • Integrated Information Management for TESLA • Asset Management for Cavities and Computers at DESY • Experience with Document Management at DESY

  5. Singletons: • Bringing Mathematical Formatting to the Web - An Overview of the Technology and SLAC's Experience • Multi-threaded, Discrete Event, Simulation of Distributed Computing Systems – CMS • CORBA Scripting in HEP and Beyond. • ATLAS DAQ back-end Software • Integrated toolset, mainly Open Source • Second use of this will be an interesting test. • Defining deliverables of “pre-design” phase. • How to move to an Open Source development environment for critical piece of Experiment architecture?

  6. Event Displays - 3 talks • Atlantis an Event Display for ATLAS: When displaying 6000 hits recorded by only 7 layers, can we recognize tracks? • WIRED - World-Wide Web Interactive Remote Event Display • HepRep: a Generic Interface Definition for HEP Event Display Representables

  7. General Analysis Packages/Frameworks • Java Analysis Studio • ROOT at RUN II • Open Scientist • A Comparison of Data Analysis Packages (Root and IDL)

  8. Experiment Analysis Packages • A Graphics Sampler for the D0 Experiment • The CMS IGUANA Project • Object Oriented Data Analysis in the DELPHI Experiment • Distributed BELLE Analysis Framework • New Data Analysis Environment in H1

  9. General Analysis Components • Abstract Interfaces for Data Analysis – AIDA – Cern/IT • A Histogramming package in Java - KEK • Lightweight Histogram Interface Layer – from NERSC/LBL

  10. Software Process and Configuration Management - 5 talks all from Europe.. • An Evaluation of Tools for Static Checking of C++ Code • Impact of Software Review and Inspection • C++ Code Analysis: an Open Architecture for the Verification of Coding Rules • CMT: a Software Configuration Management Tool • Use of a Configuration Management Tool in LHCb Software

  11. What there was Not • Software Development Methodologies and Case Tools. • 3-D and Virtual Reality. Overall Impression • Continuation of directions and initiatives from last Chep.

  12. F19 MathML Interface for Abstracts in SLAC Spires • Situation today (finally): • method for defining a mathematical expression in terms of its presentation and semantics • MathML is a standardized instance of XML • Used “IBM TechExplorer Hypermedia Browser” • A Web browser plug-in • Dynamically formats and displays documents coded with TEX/LATEX and/or MathML • Beta version applied to existing Spires TeX files works. • Try it and feed back comments ...

  13. F19 - Tool for Simulation of Complex Distributed Data Handling Systems - • Technology – in house developed JAVA toolkit. • Process driven not event driven architecture • Full presentation and easy editing of parameters • encourage many different models and users • Immediate publication of results on the web • Enable physicists to read, analyse and feedback. • Alternative architectures and designs can be commented with quantitative and analytic input • Technology is ready to be tested for “Simulator Friendliness” … • Volunteers to model the existing Babar and Run II systems would indeed be useful.

  14. F359, F65, F66, - DESY Integrated Information Management • Reminded us of the importance of need for and time required to • Peoples Use and Access Patterns change after they have experience with a working system • Current and Future Business Processes significantly affect design • Users expect latest Technology (currently video, mobile computing etc) • Demonstrated Success of collaborative development of vendor tool for HEP needs

  15. Current Tuovi Status & Statistic • 20 projects with: - 6 GB data vault- 2400 project nodes- 18600 files- 3300 registrated user

  16. Cern EDMS Extensions • Import/Export Interface - define an exchange data format in XML. • Conversion of metadata to and from various formats e.g. Excel, Html, Databases, Text Processors etc. • in use by several LHC groups: • LHC Baseline and ABS • ATLAS: Import/Export of ABS Structures • CMS: ECAL ABS • ALICE: Documents and Files check-in • Documents Management in the LHC Design office (in progress)

  17. Histogramming Packages • Histogrammer in JAVA - Successful interfacing of Java and C++ objects for in a simple Histogrammer application • Lightweight C++ Histogram Interface - Break between Statistical Data Gathering, from analysis and visualization tasks. Define Standard API and Support Object Structures. • How does it relate to transient Cern/HTL implementation

  18. Analysis and Event Display Toolkits and Interfaces • WIRED • HepRep • AIDA • Java Analysis Studio • Scientific Toolkit for Java • Open Scientist • ROOT • IDL - through comparison with ROOT

  19. Wired • Rewrite since Last CHEP • In Use by CHORUS, BaBar, DELPHI etc. • Lacks a Scripting Language. • BaBar Use shows up Need to Think before Defining what Data gets Transported between User Client and Data Server – don’t want a “Packet per Point”

  20. HepRep • Experience with Wired at BaBar led to need to layer interface between Client and Display so as to • reduce granularity in getting remote information • increase ability to for Client/Users to easily add objects and views • HepRep defines a general API for serving Representables, not Physics Objects • “Serving Representables keeps the detailed reconstruction code, swimmers and detector models on the server side where they belong. • Spatial information is assembled and shipped in an efficient manner, avoiding the overhead of too many individual method calls.”

  21. General Comments • Transport the Physics Analysis Code to the Data - JAS – No Answer Yet on Whether this will Work Well. • WIRED and JAS both Talking about Scripting Languages – no decision yet. • Once Package in Use and Responding to User Use Needs, More Core Development More Difficult • Integration with C++ Analysis Algorithms and Data Adaptors Not Here Yet.

  22. AIDA • Part of Next LHC++ Analysis Toolkit – the Start. • HepVis ’99 established Collaboration for Definition of Abstract Interface for Data Analysis • Interface Architected to Support Scripting and GUI Equivalently. • Goal of Adaptors/Interfaces to HepTuple, JAS, Openscientist, Root etc.

  23. JAS • Extensions since last CHEP: • JEdit Editor Bean • Using Swing JEditorPane JAS now supports HTML display • XML support - JASHist bean directly supports reading/writing XML • Servlet Support • Continued use in BaBar Online and LCD Simulations • Packaged and made Available in OpenSource Paradigm - Contributions starting e.g. 3D Classes from Fermilab.

  24. Scientific Toolkit for Java • Explained was a Separate Project to “Learn Technology”. • Use of JavaHelp • Based onJava 2D (JDK 1.2) • Generic Swing Console for those Interpretive Languages not supported by Swing. • Anticipate Discussing Collaboration with / Contributions to JAS. • Starting New Development Allowed use of Latest Generation Technologies

  25. Open Scientist • Work Continuing for Experiments – LHCb, Virgo, Auger etc. • Concept of “Hub” as Integration Layer. What is boundary/layering between Frameworks and Utilities - Do we need More Concepts Here? • Worklist includes Integration of CINT and Extensions to Free Open Inventor Implementation.

  26. ROOT - Run II Perspective • Progress since Run II Adoption of ROOT with Caveats. • Priorities from 1999 Workshop been in Large Part Addressed – Robustness, CINT Extensions, Pre-compilation of user classes. • Tutorials and Education Popular with Run II Users. • To meet Local Needs currently Building 23 Flavors. • Collaboration Works with Technical Commitment • Second ROOT Workshop last week at Cern resulted in New Worklist - now posted – main extension Self Describing Data format.

  27. Comparison of ROOT and IDL: • IDL command language • Concatenate Sets of Files of ntuples - chain command. • Read in and Use Variables from an “ntuple”File - Local Extension. • Event Selection Cuts on Several Variables • Plot - Static or “Liveplot”. • Large ntuple problem addressed by “mapped files”. • Needs test drive for “large scale analysis”.

  28. Experiment Specific Event Displays and Analysis

  29. Mature Experiments Retooling Technology for Analysis – Delphi and H1 reported • Issues include: • Maintenance – Technical and Resources; • Support of Multiple Generations of Technology. • OO, ROOT, C++ - Speed and Usability Benefits • Encourage New Collaborators • Allow and Provide for Curation of the Data

  30. Atlas Event Display - Atlantis • Applying Hans Drevermanns Algorithms to New/Future Experiment • “V-plot” • Z finder • Hit filtering • Plan Recode from Fortran to Java.

  31. Experiment Event Displays – D0 • Suite of Different Tools based on Legacy and New Packages. • New Tools based on Open Inventor. Will Evolve. • Finding very useful for algorithm development and starting for analysis of processed Monte Carlo data

  32. D Graphics Sampler Rest of D0 software D Graphics HEPVis Xi Graphics Exceed 3D Open Inventor Mesa OpenGL

  33. CMS - IGUANA • On the way to being a Fully Featured Data Presentation and Manipulation Framework. • Will be used in CMS Simulation and Test Beam Analysis this Year. • Pragmatic Approach to getting Prototypes and Early Production Releases. • Based on Open Toolkits and Open Inventor. • “Sharing experience may be more important than sharing code”- aligned with LHC++ strategy.

  34. 6 (b) HEPInventor & OpenInventor, Motif • 6(c) SciPlot • (Xt / Qt) 6(a) HepExplorer & IRIS Explorer IGUANA Objectivity browsing IGUANA Objectivity tag analyser (tag is similar to ntuple) 6(d) Qwt & Qt Preferred Solutions IGUANA Prototypes: Objectivity Browsers and Persistent Histograms and Tags Plotters

  35. BELLE – BASF Analysis Framework • Currently supports Single SMP Parallel Processing of Events for Users. • dBASF Extension being Designed to Support Network Connected Clusters of PCs. Package of Daemons for • Job Management • Data I/O • Resource Management • Monitoring. • Resource Managers Dynamically Reconfigures – Increase, Decrease - Number of Computing Servers Running Jobs.

  36. HEP Configuration Management Tools • Report from 2 users of CMT – Build and Release System. • Poster papers on SRT Rewrite, NT Extensions • From Marjorie’s Plenary Talk: “Everyone now using them, can’t we use the same one…”

  37. jcmt Package areas browser Packages & versions Free action onto the package editor Requirements editor Configuration monitor monitor

  38. F63 - C++ Code Rules and Checking • C++ Coding Standard – 180 Rules for Naming, Coding and Style of Varying Severity for Each Experiment. • Evaluation of Commercial tools for Code Rule Checking. Tested on Geant4 and Atlas Libraries. • Out of 5 tools considered 2 Meet the Basic Needs – about 75% of total: CodeWizard and QA C++. • Now Traditional Questions on: “What does it take to Decide?” “Will IT/CD Support It?”

  39. F361 - C++ Code Rules and Checking - Alice • Development of Code Analysis Tools - Parser, Formatter, Checker • Limitations in Checking Complete C++ syntax e.g. Templates, Namespace • Successfully used on ALICE code

  40. F331 - Atlas Software Review and Inspections • Movement in HEP from Fagan Inspections (Fermilab SDSS/DART/Run II) to Gilb Inspection Method • Bodes Well that this is Part of Formal Development of Proposed OpenSource Project..

  41. F119 - Applications using CORBA Script – continuation of work reported at Chep ‘98. • Interpreted Scripting Language which can interact with CORBA objects through IIOP and has dynamic access to any CORBA object. • OMG Standard • Interface Definitions Stored in Repository • Applications in Babar Data Handling System

  42. Summary – TechnologyJava, XML, CORBA • JAVA • Mainstream for New Presentation Applications. • Not Mainstream for Data Analysis and Reconstruction. • Work Not yet Done to Integrate with C++ Algorithm Modules and Frameworks. • Work only just Beginning on HENP Utility Libraries.

  43. Summary – Physics Analysis Packages • Not much development using Commercial Analysis Tools. • ROOT most Generally Used; Extensions still being requested to meet Experiment Long Term Needs. Open Source Collaboration without Open Source licence? • JAS, OpenScientist, WIRED, progressing steadily. • Question still Open as to How or Whether any of the Existing Analysis Technologies will Successfully meet Needs of >500 Physicists doing Analysis of Babar and Run II over the next 10 Years.

More Related