1 / 24

Panoramix

Panoramix. A visualization environment for LHCb. Goals. Handle event display and statistical visualization in a consistent way. LHCb coarse graining organization in term of “projects” :. Panoramix:vis. Gauss : sim. Brunel : rec. DaVinci : phys. LHCb : detector desc & event model. Geant4.

camille-vaughn
Download Presentation

Panoramix

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Panoramix A visualization environment for LHCb G.Barrand, eXtreme debugger at LAL

  2. Goals • Handle event display and statistical visualization in a consistent way. • LHCb coarse graining organization in term of “projects” : Panoramix:vis Gauss : sim Brunel : rec DaVinci : phys LHCb : detector desc & event model Geant4 Gaudi/SEAL/POOL OpenScientist • Panoramix is on top “of everything” (that explains the below “eXtreme debugger”). • Panoramix subjected to a software agreement between LAL and LHCb. G.Barrand, eXtreme debugger at LAL

  3. It works !!! From the collection of Thomas Ruf (done on Windows) G.Barrand, eXtreme debugger at LAL

  4. Engineering choices • Plugged directly to the data framework (Gaudi). No client / server model. • Since the data framework is in C++ (and most of native vis and GUI tools are in C / C++) use C++ for all. • Target the three laptops of the moment : Linuxes, Windows, MacOSX by using their native environments. • Used one powerful “scene manager”, Inventor, for all 3D or 2D data representation. • Try to put in “reusable elsewhere packages” things not fully dependant of the experiment. Here the “reusable elsewhere packages” being the ones put in the OpenScientist distribution. • See OpenScientist slides for more on “Engineering choices”. G.Barrand, eXtreme debugger at LAL

  5. Vis and GUI products • Rendering layer : more than ever : OpenGL. • Scene graph manager : OpenInventor. By using the Coin implementation of System In Motion (Norway). • Do all graphic with the uppers, including histogram and function plotting. • GUI : see OpenScientist slides for the problematic of a choice. Describe the GUI in XML and use the OnX package to create the GUI by using native toolkit of desktop providers. This is simple and give the best performances. • Scripted callback : mainly dld/c++ (string front end to dlopen/dlsym) : <widget class=“PushButton”> <activate exec=“C++”>OnX ui_echo hello</activate> </widget> and Python (with LHGDict wrapping) : <activate exec=“Python”>from import OnX *;ui.echo(‘hello’)</activate> G.Barrand, eXtreme debugger at LAL

  6. OnX Xt, Win, Qt, gtk+ XML OnX Dld, Python, CINT GL,Inventor G.Barrand, eXtreme debugger at LAL

  7. GUI paradigm “A la PowerPoint” : a compact GUI panel organized around a document area. This is 100% in opposition to a terminal prompt spawning viewers around the screen G.Barrand, eXtreme debugger at LAL

  8. Panoramix on a MacOSX by using native Cocoa (NextStep) / OpenGL (not released yet since all below layers not yet released on OSX). G.Barrand, eXtreme debugger at LAL

  9. Connection to the data framework • “data framework” should be understood as the software permitting to manipulate the “event model” and permitting to connect the “event model” (the “data”) to “facilities” like storage, graphic, GUI, scripting. The data framework is hereGaudi. Gaudi does not do any kind of graphic (no Gaudi::DataObject::draw method aroundand this is correct). Gaudi manipulates services that permits someone to operate the “transient store” containing, in memory, instances of the “event model”. • OnXis here the “interactivity framework”. It does the connection between GUI (from an XML description), viewers, scene manager (Inventor), renderer (OpenGL) and scripting. • The connection “data framework” “interactivity framework” is done through the Gaudi OnX service, put in a standalone package : Vis/OnXSvc. • Various elements of the event model have a “representation” (a datarep). A datarep code is in general a Gaudi “converter” for the Inventor technology (an SoConverter). An SoConverter, for example SoMCParticle, build from a data instance (here an MCParticle) an Inventor scene graph. When built, the scene graph is send to OnX to be displayed, in general, in the “current viewer”. • A “visualization request” starts from a scripted GUI callback ; the OnXSvc is searched, then a “data convertion for Inventor” request is activated for various pieces of data. • Note that, due to a massive usage of abstract interfaces, things are nicely decoupled (for example an SoConverter is not a StorageConverter. A direct consequence is that someone can get rid of a technology and migrate to a new one, without touching to everything. G.Barrand, eXtreme debugger at LAL

  10. The Vis/So packages • Physicists enter into action by programming some “SoConverter” put in some “Vis/So” package. • SoDet (G.Barrand, S.Ponce) : representation of the detector. This is very generic and is now sparely touched. Coupled to the LHCb XML det description, it offers a very flexible way to enter and view a geometry. Note that there are various interactive goodies that permits to “expand” or “contract” volumes (from J.Boudreau SoDetectorTreeKit). The GUI can view the detector and event tree in a tree widget. • SoEvent : historical. Used to gather first datareps like SoMCParticle, SoMCHit (P.Mato) and representations for the trackers (J.Van.Tilburg NIKHEF). • SoCalo (I.Belyaev ITEP) : dedicated package to gather reps for calorimeters. • SoRich (C.Jones Cambridge) : reps for the RICH. • SoStat (G.Barrand LAL) : to represent histograms of the Gaudi transient store. It uses the HEPVis SoPlotter nodekit. • SoHepMC : to visualize HepMCs. Used in conjunction of the Geant4 simulation (Gauss). • People did not seem to suffer of Inventor programming. . G.Barrand, eXtreme debugger at LAL

  11. The applications • Two packages remain Vis/Panoramix and Vis/LaJoconde. • Panoramix : it contains the “main” Panoramix.exe program which is a standard “Gaudi main”. This package contains also the Panoramix.onx and other *.onx XML files describing the GUI. It contains also “C++ callbacks” and various Python scripts sufficiently generals to be put “in the common pot”. It is in scripts that people can setup their views (number of regions with their size and position, background colors, etc…) and decide of what they put in their “scenes”. This could be done in Python or directly in C++ (in case some Python wrapping is lacking, too tricky C++ is needed or intensive speed is needed) by using the OnX “dld/C++ callback system”. • LaJoconde : historically introduced to connect to “The DaVinci code” (where the secret of LHCb physic is hidden). It had been introduced for “coarse graining” organization but will probably be “transferred” back to Panoramix in future so that physicists manipulate only “one front end program”. This will depend of the capabilities to be able to build coarse graining “Job options” files (by saying easily “I want or I do not want to use DaVinci”) and be able to access corresponding packages and libs at run time. G.Barrand, eXtreme debugger at LAL

  12. Speed • Due to basic choices (stick to desktop providers and be plugged directly to the data-framework) it is fast and the author wait anybody that claims to have something faster…. • BUT, we must always “have an eye” on Inventor. It is a powerful tool and then can be dangerous. A scene graph may become “too fat” and slow down the creation of a data representation. But there are numerous ways to optimize things (share color, line styles nodes, create dedicated nodes to handle huge collections of data, etc…). Experience shows that on local machines scene graph with 10 000 up to 100 000 nodes are ok. For more orders of magnitude, you have to think at optimizing. BUT SOMEONE MUST BE AWARE THAT A CROWDY SCENE GRAPH OFTENLY MEANS A CROWDY PICTURE ON THE SCREEN…. • Then the secret of “being fluent” is not in being able to handle big scene graph, it is in being fluent in “selection of data” (selection of what you really want to see). • Up to now, at least for what is done in LHCb, speed is sufficient. (I am more worried by what is (not) done in more bigger experiments…) G.Barrand, eXtreme debugger at LAL

  13. Problems (1) • All this works and had been put in place very early. • Problem remains around visualization of boolean operations over volumes. It is the “case of coplanar edges”. Dixit E.Chernyaev, the author of the algorithm (the BooleanProcessor of Geant4) there are cases where things cannot be decided (easily) and the best way is to avoid to have “edge to edge situations”. Fine, ok. But it is hard to explain to the detector people that they have to add or remove some matter here or there in order to “pass” a visualization algorithm !!!!! • But a non-algorithmic solution perhaps remains ; use the stencil capabilities of OpenGL to do that. Code had been found in Australia but not yet put in an Inventor node and integrated / tested in the system. A side effect ; it probably kill the vector PS production and then only “pixmap” PS production would remain for these views. • If someone has ideas ? Other algorithms to try, etc… G.Barrand, eXtreme debugger at LAL

  14. Problems (2) • Not yet fluent in fully stand alone installation on a (remote) local machine. We remember that a display is on top of everything and that the three common desktop are targeted. Then be fluent in installing the display passes by being fluent in installing “all the below”… • Today, binaries (Windows, Linux) are regularly reconstructed on the /afs/cern.ch areas (lot of thanks to Florence Ranjard) and users run them on their local machine. (Doing remote X11 on lxplus is definitely not recommended). Then we : • Use CERN as a central integration / binary construction / distribution place : THIS IS GOOD. (I strongly believe that this is the number one role of CERN). • DO NOT USE CERN at run time but use local machines seeing CERN with afs. But CERN anyway around at run time as a STRONG afs server provider. • For more, we strongly rely on efforts done around CMT (and PACMAN ?) to provide at some time straight forward source or binary local installation “of everything”. (See Christian Arnault poster about ATLAS software distribution). G.Barrand, eXtreme debugger at LAL

  15. Problems (3) • LCG : where are we going here ? • Can we build “bosonic software” with “fermionic people” ? • Panoramix on top of much more things not really deeply though to work together. • MacOSX port had been a pure nightmare… (And the story is not finished here. The only presence of ROOT induces to run in “bind_at_launch” mode that slows down the startup of everything. There are mysteries around dynamic loading and execution of static object constructors done with Gaudi and SEAL / PluginManager). cint root/io root/th pool pi gaudi/seal python lcgdict G.Barrand, eXtreme debugger at LAL

  16. A problem ? • In the software agreement, was mentioned that a physicist should be involved (at partial time) to do the sync <engineering> <physic> during the whole story. This never happened. • For example, despite that Panoramix received code contributions from numerous people and that it is used by various people, no physicist “strongly visualization driven” spontaneously emerged to survey that all element of the event model received a representation. The engineer isNOT paid to have all the LHCb detector in head along with all the event model and physic algorithms and to decide how things should be represented. • It would be welcome to avoid a “last minute rush” on that last point. G.Barrand, eXtreme debugger at LAL

  17. Problems (4) • http://www.lal.in2p3.fr/Panoramix. • html pages and paper doc produced with Doxygen from dedicated .h files (and not the ones of the code). Some of the files loadable from the program to form a help pull down menu and contextual helps. • But the content is clearly insufficient. • Nice user page of Thomas Ruf • Basement for a wiki ? • But for programmers (writing SoConverter), doc of Gaudi and Inventor is online. • But once more, is it to the engineer to document everything and in particular all what can be represented with various options, etc…? G.Barrand, eXtreme debugger at LAL

  18. Problems (5) • How to fit 48 hours in one day ? (Transplantation of a superconducting positronic brain ? (*)) • Need manpower ? Sure, the needed army that LCG will not put in place for doing graphic… * Due to the international context, the US Robots refuses to do that on a French for the moment…. G.Barrand, eXtreme debugger at LAL

  19. Next… • Next main issue is going to be the integration of “stat tools” to do “analysis”. For LHCb it will consist to operate “The DaVinci code” from the interactive tools. Here the way to deal with events is a little bit different : • Display : pick one event, view things in it, pass to another event, etc… • Analysis : pick and configure one or a bunch of algorithms (that will book histos and tuples), run over thousands of events, look histograms. Repeat again by modifying algorithms. • That’s right that what is done BY THE PHYSICIST is different but things to put in place BY THE ENGINEER ARE EXACTLY THE SAME IN BOTH SITUATIONS : a GUI, operate on the event model, produce “graphical representations” from data. • The only significant differences are around the data that must be visualized. In one case it is things in the event model, in the other case it is common things like histograms and tuples. • In the second case, then a key issue is to INTEGRATE A GOOD PLOTTER, that is to say a good piece of software able to create graphical representation of histograms. G.Barrand, eXtreme debugger at LAL

  20. HEPVis / SoPlotter, AIDA • Due to the choice of Inventor as scene manager, the author targets a plotter done with Inventor : here the HEPVis / SoPlotter one developed mainly be the author. SoPlotter covers now a lot of common cases (see OpenPAW poster for pictures). • The Vis / SoStat package does the connection Gaudi-histos / SoPlotter. This is in fact available for long but not yet heavily used due to the fact that connection to DaVinci is a little bit young. (and probably because people had taken some habits with other tools…). • Then, with SoPlotter, the same Panoramix environment can then be used also by working on DaVinci. (It opens also the way to do nice things like putting histo within events). • For scripting, someone can already get from Python the OnXSvc and do a vis request over an histo managed by Gaudi. Someone can also customize a lot of the SoPlotter. • What have to be provided now is to “standardize” the way to do that. The plan is to used the AIDA API and interfaces, mainly because it had been though and discussed by various parties and that, being pure abstract interfaces, it does not compel us to tied to another system having orthogonal backend GUI and graphic. It is crucial to keep in sync the LHCb software with AIDA G.Barrand, eXtreme debugger at LAL

  21. Integration of other tools (1) • Had been demonstrated than from a python shell, someone can spawn a DaVinci task with : • Panoramix (to display an event or plot an histo) • Hippodraw • ROOT • Then physicists will have tools. • But, shall software engineers be happy with that ? NO. All these tools have incompatible backend technologies (rendering, GUI) and there is no hope to provide one tool within a common GUI that integrate all that in a consistent interactive way. • A physicist will be let with a desktop crowded with standalone GUI windows, each having its own logic. An already announced pain. • Then, some job (low priority) is done to try to integrate some of these tools within Panoramix… G.Barrand, eXtreme debugger at LAL

  22. Integration of other stat tools (2) • Hippodraw : the design of it is so that someone can easily get rid of the default GUI and rendering (all in Qt), and comes with new ones. Had been done an OpenGL driver to hippodraw. From that it is quite easy to integrate a hippo window within Panoramix. An embryo of reimplementation of the famous “hippo inspector” done with OnX (then mainly in XML) exists now (OpenScientist / Hippo package). • ROOT : it will go through an implementation over Inventor of the painful TVirtualPad (some kind of strange mixture of GUI, rendering and scene manager virtual methods). A prototype exists (OpenScientist / Mangrove package). Someone can visualize a TGeo with OnX and Inventor and also the bins of a TH. But more work have to be done to have the full THistPainter over Inventor. But this is feasible. (Any volunteer ? No one, as usual). • Java / JAIDA / jas : hard to see how we can have jas drawing in an Inventor viewer or an OpenGL window not managed by the awt / swing. Moreover it appeared difficult to have Inventor drawing in an awt / swing window…. Strong expertise required to go on. • If no help from the developers of these tools is coming, the author will continue to put the effort on the SoPlotter (and OpenPAW) and to its AIDA front end ; the AIDA::IPlotter. G.Barrand, eXtreme debugger at LAL

  23. Must be mentioned : Agora • Agora is a prototype of visualization environment for ATLAS using the same technologies than Panoramix (OpenGL, Inventor, OnX, AIDA). • It makes sense since the data-framework is the same than LHCb (Gaudi). • Proof of feasibility done. In fact here, it is a shame that LHCb and ATLAS had not been able to converge on XML for the det desc…. • Anyone interested ? G.Barrand, eXtreme debugger at LAL

  24. Conclusions • Panoramix is regularly released and used and had already given services (for the XML det desc debugging, reconstruction debugging). • Even if not yet an overflow, views appear in meetings and workshops ; and some are very very nice. • Engineering choices are done (let us say up to the startup). But the overall architecture will permit to migrate parts if needed. • More work around the release building and distribution are needed if the three common user laptops are targeted (Linux, Windows, Mac). • At the engineering level, efforts will be put now on the final (!) integration of statistical tools : AIDA / OpenPAW for sure, hippodraw probably, ROOT with a gun in the back. Rendez vous at next CHEP for that. • Are physicists happy ? (is a physicist ever happy ?). It is not to the author to give an answer but there are not too much screams, shouts, laptop thrown on the wall for the moment… G.Barrand, eXtreme debugger at LAL

More Related