Updates from sPHENIX

1. Updates from sPHENIX TK Hemmick

2. sPHENIX? • One year from now, PHENIX will be destructively decommissioned (blowtorch…) • sPHENIX is a concept originating w/in PHENIX for a new experiment in the 8 O’Clock Hall. • Steps to form a brand new collaboration (which will not be called sPHENIX) started in June 2015. • Steps to divorce the desirable portions of the PHENIX code from the “PHENIX Universe” were starting Spring 2015. • Meeting to discuss divorced code base held 3 weeks ago…these slides are partly from there.

3. Intended/ Not Intended • This talk is a description of sPHENIX code. • This talk is NOT a sales-pitch! • As always, there are bad/bad features. • I will both summarize what is there and also summarize what maps well (or not) onto our evolving model for SoLID. • Slides mostly from C. Pinkenburg. NOTE: ENTIRE codebase (offline, simulations, nightly rebuilds) is available via github and is likely most valuable while most generic! Code: https://github.com/sPHENIX-Collaboration Meeting: https://indico.bnl.gov/conferenceDisplay.py?confId=1237

4. Fun4All

5. Coding Paradigm • There is ONLY ONE executable: root.exe (unaltered from root.cern.ch) • Root becomes useful by two methods: • Compile system/user code into *.so • rootcint makes dictionaries of user objects so that these can be instantiated and manipulated at the root prompt. • Execute macro at the root prompt. • User *.so attaches to core *.so via libadd in their package’s Makefile.am file. • Core codebase maintained fresh by nightly rebuilds that include complete compilation and test runs of the core code over old datafiles.

6. Input Managers Output Managers DST DST Analysis Modules Raw Data (PRDF) Raw Data (PRDF) Simulated PRDF HepMC/Oscar Histogram Manager Empty Root File Structure of Fun4All Fun4AllServer That’s all there is to it (as long as you don’t look under the hood)

7. The Phool Node Tree • The Node Tree is at the center of the Phenix software universe (but it’s more or less invisible to you). It’s the way we organize our data. • It is NOT a Root TTree • We have 3 different Types of Nodes: • PHCompositeNode: contains other Nodes • PHDataNode: contains any object • PHIODataNode: contains objects which can be written out to DST • PHCompositeNodes and PHIODataNodes can be saved to a DST and read back • This DST contains root TTrees, the node structure is saved in the branch names. Due to Roots limitations not all objects can become PHIODataNodes (e.g. anything containing BOOST). • We currently save 2 root trees in each output file, one which contains the eventwise information, one which contains the runwise information • Input Managers put objects as PHIODataNodes on the node tree, output managers save selected PHIODataNodes to a file. • Fun4All can manage multiple independent node trees **ALL items in PHIODataNode move seamlessly to Database and File storage. AUTOMATIC FORMATING

8. **Two Tickets to Paradise • “Streaming” is root’s name for taking a live object in memory to and from static content. • Root considers static content as TFile(). • PHENIX adds psql IO for root streamers. • User only defines CONTENT of object. • IO to Database and TFile() automatic. • Huge burden lifted. • Analysis code identical for: • You are running GEANT and reconstruction at same time. • You saved simulation to file. • You read data from file. Inherit from TObject Add to header:

9. It’s all about choices - Input • Fun4AllDstInputManager: Reads DST’s, if reading multiple input files it makes sure all data originates from the same event • Fun4AllNoSyncDstInputManager: Reads DST’s but doesn’t check events for consistency (needed when reading simulated input together with real data for embedding or if you just feel like screwing up) • Fun4AllHepMCInputManager: Read HepMC and Oscar formatted ASCII files • Fun4AllPrdfInputManager: Reads PHENIX Raw Data Format (PRDF) • Fun4AllDummyInputManager: just drives the event loop NOTE: Should be easy to write Fun4AllEVIOInputManager()

10. It’s all about choices - Output • Fun4AllDstOutputManager: Write DST’s • Fun4AllEventOutputManager: Writes Events in PRDF (packet selection possible) • Fun4AllPrdfOutputManager: Write simulated raw data file (needs probably work). • Fun4AllHepMCOutputManager: Write HepMC ASCII files Caveat: You can only use the Fun4AllEvenOutputManager if you have an Event object and a Fun4AllHepMCOutputManager if you have a HepMC object on the node tree

11. Your Analysis Module You need to inherit from the SubsysReco Baseclass (offline/framework/fun4all/SubsysReco.h) which gives the methods which are called by Fun4All. If you don’t implement all of them it’s perfectly fine (the beauty of base classes) • Init(PHCompositeNode *topNode): called once when you register the module with the Fun4AllServer • InitRun(PHCompositeNode *topNode): called whenever data from a new run is encountered • Process_event (PHCompositeNode *topNode): called for every event • ResetEvent(PHCompositeNode *topNode): called after each event is processed so you can clean up leftovers of this event in your code • EndRun(const int runnumber): called before the InitRun is called (caveat the Node tree already contains the data from the first event of the new run) • End(PHCompositeNode *topNode): Last call before we quit If you create another node tree you can tell Fun4All to call your module with the respective topNode when you register your modue

12. Okay, How do I navigate the Node Tree which is in every argument???? You need to know the name of the node and the class of the object you want (e.g. some PHG4HitContainer version in the node called G4HIT_HCALIN (that’s where the Fun4All printout of the Node Tree comes in handy) #include <g4hit/PHG4HitContainer.h> #include <fun4all/getClass.h> Myanalysis::process_event(PHCompositeNode *topNode) { PHG4HitContainer *g4hits = findNode::getClass<PHG4HitContainer>(topNode,”G4HIT_HCALIN”); if (g4hits) … } topNode of the node tree your module is registered with Caveat: getClass will return the first node of a given name, if you have have multiple identically named nodes you need to search differently

13. Blah Blah Blah – Very nice, but how do I run the show? Fun4All provides only building blocks, you need to put them together yourself in your root macro according to your needs. void G4HitTTree(const char *fname, const int nevnt = 0, const char *outfile="hits.root") { gSystem->Load("libg4histos"); // load library (Makefile.am makes sure dependencies are resolved) Fun4AllServer *se = Fun4AllServer::instance(); // Get pointer to the instance of the Master of the Beast G4HitTTree *tt = new G4HitTTree(); // Make my analysis module tt->Detector(“InnerHcal”); // set my module up using it’s methods se->registerSubsystem(tt); // register with Fun4All – this determines the order in which modules are called Fun4AllInputManager *in = new Fun4AllDstInputManager("DSTin"); // Make an input manager – we read a DST in->fileopen(fname); // open the input file se->registerInputManager(in); // register the Input Manager with Fun4All Fun4AllOutputManager *out = new Fun4AllDstOutputManager("DSTout",outfile); // Make an output manager – we write a DST out->AddNode(“MyInnerHcal”); // We only want to write out this node se->registerOutputManager(out); // register the Output Manager with Fun4All se->run(nevnt); // Run over N events (0 means all events of input file) se->End(); // Call the End() method of analysis modules, close input and output files, write out runwise T1 TTree delete se; // cleanup, free up all memory }

14. Detector class design PHG4Subsystem: SubsysReco, interface to Fun4All PHG4Detector: Detector setup – implements GEANT construct method PHG4SteppingAction: implements stepping action for each detector PHG4EventAction: implements actions taken at end of event (e.g. removing G4Hits with zero energy loss)

15. GEANT steps GEANT propagates particles one step at a time. The step size is determined by the physics processes associated with the current particle or when a boundary between volumes is crossed After each step the user stepping method is called with a pointer to the current volume which has access to the full information (energy loss, particle momentum at beginning and end of step, …)

16. G4Hit Our G4Hits In our stepping method we add the energy loss in each volume and store the entry and exit coordinates (and for tracking detectors the momentum at the entry and exit) We also keep the ancestry for G4Hits so any hit can be traced back to a primary particle. To reduce size we do not store particles which do not leave G4Hits and are not in the ancestry of a particle which created a G4Hit

17. PHG4Reco • Fun4All Interface to GEANT4 • Sets features of the world (size, shape, material, magnetic field, physics list) • Provides interface to GEANT command line (especially useful for event display) • Manages our detectors • totally configurable on macro level

18. Why is it called Fun4AllServer when it’s not a Server? The parent of Fun4All is the PHENIX online monitoring and that one is a real server to distribute histograms. If it turns out to be useful some day this capability can be put back

19. History • Development started in 2002, in use by PHENIX from 2003 on (reconstruction and analysis of Run3 data) • Underlying data structure – the phool node tree – was part of day1 of PHENIX software (PHENIX Object Oriented … it predates stl) • Needed to get many subsystems who developed their code independently and without coordination under one umbrella • Development driven by reconstruction and analysis needs – not by “beauty” or some abstract design considerations (or “rules”)

20. Phool Node Tree for sPHENIX Print it from the cmd line with Fun4AllServer *se = Fun4AllServer::instance(); se->Print("NODETREE"); Node Tree under TopNode TOP TOP (PHCompositeNode)/ DST (PHCompositeNode)/ G4HIT_HCALIN (PHIODataNode) G4HIT_ABSORBER_HCALIN (PHIODataNode) G4HIT_HCALOUT (PHIODataNode) G4HIT_ABSORBER_HCALOUT (PHIODataNode) SVTX (PHCompositeNode)/ SvtxHitMap (PHIODataNode) SvtxClusterMap (PHIODataNode) SVTX_EVAL (PHCompositeNode)/ SvtxClusterMap_G4HIT_SVTX_Links (PHIODataNode) RUN (PHCompositeNode)/ CYLINDERGEOM_SVTX (PHIODataNode) CYLINDERGEOM_SVTXSUPPORT (PHIODataNode) CYLINDERGEOM_EMCELECTRONICS_0 (PHIODataNode) CYLINDERGEOM_HCALIN_SPT (PHIODataNode) PAR (PHCompositeNode)/ SVTX (PHCompositeNode)/ SvtxBeamSpot (PHIODataNode) TOP: Top of Default Node Tree Creation and populating of other node trees is possible (used for embedding)

21. Phool Node Tree for sPHENIX Print it from the cmd line with Fun4AllServer *se = Fun4AllServer::instance(); se->Print("NODETREE"); Node Tree under TopNode TOP TOP (PHCompositeNode)/ DST (PHCompositeNode)/ G4HIT_HCALIN (PHIODataNode) G4HIT_ABSORBER_HCALIN (PHIODataNode) G4HIT_HCALOUT (PHIODataNode) G4HIT_ABSORBER_HCALOUT (PHIODataNode) SVTX (PHCompositeNode)/ SvtxHitMap (PHIODataNode) SvtxClusterMap (PHIODataNode) SVTX_EVAL (PHCompositeNode)/ SvtxClusterMap_G4HIT_SVTX_Links (PHIODataNode) RUN (PHCompositeNode)/ CYLINDERGEOM_SVTX (PHIODataNode) CYLINDERGEOM_SVTXSUPPORT (PHIODataNode) CYLINDERGEOM_EMCELECTRONICS_0 (PHIODataNode) CYLINDERGEOM_HCALIN_SPT (PHIODataNode) PAR (PHCompositeNode)/ SVTX (PHCompositeNode)/ SvtxBeamSpot (PHIODataNode) • DST and RUN Node: default for I/O • DST – eventwise • RUN - runwise • Objects under the DST node are reset after • every event to prevent event mixing. You • can select the objects to be saved in the • output file. Subnodes like SVTX are saved • and restored as well. DST/RUN nodes can • be restored from file under other TopNodes • ROOT restrictions apply: • Objects cannot be added while running to • avoid event mixing

22. Phool Node Tree for sPHENIX Print it from the cmd line with Fun4AllServer *se = Fun4AllServer::instance(); se->Print("NODETREE"); Node Tree under TopNode TOP TOP (PHCompositeNode)/ DST (PHCompositeNode)/ G4HIT_HCALIN (PHIODataNode) G4HIT_ABSORBER_HCALIN (PHIODataNode) G4HIT_HCALOUT (PHIODataNode) G4HIT_ABSORBER_HCALOUT (PHIODataNode) SVTX (PHCompositeNode)/ SvtxHitMap (PHIODataNode) SvtxClusterMap (PHIODataNode) SVTX_EVAL (PHCompositeNode)/ SvtxClusterMap_G4HIT_SVTX_Links (PHIODataNode) RUN (PHCompositeNode)/ CYLINDERGEOM_SVTX (PHIODataNode) CYLINDERGEOM_SVTXSUPPORT (PHIODataNode) CYLINDERGEOM_EMCELECTRONICS_0 (PHIODataNode) CYLINDERGEOM_HCALIN_SPT (PHIODataNode) PAR (PHCompositeNode)/ SVTX (PHCompositeNode)/ SvtxBeamSpot (PHIODataNode) Users can add their own branches. Resetting the objects is their responsibility. The PAR node will probably turn into the node for calibrations which we want to keep on the DST

23. Phool Node Tree for sPHENIX Print it from the cmd line with Fun4AllServer *se = Fun4AllServer::instance(); se->Print("NODETREE"); Node Tree under TopNode TOP TOP (PHCompositeNode)/ DST (PHCompositeNode)/ G4HIT_HCALIN (PHIODataNode) G4HIT_ABSORBER_HCALIN (PHIODataNode) G4HIT_HCALOUT (PHIODataNode) G4HIT_ABSORBER_HCALOUT (PHIODataNode) SVTX (PHCompositeNode)/ SvtxHitMap (PHIODataNode) SvtxClusterMap (PHIODataNode) SVTX_EVAL (PHCompositeNode)/ SvtxClusterMap_G4HIT_SVTX_Links (PHIODataNode) RUN (PHCompositeNode)/ CYLINDERGEOM_SVTX (PHIODataNode) CYLINDERGEOM_SVTXSUPPORT (PHIODataNode) CYLINDERGEOM_EMCELECTRONICS_0 (PHIODataNode) CYLINDERGEOM_HCALIN_SPT (PHIODataNode) PAR (PHCompositeNode)/ SVTX (PHCompositeNode)/ SvtxBeamSpot (PHIODataNode) Caveat: You loose ownership once an object is put on the node tree. Fun4All deletes the node tree when cleaning up. Deleting nodes is not supported (yet – but ROOT prevents this for the DST node)

24. Features • Reading of various formats (DST’s, PHENIX raw data, HepMC, Oscar) • Synchronization, supports parallel reading of multiple input files with only common events handed to the event loop • User modules inherit from common base class and have to implement the methods they want to use • Writing various output formats (DSTs, raw data, HepMC)

25. It’s all about choices - Input Caveat: If you read from more than one input file containing identical nodes, nodes from the last input manager will overwrite the others. Input Managers can be given multiple files or file lists. They will transparently open a new file when the previous file is exhausted But files have to be run over in ascending segment order otherwise the synchronization will get lost Input Managers can run user modules on “their data” allowing you to deal with identical nodes somehow before they get wiped

26. Histogram OutPut Root’s pathetic TDirectory handling makes it unlikely that you figure out how to create and save histograms on your own reliably (at least that’s my experience). Use the Fun4AllHistoManager for this #include <fun4all/Fun4AllHistoManager.h> MyAnalysis::MyAnalysis() { HistoManager = new Fun4AllHistoManager(“MyHistos"); HistoManager->setOutfileName( “myhistos.root”); } MyAnalysys::Init(PHCompositeNode *topNode) { myhist1 = new TH1F(….); // myhist1 should be declared in MyAnalysis.h HistoManager->registerHistogram(myhist1); } This will save your histograms when executing the Fun4AllServer::End() in the file “myhistos.root”. If you don’t set the filename it will construct a name from the name of your analysis module. If you don’t register it with Fun4All you have to call the writing out yourself – e.g. saving snapshots in separate files or in the End() method and delete it in your dtor.

27. Create your own TTree My biased opinion: creating my own root TTrees is a very painful experience and analyzing them is even worse it’s so much fun to Write your own eventloop and try to debug those crappy CINT macros But aren’t our node trees saved as root trees? Wouldn’t it be nice to just piggy back on its very generic mechanism to do this? It is actually possible – the only thing you need to do is write a class which contains the variables you want to save (on an event by event basis, but you can select which event should be saved). You put this object on the node tree and tell an output manager to write it out. Now you’ve got your own root TTree which you can either analyze like you analyzed your old tree or you can feed it back into Fun4All and use another SubsysReco module to analyze it

28. Create your own TTree An example how to do this is in the sPHENIX tutorials under MyOwnTTree Don’t worry about the ugly TClonesArray gymnastics you just have to modify MySimpleTree

29. So you finally admitted to yourself that writing bugfree code is just impossible and you are tired of adding couts Here are some short tutorials how to run code checkers: The ever popular and easy to use and understand valgrind (what more than the line number were your code is doing the dirty deed do you want): https://wiki.bnl.gov/sPHENIX/index.php/Tools#valgrind Insure is more cryptic but it does find problems valgrind cannot pick up: https://wiki.bnl.gov/sPHENIX/index.php/Tools#insure And if your code just bombs out on you, don’t use print statements, use gdb instead which will get you to the problem in no time: http://www.phenix.bnl.gov/WWW/offline/tutorials/debuggingTips.html (this one is old and needs to be brought up to date but it is still valid)

30. What’s next? • Fun4All is a work in progress – it will continue to evolve for use by sPHENIX – the current version is already incompatible with what we have in PHENIX • Keep it simple – it’s a tool – not a monument • Next project is a port of our calibrations DB (PdbCal) and development of DB capabilities based on it