Persistent Object References in ROOT I/O Status & Proposal

1. Persistent Object Referencesin ROOT I/OStatus & Proposal CMS-ROOT meeting CERN- November 27 René Brun ftp://root.cern.ch/root/refs.ppt Persistent Object References in ROOT

2. Plan of talk • Persistent C++ pointers • Using TRef & TRefArray • Action on Demand • Combining TRef & Action on Demand • Proposal for changes & extensions Persistent Object References in ROOT

3. Normal Streaming modeReferences using C++ pointers Only one copy of each object in the graph saved to buffer A TBuffer b; A.Streamer(b) Persistent Object References in ROOT

4. A Normal Streaming modeReferences using C++ pointers TBuffer b1; A.Streamer(b1) TBuffer b2; B.Streamer(b2) Objects in red are in b1 and b2 C++ pointer B Persistent Object References in ROOT

5. A Normal Streaming modeReferences using TRef pointers TBuffer b1; A.Streamer(b1) TBuffer b2; B.Streamer(b2) Objects in blue are only in b1 C++ pointer TRef B Bz z Set pointer to z with: TRef Bz = z; Get pointer to z with: z = Bz.GetObject() Persistent Object References in ROOT

6. Setting a TRef pointer • Assuming obj = pointer to a TObject* • TRef ref = obj; • TRef is itself a TObject • Its fUniqueID is set to obj - gSystem • The objkIsReferenced bit is set (fBits of obj) • Get pointer obj with obj = ref.GetObject() • returns fUniqueID + gSystem Class TObject { unsigned int fBits; unsigned int fUniqueID; Class TRef : public TObject { TProcessID *fPID; //!pointer to process id Persistent Object References in ROOT

7. Writing TRefs to a buffer • A TRef is written by TRef::Streamer • Writes uid(8 bytes) + pid(4 bytes) • 12 bytes in non compressed mode • 2.4 bytes in compressed mode 1 (default) • uid = object unique identifier • default uid = pointer - gSystem • (see proposal) • pid = Process identifier • Each process has a unique pid (TProcessID) • A file contains the pids of all processes that have written objects to it. Persistent Object References in ROOT

8. Writing Referenced objects • A referenced object is written by obj->Streamer • This Streamer at some point calls its TObject::Streamer • In TObject::Streamer, if the kIsReferenced bit is set in fBits, the following additional info is also written: • uid (8 bytes) = obj - gSystem • pid (4 bytes) = TProcessID of current process A Referenced object may be written multiple times in the same file as the TRef or in other files Persistent Object References in ROOT

9. Reading Referenced objects • A referenced object is read by obj->Streamer • This Streamer at some point calls its TObject::Streamer • In TObject::Streamer, if the kIsReferenced bit is set in fBits, the following additional info is also read: • uid (8 bytes) = obj - gSystem • pid (4 bytes) = TProcessID of current process • the fUniqueID is set to uid • The pair (uid,obj) is added to the TExMap of the TProcessID corresponding to pid • When obj is deleted, its pair (uid,obj) is also removed from the TProcessIDTExMap. Persistent Object References in ROOT

10. Reading TRefs from a buffer • A TRef object is read by TRef::Streamer • The pair uid,pid is read • the fUniqueID of TRef is set to uid • The transient pointer fPID is set to the TProcessID corresponding to pid • via a direct access table in the gROOT object • The bit 1 of fBits is set Persistent Object References in ROOT

11. Using a TRef • To get a pointer to the referenced object, do: • Myclass *obj = (Myclass*)ref.GetObject() =0 Returns obj = fUniqueID + gSystem fBits[1] GetObject fPID = 0 or fPID>GetObjectWithUniqueID() = 0 =1 execid = fBits[8/8] YES NO Set fBits[1]=0 return obj YES return null execid = 0 NO Execute TExec with execid Obj = 0 YES Persistent Object References in ROOT

12. TRef example: Event.h class Event : public TObject { private: char fType[20]; //event type char *fEventName; //run+event number in character format int fNtrack; //Number of tracks int fNseg; //Number of track segments int fNvertex; int fMeasures[10]; float fMatrix[4][4]; float *fClosestDistance; //[fNvertex] EventHeader fEvtHdr; TClonesArray *fTracks; //->array with all tracks TRefArray *fHighPt; //array of High Pt tracks only TRefArray *fMuons; //array of Muon tracks only TRef fLastTrack; //reference pointer to last track TRef fWebHistogram; //EXEC:GetWebHistogram TH1F *fH; //-> public: ... TH1F *GetHistogram() const {return fH;} TH1F *GetWebHistogram(Bool_t reload=kFALSE) const { return (TH1F*)fWebHistogram.GetObject(reload);} Persistent Object References in ROOT

13. TRef & Action on Demand • When the keyword “EXEC:” is found in the comments of the data member as in: • TRef fWebHistogram; //EXEC:GetWebHistogram • The information in the comment field is kept in the dictionary. Execid is saved in TStreamerElement • When the TRef object is read, the execid is stored in the fBits on one byte (from TStreamerElement). • When TRef::GetObject is called, • TObjArray *lexecs = gROOT->GetListOfExecs(); • TExec *exec = (TExec*)lexecs[execid]; • exec->Exec(); Action on Demand is Persistent fWebHistogram.GetObject() executes the action GetWebHistogram Persistent Object References in ROOT

14. What a TExec can do • TExec is a CORE ROOT class that can be used to execute: • a call to a compiled or interpreted function • example: Exec:LoadHits() • an interpreted script • example: Exec:GetWebHistogram • If GetWebHistogram is not a function (compiled or interpreted), then TExec::Exec will try to execute the script GetWebHistogram.C void GetWebHistogram(){ // example of script called from an Action on Demand when a TRef object // is dereferenced. See Event.h, member fWebHistogram const char *URL = "http://root.cern.ch/files/pippa.root"; printf("GetWebHistogram from URL: %s\n",URL); TFile *f= TFile::Open(URL); f->cd("DM/CJ"); TH1 *h6 = (TH1*)gDirectory->Get("h6"); h6->SetDirectory(0); delete f; gROOT->SetSelectedPrimitive(h6); } Persistent Object References in ROOT

15. A Working Example Action.C { gSystem.Load("libEvent"); TFile f("Event.root"); Event *event=0; T.SetBranchAddress("event",&event); T.GetEntry(45); event->GetWebHistogram()->Draw(); } Root >.x Action.C GetWebHistogram.C void GetWebHistogram(){ // example of script called from an Action on Demand when a TRef object // is dereferenced. See Event.h, member fWebHistogram const char *URL = "http://root.cern.ch/files/pippa.root"; printf("GetWebHistogram from URL: %s\n",URL); TFile *f= TFile::Open(URL); f->cd("DM/CJ"); TH1 *h6 = (TH1*)gDirectory->Get("h6"); h6->SetDirectory(0); delete f; gROOT->SetSelectedPrimitive(h6); } Persistent Object References in ROOT

16. Status with Persistent pointers • C++ persistent pointers • In all versions of ROOT since day1 • TRef, TRefArray • implemented in dev version 3.02/00 • TRef with Action on Demand • implemented in 3.02/05 (this week) • TRef simplification & extensions • see proposal Persistent Object References in ROOT

17. Simplification • Instead of storing obj-gSystem as unique id for a given process id, I propose to store the object number within a process. • This is a trivial change to the current scheme in dev. • Many advantages: • simpler to manage • no TExMap, but a simple TObjArray with direct access for each TProcessID • One could imagine a very simple object catalog in the RDBMS • Disadvantages; • Require a table of pointers in the process writing the objects • This problem could be solved by an Wipe_Event function Persistent Object References in ROOT

18. Setting a TRef pointer • Assuming obj = pointer to a TObject* • TRef ref = obj; • TRef is itself a TObject • If the obj::kIsReferenced bit is not yet set, the obj::fUniqueID is set to the CurrentNumber+1 and obj::kIsReferenced is set to 1. • Its fUniqueID is set to obj::fUniqueID • CurrentNumber is managed by TProcessID • fObjs[CurrentNumber] is set to obj in fPID Class TObject { unsigned int fBits; unsigned int fUniqueID; Class TRef : public TObject { TProcessID *fPID; //!pointer to process id Persistent Object References in ROOT

19. Writing TRefs to a buffer • A TRef is written by TRef::Streamer • Writes uid(4 instead of 8 bytes) + pid(4 bytes) • uid = object unique identifier • uid = ref::fUniqueID = obj::fUniqueID = current object nr • pid = Process identifier • Each process has a unique pid (TProcessID) • A file contains the pids of all processes that have written objects to it. Persistent Object References in ROOT

20. Writing Referenced objects • A referenced object is written by obj->Streamer • This Streamer at some point calls its TObject::Streamer • In TObject::Streamer, if the kIsReferenced bit is set in fBits, the following additional info is also written: • uid (8 bytes) = obj - gSystem already in obj::fUniqueID • pid (4 bytes) = TProcessID of current process A Referenced object may be written multiple times in the same file as the TRef or in other files Persistent Object References in ROOT

21. Reading Referenced objects • A referenced object is read by obj->Streamer • This Streamer at some point calls its TObject::Streamer • In TObject::Streamer, if the kIsReferenced bit is set in fBits, the following additional info is also read: • uid (8 bytes) = obj - gSystem • pid (4 bytes) = TProcessID of current process • In TProcessID::fObjsfObjs[fUniqueID] = obj • When obj is deleted, fObjs[fUniqueID] = 0; Persistent Object References in ROOT

22. Reading TRefs from a buffer • A TRef object is read by TRef::Streamer • The pair uid,pid is read • the fUniqueID of TRef is set to uid • The transient pointer fPID is set to the TProcessID corresponding to pid • via a direct access table in the gROOT object • The bit 1 of fBits is set Persistent Object References in ROOT