Intro to SAM DbServer Rewrite Toolkit and Components

1. Intro to SAM DbServer Rewrite Toolkit and Components Lauri Loebel Carpenter November 2003

2. Last year, if you recall… DbCorbaClient / SamApi (or, what I did on my Thanksgiving vacation) Lauri Loebel Carpenter December 10, 2002

3. Where did this lead? • Complete IDL redesign • Infrastructure to support new concepts: • Struct Wrappers • Automarshalling DbServerProxy • Automarshalling DbServantImpl • DbAware Objects • ProtectedInterfaces • Exception handlers, db utilities, etc.

4. IDL: Interface Definition Layer • Defines anything and everything that needs to be known by two or more code packages. • Constants • Data Structures • Available Interfaces • Allows communication between arbitrary programming languages on arbitrary platforms • Powerful, but error-prone and difficult to debug.

5. Marshalling Data is complicated; Unmarshalling is worse! • Marshal: take my object, convert it to a byte stream and send it to another application • On a different platform • In a different programming language • Unmarshal: receive this arbitrary bag of bites, and figure out what it is and what it is supposed to do, and send back some results. • And handle all of the exceptions.

6. Bottom Line: We really do not want arbitrary pieces of code to be marshalling and unmarshalling data structures.

7. Encapsulate! StructWrappers • For each IDL data structure, create a representation that knows how to marshal/unmarshal itself. • Perform all marshalling/unmarshalling in one place (dbProxy, implCreation) • Treat exceptions as just another type of object to be marshalled/unmarshalled

8. Struct Wrappers in a nutshell • initialize_fromPython(self, *args, **kwargs) • initialize_fromCorba(self, corbaObject) • pyRepr() • corbaRepr() • __repr__() • __str__()

9. Struct Wrappers: the tricky part: Many initialization signatures. • Valid SamSizes: s1 = SamSize('10g') s2 = SamSize(s1) s3 = SamSize(0) s4 = SamSize('100', 'bytes') • Valid RunDescriptors: r1 = RunDescriptor(runId=10) r2 = RunDescriptor(r1) r3 = RunDescriptor(runNumber=12, runType='monte carlo') r4 = RunDescriptor(12, 'physics') r5 = RunDescriptor('physics', 12)

10. Automarshalling Proxies/Servers: # get a proxy to the interface of interest: dbProxy = proxyMgr.getProxy('DataFile') # create the wrapped object representing the # high level data structure theData = SamStructWrapper('input', 'args') # pass the high-level data structure to the method; # result will be a high-level data structure also result = dbProxy().interfaceMethod(theData)

11. Db (and other) Proxies: • Standard SamServerProxyManager utility for managing proxies • Marshals all args (via corbaRepr()) • Unmarshalls all return values (via pyRepr()) • Client code needs ONLY high-level (auto-documentable) data structures

12. ServantFactory and MethodWrappers • ServantFactory creates an instance of an Implementation • Then takes that Implementation and wraps the LOCAL, PUBLIC methods in a ServantMethodWrapper • ServantMethodWrapper unmarshals all incoming data (via pyRepr()), marshals all return values (via corbaRepr()) • ServantMethodWrapper also handles marshalling/unmarshalling of exceptions raised during execution of the method

13. Implementation Layer: • Receives the high-level data representation • No CORBA marshalling/unmarshalling code to distract the developer from SQL nightmares. • Returns another high-level object • ServantMethodWrapper marshals it • dbProxy unmarshals it… • Server code needs ONLY high-level (auto-documentable) data structures

14. Conceptual Example:What the User Sees from SamFile.SamDataFile import SamDataFile rawFile = SamDataFile(fileName='demo.raw', fileType='colliderImport', startTime='10-oct-2003', endTime='12-oct-2003', runType='physics data taking', runNumber=2001, firstEvent=1, lastEvent=10, eventCount=10, datastream='physical generic', dataTier='raw', filePartition=200, appName='datalogger', appFamily='datalogger', appVersion='0.0') fileId = rawFile.declareMetadata()

15. What the client code looks like (simplistically): class SamDataFile(DataFileAttributes): …. def declareMetadata(self): proxy = self.proxyMgr.getProxy('DataFile') fileId = proxy().declareMetadata(self) self.setFileId(fileId)

16. What the Server code looks like(very simplistically): class DbAwareSamDataFile(SamDataFile): …. def declareMetadata(self): self.validateMetadata() fileId = self.dbCore.nextId('data_files_id_seq') self.dataFiles.insertOne({'fileId':fileId, 'fileName' : self.getFileName(), 'fileSizeInBytes': self.getFileSize().getSizeInBytes(), …. } )

17. Implementation: Summary • Define data structure • Sam_idl/sam_corba_base/ • Sam_common_pkg/SamStruct/ • Add methods • Client: calls dbproxy().method • Server: DbAware object implements the guts of the method

18. Other DbServer Tools and Utilities: • Protected Interface/Implementation • Samadmin, shifter commands • nameAndIdUtility • Standard utilities for NameOrId objects • relationshipUtil • Standard utilities for validating relationships • DbServer ExceptionHandler • Raise ONLY what is declared in the IDL; notify developer of any undeclared exceptions and convert them to InternalError

19. Other General Tools and Utilities: • SamServerProxyManager • Maintains all proxies • SamExceptionHandler • Exception auditor • formattedTracebackString • Additional information in argDict

20. Where to find more info:IDL Sam_idl/ /sam_corba_base/Typedef.idl /sam_corba_base/<dataStruct>.idl /sam_db_srv/<interface>.idl

21. Where to find more info:Struct Wrappers Sam_common_pkg/ SamCorba/SamIdlStructWrapperBase.py SamCorba/SAMToSamStructCodeGenerator.py Sam_common_pkg/ SamStruct/<dataStruct>.py

22. Where to find more info:Automarshalling Sam_common_pkg/ SamCorba/DbServerProxy.py Sam_db_srv/ src/ServantMethodWrapper.py Sam_common_pkg/ SamUtility/MethodWrapper.py

23. Where to find more info:Misc. Sam_common_pkg/ SamException/SamExceptionHandler.py SamExceptions.py Sam_common_pkg/ SamFile/SamDataFile.py