1 / 19

Schema Issues

Schema Issues. Schema Exchange between Federations Runtime Schema Access. Schema Exchange. Two areas where currently separate FDs are needed for robustness reasons Package developers will use separated development federations

kyria
Download Presentation

Schema Issues

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Schema Issues Schema Exchange between Federations Runtime Schema Access

  2. Schema Exchange • Two areas where currently separate FDs are needed for robustnessreasons • Package developers will use separated development federations • Separated production sites will run against separated federations until data replication is feasible • This separation introduces schema management problems • Schema changes have to be propagated between disconnected Federations • Type numbering must be controlled if one wants to exchange any data between federations • Type number allocation for different packages needs to be de-coupled • e.g. using named schema feature of Objectivity/DB

  3. Schema Exchage using Prototype Databases • Strategy used in the first releases of HepODBMS/LHC++ • Packages are sequentially compiled against an empty database (prototype fd) • this federation accumulates the schema • Only the first package creates a federation using oonewfd • All following packages start from the schema prototype database of the previous package • Disadvantages • Constrained build sequence even if packages are not dependent on each other • First package determines database parameters like the page size • User can not choose just a set of packages: (either all or nothing) • Difficult procedure for propagating schema changes back • We asked for a tool to exchange schema

  4. Schema Exchange Tool • Introduced with Version 4.0.10 • Not included in the V5 release ? • Set of two command line programs ooschemadump -outfile MyFd.schema MyFD • Creates ASCII representation of a FD schema • This allows to manage schema using CVS ! • Class shape • Type- and association numbers • Named schema info ooschemaupgrade -infile MyFD.schema newFD • Loads schema from an ASCII file • Allows also to merge schema

  5. Example Output class FieldTabElem { private: enum {maxFieldNameLen=64}; // a const char name[maxFieldNameLen]; HepExplorableFieldType type; d_Long ofs; public: ... }; declare(ooVArray,FieldTabElem) class FieldTab : public ooObj { ooVarray(FieldTabElem) tab; .. }; schema HepODBMS 16 27 8 { class FieldTabElem UNKNOWN 1048584 1048584 1 0 3 0 3 0 0 { Basic private int8 name[64] 1 0 0 0 Basic private int32 type 2 1 0 0 Basic private int32 ofs 3 2 0 0 } class FieldTabElem_ooVArray UNKNOWN 1048585 1048585 1 0 1 0 1 0 0 { VArrayStruct private ooVArray(FieldTabElem) _vs ...

  6. Functionality Provided • The tools allow to: • exchange schema definitions between different FDs • either the complete schema, a single named schema (e.g. a package) or a single class • last two options would profit from an additional tool option to specify a schema or class name • remove obsolete type definitions • create an empty federation • applying the edited schema definition • re-attach existing databases • clone federations e.g. with different page sizes • data may be moved across using ooload/oodump • renumber schema • Probably also allows to mess up the schema ! • Not to be used by end users!

  7. New Schema Exchange Recipe • Typical checkout, update, commit cycle • Checkout package sources and schema from CVS • All schema of a package resides in one (or more) named schema • Install all used named schemata from ASCII files into an empty federation • Attach or import (e.g. ooload) any test data • Modify any C++ source or ddl files • ooddlx may be run and may be used to do schema evolution • Compile the package and test it • Extract the schema • Check in source and schema modifications • Install library, header files and schema description

  8. Minor Problems • In a few cases the output of ooschemadump can not be parsed by ooschemaupgrade • Example: Class HitList : public ooObj, public ooVArray(ooRef(Hit)){ … }; • results in a reference to a type named opiRef<Hit >_ooVarray • neither a valid C++ type, nor can it be parsed by ooschemaupgrade • renaming to opiRef_Hit_ooVarray seems to be a workaround • likely to be a bug in ooddlx

  9. Schema Handling Tools in HepODBMS • Set of scripts in ${HEP_ODBMS_DIR}/etc • defineNamedSchemata <fd> • defines the common set of named schemata to de-couple type numbers allocation in different packages • cmpSchema <fd1> <fd2> • compares the type numbering of two federations • returns 0 if both numbering schemes agree • prints differences • Under development: • Dump/load of a single named schema (perl wrapper) • Check whether a federation is compatible with a schema description file.

  10. Schema Read Access • Problem: How to access objects without “knowing” their schema at the compile time of the accessing program ? • Program is not being linked against Class_ddl.C/Class.h definition files • Examples: • oodump/ooload • oobrowse/ootoolmgr • HepExplorer access to attributes in collections of tags (HepExplorableCollection class in HepODBMS) • Access via schema description obtained from the database at runtime

  11. Type Manager Interface (TMI) • Objy made a type manager interface library available • set of header files and a library ( libtmi.a in earlier versions of Objy) • Read access to the federation schema • describe named schemata • describe classes • describe fields of a given class • describe associations of a given class • Read/write access to the data • read/write access to class members • read/write access to elements of a Varray member • read/write access to associations

  12. First Results using TMI • Good documentation • Objy style reference manual • Simple, intuitive and complete interface • Largely compatible to what was used in HepExplorable • Complete, non-trivial code example • sources of the ooshow tool • Used in a HepExplorable re-implementation • Obtains the tag description of “concrete tags” directly from the database • Users will not have to describe their concrete tag objects • HepExplorer Modules code stays unchanged • Up to now only compiled using Objy V4 on Solaris

  13. Summary • Schema exchange tools have been provided by Objectivity • Remove many constrains of the “old” prototype database approach • Functionality is sufficient to support larger number of packages & developers • Intend to switch to this tool for the next release of HepODBMS • Integration with software release tools (SRT) could make this even “user-friendly” • Type manager interface has been provided • Provides sufficient functionality to implement user data browsing capabilities for HepExplorer modules • Use for dynamic type binding needs more investigation (e.g. during my visit in Mt. View)

  14. HepODBMS News & Plans

  15. Build Process • Windows NT support moved from VC++ projects to GNUmakefiles • One set of portable GNUmakfiles on Windows and Unix • Should not affect existing Unix GNUmakefiles • Expect to move to SRT as soon as possible • Support for platform independent makefiles for end users takes a lot of time • The HepODBMS makefiles are not integrated with any experiment packages • Integration of SRT with Objectivity/DB may need some special rules and targets

  16. Changes in ooSession • Semantic of nested calls start transactions • currently: nested calls return false, requested: inner calls return true: ????? • Semantic of ooSession::container(const char *name) • current semanticsThe creation of two containers with the same name in different databases fails. ooSession returns the first container also from the second call. • Request: Drop the lookup in the container dictionary kept by ooSession and allways act on the current database. • Intend to move to the new d_session class from Objectivity • Need to investigate multi-threading implications • Windows NT port? • Planned work item for Mt. View visit

  17. HepDbApplication changes • HepDbApplication will provide a subscriber interface to allow the user to provide callback functions for • Init (after Objy &HepODBMS init) • Transaction start, commit and abort • Exit (before HepDbApplication is destructed) • Changes to application defaults: • Read transactions will be started in MROW mode • User may override to switch MROW off • AMS connection will be AMS required • no silent fallback to NFS

  18. Event Collections • Existing Event Collections will be superseded by the new templated collections (see talk tomorrow) • don’t start new developments based on the old classes • HepEvent and HepEventTag base classes will be replaced by a set of interface requirements for tagged objects • Integration with templated collections • Support for other tags than event tags • Functionality of ClusteringHint classes will be provided by container based collections • Only a single type is used to implement clustering hints • Delegation allows to change clustering strategy at runtime • Collections provide proper iterators

  19. Please make your request now! • Important to agree on shared classes • Using multiple versions of classes like ooSession will lead to problems • Have a look at the HepODBMS interfaces • Do they fit into your framework? • How would you like to inherit / use HepODBMS classes? • Why can’t you use them now?

More Related