1 / 11

Bulk XFDU Creator Sergey Nikhinson Lou Reich

Bulk XFDU Creator Sergey Nikhinson Lou Reich. Requirements. create an XFDU manifest and save it as template create new template or modify existing template specify pattern for file names that will be used as data and metadata objects

mairi
Download Presentation

Bulk XFDU Creator Sergey Nikhinson Lou Reich

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Bulk XFDU Creator Sergey Nikhinson Lou Reich

  2. Requirements • create an XFDU manifest and save it as template • create new template or modify existing template • specify pattern for file names that will be used as data and metadata objects • specify relationships between data and metadata (content unit info) • specify necessary attributes for data/metadata/content units • specify other static information for XFDU package (e.g. header)

  3. High Level Design Three major logical components: • XFDU Engine (XFDU Toolkit with some additions) • Templating engine • Templates

  4. Design Diagram Templates Templating engine Files XFDU Engine Manifest Bulk Package Creator XFDU Package

  5. Basic Flow • Templating engine consumes template(s) • Templating engine parses the template(s) and passes information about files to be used to XFDU Engine • XFDU Engine uses the passed information to create XFDU structures • The created structures are merged with the rest of the information in the template to create XFDU manifest • The created manifest is consumed by XFDU Engine • XFDU package is created by XFDU Engine

  6. Use Cases Use Case Scenario 1 • User creates an XFDU instance (manifest) and saves it as template • User modifies the template to include patterns for file selection and other desired attributes • User runs bulk creator and produces XFDU package Use Case Scenario 2 • User uses existing XFDU manifest template • User modifies the template to include patterns for file selection and other desired attributes • User runs bulk creator and produces XFDU package

  7. First Implementation • Apache Velocity as templating engine • Velocity template as input • Ant-style wild-carded path specification for files • Nested content units • Other information via velocity variables

  8. Sample template <?xml version="1.0" encoding="UTF-8"?> <xfdu:XFDU xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.ccsds.org/xfdu/2004 xfdu.xsd" xmlns:xfdu="http://www.ccsds.org/xfdu/2004"> <packageHeader ID="packageHeader"> <volumeInfo> <specificationVersion>1.0</specificationVersion> <!-- sequence information attribute is specified by producer--> <sequenceInformation sequenceSize="10" sequencePosition="1">producer1.seq10</sequenceInformation> </volumeInfo> <environmentInfo> <xmlData> <platform>Linux2.4.22-1.2129.nptl</platform> </xmlData> </environmentInfo> </packageHeader> <informationPackageMap ID="informationPackageMap"> <xfdu:contentUnit ID="$cuIdPrefix" anyMdID="$anyMdFilePattern> #set($tmp=!$cuContext.put('cuIdPrefix','blk')) #set($tmp=!$cuContext.put('anyMdFilePattern','usa_nasa_pds_mgst_1230/DATA/**/*.LBL')) #set($tmp=!$cuContext.put('dataObjectFilePattern','usa_nasa_pds_mgst_1230/DATA/**/*.TAB')) <dataObjectPointer dataObjectID = "$dataObjectFilePattern"/> </xfdu:contentUnit> </informationPackageMap> </xfdu:XFDU>

  9. Partial sample output <informationPackageMap> <xfdu:contentUnit anyMdID="meta1 meta2" ID="blk_contentUnit1"> <dataObjectPointer dataObjectID="dataObject1"/> </xfdu:contentUnit> <xfdu:contentUnit anyMdID="meta1 meta2" ID="blk_contentUnit2"> <dataObjectPointer dataObjectID="dataObject2"/> </xfdu:contentUnit> <xfdu:contentUnit anyMdID="meta1 meta2" ID="blk_contentUnit3"> <dataObjectPointer dataObjectID="dataObject3"/> </xfdu:contentUnit> </informationPackageMap> <metadataSection> <metadataObject category="OTHER" classification="OTHER" ID="meta1"> <metadataReference mimeType="text/xml" vocabularyName="OTHER" locatorType="URL" href="file:usa_nasa_pds_mgst_1230/DATA/MARS/DATASET.LBL"/> </metadataObject> <metadataObject category="OTHER" classification="OTHER" ID="meta2"> <metadataReference mimeType="text/xml" vocabularyName="OTHER" locatorType="URL" href="file:usa_nasa_pds_mgst_1230/DATA/MARS/VAR.LBL"/> </metadataObject> </metadataSection>

  10. Partial sample output (cont.) <dataObjectSection> <dataObject repID="" ID="dataObject1"> <byteStream size="937398" mimeType="application/octetstream"> <fileLocation locatorType="URL" href="file:usa_nasa_pds_mgst_1230/DATA/MARS/OBS23002.TAB"/> </byteStream> </dataObject> <dataObject repID="" ID="dataObject2"> <byteStream size="2298307" mimeType="application/octetstream"> <fileLocation locatorType="URL" href="usa_nasa_pds_mgst_1230/DATA/MARS/TLM23004.TAB"/> </byteStream> </dataObject> <dataObject repID="" ID="dataObject3"> <byteStream size="3835230" mimeType="application/octetstream"> <fileLocation locatorType="URL" href="usa_nasa_pds_mgst_1230/DATA/MARS/BOL23005.TAB"/> </byteStream> </dataObject> </dataObjectSection>

  11. Future direction • Various pluggable strategies for file selection • User interface for template creation

More Related