1 / 9

ALMA CPM3 2014 cominf (archive) Alisdair Manning

ALMA CPM3 2014 cominf (archive) Alisdair Manning. 1. The Plan. 2014.3 AQ incremental release every release Easy performance improvements for RH DataPacker NGAS 2014.4 Pipeline Ingestion and meta-data extraction Relational Asdm (no delivery) 2014.5 DataPacker : re-assemble

benjy
Download Presentation

ALMA CPM3 2014 cominf (archive) Alisdair Manning

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ALMA CPM3 2014cominf (archive)Alisdair Manning 1

  2. The Plan • 2014.3 • AQ incremental release every release • Easy performance improvements for RH DataPacker • NGAS • 2014.4 • Pipeline Ingestion and meta-data extraction • Relational Asdm (no delivery) • 2014.5 • DataPacker: re-assemble • RelAsdm delivery • 2014.6 • Harvester • APDM XML (no delivery) • 2015 • SourceCatalogue • Adapt 2

  3. Priorities and Resources me Christophe CADC Heiko Schemas DataTracker ALMA schema Harvester SourceCat ASDM Updates NGAS Pipeline Int. RH - DtD RH - OODT Harvester DataPacker NGAS Download Manager AQ Monitoring ALMA schema NGAS DB Docs APDM SourceCat Pipeline Int. RH - DtD RH - OODT Harvester Download Manager AQ Monitoring 3

  4. Priorities Update I ASA – close loop on deliveries with DataTrackeraccepted 201405-CYCLE2-OFF SCO 7: write DB DT 6: start PP 1, 8: data transfer ARC DT 5: email 2: poll 3: PI from ARC? 4 :all there?

  5. Priorities Update II • Pipeline Integration (including OODT into the RH)stalled – missing XML, RH tasks. To resume ASAP. • Monitoring Databaseend of 2014 • ASA incremental updates • Harvester extensionsfixes, +spatial resolution, -gal, -abstract, -bib • AQ featuresauto-complete, freqSupport, export, API, state • RH featurescycle2, GOUS, hierarchical display • Performance-aq, +rh, -rh 5

  6. Priorities Update III SourceCat schema restructuring201403-CYCLE2-OFF deployed SourceCat refactoring / VO re-architecture NGAS maintenance / performance / merge with ESO codebasemeeting with Andreas at ESO AsdmUpdate 6

  7. Relational ALMA • DelayHarvester and Sourcecat work (2015) • ASDM required for AQUA / ERMA • create insert / retrieve performance tests • Control’s schema in Garching dev env. • Cost / benefit. Suitable? Performant? Iterate. 16/03 • Review by DBAs • Modify module build order ARCHIVE / ICD 31/03 • Generate hibernate mappings from model 18/05 • Migrate online code, harvester, data packer 20/07 • APDM for ComInf tools (or use Oracle 11gr2 features?) • Feasible? Performance tests 30/09 • Hibernate mappings from model. 09/11 • Migrate DP, Harvester, online code. 31/12 • https://ictwiki.alma.cl/twiki/bin/view/CommonInf/XmlToRelational 7

  8. Plan from CPM2 • 10.8 • Harvester corrections, galactic longitude / latitude, spatial resolution • DataTracker 1st release at ARCs • Detect ingestion of pipeline product from JAO • Does products PI belong to ARC? • Have all pieces made it to the ARC? • Email to PI • Another email one month before end of proprietary period • 2014.1 • Pipeline ingestion into Archive • 2014.2 • SourceCatalogue bugs, refactoring, data model changes, history • AsdmUpdate • 2014.3 • DataPacker expansion to re-assemble pipeline products for delivery 8

  9. Lost Time • 201403-CYCLE2-OFF cost a lot of effort • Need to verify against base configuration – better instructions. Scripting. • DBA work: importing data for debugging / testing • Scripted • Firefighting – performance of RH, NGAS issues in production • Broken unit tests in ONLINE system in particular – needs refactoring • VLT RH issues needed investigation – unanticipated usage

More Related