1 / 25

Database Competence Centre

Database Competence Centre. openlab Major Review Meeting 2012 2 nd February 2012 Maaike Limper Zbigniew Baranowski Luigi Gallerani Mariusz Piorkowski Anton Topurov Nicolas Bernard Marescaux. Outline. Progress on - Integrated Virtualization, - Enterprise Manager 12c,

baka
Download Presentation

Database Competence Centre

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. DatabaseCompetence Centre openlab Major Review Meeting 2012 2nd February 2012 Maaike Limper Zbigniew Baranowski Luigi Gallerani Mariusz Piorkowski Anton Topurov Nicolas Bernard Marescaux

  2. Outline • Progress on - Integrated Virtualization, - Enterprise Manager 12c, - Replication Technologies, - 11g deployment for WLCG, • Outlook for openlab IV new projects - New openlab fellow – Maaike Limper, - Accelerator logging data-mining. CERN openlab major review February 2012

  3. Oracle Enterprise ManagerInfrastructure Provisioning, Self Service Portal tests

  4. EM12c Infrastructure Provisioning • Clean install of EM 12 (released version) • OVM 3.0 installed on 3 hosts • Re-testing of problematic features • Most of the reported errors fixed • Much faster setup then in Beta CERN openlab major review February 2012 4

  5. EM12 Structure CERN openlab major review February 2012

  6. EM 12 Self Service Portal • Not tested during Beta • Allows end users to provision own VMs • Self Service Admin controls available actions • VM images available for deployment • Disk, memory and CPU allowances • Scheduling windows, VMs expirations, etc • Works well from both admin and end user side CERN openlab major review February 2012

  7. EM 12 Self Service Portal

  8. Self Service Portal CERN openlab major review February 2012

  9. Replication technologies update

  10. Continuation of replication testing • Tested technologies • Streams 11.2.0.3 • GoldenGate11.1.1.1.1 • Active DataGuard11.2.0.3 • Goal • Validation of above technologies with all CERN’s streamed production workload • Direct performance comparison • On exact hardware and software configuration • Same that will be used in production CERN openlab major review February 2012

  11. Final results LOWER - BETTER LFC for LHCB workload description: -13 days of production data - redo volume 15G - 7,235,172 changes - ~2M transactions ATLAS COOL workload description: - 3 days of production data - redo volume 22,4 G - 15,313,748 changes - 63,481 transactions ATLAS PVSS workload description: - 3 days of production data - redo volume 11,1 G - 8,162,001 changes - 567,227 transactions CERN openlab major review February 2012

  12. Final results HIGHER - BETTER LFC for LHCB workload description: -13 days of production data - redo volume 15G - 7,235,172 changes - ~2M transactions ATLAS COOL workload description: - 3 days of production data - redo volume 22,4 G - 15,313,748 changes - 63,481 transactions ATLAS PVSS workload description: - 3 days of production data - redo volume 11,1 G - 8,162,001 changes - 567,227 transactions CERN openlab major review February 2012

  13. Summary • All Oracle replication solutions had been validated with CERN’s production datasets • ADG was the fastest one in each case • GoldenGate software is developing very fast • all data types used by CERN are now supported! • We could not benefit from parallelism optimization • Not compatible with production sets • Streams issues discovered during previous testing have been reported to Oracle Support • fixes are included in latest Oracle version (11.2.0.3) and we are ready for deployment… CERN openlab major review February 2012

  14. Next steps – Active Data Guard • ADG appears to be not only the fastest replication technology but also the easiest to establish and maintain • It makes it the best candidate for online – offline replication • First production setup will be deployed for CMS and ALICE in February • Opens possibilities for offline analysis (data mining) of large data sets • like accelerator logs CERN openlab major review February 2012 14

  15. 11g Deployment for WLCG

  16. 11g deployment overview • Narrow upgrade window • during LHC shutdown (January – March) • all experiments databases will be upgraded to 11g until • middle of February at T0 • beginning of March at T1 • Additionally tested with RAT (Real Application Testing) • Replication related challenges • due to tight schedule upgrades cannot be performed in order recommended by Oracle • upgrades of T1s targets requires coordination and actions on T0 • Report covering problems experienced during Streams replication upgrades will be provided to Oracle CERN openlab major review February 2012

  17. 11g deployment in worldwide databases distribution • ATLAS – the core is already running 11g T1 T0 24.01.12 11g 11g 11g 11g 11g 11g 18.01.12 10.01.12 17.01.12 14.12.11 23.01.12 STREAMS11g STREAMS11g STREAMS11g STREAMS11g CERN openlab major review February 2012

  18. 11g deployment in worldwide databases distribution 31.01.12 • LHCB LHCB online database – 6th of February 01.02.12 31.01.12 11g 11g 11g 11g 11g 11g 11g 11g 31.01.12 14.12.11 30.01.12 01.02.12 24.01.12 CERN openlab major review February 2012

  19. Openlab IV projects

  20. openlab IV • New Oracle Openlab fellow: • - Maaike Limper, started January 2012 • Project outline: • - Investigate possibility of doing LHC-scale data reconstruction and physics analysis within an Oracle database • Focus on Physics Analysis • Analysis mainly done from (root)-n-tuples • Tree structure of root-n-tuples could be (easily?) converted into database structure • External procedure agent on database to handle C++ physics analysis code CERN openlab major review February 2012

  21. Physics Analysis and Databases • Challenges: • Optimize database structure: physics objects can be defined as vectors-of-vectors with varying length per event, could be stored as blobs or separate tables • Flexibility for different types of analysis: subsets of data objects could be defined through “views” for different physics groups (SUSY, Higgs, etc.) • Analysis time: large number of users to access same data at the same time, optimize performance to be competitive with current analysis scheme • Database size: could take significantly more space than root-files; data for results with many different versions of the reconstruction software to be stored CERN openlab major review February 2012

  22. Accelerator logging data mining • Problem solving for early detection of the accelerator • Identify non-optimal response: discharge trajectory different from expected envelope • Response of valves • etc. • Make use of technology • Active DataGuard to analyze data at the same time as it is captured/loaded, • RAC, parallel query for processing • Large cache and high number of cores with fast interconnect CERN openlab major review February 2012

  23. Outreach

  24. Presentations • “GoldenGate vs Oracle Streams for Worldwide Distribution of Experimental Physics Data” Eva Defonte Perez & Zbigniew Baranowski, UKOUG, Birmingham 5-7 December 2011 • “Testing Storage for Oracle RAC 11g with NAS, ASM and SSD Flash Cache” Luca Canali & Dawid Wojcik, UKOUG, Birmingham 5-7 December 2011 • “Oracle Enterprise Manager: The Key to Building a Virtualized Stack” Anton Topurov, UKOUG, Birmingham 5-7 December 2011 • “Going deeper into Real Application Testing” Mariusz Piorkowski, UKOUG, Birmingham 5-7 December 2011 CERN openlab major review February 2012

  25. Questions? CERN openlab major review February 2012

More Related