1 / 32

Comprehensive Large Array-data Stewardship System (CLASS) class.noaa

Comprehensive Large Array-data Stewardship System (CLASS) http://www.class.noaa.gov. GOES-R -- Overview and Plans. Richard G. Reynolds CLASS Project Manager May 12, 2004. Agenda. CLASS System Overview Project Management Accomplishments to Date Current Hardware Architecture FY04 Goals

abena
Download Presentation

Comprehensive Large Array-data Stewardship System (CLASS) class.noaa

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Comprehensive Large Array-data Stewardship System(CLASS)http://www.class.noaa.gov GOES-R -- Overview and Plans Richard G. Reynolds CLASS Project Manager May 12, 2004

  2. Agenda • CLASS System Overview • Project Management • Accomplishments to Date • Current Hardware Architecture • FY04 Goals • “Final” Architecture • Budget … Resources and Commitments • Issues and Risks

  3. CLASS Overview NOAA's National Data Centers and their world-wide clientele of customers look to CLASS as the sole NOAA IT infrastructure project in which all current and future large array environmental data sets will reside. CLASS provides permanent, secure storage and safe, efficient access between the Data Centers and the customers.

  4. CLASS Overview • CLASS is a web-based data archive and distribution system for NOAA/NESDIS environmental data • Archive … ingest, storage, metadate management, and data quality assurance • Distribution … access, visualization, and data delivery • CLASS is an extension of an 1995 operational system … SAA (Satellite Active Archive) • Transition to the CLASS architecture began in 2001 • “Dual-Site” Operational CLASS began 02 April 2004 • CLASS currently supports POES, DMSP, and GOES data sets, plus RadarSat (Synthetic Aperture Radar) and SeaWiFS (Ocean Color Product) • CLASS will support additional campaigns, broader user base, new functionality currently being defined • CLASS concurrently support ongoing operations and new requirements implementation

  5. System Overview – Functional Flow Diagram Visualize Data Visualization Data Data Products And Metadata Ingest and Store Data Data Set Inventory Access Data Interface with Users USERS Data Caches Data Providers Process Orders Archive Maintain, Monitor, Control Orders CLASS Internet/Intranet CLASS Operators

  6. System Overview – System Design • Data Storage and Distribution Subsystems • Ingest, Subscription and Bulk Order Generation, Data Recall, Delivery •  Servers: • Inventory, Visualization, Order • Socket messages in XML format • Object oriented design implemented in C++

  7. System Overview – System Design • User Interface • Java/XML-based web interface uses COTS components • Apache – Web Server; • Cocoon - publishing framework • Avalon/Excalibur - database connection pooling • Tomcat- servlet engine • LogKit - message logging • Informix Java Database Connectivity (JDBC) Driver • Monitoring and Maintenance Tools • Log Monitor, Cache Cleanup, Work Space Cleanup, Independent Monitoring, Operator Interface • C++ • Perl

  8. System Overview– Distributed Redundant Sites Supplier Supplier Supplier Supplier Supplier Supplier Asheville Suitland Archiveinterchange Ingestprocess Ingestprocess Archiveinterchange Operationalinventory Operationalinventory Archiver Archiver Operationaldatastore Operationaldatastore Roboticstorage Roboticstorage USER USER USER USER USER

  9. System Overview– Servers

  10. System Overview– User Interface USERS

  11. System Overview– Maintenance and Monitoring

  12. CLASS Overview (Continued) NOAA Observing System -- Notional Architecture -- The “End-to-End System” Observation Control System is controlled by Larger System Owner Support part of provides data directly to supported by owned by Data Handling System Observing System Operator User operated by provides info to provides data to is type of contains Processing Element Platform / Station situated on < drives located at Stake-holder Sensing Element measures Location has Environmental Parameter is a is Sensor Human Stakeholder Requirement Fixed Mobile characterizes Environmental Phenomenon is type Space Space Air Air Basic Service Requirement In Situ Remote Ground Ground < drives < drives Ocean Ocean CLASS

  13. Project Management ITAT NESDIS Data Archive Board CLASS Project Richard G. Reynolds Archive Requirements Council CLASS Project Management Team (CPMT) OSDPD Development Team (Suitland, MD) NCDC Development Team (Fairmont, WV) NGDC Development Teams (Boulder, CO) OSDPD Operations (Suitland, MD) NCDC Operations (Asheville, NC) System Integration & Test Team (Suitland, MD)

  14. Boulder Team • Eric Kihn (NGDC) • SABR • Integration • DMSP • Data Mining • Ted Habermann (NGDC) • Geospatial • databases • Geotiff images • Metadata • Suitland Team • Alex Kidd (OSD) • Robert Rank (OSD) • Tino Cremidis (CSC) • David Bowman (NCDC) • Baseline system • Processes/policies System Architecture & Design • System Integration & Testing • Support Suitland • operations • West Virginia Team • Carlos Martinez (TMC) • GOES Lead • CCR Implementation • NPP Implementation • EOS Implementation • Support Asheville • operations • COAST • Chung Wu (OSD) • E-commerce • Order • Management • System Project Management (Continued) Richard G. Reynolds (OSD) Charles Bryant (OSD), Chung Wu (OSD)

  15. DataArchive Board (DAB) (Tom Karl/NCDC) Archive RequirementsCouncil (ARC) (John Bates/NCDC) CLASS PROJECT Information Exchange OSD (w/ CSC & TMC) & Data Centers (w/ Contractors)

  16. CLASS Accomplishments to Date • Completed overall design of CLASS top-level architecture • Prepared Key System Documentation • Requirements • ICDs • CONOPS • Management Plans and Procedures • Established operational, integration and test, and development environments in Suitland • Completed migration from SAA to CLASS • Migrated about 30TB of data from old tape archive system • Added more than 50 new derived products to the archive • Consolidated three web sites in one web-based user interface

  17. CLASS Accomplishments to Date (Continued) • Enhanced ingest system to be independent of file type • Delivered baseline systems to Suitland and Asheville • Suitland CLASS Operational with POES and DMSP data sets • Completed IJPS/Metop – Archive and Access Segment Preliminary Design Review • Coordinated with NPP/NPOESS for defining the IDPS to CLASS Interface Control Document (draft) • Worked with NASA personnel to define initial requirements to archive EOS/MODIS Level-0 data.

  18. CLASS Accomplishments to Date (Continued) • Established the CLASS Operations Team (COT) • Includes OSDPD and NCDC personnel • In preparation for Baseline Dual-Site Operations • Established a project wide risk management program • Completed documentation of business case for CLASS • Suitland CSC Development Team Certified • SEI-CMM Level-3 • Worked with IBM-China for possible implementation of CLASS for National Satellite Meteorological Center (NSMC)

  19. CLASS Accomplishments to Date (Continued) • Completed … Summary “10-year” CLASS Project Budget • FY04/05, plus $25.6M Flat-line …. FY06-FY14 • Completed … NPP/NPOESS Campaign Plan • Completed … FY04 Spendplan • Against the Approved $10.5M NESDIS funding allocation • Completed … Initial CMMI Training at TMC • Completed … CLASS Developers Workshop • Held 03/04 March 2004 in Suitland • Brought together all CLASS team members

  20. CLASS Accomplishments to Date (Continued) • Completed … Software Release 2.0 / 2.1 / 2.2 • Operational Dual-site configuration • Operational 02 April 2004 • Completing GAA Activities • Final McIdas-less software design in final preparation at TMC • Planning for EOS/NPP Implementation at TMC • Task SOW Completed, sent to TMC, and Proposal received • NASA/IV&V Facility MOU in preparation for floor space

  21. CLASS -- Near-term Plans • CLASS Release 3.0 • Scheduled for 24 May 2004 • Provides: Delivery Manifest and Web Enabled Subscription Management • CMMI Activities • Preparing all CLASS development teams for Internal Evaluation in July 2004 • CLASS Release 3.1 • Scheduled for August 2004 • Provides: Ingest Enhancements to support IJPS NOAA data

  22. CLASS -- Near-term Plans (Continued) • System Storage Capacity Upgrade • Scheduled for September 2004 • Additional disk space at both CLASS operational sites • CLASS Release 3.2 • Scheduled for November 2004 • Provides: Ingest enhancements to support IJPS EUMETSAT data, and IJPS Test readiness

  23. FY04 CLASS Goals • Prepare a “5-Year” Budget and Program Plan for CLASS (COMPLETED) • Focus CLASS resources on CLASS Activities (COMPLETED) • CLASS, GAA, EOS, SAA Projects combined • Initial test of CLASS enhancements for IJPS … NOAA-N (October 2003 - COMPLETED) • Configure and Test CLASS at NCDC and transition to Stand-alone Operations (December 2003 - COMPLETED) • Complete testing and transition to operations of CLASS dual site capabilities (April 2004 - COMPLETED) • Complete testing and transition to operations of the GOES Campaign (April 2004 - COMPLETED) • Complete IJPS/Metop Critical Design Review (21 April 2004 - COMPLETED)

  24. FY04 CLASS Goals (Continued) • Configure and Test CLASS for support of IJPS/Metop, and transition to Operations (September 2004) • Establishment of an integrated Design-Implementation team between Suitland and TMC/Fairmont • Design NPP Campaign Data Ingest as initial phase of NPP Campaign, and with partial implementation • Design and Implement EOS Data Ingest as initial phase of EOS Campaign • Design of the JASON Campaign Data Ingest, as initial phase of the JASON and OSTM Campaign(s) • Completion of the GOES (development) Campaign • Continue migration of GOES Retrospective Data Sets into CLASS • Continue migration of POES Retrospective Data Sets into CLASS • O&M of Dual-site CLASS System at Suitland and Asheville

  25. Current CLASS Hardware Architecture • CLASS hardware for Suitland and Asheville sites: • Operational Servers, 4 at each site • IBM P660, 4GB RAM 2CPUs • Integration and Test Servers, 2 at each site • IBM P660 • Development Servers, 2 at Suitland • IBM P660 • Storage Area Network (SAN) • SUN 9980, 5.2TB • Tape Archive System & Hierarchical Storage Manager (HSM) • 9522 LTO Tape Slots, 12 Drives, 8PB capacity @ LTO-3

  26. Suitland System Hardware

  27. The Future CLASS

  28. CLASS Budgets • FY06 $11.9M • FY07 $9.9M • FY08 $9.9M • FY09 $22.1M • FY10+ $27.9M/yr • FY00 and prior N/A • FY01 $1.995M* • FY02 $3.599M* • FY03 $2.881M* • FY04 $10.5M • FY05 $11.1M * NET

  29. “GOES-R Campaign” • Requirements Definition and ICDs • Data and Products Ingest • Storage, Processing, and Communications Upgrades • Metadata extension • Catalogue extension • Visualization extension • Reprocessing extension

  30. CLASS “GOES-R” Budget • FY07 $2.0M • FY08 $3.8M • FY09 $4.1M • FY10 $7.6M • FY11 $8.5M • FY12 $6.9M • FY13 $6.2M • FY14 $2.0M $41.0M TOTAL

  31. THANK YOU!

More Related