330 likes | 500 Views
Satellite Data Provided by the NASA Ocean Biology Processing Group (OBPG). Deep-Water Chlorophyll Images common-bin 12-day composite, Winter 2002. 0.01-1 mg/m^3. SeaWiFS R5. MODIS/Aqua R1. OBPG Satellite Data. - MODIS Aqua (2002-present) - MODIS Terra (2000-present)
E N D
Satellite Data Provided by the NASA Ocean Biology Processing Group(OBPG)
Deep-Water Chlorophyll Images • common-bin 12-day composite, Winter 2002 • 0.01-1 mg/m^3 SeaWiFS R5 MODIS/Aqua R1
OBPG Satellite Data - MODIS Aqua (2002-present) - MODIS Terra (2000-present) - SeaWiFS (1997-present) - OCTS (1996-1997) - CZCS (1978-1986)
HDF File Format • Hierarchical Data Format (HDF) is a data file format • OBPG data files stored as HDF (L1A, L1B, L2, L3) • HDF can be thought of as a “container” to store a variety of data types and meta data in a single file
HDF-EOS • OBPG files stored in HDF-EOS format: • specialized form of HDF • created by NASA in 1993 • standard format for all EOS data • SeaDAS is compatible with OBPG HDF-EOS
MODIS Data Levels Level 0 Level 1A Level 1B Level 2 Level 3 binned Level 3 mapped
Level 0 raw digital counts native binary format Level 1A raw digital counts HDF formatted Data Levels & Flow • ATT & EPH • spacecraft attitude • spacecraft position • GEO • geolocation • radiant path geometry • Level 1B • calibrated reflectances • converted telemetry • Ancillary data • wind speed • surface pressure • total ozone • Reynolds SST • Level 2 • geolocated geophysical products for each pixel
Level 0 raw digital counts native binary format Level 1A raw digital counts HDF formatted Data Levels & Flow • ATT & EPH • spacecraft attitude • spacecraft position • GEO • geolocation • radiant path geometry • Level 1A Subset • reduced to standard ocean bands only • Level 1B • calibrated reflectances • converted telemetry • Ancillary data • wind speed • surface pressure • total ozone • Reynolds SST • Level 2 • geolocated geophysical products for each pixel
Level 1A (HDF) • raw radiance counts • Onboard calibration data • navigation data • spacecraft telemetry • instrument telemetry • A2006027191500.L1A_LAC
Level 1B (HDF) • calibrated, geolocated at-aperture radiances i.e. Top of Atmosphere (TOA) radiances • quality flags • error estimates • calibration data A2006027191500.L1B_LAC A2006027191500.L1B_HKM A2006027191500.L1B_QKM
Level 2 (HDF) • geophysical values for each pixel • products include chlorophyll-a, SST, nLw’s, K490, flags • geolocation data A2006027191500.L2_LAC A2006027191500.L2_LAC_SST A2006027191500.L2_LAC_SST4
Chlorophyll-a SST
Level 2 Standard MODIS Ocean Products • Ocean Color (day only) • Normalized water-leaving radiances - nLw() • Chlorophyll - Ca • Diffuse attenuation - Kd(490) • Aerosol type and concentration • Optical thickness - a • Ångström exponent • Atmospheric epsilon • Processing flags • Cloud, land, glint, atmfail, atmwarn, chlfail, chlwarn, etc. • Ocean Temperature • Long-wave SST (11-12 m) (day and night) • Short-wave SST (3.9 - 4.0 m) (night only) • SST quality level (0-4)
Level 2 Non-standard Ocean Products • Ocean Color • Alternate Ca and Kd algorithms • Chlorophyll fluorescence, FLH • Particulate inorganic carbon, Calcite • Inherent optical properties (various bio-optical models) • absorption (total, phaeophytin, dissolved matter) • backscatter (total, particulate) • Photosynthetically active radiation, iPAR, PAR (coming) • Euphotic depth (Zeu, Zsd) • Subsurface PAR at 1st optical depth, Kd(PAR) • Intermediate atmospheric correction products • Ocean Temperature • Brightness temperatures
L3 Binning Level-2 After Level-2…
Level 3 Binned data products (HDF) • data composites • binned Level 2 data (spatially and temporally) • arithmetic means with statistics for each bin • daily, 8-day, monthly, yearly periods • 0.5, 1, 2, 4, 9, 36 km2 resolutions • stored as one file or a main file with subordinates • useful for scientific research A20071822007212.L3b_MO.main
Level 3 SMI (mapped) (HDF) • Mapped L3 binned product • Data is scaled to 16-bit values • Equidistant cylindrical projection A20071822007212.L3m_MO_CHLO_4
Ancillary data products (HDF and other formats) • NRT data and refined ancillary data • used in various stages of processing: • Geolocation (attitude and ephemeris) • Calibration (coefficient look-up tables) • atmospheric correction (MET/OZONE/SST)
Operational MODIS Ocean Band Suite VIS/NIR Ocean Color Thermal SST
Operational MODIS-Aqua Data Flow NASA EDOS System Software process April 1, 2014 Hardware system Database table NOAA Realtime System Ancillary input data User Community MET Full-resolution day- and nighttime via SEN (~60 GB per day) MET, Ozone, and OISST data are dynamically selected for each L1A granule Ozone MODISA L2 table Level-0 Ingest L1B-L2 (MSl12) Ocean Color Web Server OISST Archive - Distrib Server Sensor CAL L1A-L1B (MOD_PR02) Ingest queue table Sensor attribs Browser CGI / httpd MODISA L3-bin table L3BIN (l2bin) Atm corr Archive - Distrib Server ATT/EPH Ingest process Archive - Distrib Server MySQL DB Geo-Location (MOD_PR03) MODISA atteph table MODISA L3-map table Product meta data are populated from production DB L3MAP (smigen) Archive - Distrib Server L0 Ingest process Archive - Distrib Server Archive - Distrib Server L0-L1A (MOD_PR01) MODISA L0 table MODISA L1 table
ODPS Data Processing System Current Components Distribution Servers (ftp) 6 storage nodes 7.2 TB Distribution Servers (web)1 large server – 3 TB 58 storage nodes 69.6 TB Ingest Servers 2 SeaSpace groundstations 5 storage nodes 6 TB Cal/Val & QC Systems Processing Cluster 34 processing nodes 1.5 TB Extreme Networks Black Diamond 6816 Gigabit Ethernet switch Mission Operations Systems Database Server 1 large server 876 GB Development Servers 1 processing node 2 storage nodes 2.4 TB Testing Cluster 13 test nodes 2 TB Backup Servers 1 large server 876 GB+2 TB tape 5 storage nodes 6 TB User Desktops Network Support Systems
Data Processing and Storage gigabit ethernet • database controlled, distributed processing and storage system • many Linux-based, processing nodes (34 dual 3GHz Xeon) • 200X processing for MODIS/Aqua ocean color • 4000X processing through-put for SeaWiFS ocean color • 9-month OCTS mission can be reprocessed over lunch • all data online for direct access (100TB RAID-5 array) • capacity for global mission testing prior to reprocessing, allowing opportunity for community participation in reprocessing decisions • direct and near real-time user access to data archives database server processing node storage node
Ocean Biology Data Processing System - Processing Node Supermicro 6024 rack mount server 2 Intel Xeon CPUs 3.06 Ghz 8 1 GB DDR2 RAM modules 2 Intel Gigabit Ethernet network interfaces 1 Seagate 80 GB EIDE disk (system) 5 Maxtor 73.4 GB Atlas 10K IV Ultra320 SCSI disks (processing) 1 CD-ROM (low profile) 1 Supermicro X5DP8-G2 motherboard with built-in Ultra320 SCSI controller 1 Supermicro 6024H 2U rack mount server case with dual power supplies Operating system: Linux
Ocean Biology Data Processing System - Storage Node NAS 1.2 TB rack mount server 1 Intel Pentium-4 CPU 3.06 Ghz 2 512 MB DDR2 RAM modules 1 Gigabit Ethernet network interface 1 Seagate 80 GB EIDE disk (system) 8 Maxtor 250 GB MaxLine Plus II EIDE disks (RAID 5, 6 data + 2 hot spares) 1 CD-ROM (low profile) 1 Asus P4C800-E Motherboard 1 3ware Escalade RAID controller for 8 drives 1 Comix Computer RM-3U9SCA (IDE) ATX rackmount chassis 3 U, 9 hot swappable drive bays Operating system: Linux
Ocean Biology Data Processing System - Large Server SunFire V880 rack mounted 4-8 UltraSPARC-III+ processors 8-16 GB RAM 6-12 73 GB 10K RPM FibreChannel disks 2 Gigabit Ethernet nework interfaces 1-5 Fast Ethernet network interfaces 1 External FibreChannel adapter Operating system: Solaris
Ocean Biology Data Processing System - Network Switch Extreme Networks Black Diamond 6816 768 Gbps total switching capacity Route/filter/forward 192 million pps 160 GigE ports in current configuration All modules are hot swappable 4 MSM management modules 2 G8Ti 8 port Gigabit Ethernet modules 6 G24T3 24 port Gigabit Ethernet modules 4 Hot-swappable power supplies