1 / 13

Atmospheric Science Data Center HPC User Forum

Atmospheric Science Data Center HPC User Forum. Michael Little September 10, 2009 Broomfield, CO. Agenda – September 09, 2009. Recent NASA Science Mission Directorate Activities SMD Modeling Needs Study 2008/2009 – Tsengdar Lee http://www.hec.nasa.gov/news/reports.html

ace
Download Presentation

Atmospheric Science Data Center HPC User Forum

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Atmospheric Science Data Center HPC User Forum Michael Little September 10, 2009 Broomfield, CO

  2. Agenda – September 09, 2009 • Recent NASA Science Mission Directorate Activities • SMD Modeling Needs Study 2008/2009 – Tsengdar Lee • http://www.hec.nasa.gov/news/reports.html • Decadal Survey Data Systems (DSDS) –Martha Maiden • June, 2009 • http://dsds.nasa.gov • Atmospheric Science at NASA Langley Research Center • Science Focus • Atmospheric Science Data Center • NextGen Net-centric Operations NASA ASDC HPCU Forum

  3. Atmospheric Science @LaRC • Remote Sensing Instruments &in situ measurements • Earth Radiation Budget (ERBE, CERES) • 40 years of internally consistent measurements (<1% err) • TRMM, AQUA, TERRA, NPP*, NPOESS* • Atmospheric Chemistry (SAGE) • Clouds and aerosols (CALIPSO) • Analysis of measurements • Radiation and Clouds in Climate • Clouds properties • Data Fusion is key to all higher order data products NASA ASDC HPCU Forum

  4. Earth Radiation Budget CERES Measures the Earth’s Radiant Energy Balance NASA ASDC HPCU Forum

  5. CALIPSO LIDAR Data NASA ASDC HPCU Forum

  6. Climate Science Needs for Computation • High accuracy (reduce error to <1% in measurement) • Calibrate, calibrate, calibrate • Current weather measurements are ~15% • Re-processing long time series • Internal consistency • 100x (100 days of observations re-processed in 1 day of compute time) • Cannot guarantee re-processing will give same data • Easy to Use Development Environment • Physical Scientists frequently have weak computer science skills (no formal training) • Resistance to change • Scientists are more valuable than compute cycles • MATLAB and IDL/ENVI NASA ASDC HPCU Forum

  7. Atmospheric Science Data Center • ESDIS Digital Active Archive Center • Data Sets • ERBE, CERES, CALIPSO, SAGE, TES, MISR • 6 Functions: Ingest, Archive, Produce, Distribute, Report, Support • Produce/re-process instrument data products • CERES, CALIPSO, MISR, SAGE • Distribute Released Data Products • Science Computing Facility • Local research and missions(300 users) • Develop/test code to produce data products • Instrument calibration (including pre-launch ground cal) • Data Product calibration • Analysis of data • Commercial Applications: MatLab and IDL/ENVI NASA ASDC HPCU Forum

  8. Recent Changes at ASDC • Replace legacy systems with computational environment with high I/O to disk systems • Big endian and little endian due to code migration issues • Increase processor availability with minimum system-wide outages • Fiberchannel interconnect • Improve reliability, maintainability • Reduce SysAdmin workload • Put all interesting data online (2PB) • IBM x3950, x3650, P6 processors • IBM DS4800, DS5300 Disk Arrays with lots of redundancy • Brocade M48k Director • Sun Grid Engine • Improve stewardship and data protection • All sites have local environmental risks • Key products stored offsite on tape • Tape archive and backup • SunSTK 8500 with Quantum StoreNext • Code conversion took > 5 years • Scientifically identical data products NASA ASDC HPCU Forum

  9. ASDC/SCF Fiberchannel Connectivity ASDC SDP Production 56 P6 256GB 56 P6 256GB LB1 LB1 ASDC JBOSS Server & TS-1 & Ingest Common SANhead (NFS) (AFP) (Samba) SCF Cluster (Access through Sun Grid Engine) 56 P6 256GB SCF Interactive Processors Db Server Db Server 56 P6 256GB 14 4P6@8GB Clouds Clouds SOFA 64 x86 SMP 256 GB WWW WWW SRB/Power SRB/Power Instrument ERBE-like Instrument SARB Consodine TISA TISA 64 x86 SMP 256 GB 64 x86 SMP 128 GB 16 x86 HS12 64 GB 64 x86 SMP 256 GB 56 P6 256GB Fiberchannel Switch 220TB DS4800 220TB DS4800 220TB DS4800 220TB DS4800 220TB DS4800 220TB DS4800 64 x86 SMP 128 GB 220TB DS4800 220TB DS4800 64 x86 SMP 128 GB ECS DUE Processing SCF Disk Array ASDC Disk Archive (R/W Production) (R/O SCF) Cloud Modeling Cluster (Xu) FC Monitor GPU Experiment (YongHu) Sim Server 3755 Tesla 3755 Tesla NASA ASDC HPCU Forum

  10. Projected Work at ASDC • Improve performance of processing and storage • Tune SGE • Fiberchannel switch configurations • Improve quality of SysAdmin Support • Train Users to leverage investment • Model computational environment • SIMSCRIPT III • Improve Distribution Function • Google Earth • Permit direct access of data by non-local research community • Service Oriented Architecture (SOA) NASA ASDC HPCU Forum

  11. NextGen National Airspace System • Trajectory Based Operations requires Common Operating Picture among everyone using NAS • Avionics Roadmap created in November, 2008 • Scope is curb-to-curb instead of gate-to-gate • NextGen Joint Planning and Development Office • Created by Public Law 108-176, VISION 100 Act • Coordination function, not PMO • 3x air traffic, safer, cheaper with less environmental impact by 2025 • http://www.jpdo.gov • Industry participates via NextGen Institute to avoid Conflict of Interest • Governance by Collaboration among Partner Agencies and Companies • FAA, Defense, Homeland Security, Commerce, NASA, EPA • Airlines, Airports NASA ASDC HPCU Forum

  12. NextGen Net-centric Operations • Leverage DoD experience with Net-centric Operations • JPDO Net-centric Operations Division leads collaboration • USAF CIO, Electronic Systems Center, AF Weather Agency • FAA SWIM PM, FAA Telecommunications Infrastructure • NOAA Office of the Federal Coordinator for Meterological Services and Supporting Research • NWS Aviation Weather Office • NASA LaRC, GSFC and eventually ARC and DFRC • DHS Chief Technology Officer and CIO • NASA, DARPA and NSF supporting R&D • Industry partnership via Net-centric Working Group • Membership cleared through NextGen Institute • https://jpdo-ncowg.larc.nasa.gov/wiki/index.php/Main_Page • Lead business process re-engineering, info exchange design • Info owners to implement information exchanges • Ensure access to information for all parties in NAS • Semantic Service Oriented Architecture (SOA) • Machine-to-machine readability is essential NASA ASDC HPCU Forum

  13. NextGen Semantic SOA • 4D Weather Cube is first implementation • 2013 Initial Operational Capability for Info Exchange Services • Development at MIT Lincoln Labs and NCAR • FAA NNEW and Weather in the Cockpit programs • NWS Aviation Weather Office • Other Business Areas with Information for Exchange • Safety • Aircraft • Airports • Flight/Flow • Unmanned Aerial Vehicles • Key Issues • Cultural Change • NCO Governance • Cybersecurity • Infrastructure: computation and data communications • Integrated Surveillance • Environmental Impact • Airports • Special Use Airspace NASA ASDC HPCU Forum

More Related