Current Air Quality Information ‘Ecosystem’ (Draft for Feedback)

1. Current Air Quality Information ‘Ecosystem’ (Draft for Feedback) AQ information includes emissions, ambient & satellite data and model outputs The distributed data are produced and provided by agencies, mostly through portals Providers have different access protocols, formats, and information usage conditions This lack of interoperability causes the under-utilization of the rich data resources ESIP AQ Cluster, rhusar@me.wustl.edu

2. Future Integrated AQ information System (Draft for Feedback) Data Federation Distributed, Virtual, Uniform Data Processing Filtering, Aggregation, Fusion Info ProductsReports, Websites NEI DAACs AQ Forecasting NEISGEI IDEA Data are maintained by custodians and exposed through ‘portals’ Mediators uniformly ‘wrap’ data and provide processing services Analysts program the services to create application-specific products Responsibility is shared among data providers and mediator/ integrators ESIPFed can provide the infrastructure and tools for the AQ info system Emission Missions AQ Compliance FireInv GASP DataMart AQMod Network Assess. AIRNow Forecast Status and Trends VIEWS WeaMod Mediators ASOS GloMod ESIP AQ Cluster, rhusar@me.wustl.edu

3. EPA NEI EPA-AQS DataMart NOAA Forecast EPA AQModel NASA GloModel NOAA GASP NOAA FireInv State/Local Emission NASA DAACs NASA Missions NOAA WeaMod EPA AIRNow NASA IDEA EPA NEISGEI NOAA ASOS RPO VIEWS Air Quality Information Providers • AQ information includes emissions, ambient (surface) and satellite data and model outputs • The information is provided by multiple Agencies, have different form and is • AQ data usage requires considerable processing and integrating Form | Content Emission Ambient Satellite Model ESIP AQ Cluster, rhusar@me.wustl.edu

4. DataFed WS Output Data Types • UrlGranuleType • TimePointType • TimeDimensionType • MapVectorType • MapTrajectoryType • MapTimePointType • MapPointType • MapLocationTableType • MapImageLatLonType • MapGridType • ImageType • HtmlType • DotNetTableType • DataSetType ESIP AQ Cluster, rhusar@me.wustl.edu

5. Interagency Working Group for Earth Obs. (IWGEO) Global Earth Observing System of Systems (GEOSS) T. Karl, NOAA, NCDC ESIP AQ Cluster, rhusar@me.wustl.edu

6. T. Karl, NOAA, NCDC Integrated Observing Systems 21st Century Ocean Observations Space Observations Innovations Innovations Mass Productions Breakthrough Efficiencies Cost Efficiencies Cost Breakthrough Mass Productions OBSERVING SYSTEM TIMELINE 6

7. Tools for Users • Pare down large file sizes of high resolution data and products. • (re-) Group different data sets to create needed products – such as initialization files for model development, analysis, and intercomparison. • Subset the data: • in parameter space • in physical space • in temporal space ESIP AQ Cluster, rhusar@me.wustl.edu

8. Collaborations: How do we get there? G. Rutledge: Emerging Tools for Distributed Data Access and Collaborations • Data systems based on the integration of independently developed system elements offer many more opportunities than more traditional centrally developed ones. • P. Cornillon • Data transport is being actively pursued: OPeNDAP, SOAP, ... • Earth System Partners need to be able to find and use various data sets, wherever they may be, whatever format... • THREDDS can provide dynamic access and generate catalogs • GCMD is a major resource for metadata management for the entire GeoSciences community- this activity must evolve! • Ontology projects such as SWEET in conjunction with THREDDS and GCMD can provide individual data sources, data variables and metadata management for the community. ESIP AQ Cluster, rhusar@me.wustl.edu

9. GIS is Evolving to a Web Services Environment Web Services Networks Information Systems Tools & Data Shared Services Information Management Professional Productivity . . . Becoming More Intelligent And Distributed ESIP AQ Cluster, rhusar@me.wustl.edu

10. GIS Networks Will Allow Us to Connect andIntegrate Distributed GIS Resources Models Maps Metadata GeoData Sets Peer-to-Peer GIS Data Models . . . Making Virtual Collaborations Possible ESIP AQ Cluster, rhusar@me.wustl.edu

11. Enabling Technology • Faster Hardware • Distributed Computing • Mobile/Wireless • Services Oriented Architecture • Large Data Repositories • GIS Software Pervasive Computing Capacity In 10 Years • 100x Computing • 1000x Storage • 5000x Networks Terabyte/Second Communications ESIP AQ Cluster, rhusar@me.wustl.edu

12. GIS Portals Support Data Dissemination Many Standard Formats And . . . Clip/Zip/Ship Data Conversion Internet Web Services Select Format VPF Zoom to Extent GML DGN DXF CAD XMC IMS Server TIGER S57 MIF DLG DWG Geomedia Support Many Formats SDTS . . . Supporting Interoperability ESIP AQ Cluster, rhusar@me.wustl.edu

13. DBMSIntegration Direct Read (API) Conversion Interoperability Is Important There Are Many Standards . . . XML/SOAP GIS Server Web Services . . . Focus Is On Simple and Practical Approaches That Work ESIP AQ Cluster, rhusar@me.wustl.edu

14. MIF GML M.S. MIF Interoperability Technology Is A Fundamental Part Of GIS Products Standards And Direct Proprietary Interfaces Dynamic Read/Conversion/Use Direct Read & Use Custom Format Converters . . . Supporting Complex Data Transformation ESIP AQ Cluster, rhusar@me.wustl.edu

15. Geoprocessing Models Interoperability Is Important For GIS Networks to Work Either Everyone Uses Same Software, Data Formats, and Data Models . . . . . . OR They Use Interoperability Procedures . . . Geoprocessing Models Can Transform Data Automatically • Format Conversion • Schema Reorganization (ETL) • Scale Projection Changes • Generalization • Merge . . . Enhancing Collaboration ESIP AQ Cluster, rhusar@me.wustl.edu

16. Geoprocessing On Servers Distributing Spatial Analysis And Modeling Now Future Browser Desktop GIS GIS Data Sets . . . Distributed Workflow & Process Models ESIP AQ Cluster, rhusar@me.wustl.edu

17. Future T1 Managing Multi Dimensional Geographic Data Sets And Simulation Modeling New Folder\ELNINO_Final.avia With Particular Focus on Time • Data Modeling • Tools for Manipulation • Query • Change Analysis • Iterative Processing • Visualization • Animation • Charting Simulation / Time Looping . . . Iterative/Recursive Modeling ESIP AQ Cluster, rhusar@me.wustl.edu

18. GIS Will Maintain Distributed Geographic Knowledge Geodatabases Will be Distributed and Federated Relationships Will be via “Messaging” (Sending/Receiving Web Services Messages) ESIP AQ Cluster, rhusar@me.wustl.edu

19. Serving Globes Over the Web Globe Web Server . . . Serving 3D Virtual Geography ESIP AQ Cluster, rhusar@me.wustl.edu

20. Web Service GIS Desktop Personal Server Metadata Geodata Sets Data Models Maps Geodatabase Models Personal GIS Server Will Allow Peer to Peer Collaboration • Easy to Use • Simple to Install Supporting • Map Services • Metadata Catalog (Searching & Harvesting) • Download • Data • Models • Data Models . . . Users Will ShareAnd Serve Their Knowledge ESIP AQ Cluster, rhusar@me.wustl.edu

21. T T T T T T Geodatabases Will Support Distributed Data Management • Replicated • Periodically Updated • History/Archiving = Transactions Update Messages National State Local ESIP AQ Cluster, rhusar@me.wustl.edu

22. Infusion Confusion Solutions:Putting Technology to Work Earth Science Data System Working Group on Technology Infusion Karen Moe, NASA/ESTO Rob Raskin, NASA/JPL

23. What is the Technology Infusion Working Group? ESE Strategic Plan • One of four groups established by the REASoN CAN • Standards & Interfaces • Metrics Planning & Reporting • Reuse Frameworks • Technology Infusion • Outgrowth of SEEDS • Strategic Evolution of ESE Data Systems • Explored ways to support NASA ES strategy • More PI production processing • Measurement-oriented systems SEEDS REASoN CAN Projects Projects • ESDSWG • Standards • Metrics • Reuse • Infusion Projects • Data Life Cycle New in 2005 REASoN = Research, Applications, and Education Solutions Network CAN = Cooperative Agreement Notice ESDSWG = Earth Science Data System Working Groups ESIP AQ Cluster, rhusar@me.wustl.edu

24. Why is Technology Infusion Important?Drivers and Opportunities • Enterprise Context • Constrained budgets • Broad data service provider community • Organizational Goals • Lower system costs • Increase community participation • Increase flexibility & responsiveness Drivers Effective Technology Infusion External Internal Opportunities • Emerging Technologies • Technology investments • Web and grid computing • Linux clusters • Pragmatic Infusion Approaches • Information sharing • Demonstration testbeds ESIP AQ Cluster, rhusar@me.wustl.edu

25. Why is Technology Infusion Important?Meeting ESE Goals Requires Tech Infusion • Science and application needs • Faster & better models • Near-real-time data • Easier data fusion • Science data system needs • Enable open distributed architecture for PI processing • Fill capability gaps in current systems • Support evolution New Research New Applications New System Capabilities Technology Infusion Technology Identification / Development Science & App Needs System Capability Vision ESIP AQ Cluster, rhusar@me.wustl.edu

26. Technology Infusion is Part of a Larger System Evolution Process • Think globally, act locally • How can we improve technology infusion across the community? • How can you successfully infuse technology in your own projects? Peer Review & Competitive Selection Technology Development Technology Infusion Solicitation Formulation Operational Systems Capability Needs Capability Vision Identified Gaps Technology Roadmaps Technology Projections ESIP AQ Cluster, rhusar@me.wustl.edu

27. What Capabilities are Needed? ESIP AQ Cluster, rhusar@me.wustl.edu

28. THREEDS THREEDS - Topics • Traditional Unidata Approach • Mainly meteorological data • Subscription system pushes data to user sites • Unidata Program Center provides data analysis tools for use on data at user sites • THREDDS Enhancements • Broader menu of Earth system data • Local client access from remote servers • Less arcane, more general and accessible tools • Integration of data and analysis tools into educational modules and digital libraries The THREDDS (Thematic Realtime Environmental Distributed Data Services) project is developing middleware to bridge the gap between data providers and data users. The goal is to simplify the discovery and use of scientific data and to allow scientific publications and educational materials to reference scientific data. The mission of THREDDS is for students, educators and researchers to publish, contribute, find, and interact with data relating to the Earth system in a convenient, effective, and integrated fashion. Just as the World Wide Web and digital-library technologies have simplified the process of publishing and accessing multimedia documents, THREDDS is building infrastructure needed for publishing and accessing scientific data in a similarly convenient fashion. ESIP AQ Cluster, rhusar@me.wustl.edu

29. People Documents Data THREDDSTHematic Real-time Environmental Distributed Data Services Connecting people, documents and data ESIP AQ Cluster, rhusar@me.wustl.edu

30. Summary • Universities have used Unidata tools to acquire, analyze, and display real-time atmospheric data for nearly 20 years • THREDDS – along with related client/server access and display technologies-- makes an even broader menu of Earth system data to a more diverse community of users • THREDDS technologies enable the creation of compound educational modules and scientific publications with embedded pointers to datasets and tools. ESIP AQ Cluster, rhusar@me.wustl.edu