Interoperable Information System of Systems for HTAP

1. Interoperable Information System of Systems forHTAP Rudolf B. Husar and Rich Scheffe With Erin Robinson Presented at HTAP Workshop, WMO, Geneva, January 26, 2006

2. Intercontinental Transport Events have Influenced AQ Regulations ..and now Policy? –> Air Quality ManagementWeb-Based Collab. in Virtual Workgroups –> Science, the ProcessNear-Real-Time Data Access & Integration –>Technologies and Tools US-Daily Avg. PM25 Dust: Seasonal + Events

3. HTAP Characteristics • Stakeholders are autonomous countries, organizations • Diverse perspectives on the issues, solutions • Pollution sources and impacts are hemispheric • Information integration is key to success • Observations and models are ready for integration

4. The HTAP - GEOSS • Stakeholders are autonomous countries, organizations • Diverse perspectives on the issues, solutions • Recognition of the need to understand, act • Pollution sources and impacts are hemispheric • Observations and models are ready for integration • Information integration is key to success Hence GEOSS is a good system model for HTAP HTAP is a suitable use case for GEOSS Some Characteristics of System of Systems (SoS) • Autonomous constituents managed independently • Independent evolution of each constituent • Displays emergent (novel, unpredictable) behavior

5. Summary There is an ‘alignment of stars’ for integrating heterogeneous HTAP data • System of Systems architecture is suitable for integrating HTAP data • Standard data access is a key interoperability protocol • Heterogeneous data can be non-intrusively standardized by mediators • Service-based software architecture delivers tailored products to diverse uses • Federated data and shared web-based tools are in use • DataFed already includes over 100 datasets (emissions, ground, satellite) • The system has been applied to EPA regulatory development • However, • DataFed is just one of the many mediator nodes, but these need to be connected • Much more data would need to be federated • HTAP model-data comparison would be an attractive use case WMO-GAW could be an effective neutral facilitator for global data integration using GEOSS (system of systems) principles

6. GEOSS Architecture Framework - DSS for HTAP HTAP Reports:Model Estimates, Obs. Evidence Models Knowledge into the Minds of AQ Analysts HTAP Knowledge into the Minds of Policy Makers HTAP Knowledge into the Minds ofPolicy Analysts Observations Decisions Decision Support System GEOSS Architecture Framework

7. System of Systems: InteroperabilityConnecting Machines and People Interoperability Stack: Key concept of the Web System components have to be interoperable at each layer Amplify Individuals Connect Minds Open Architecture Data Standards Service Based IP – Internet Protocol

8. How do you agree on Interoperability?

9. App App App App WCS/WMS Server GetCapabilities Client Std. Interface Std. Interface Capabilities, ‘Profile’ Back End Where? When? What? Which Format? Front End GetData Data What few things must be the same so that everything else can be different? Are WCS/WMS Space-Time-Parameter queries T1 T2 Convergence Protocols

10. Lets agree onSpace-Time-Parameter Data Access Query Protocol

11. Data Providers Info System Info Users Public AIRNow AIRNow Model Manager Compliance Scientist DAACs Science • Current info systems are project/program oriented and provide end-to-end solutions • Part of the data resources of any project can be shared for re-usethrough DataFed • Through the Federation, the data are homogenized into multi-dimensional cubes • Data processing and rendering can then be performed through web services • Each project/program can be augmented by Federation data and services From ‘Stovepipe’ to Federated Architectures

12. Service Chaining & Workflow Web Services and Workflow for Loose Coupling Obs. & Models Decision Support System Control Std. Interface Std. Interface Data Reports Datasets Data Views Integrated Data Reports Web Service Interaction Service Broker Find Publish Bind Service User Service Provider

13. Co Writing - Wiki Screencast Collaborative Reporting and Dynamic Delivery Obs. & Models Decision Support System Control Std. Interface Std. Interface Data Reports Datasets Data Views Integrated Data Reports Analysis Reports: Information supplied by manyNeeds continuous program feedbackReport needs many authorsWiki technologies are for collaborative writing Dynamic Delivery: Much of the content is dynamicAnimated presentations are compellingMovies and screencasts are for dynamic delivery

14. WMS, WCS OGC Services Model-Data Comparison Workflow Software VIEWSChemical Data Ft. Collins, CO Std I/O Model-Data Comparison Workflow AeroCom Chemical Models Paris, FR Std I/O Workflow Flow ProgramLego-like assembly of component

15. Sample of Federated Datasets

16. HTAP Model-Obs “Test/Benchmark”ed • A virtual data center design to meet the needs of the TF HTAP model intercomparison and evaluation • Archives of key observations and model data • Tools for model intercomaprison

17. Tools..Analyst ConsoleUser Layout

18. Imagine…More Shared Global Obs & Models 2007 Global Data & Models 2007++ More Global Data & Models

19. AeroCom CAPITA Obs &Services Obs & Model Models Services Obs Federated Information System Observations, Models, Services Other Federations VIEWS Models & Services The Network Effect:Less Cost, More Benefits through Data Reuse Programs ask/get Data Orgs Develop Programs Public sets up Orgs Data Data Program Data Organization Program Data Re-Use Network Effect Public Program Organization Data Data Program Data Pay only once Richer content Less Prog. Cost More Knowledge Less Soc. Cost More Soc. Benefit

20. Agile Information SystemDesign Principles Guiding Idea: System of Systems GEOSS Domain of Interaction WMO/GAW & Stakeholders Infrastructure: Internet, Middleware, Web 2.0 Methods, Tools: Standards, Use Cases Based on P. Senge et. al, 1994: Architecture of Learning Organizations(Link)

21. R. Scheffe NPS Protect ecosystems, AQ WQ USDA Protect/optimize Ag and forest resources CDC To promote health and quality of life …. Apparent divergences? Organizations have different missions NASA Explore fundamental Earth System Properties EPA (Protect human health & envi. Improve air, water, ecosystem NOAA To understand and predict changes in the Earth’s environment and …

22. R. Scheffe Collaboration (culture) empire building? Observation technologies {e.g., satellites} Computational power Science, talent {embodied in AQ models and young geniuses} Accountability,↓ regulatory assessments {e.g., NAS, CASAC} Information technologies {e.g., data sharing protocols} Alignment Budgets, agency collaboration resource/program accountability Stars aligned? Confluence again..

23. Summary There is an ‘alignment of stars’ for integrating heterogeneous HTAP data • System of Systems architecture is suitable for integrating HTAP data • Standard data access is a key interoperability protocol • Heterogeneous data can be non-intrusively standardized by mediators • Service-based software architecture delivers tailored products to diverse uses • Federated data and shared web-based tools are in use • DataFed already includes over 100 datasets (emissions, ground, satellite) • The system has been applied to EPA regulatory development • However, • DataFed is just one of the many mediator nodes, but these need to be connected • Much more data would need to be federated • HTAP model-data comparison would be an attractive use case WMO-GAW could be an effective neutral facilitator for global data integration using GEOSS (system of systems) principles