Federation Past, Present and Future: a personal view

1. Federation Past, Present and Future:a personal view John Townshend, Past President of the ESIP Federation, 1999-2000 Global Land Cover Facility ESIP University of Maryland

2. Presentation Outline • The goals, scope and operating mode of the Federation. • A brief history of the Federation. • What is the relationship of the Federation to EOSDIS and NewDISS? • How successful has the Federation been in reaching communities? • Future of the Federation

3. What is the Earth Science Information Partnership? • Set up as an experiment in science governance environmental data products and services. • Cross-cutting involving government, academic, commercial and non-profit participants. • Brings together data and rapidly evolving technologies to meet the needs of many Earth Science missions. • Acts as a framework for coordinating and expanding upon pioneering research set forth by the Partners. • Prototype for the next generation Earth Science Enterprise model.

4. 1) The goals, scope and operating mode of the Federation. • The Goals of the federation are to increase the quality and value of Earth Science products and services throughout their life cycle. • The Beneficiaries are all the Federation’s stakeholders. • Achieving the goals will be by continuously improving all of the science-based processes underpinning its goods and services.

5. Federation Scope: Core Activities • Encouraging the use of best science practices to ensure the quality and breadth of data and resultant information, products and services. • Ensuring that data and information can be readily exchanged and integrated to improve Earth science data, information, products, and services. • Contributing to the development of an Earth science information economy through application research and commerce. • Increasing the diversity and breadth of users and uses of Earth science data, information, products and services.

6. Federation Partner Types • Type 1 ESIPs (principally the DAACs) • standard data and information whose production, publishing/ distribution, and associated user services requires considerable emphasis on reliability and disciplined adherence to schedules. • Type 2 ESIPs • data and information products and services in support of Earth System Science that are developmental or research in nature or where emphasis on flexibility and creativity is key to meeting the advancing research needs. • Type 3 ESIPs • providing ESE-based data and information products and services to users beyond the Earth System Science research community. • Type 4 ESIPs • Large institutional sustaining members – currently only NASA.

7. ESIP Types 2s Creating and distributing remote sensing data using a variety of innovative tools and interfaces. • The Distributed Oceanographic Data System (DODS) • Terrestrial Environmental Research • The Earth Science Partners Private Network • The Earth System Science Workbench • The GPS Environmental and • Earth Science Information System • The Ocean ESIP • The Passive Microwave ESIP • The Numerical Terradynamic Simulation Group • The Global Land Cover Facility • SnowSIP • The Seasonal to Inter-annual ESIP • IBM Watson Research Center Public Health ESIP • The Southwest RESAC • The Tropical Rain Forest Information Center • Great Plains Regional Earth Science Applications Center

8. ESIP Types 3s Working directly with user communities to provide targeted environmental products and services. • Bay Area Shared Information Consortium • California Land Science Information Partnership • The Earth Data Analysis Center • The Environmental Legal Information System • Museums Teaching Planet Earth • Northeast Regional Earth Science Applications Center • The Earth Data Discovery Consortium • Reading Information Technology Incorporated • Scientific Fishery Systems • TerraSIP • Veridian MRJ Technology Solutions • The Upper Midwest Aerospace Consortium • Stormcenter.com • TERC • Mid-Atlantic Regional Earth Science Applications Center • Southern California Wildfire Hazard Center

9. Federation Beginnings: • NRC recommendation • USGCRP found it controversial • NASA took on the Federation Experiment • NASA DISS evolves – PI processing, NewDISS • Initial Working Prototype: • No Single contractor • Exploration of self-governance • Self-organizing elements, “cluster” • DAACs become part of Federation • Exercise in “Coopetition” • Ongoing Evolution: • Technological, scientific and structural innovation • Heterogeneous but highly communicative structure leads to rapidly adaptable systems which can swiftly meet changing ESE needs • Begin Mapping to NewDISS • Foundation initiated 2. A brief history of the Federation ESIPs Types 2 and 3 initiated in March 1998 7th Federation Meeting, Grand Forks, ND July 2001 NRC meeting held at La Jolla 1995

10. Organizational Structures of the Federation • Development of Organizational Structures • Formal structures (such as Committees and Working Groups, e.g. for Interoperability and developing the information economy). • Informal structures (namely Clusters to allow members of the Federation to organize as they please) • Formal rules of governance set up in January 2000 • Not-for-profit legal structure July 2001

11. 3. The Relationship of the Federation to EOSDIS and NewDISS

12. DAACs

13. Federation Extensions to EOSDIS MODIS Vegetation Continuous Fields MODIS Land Rapid Response for US Forest Service National Interagency Fire Center HDF-EOS, Integerized Sinusoidal standard products Calibration and QA Terra Satellite Raw Data Received Re-projected, Re-processed products Remote Sensing of the Environment Observation Collection Assessment Source Data Entry Data Processing Value-added Products Models Publications Targeted Applications Type 2 Type 3 Type 1 • ESIP I’s ensure data availability and long-term archiving • ESIP 2’s add value to and ensure the scientific utility of key EOS data and products • ESIP 3’s apply EOS data for commercial and non-profit uses EOSDIS

14. Proposed NewDISS Characteristics • NewDISS can be regarded as the logical evolution progression from the Federation • Structure is heterogeneous • participants (investigators & institutions) • technological approaches (many capabilities exist and many more to be developed) • Distributed and flexible, responsive systems • Smaller, more manageable pieces • Driven by competition • Establish a framework to integrate ESIP activities. • Define a core set of interface standards and practices • Utilize community-wide interface standards

15. Evolution of NewDISS Inclusion of PI data processing in heterogeneous system processing, archive and distribution Individual researchers, no data management coordination Highly structured data system connecting various data repositories

16. Relationship to NewDISS System Wide Interoperability Layer

17. Federation as a Testbed • The Federation by definition is a broad spectrum of Earth science data centers. The Federation is a ready-made laboratory for testing NewDISS concepts and reporting on the next set of “lessons learned”: • Federation focuses on tasks that involve cross center issues that strengthen and build the Federation from within. • Each ESIP is independent - yet they work toward the common good; the whole is greater than the sum of its parts • Federation is lightweight, nimble and thus can conduct rapid prototyping: • MODIS cluster – collaboration between backbone centers and science and mission centers to improve dissemination of MODIS-based products • LBA cluster – provides assistance to LBA project in acquiring and disseminating products

18. Federation Data and Information Interoperability • A critical aspect of the evolution of the Federation focuses resources and expertise toward improving data interoperability. The Federation has adopted and produced solutions including: • Catalog-level data searching via the Internet (Using GCMD and Mercury) • Allowing distributed data exchange using novel systems (DODS) • Numerous working prototypes for data discovery and access (MOCHA, GLCF ESDI, TRFIC Landsat Portal) • The adoption of both ‘core’ (HTTP, XML, FGDC metadata) and ‘community’ (DIF, OpenGIS, Z39.50) standards for data exchange.

19. 4) How successful has the Federation been in reaching communities? • September 2000 QuarterMarch 2001 Quarter • Ratio of Downloads/Data Available 91.3 % 139.3% • Average Data Delivered per ESIP 324,150 MB 339,305 MB • Number of New Data Sets 153 204 • Institutions Using ESIP Software 537 632 • Select Metrics to Date • An average of 113.16 data sets available per ESIP • 204 data sets not available before the Federation • An average of 9.58 services available per ESIP • 65 services not available before the Federation • 1,910,591 Products delivered to users • 13.7 Terabytes data provided to users • 84,718 Services provided to users

20. How successful has the Federation been in reaching private sector, policy, and general public users? The Federation has been challenged by NASA management to broaden the user base for remote sensing data and value-added derived products. To date there are numerous examples of significant progress in reaching non-traditional user communities. COMMERCIAL DATA & INFO SUPPLIERS • The commercial market place has been a particular target for Federation partners. The Type 3 ESIPs are the Federation partners who move environmental data into new for profit market areas. RITI Marine Solutionsoffers the integration of dynamic resource (vessel) with internal/external databases, visual geographic presentation, and store-forward full length ship tracks (proprietary algorithm for effective storage). Transferring Earth science data to the public sector by Federation partners. StormCenter, for example, draws from existing and emerging remote sensing data and passes these on to numerous users. Interfaces for viewing remote sensing data are being developed for commercial partners. COMMERCIAL EDUCATION DEVELOPERS • There are over eleven Federation partners adding value to Earth science data so that new markets for these data are created. The Federation also reaches non traditional commercial interests through consulting and environmental legal services. Ocean Expeditions: El Niño is the 1998 CODIE award winner for Best Curriculum Software for Secondary/Post-Secondary Schools. The Challenger Center has now been replicated at 29 other sites in the U.S. and Canada.

21. How successful has the Federation been in reaching private sector, policy, and general public users? METRICS of REACHING NEW USERS • To assess the impacts and reach the Federation is having on the commercial and general public access and use of environmental data and services the following data is useful: • Percentage of web users from known domains reveal for the latest reporting Quarter 2001. These numbers demonstrate how Federation partners are attracting users from both commercial and the general public. • BASIC 20% Commercial users • SnowSIP 48% General Public access • Passive Microwave 49% General Public access • Ocean ESIP 44% Commercial users Success stories from non-traditional user communities. The Federation collects ‘nuggets’ from our partners that address particular data and information questions. The following demonstrate the number of entries in a few relevant categories since September 2000. • Broadening the Earth science user community to include non-traditional users. 86 entries • Advancing Earth science data and information to the commercial market. 37 entries • PR events (e.g. Presentation by ESIP staff at conferences or community events). 35 entries Thanks from a John Q. Public citizen for real time global satellite data, Passive Microwave ESIP, Third Quarter 2000 “Thank you so very much for making this type of information so accessible to the public. It's just so wonderful to be able to see the patterns on the "whole" globe in real time.”

22. Distribution of data and service users by domain, First Quarter 2001 Commercial: 11%  14% Education: 23%  34% August 2000 to March 2001

23. Federation Services and Applications Ranching K-12 Instruction Energy Precision Farming Urban Planning Application Providers Commercial Fishing Products and Services Providers Data and Information Providers The ESIP Federation Land Management Weather Science-driven Processing

24. The Federation Aids Many User Communities • Development of commercially viable products such as diverse as Palm applications to data brokering. • Provision of products, services and technologies to critical elements of our nation’s economy and infrastructure: • Farming: BASIC’s Vineyard Infestation Defense Map • Ranching: EDAC’s Rangeland Toolkit • Commercial Fishing: Scientific Fishery Systems, Inc.’s FishTrek • Weather: RITI’s WeatheRoute GIS Software • Energy: SnowSIP’s Hydroelectric Power Flow Estimates • Media: Stormcenter.com’s Meteorological Information for the Media

25. UMD, GSFC and USFS • Fire Rapid Response • Rapid response provision of Active Fires and Reflectance(1 to 6 hrs) • Operational prototyping of standard MODIS L2+ products. • Automatic system developed since February 2001 • Daily Reflectance and Active Fire mapped to standard NIFC regions • Builds upon IMAPP software developed by Univ. of Wisconsin • Transfer of Rapid Response software to USFS • Remote Sensing Applications Center (Salt Lake City) • Fire Science Lab (Missoula) • 2001 bridging activity to provide MODIS products while DB stations are built • Enhanced VCC Burn Severity and Smoke / Aerosol products VCC Burned Area, ID/MT 25km

26. MODIS FIRE PRODUCTS Sicily, July 23, 2001 500m Northern Australia, July 29, 2001

27. Fires in Eastern Siberia, a.m. July 30th 2001 Fires

28. 5) Plans for evolving the Federation • Widening the Federation: several new ESIPs have already joined. • Focussing on the latest EOS Data, e.g. MODIS Cluster. • http://www.modis-cluster.org • Improving interoperability - focussing on the data level rather than directory and catalog levels. • Developing strategic alliances outside of the Federation e.g rapid fire response, links to international programs, e.g TRFIC with GOFC and START. • Setting up a parallel Foundation: a legal entity so that funds can be received by the entity rather than individual ESIPS.

29. New sources of funding • Note: all ESIPs leveraged off other funds; ESIP 3s only 50% funded; cost shares in ESIP 2s. • Some ESIPs are developing using private capital • Inclusion of NCDC means that NOAA funds are now included. • No figures directly available but all ESIPs are seeking extra funding by multiple routes: other NASA sources, other agencies (e.g. USFS), not-for-profits (e.g. WRI). • But without some core funding the future of the Federation will be difficult and may disappear. • Funds for DISS are difficult to obtain.

30. Current Senate Mark Up 1998 1999 2000 2001 2002

31. Current Senate Mark-up University Pork Algorithm Dev. Federation “Synergy” ECS ECS Increase

32. Explore the ESIP Federation www.esipfed.org www.esipfed.net www.modis-cluster.org

33. Extra slides

34. Bay Area Shared Information Consortium: Vineyard Infestation Defense Map • Infestation mitigation through species monitoring • Monitoring of glassy-winged sharpshooter • Mapping of vineyards and other potential habitats • A partnership between BASIC, county government, academia and Central Coast Pierce’s Disease Task Force The initial study area

35. Earth Data Analysis Center:Rangeland Toolkit • A comprehensive rangeland management toolkit, integrating: • Satellite Data • Landscape attributes • Socioeconomic data • Data, metadata and previews are available for download • A testbed for value-added products based on EOSDIS data and technologies Examples of data available through the Toolkit: Bareness (top) and drainage (below)

36. Scientific Fishery Systems, Inc.:FishTrek • A comprehensive software package for mapping fisheries • Integrates bathymetric maps, GPS and automated catch logging and mapping • Inputs largely taken from NOAA ship sounds • Vector-based datasets increases usability for the novice • Fully commercialized product FishTrek Commercial Fishing Software

37. Reading Information Technology Inc.:WeatheRoute GIS Software • An Electronic Navigation system (ENC) designed to help mariners avoid heavy weather at sea • Based on NOAA Significant Sea Height forecasts • Provides near real-time forecasts to captains: • Up to 5 days in advance • Improves safety and efficiency • Increases fuel efficiency • Decreases hull stress A value-added GIS with full navigation capability

38. SnowSIP: Hydroelectric Power flow Estimates • Population increase and agriculture make water resource management critical, especially for the State of California • SNOWSIP has calculated hydroelectric power flow estimates based on snowmelt • Additional measurements: • Daily/diurnal variations in radiative inputs to snowpak • Effect of climate variability on snowmelt The use of remotely sensed satellite data allows for the accurate (97%) modeling of snowpack and the corresponding volume of runoff

39. Stormcenter.com:Meteorological Information for the Media • A subscription service for news stations, nationwide • Current issues in weather and nature addressed with satellite and in-situ imagery: • Visualization of breaking environmental news • Broadcast graphics print, web and broadcast-ready • Local to national-scale imagery • Automatic notification for subscribers Examples of services available through Stormcenter.com

40. Upper Midwest Aerospace Consortium: Crop and Range Alert System • Assessment of forage quantity and quality by tracking rangeland biomass and nitrogen • AVHRR-NDVI used as primary indicator • Near real-time • Provides a more synoptic view than traditional range assessment methods Anomalies reported by the alert system

41. Planet Earth Science, Inc.:Ocean Expeditions: El Niño • Students become the crew on a virtual ship exploring the World’s Oceans • Using virtual reality, students participate in an authentic science project • Real-time satellite data is used to monitor interactions between ocean temperature, currents,winds and atmospheric pressure One of the resources available to students: a Sea Surface Temperature Anomaly Map

42. Global Land Cover Facility:Provides Imagery for Disney’s Animal Planet • What makes image brokering attractive to the public sector?: • Speed of delivery (FTP) • Cost of product (below market) • Custom formats, projections • Science and technology support • Animal Planet required a scientifically valid means to map the habitat of the Golden Tamarin but had limited funds • Through a cooperative arrangement imagery was expeditiously provided The Golden Tamarin Habitat as imaged by Landsat 7

43. MODIS FIRE PRODUCTS Sicily, July 25, 2001 500m Northern Australia, July 26, 2001

44. Relationship to EOSDIS: Lessons Learned • Information technology outpaces the time required to build large, operational data systems and services • Data systems and services should leverage off emerging information technology; and not try to drive it • A single data system should not attempt to be all things to all users • A single, large design and development effort stifles creativity • Future information systems will be distributed and heterogeneous in nature

45. Distribution of Data and Service Users by Domain, Third Quarter 2000