Global Imagery Browse Services (GIBS): Project Management Review

1. Global Imagery Browse Services (GIBS):Project Management Review Charles Thompson Matthew Cechini Tuesday, September 10, 2013

2. Outline • Introduction • Project Overview • Description • Organization Chart • Objectives • Integration with EOSDIS • Use Cases • Applications • Development Details and Status • TIE and OnEarth • Introduction • Workflows • Current status • Open source status • Current cost and labor summaries (JPL) • Project Schedule Review • Evolution • Completed Activities • Planned Activities • Transition Planning • Issues and Risks • Questions • Worldview demo

3. Project Overview

4. What is GIBS? • Value-added image archive and access services for NASA Earth Science data products • Collaboration between JPL and GSFC • JPL: server-side software • Image management and workflow: The Image Exchange (TIE) • Image processing and image access: OnEarth • GSFC: hardware and infrastructure • Configuration • System administration • Testing • Operation and maintenance • Both • Project management • System engineering

5. GIBS Organization Chart

6. GIBS Objectives: Vision • Influence the EOSDIS data search and discovery process • Transform • Streamline • Enhance • Make it visual

7. GIBS Objectives: Drivers • Necessity • Rapid expansion of Earth Science data holdings • Current search & discovery paradigm does not scale well • Need to incorporate data reduction/characterization/summary techniques, especially in data-as-a-service (DaaS) endpoints • User feedback • Need common form of data representation • Web services

8. GIBS Objectives: Benefits • Intuitive interface to science data • Minimal user intimidation • Extended customer base • Wide array of applications • Cohesive approach to EOSDIS image management • Facilitates cross-discipline research • Publicity

9. EOSDIS Components:Where GIBS fits in Web Applications (Reverb, Worldview) Web Portals (Standard and Mobile) Web Services Web Infrastructure Collection and Inventory Metadata ECHO, GCMD User Registration EOSDIS URS Global Image Browse Services GIBS Content Management Drupal, CMSs User Support System Kayako Service Metadata Metrics EMS Content Discovery Google Bug Issue Tracking Jira Collaboration Tools Doc. Mgt., Wiki Code Repository GIT, Stash Direct Data Access HTTP, FTP Data as a Service (DaaS) OPeNDAP, WCS, Live Access Server Ingest and Archive Systems ECS, DMAS, S4PA… EDOS, MEaSUREs, SIPS, External Data

10. GIBS: Core Use Scenarios Metadata First Imagery First ESDSWG Visualization Working Group Recommendations October 2013

11. Environmental Forecasting • EPA’s AirNow-Tech Navigator • Air Quality Data • Surface Weather • Satellite Data AOD 0.70 Smoke 24-hr average PM2.5 – AirNow WMS HYSPLIT Trajectory – NOAA ARL Terra True Color – NASA GIBS WMS 24-hr average PM2.5 – AirNow WMS HYSPLIT Trajectory – NOAA ARL Terra & Aqua AOD – NASA GIBS WMS 0.27

12. GIBS Applications:Mission Planning

13. GIBS Applications:Education • Science on a Sphere • PBS NOVA Labs – Cloud Lab • American Museum of Natural History

14. Worldview & SOTO • Worldview • Reference client for GIBS • Browser-based, developed using open standards • Currently tailored toward NRT data community • Supports polar viewing, full-res image download • State of the Ocean (SOTO) • Enables interactive viewing of ocean parameters • Includes external contextual overlays (buoys, currents) • Moving 30-day viewing period including NRT data • Raster and vector overlays • SOTO 3D: Google Earth client • SOTO 2D: OpenLayers client, under development • Demonstrations of Worldview and SOTO 2D Wednesday 10-11 here!

15. Development Details and Status

16. GIBS Software Constituents

17. OnEarth • Image transformation and serving software • Emphasis on extremelyefficient (fast)image serving capabilities • Keys off a specialized image pyramid encoded into the Meta Raster Format (MRF) • Consists of Layer Generation and Layer Access components • Developed at JPL by Lucian Plesea

18. The Image Exchange (TIE) • Image management and workflow package • Based upon PO.DAAC’s Data Management & Archive Software (DMAS) • Ingests and archives provider imagery and associated “image metadata” • Triggers and monitors OnEarthprocessing

19. TIE Workflow Overview Imagery Data Handler Imagery Internet Image provider Input Image Extract metadata Validate SIP Inventory Register Layer Generation MRF Image yes Check for new files Archive Copy Data Subscriber Manager & Ingest Engine Layer Access Legend OnEarth TIE Internet Image user

20. OnEarth Workflow Overview

21. OnEarth: Current Work • Understanding internal processes and roles of each routine • Documentation and diagramming • Code reorganization • Merging multiple instances of source code • Establishing formal version control • Developing TIE interface specifications • Purpose: Setting the groundwork for software development phase

22. TIE: Current Work • Extending and enhancing DMAS for GIBS • Building data handlers for imagery • Solidifying interfaces with OnEarth • Establishing some level of end-to-end throughput

23. Open Sourcing GIBS Software • Why? • Facilitate continued relationship with original OnEarth developer • Widen possible collaboration possibilities • Promotes open standards • Cost effective • Status • OnEarth • License type and repository determined, all required signatures gathered, now awaiting JPL authorization process to play out • TIE • Will follow OnEarth process after it has been smoothed out

24. GIBS (JPL): Planned vs. Actual Cost $K OnEarth Software Engineer hired Month

25. GIBS (JPL): Planned vs. Actual Labor FTE OnEarth Software Engineer hired Month

26. Project Schedule Review

27. GIBS Evolution 2010 • Infancy • Developed as Rapid Response extension, fast access to NRT imagery • Demonstrated overlay utility (MODIS True Color & AMSR-E Rain Rate) • JPL Involvement • State of the Earth (SOTE) pilot project testing image serving software • White paper recommending JPL software as best option for serving imagery • Operating Prototype • Standardized (OGC WMTS/TWMS/KML) service endpoints • Access to ~100 imagery products • Multiple clients: Worldview, World Wind, EPA, iPhone App, SOTO • Beginning of Formal Development • ConOps, design documents, schedule, v0.2 “Operationalization” • V1.0 release • Improved Reliability, Fault Tolerance, Scalability, and Performance • Improved Governance Processes • Increased Ingest of new Products 2011 2012 2013 2014

28. Completed Activities • Systems Engineering • System Requirements • “Web Layer” Trade Study • System Architecture • Concept of Operations (Public Draft) • Development • Software Development Plan • TIE Software Detailed Design • OnEarth Software Detailed Design (Draft)

29. Completed Activities (cont) • Infrastructure • SAN Initial Configuration • Development Environment Installation • Procurement for Dev/Test/UAT/Prod Hardware • Testing • Performance Test Plan (Draft)

30. Planned Activities 1/14 9/9 10/11 12/13 Dev Sprint v0.2a Dev Sprint v0.2b Test Support & Bug Fixes Bug Fixes Dev v0.2b Functional/Performance Testing User Acceptance Testing v0.2a Functional Testing Test “Go Live” Concurrent Operations Ops Network, Storage, Backup, Database Testing/Implementation

31. Transition Planning • Concurrent Operations w/ “Old GIBS” • Reduced imagery migration • Increased period of performance and stability testing • Additional enhancements to address functional issues or improvements (TBD) • “Seemless” Cut-Over • Ensure imagery synchronization • Redirect web traffic to new endpoints • Shut down ingest into “Old GIBS”

32. Issues

33. Risks: Open

34. Risks: Closed

35. Questions

36. Worldview Demo