1 / 40

SENSOR TO USER Progress Report - 2 nd Year (Q2-Q4) Prepared by

SENSOR TO USER Progress Report - 2 nd Year (Q2-Q4) Prepared by Dr. Erick Malaret, ACT Corp. (703-742-094) Email: malaret@actgate.com. Description and Objectives

tobit
Download Presentation

SENSOR TO USER Progress Report - 2 nd Year (Q2-Q4) Prepared by

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. SENSOR TO USER Progress Report - 2nd Year (Q2-Q4) Prepared by Dr. Erick Malaret, ACT Corp. (703-742-094) Email: malaret@actgate.com

  2. Description and Objectives Applied Coherent Technology Corporation (ACT), the Naval Research Laboratory (NRLSSC) Oceanographic Division, and our NOAA partners, are under a cooperative agreement with NASA to maximize the value of NASA/EOS Earth science observations to decision makers related to coastal, marine, and ocean environments. ACT and NRLSSC are integrating real-time ocean systematic measurements from NASA and NOAA satellites and available coastal observations along with coastal ocean model output into an automated real-time database of ocean weather in the Gulf of Mexico. This project focuses on monitoring coastal ocean properties and it is aimed at providing new real-time support to NOAA efforts initially in the following focus areas: Harmful Algal Blooms (HABs), Coastal Flooding /freshwater discharge, and Hypoxia. ACT in collaboration with NOAA will expand the project to other U.S. coastal areas and add additional NASA,NOAA,DOD and foreign data sources. Note: Big Scope of Cooperative Agreement Scope and participants remains unchanged Initial Implementation Approach The on-going effort is divided into an application component and a and product research/development component. • The Application Component [by ACT] – provides an architecture that addresses issues related to data ingestion, processing, distribution, access, and decision support. The key Decision Support Server and Fusion Server will be installed at NRL/SSC, followed by other servers at different NOAA sites. • The Product Research/Development component[by NRLSSC] – provides state of the art oceanographic expertise and research addressing issues related to generation of new geophysical data products as part of a production pipe-line. Schedule and Key Deliverables Year 1: Application Component – a) Req. Gathering, b) Install high end HW/SW to ingest/process/serve data, c) enhance current Fusion server & Decision Support Tool Technology. Product R&D Component - …. Year 2: Application Component – a) Demonstrate widely available internet access to all data levels, b) develop initial Hazardous Algal Bloom product based on satellite/model data in GIS and other compatible formats, c) add new data streams (e.g. NPP VIIRS) Product R&D Component - …. Year 3-5: Application Component – a) Expand to other application areas, geographic areas, and data sources, b) identify and customize architecture to provide additional decision support tools, assist in transition of application to operation. Product R&D Component - …. 2nd year Started May 2005

  3. The next slides will concentrate on Application Component progress in the last 7 months De

  4. HAB Test-bed for NOAA: Present Network Configuration Fusion Server atNAVY/NRL-SSC NOOA Fusion Servers at ACT • NOAA/CW • MODIS data • NOAA/NDBC • Buoy data • FNMOC • Many models • OTHER • +in-situ obs. • +satellites Obs. … • NRL-SSC • MODIS OBS • MODELS • MODIS VAPs WIPE(servers) WIPE(server) REACT SOAP WSDL HTTP/ CGI/XML HTTP/ CGI Data providers(user Type 1) FTP & HTML xfers Local +Satellite Imagery +MODELS + … REACT (Field Analyst) NOAA/CW NOAA/NDBC NOAA/NCEP optional access directly by REACT Web Browsers (HTML and Applet Access) REACT (NOS User) REACT(CSC User) SOAP WSDL Final HAB Bulletin For Decision Maker (user Type 3) HAB bulleting creation Wizard NOAA/NOS/CSC Analyst Team (user Type 2)

  5. Apply WIPE/REACT To HAB’s needs by NOAA

  6. DATA INGESTION PROGRESS

  7. Ingestion Data Aspects and Engines (continue) NRL Model Data Model data ingestion engines were enhanced in REACT to rapidly open any of the NRL volumetric NetCDF files. The files can be open via the REACT interface. Visualization tools were added to REACT to simplify the inspection of volumetric model data. Layer animation File animation Data inspection with row/col/depth proves Interactive Intensity Mapping NRL Level 2-4 HDF Products Integrated the ability to read any of the sub-datasets included in the NRL HDF files. The REACT interface provides a hierarchical file layout over the file system and the HDF file User can load desired set of sub-dataset layers simultaneously, and remap into desired cartographic projection Visualization capabilities similar to the model data were added. Done: multiplicative and additive scaling constants are applied. NOAA Coast Watch After Meeting with Coast Watch it was clear that ALL NOAA/CoastWatch products are being saved in CW HDF format. Added capabilities to read/load all the sample data provided by CW to ACT NOS Needed Data Incorporated the Buoys data and filtering algorithms In-situ observations can not be accessed automatically at this time DATA INGESTION PROGRESS (continue)

  8. Data Ingestion and Status of WIPE Server(s) NRL/SSC The WIPE server at NRLSSC has been working since October 05. NRL/SSC has given security approval to the machine, hence it can be access over the internet as a web server. The web page is: http://cesium.nrlssc.navy.mil Met with NRL/SSC at ACT, and identified automatic process to populate the WIPE server at NRL/SSC with both MODIS L2-L4 products and with MODEL data. The advantage of the approach taken is that it does not replicate in any way Hark Disk storage at NRL/SSC. It uses the same hard disk area that their DODS server is using. Training on WIPE was done in November ACT is presently incorporating the suggestion received from NRL Basemap MODIS COMPOSITE NCOM TEMP

  9. Data Ingestion and Status of WIPE Server(s) (continue) NOAA CW Servers at ACT: Working. Tested with Sample CW-HDF format data. All modes working, except the Swath data sets. Bow-tie projection problems in the MODIS L1B data. To save computational time we were not using all the ground control points in the MODIS data. This resulted in visible bow-tie projection problems that were not being handled correctly. Investigated used of third party SW  decided against it after discussing with NRL/SSC that all that is needed is to use ALL the GCP information. Made significant modifications to the processing of Ground Control Point to generate a properly corrected geo-referenced image. The new approach is more computationally intensive, hence it takes longer to generate a full area mosaic at full resolution. 5-10 minutes.

  10. Status of HAB WIPE Server at ACT Presently working on getting all the data automatically pushed into the WIPE server at ACT. Some of the cell count data is at FTP sites. Presently embedding at FTP library into WIPE to simplify process of getting data from remote servers via FTP. Data Ingestion and Status of WIPE Server(s) (continue)

  11. SOFTWARE DEVELOPMENT PROGRESS

  12. CORE SW Development Progress • MSHELL - Image Processing Interpreter Language developed by ACT. The language has a high level syntax similar to Matlab and IDL. • ProVIEW - Professional Windows Environment for Windows (ProVIEW). ProVIEW has extensive capabilities in image processing, analysis, and map generation. • WIPE – WWW Information Processing Environment. Data Assimilation, Processing, and distribution system with batch and interactive capabilities. Rapid Environmental Assessment Composition Tools (REACT) has been implemented and continues under rapid development. It captures elements of MSHELL+ProVIEW+WIPE all under an easy to deploy stand alone application, and portable GUI environment. During 2nd-3rd quarter continue expanding the tool space in REACT in response to needs of the HAB needs MSHELL Provides the Computational Engine Used by ProVIEW & WIPE ProVIEW Provides Many of the Graphical Capabilities Used Under WIPE WIPE’s building blocks have been used in multiple mission critical operations

  13. As part of our NASA Cooperative Agreement, ACT has been participating in the Technology Infusion working group. The Technology Infusion has identified many capability blocks required in systems to support the EOS needs. REACT inherits key components of the ACT’s SW remote sensing SW tools MSHELL - interpreter language with extensive capabilities for multidimensional data manipulation ProVIEW - graphical user interface around MSHELL WIPE - provides fusion center architecture, with over 200 streams ACT has been using REACT and WIPE as a platform/architecture to demonstrate technology infusion from the OpenSource community, i.e. Qt – provides a portable library in C++ that allows GUI and system level support (A prototype version of REACT has been implemented for LINUX and Windows.) GDAL – the Geographic Data Abstraction Library is used by REACT to perform raster IO over many geo-graphic data formats new formats can be easily added following an OpenSource approach empowering developers to rapidly contribute in the adaptation of new formats. GDAL has been extended tosupport : CW-HDF, NRL-HDF, PCI-AUX (extended SRS support) gSOAP – used to SOAP enable REACT  the developer needs to primarily concentrate on the core subroutine functionality. CONTINUED EXPANDING THE SOAP INTERFACE PROJ4 – a cartographic library with support to most of the projections used by USGS SQLITE – a built-in SQL engine OpenGIS/OGC – describes protocols/methods used for the request of products (This is work in progress) REACT MSHELL (Interpreter+ scripts!) GDAL + PROJ4 + GSOAP+… CORE SW Development Progress (continue)

  14. REACT is been configured to provide Web Services for geo-spatial data Web Services Requirements are: Available over the Internet Not tied to any operating system or programming language Self-describing via a common XML grammar Discoverable via a simple find mechanism REACT captures a WIPE like type of interface for accessing inventory queries and generating mosaics, a set of five core functions are being added to the REACT/MSHELL scripting language resulting in a LARGE space of remote computing capabilities/options! (implemented using SOAP) $result = rmshell_execute($server, $scriptCommandList ) Data = rmshell_getVar( $server, $variableName) $result = rmshell_putVar( $server, $variableName) $result = rmshell_getFile( $server, $inFile, $outFile ) $result = rmshell_putFile( $server, $inFile, $outFile) ADDITIONAL FUNCTIONS WERE ADDED TO HANDLE ASYNCHRONOUS REQUESTS The above functionality can be exported to other applications via WSDL (an XML grammar for specifying public interface to the web service) GAVE A PRESENTATION TO THE ESIP FEDERATION MEETING IN SAN DIEGO, ON WEB SERVICES USING WIPE&REACT. CORE SW Development Progress (continue)

  15. CORE SW Development Progress (continue) Client side (Field System) Server side (Field or Fusion Center System) REACT REACT Script/algorithm or data result or data WIPE REACT Script/algorithm or data result or data WIPE Or REACT Web Enabled Application Script/algorithm or data result or data WIPE and REACT share the same processing engine REACT is configured to handle local user and a small number of remote applications. REACT can support remote applications as a WIPE-LITE system

  16. REACT can be seen as a WIPE-LITE system with more flexible visualization capabilities, and network centric capabilities. A prototype windows version of REACT can be downloaded from ACT, i.e. http://www.actgate.com/reason CORE SW Development Progress (continue)

  17. DECISION SUPPORT TOOLS PROGRESS

  18. Decision Support Tools means different things to different users, e.g. Data Provider  need coverage and cartographic maps/mosaics with simple to use query engines Analyst at Fusion Center Visualization Tools, e.g animations of layers, volumetric vis. Implementation of a Process in the form of a Wizard Analysis: Multivariate Detection/Estimation/Prediction/… High Level Decision Maker, e.g. Coastal Manager GO/NoGo results with access to supporting data ACT has been working during the last year in the above end-user categories with requirements driven by the needs of: NRL, NOAA/CW NOAA/NOS NOAA/CSC NOAA’s end-users DECISION SUPPORT TOOLS …. The Challenge

  19. NOAA/NOS, NOAA/CSC, NASA/SSC, NOAA/NCDDC, and ACT met in February-05 to discuss the HAB Bulleting creation process and future architecture developments. As the result of the meeting a number of concrete suggestions were provided by NOS and CSC. (see following slide). DECISION SUPPORT TOOLS PROGRESS (continue)

  20. Development Operations Delivery • [done]Use WIPE for • new data stream • ingestion • [done]Easy Incorporation of other geographies • [done]Mosaic Images • [done]Alternate look from same data => flicker • [done]View data values from multi-sensors at same time • Skill assessment match in-situ data and imagery • [done] Output data • [done] Oriented geo-tiff • Requirements • [done] Save view settings • [done] Archive data used in view • [done] Access from multiple sites • [done] Use of consistent color tables (use same tables as CW) DECISION SUPPORT TOOLS PROGRESS (continue) • Outcome: NOAA/NOS/CSC requested ACT to demonstrate how WIPE/REACT could be used reproduce current HAB Bulleting • creation process in REACT/WIPEPossible Advantages: • Proposed approach could provide clear view of process and problems • Need web centric creation process (can be a browser or application) • Explore REACT/WIPE as next generation environment

  21. HTML output + PDF output

  22. SPEED Data Cube Generated on the Fly from U&V components

  23. EXAMPLE OF READING AND DISPLAYING: generic formats, NRL’s MODEL DATA, MODIS L1B, CW Products, …

  24. EXAMPLE OF READING AND DISPLAYING NRL’s MODEL DATA DATA Probes On Active Image

  25. EXAMPLE OF READING AND DISPLAYING NRL’s MODEL DATA Easy Access to Any Layer, Including Volume Animation Metadata information

  26. EXAMPLE OF NRL L2-L4 File System Navigation and HDF Data Structure Navigation

  27. Meetings/Training/Demos Supported Different ESIP Federations Meetings Gave Training on WIPE at NRL (2 days) Explore Training on REACT (over the WEB) For Example, Gave Presentation to CSC on REACT using VNC

  28. ACT’s REASON Web page Is up an running • http://www.actgate.com/reason • Provides: • Report Summaries • Actual Data Access • DST Demos Products • Tutorials • … • …LIVE DEMO …

  29. NRL Related Work over WIPE enhancements/customization requested by NRL-SSC Custom product generation Virtual products on the fly… Legend … Further Training to NRL on Use of WIPE/REACT NOAA Related Fine-tune HAB Wizard based on inputs from NOAA/NOS/CSC/… Provide Training as needed to NOS and CSC ESIP Federation Related Demonstrate Web Services Capabilities of WIPE/REACT (SOAP/WSDL) Improve interfaces with OGC and OpenDap Demonstrate with ESIP Federation Web Services Chaining (using WIPE as a prototype) Use REACT as a test case for Technology Infusion Process being Developed by the SEEDS Working Group on Technology Infusion Investigate interface WIPE within NOAA/CLASS Architecture Start Investigation Next Year Target Application Areas Based on Team Recommendations Next Quarter Main Effort …

  30. Summary All ACT WIPE/REACT resources are accessible via ACT’s reason web page, i.e. http://www.actgate.com/reason On-line demos are being constructed to simplify end-user training NRLSSC WIPE Server is actively receiving NRL data streams NOAA can access all NRL data directly from WIPE/Applet interface or from WIPE/REACT interface! (over 1,200 different fields populated daily) REACT has matured to a state that NOAA/NOS/CSC can: Test its use for HAB analysis tools Test its use for HAB bulleting creation ACT WILL WORK WITH THEM IN SIMPLIFYING USAGE OF TOOL WIPE Axon Architecture for Decision Support Tools is working Need testing and use …

  31. Possible Items for Discussion Are we heading (REACTing) in the right direction? How can we help make the NRL products easier to evaluate by NOAA? How can we make the emerging architecture more useful to NOAA? How can we make the large volume of unique data more useful to a larger audience? Can the data in the WIPE servers be accessed by non-team members? The goal is to raise awareness of the many products available and the web-services infrastructure Can we interface WIPE better with NOAA’s CLASS architecture? …NRL and NOAA suggestions go here …

More Related