1 / 31

An Extensible System for Tracking Custom-Defined Atmospheric Events Final Review

An Extensible System for Tracking Custom-Defined Atmospheric Events Final Review. Thomas Clune(PI) Kwo-Sen Kuo(Co-I) Yang Hong(Co-I) AIST-QRS-11-0002 Final Review January 24, 2012. Presentation Outline. Cover page Quad Chart (updates, TRL current) Team Members Matrix Presentation outline

carina
Download Presentation

An Extensible System for Tracking Custom-Defined Atmospheric Events Final Review

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. An Extensible System for Tracking Custom-Defined Atmospheric EventsFinal Review Thomas Clune(PI) Kwo-Sen Kuo(Co-I) Yang Hong(Co-I) AIST-QRS-11-0002 Final Review January 24, 2012

  2. Presentation Outline • Cover page • Quad Chart (updates, TRL current) • Team Members Matrix • Presentation outline • Optional objectives, background • Primary technical findings for this review period • Accomplishments (this report period) • Schedule chart (full project period) • Costs chart (full project period) • Cost issues/recovery plans (if any) • Publications (if any; upload to e-Books) • Acronyms

  3. Automated Tracking of Earth Science Phenomena for Ingest into a Moving Object Database to Enable Systematic Studies PI: Thomas Clune, NASA GSFC Objective • Develop tracking algorithm to automate identification and characterization of spatially distributed coherent events • Provide output suitable for ingest into the ESTO funded Moving Object Database (MODB) • Demonstrate capability by identification of tornado producing mesoscale convective systems (MCS’s) • Synthesize data from many sources including 1-km 5-min quantitative precipitation data, 4-km ½-hourly GOES-East Rapid Scan Operation imagery x t Identification of coherent events my merging overlapping threshold regions Intensification merging Dissipation splitting Approach Key Milestones • Develop capability to uniquely associate spatially discontiguous events • Establish characterization parameters which provide useful description of MCS events and are suitable for ingest by the MODB. • Optimize identification criteria for MCS to allow automated tracking of MCS events • Integrate identification, characterization and tracking capabilities • Demonstrate capability by applying integrated system to multiple years of high resolution meteorological data • Implement tracking algorithm for spatially discontiguous events • Sequential implementation 07/11 • Parallelize across time slices 08/11 • Establish characterization parameters suitable for MODB ingest 08/11 • Tune MCS identification criteria 10/11 • Integrate tracking, characterization and identification 11/11 • Demonstrate automated system on MCS data 12/11 fill Co-Is/Partners Co-I Kwo-SenKuo, NASA GSFC; Partner Yang Hong, Ohio State TRLin=4 TRLfinal=5 06/09

  4. Team Members

  5. BackGround • A moving-object database, MODB, is a database with spatiotemporal capabilities, which are often extensions to stock relational database management systems. • An Earth Science (ES) event is an episode of a geophysical phenomenon, such as a hurricane, a tornado, or a thunderstorm shower. • A mesoscale convective system, MCS, is an organized ensemble of convective elements, whose lifecycle is longer than that of the individual convective elements.

  6. Background (1/2) • Earth Science MODBs facilitate easier discoveries of relevant data associated with ES phenomena and enable systematic studies of these phenomena. • Populating the databases, however, is currently largely manual and time-/labor-intensive. • Motivation: Automatic population of MODBs with Earth Science events. • Using MCS as an example, we have developed a system to automatically • Identify and track tornado-producing MCSs using • surface precipitation rate (R)and • thermal infrared brightness temperature (TB) • Extract MCS spatiotemporal parameters for populating MODB • Obtain MCS morphological parameters to enhance MODB capabilities

  7. Background (2/2) We use tornado-producing MCSs to demonstrate the capabilities of the system. The system is easily configurable to apply to other phenomena. • Why Mesoscale Convective System (MCS)? • MCSs produce a broad range of severe convective weather events. • MCSs account for about half of the annual warm-season rainfall over the Great Plains. • Understanding MCSs will improve weather prediction models and other tools to protect populations and their properties.

  8. Design PrinciplesLots of -abilities • Portability • Extensibility • Customizability • Configurability • Usability • Testability • Reliability • …

  9. Primary Findings • For now everything is under “Accomplishments”

  10. Accomplishments • Implemented Connected Component Labeling to identify events within 4D data • Extended to be parallel on distributed cluster • Implemented flexible processing system • User-specified event criteria (e.g. TB < 240K) • User-specified metrics (e.g. area, min TB, …) • User-specified filters (e.g. exclude non-tornadic events) • Results collected in DB40 (Database for Objects) • Created flexible serializer to export data to other tools (e.g. IDL, Excel) • Implemented IDL utility to create animations of MCS pulled from system output • Processed 5 years of GOES data • All MCS and tornadic MCS • Processed ???? Years of NMQ data • All MCS and tornadic MCS

  11. Implementation (1/4)Process Flow Go though each time step and determine event track Identify event component data points, e.g.TB < 240 K Driver loads required modules based on a configuration file and initiates the process. Identify event component with criteria, such as minimum area coverage required Filter event tracks based on additional criteria, e.g. tracks containing tornado(s)

  12. Implementation (2/4) • Test-Driven Development (TDD) method • Java-based for cross-platform compatibility • Multi-threaded parallelization • Controllable with simple configuration files • Pluggable components for customization • Identifiers • Apply grid-cell/pixel level identification criteria (e.g. TB and/or precip. thresholds) • Specify identification criteria at the component level (area, position, etc.) • Trackers • Determine event component relationships to establish event tracks • Track Identifiers • Specify event track identification criteria (e.g. only include tornadic events, only include events from specific time or place, etc.) • Serializers • Customize output generation

  13. Implementation (3/4) • Use Connected Component Labeling (CCL) algorithm for event identification • Accept script-based component modules • Allows for experimentation without recompiling the system code. • Flexibility and customization • Groovy now, Jython planned • Object database (DB4O Java API) • Stores events as objects • Allows for object-based queries using complex criterion (such as spatial/temporal queries) • Queries are programmable/scriptable • Groovy now, Jython planned • Previous results can be used as input for further refinement • Example: A more stringent new threshold of TB <235 K may be applied to previous results obtained using TB < 240 K without going through the voluminous input data again. • “Previous results” can be either in files or from database

  14. Implementation (4/4)Example Configuration File #Basic settings [General Tracker Configuration] #Specifies the "name" of the run. scenario.name = GOES Test #Specifies a description of the run. (optional) scenario.description = Test scenario for finding events within GOES IR data. #Specifies the identifier class to use for event identification. identifier.class = nasa.goes.GOESMCSTornadicEventIdentifier #Specifies the tracker class to use to track events. tracker.class = nasa.tracker.ccl.trackers.CCLEventTracker2DST #Specifies the serializer class to use for outputting event tracks. (optional) serializer.class = nasa.goes.GOESEventImageSerializer #Output location for output generation. This is a top level property that can be overridden by implementation specific properties. output = output/test #Enables overwriting. When set to true, the event tracker will overwrite an existing file/DB entry with the same nameoverwrite = true#Database Configuration …

  15. Data • NMQ – National Mosaic & Multi-Sensor QPE • NMQ Q2: Surface rain rate, 1-km spatial, 5-min temporal • Continuous coverage in time • http://www.nssl.noaa.gov/projects/q2/ • Geostationary Operational Environmental Satellite (GOES) Imager • Rapid Scan Operations (RSO) • 10.8 μm Brightness Temperature, 4-km spatial, irregular (~15-min) temporal • Episodic coverage in time; data dropouts • http://www.class.noaa.gov • Archived NWS Warnings/Watches Database • GIS shapefile data • http://mesonet.agron.iastate.edu/request/gis/watchwarn.phtml • Tornado History Project • CSV data, tornado geo-locations, no time information except for initiation • http://www.tornadohistoryproject.com

  16. VisualizationGOES TB (10 May 2010 14:40 UTC) Brightness Temperature (10.8 μm)

  17. All MCS Events GOES TB < 240K Tornadic MCS Events

  18. Aggregated GOES data (1 of 2)

  19. Aggregated GOES data (2 of 2)

  20. Mesoscale Atmos. Processes Lab Meeting Event QueryExample GOES Results

  21. Remaining Issues • Handling missing data • Missing frames • Missing tiles

  22. Schedule (for duration of project) Complete Complete Not necessary to add required ESTO reporting as we know these!

  23. Optional Schedule issues, recovery plans • Describe schedule impact and • Recovery plan

  24. Costs (for duration of project) Planned vs Actual • Deviation: 22K under-run to date (PI has not charged to plan) • Impact: None to technical progress • Mitigation: Increase staff/org support in FY10 Optional cost issues and recovery plans as needed added here or on subsequent chart as needed

  25. Obligations This chart is required ONLY for projects with NASA & JPL PI’s or Co-I’s (per requirements of ESTO financial analysts). • Same as Cost -- $22K deviation

  26. Workforce • Deviation: 0.5 WM cumulative • Impact: None • Mitigation: None; team is appropriately staffed

  27. Publications, include title, authors, date, where presented and/or published • Presentation at 2011 AGU • “Automated Tracking of Tornado-Producing Mesoscale Convective Systems in the United States” (K. Kuo et al) • Accepted abstract for 2012 IGARSS • “Leveraging Data Intensive Computing to Support Automated Event Services” (T. Clune et al) • Accepted abstract for 2012 AGU Chapman Conference on Remote Sensing of the Terrestrial Water Cycle. • “Precipitation Characteristics of Tornado-Producing Mesoscale Convective Systems in the Continental United States” (K. Kuo)

  28. Mesoscale Atmos. Processes Lab Meeting Conclusions • Prototype of a flexible system for tracking ES events. • Initial success of applying the system to tornado-producing MCSs. • An example of a new class of tools for data-intensive science.

  29. Mesoscale Atmos. Processes Lab Meeting Thank you!

  30. Acronyms • GOES • MCS – Mesoscale convecting system • MODB- Moving Object Data Base • NMQ

  31. Mesoscale Atmos. Processes Lab Meeting Authors K.-S. Kuo1,2,3 T. L. Clune4 Y. Hong5 S. M. Freeman4,6 C. A. Cruz4,6 J. Kouatchou4,7 R. W. Burns4,6 Affiliations • Mesoscale Atmospheric Processes Laboratory NASA Goddard Space Flight Center • Goddard Earth Science Technology and Research (GESTAR) Cooperative Program • Caelum Research Corporation • Software Systems Support OfficeNASA Goddard Space Flight Center • Department of Civil EngineeringUniversity of Oklahoma • Northrop Grumman Information Systems • Tetra Tech AMT Acknowledgement: NASA Earth Science Technology Office (ESTO)

More Related