Machine Learning and Data Mining for Automatic Detection and Interpretation of Solar Events

1. NASA AISRP NASA AMES, Moffett Field, CA April 4 – 6, 2005 Machine Learning and Data Mining for Automatic Detection and Interpretation of Solar Events Jie Zhang (Presenting, Co-I, SCS*) Art Poland (PI, SCS*) Harry Wechsler (Co-I, Computer Science) Kirk Borne (Co-I, SCS*) Oscar Olmedo (student, SCS*) (George Mason University) *SCS: School of Computational Sciences at GMU

2. Introduction • Why solar events? • Great interests of scientific understanding • Great interests of practical use: the space weather • What are solar events? Examples CME FLARE Dimming (coronal mass ejection)

3. CME/Flare Statistics Year 1996 – 2004 Flare Count 19176 CME Count 8852 Daily Min Max (1996) (2002) Flare 1.0 10 CME 0.5 5 Sunspot 20 200

4. Objectives • Our main objective is to develop an automatic system for CME detection, tracking, characterization and source region location • An automatic system is needed • Timely detection, necessary for space weather forecasting • Objective characterization, removing human bias • Reducing human cost • Data volume and number of events are enormous • Explosively growth of data (SOHO, STEREO and SDO)

5. Methods • Image Processing (current work) • Pre-processing • Detection and Tracking • Characterization • Machine Learning (future work) • Develop robust and efficient algorithms for event detection • Learning Methods • Statistical learning theory, e.g., Support Vector Machine (SVM) • Performance Evaluation • Benchmark (catalog by human) • ROC (Receiver Operating Characteristic) curve: hit, miss, or false-detection • Data Mining (future work) • Association of events from different sets of observations • Space, and Time • Physical parameters, e.g., intensity

6. Image Properties • Find a faint moving object against a strong slow-varying background • CME, like other astrophysical objects, is optically thin; no hard surface • An object without fixed shape; an expansion flow

7. Image Processing: Pre-processing • Calibration • Filtering and Smoothing • Differencing • Polar Transformation

8. Image Processing: Initial Detection • Finding CME angular expansion • Projection • Threshold : get core angles • Morphology analysis • Region Growing • Closing (Dilation + Erosion): join features with narrow gaps • Opening (Erosion + Dilation): remove narrow features • Finding CME Height • Thresholding on the area of selected angular expansion • Projection along the height

9. Image Processing: a Demo 2002/12/01 – 12/07: 431 images

10. Image Processing: Detection and Tracking • After the first detection • Set the time stamp, expire after 5 hours • set the targeted tracking region • Targeted-tracking reduces false detection significantly, e.g., remove contamination of CME trailing outflow • Cleaning • Remove sporadic detection • Preliminary Statistics: 2002/12/01 – 2002/12/07 • 19 CMEs in human catalog • 19 CMEs in machine catalog (25 before cleaning) • hit: 14 (74%) • miss: 5 (26%) • false detection: 5 (26%)

11. Future Plan of this Project • We are only a few months into this project, which is supported for only one year • We are seeking a full 3-year funding to fulfill the proposed objectives • Finish all image processing tasks • C2 (almost done) • C3 (under development) • EIT (under development) • Use machine learning methods to develop robust algorithms (future) • Use data mining methods to integrate detections, for the ultimate goal of space weather prediction (future) • Make a computer-generated event catalog

12. The Future • Automatic detection of all relevant events in the integrated Sun-Earth connection system • Sun • Solar Corona • Heliosphere • Magnetosphere • Ionosphere • Virtual X Observatories: contributor and user • Machine learning and Data Mining for general science discovery