Scientific Data Mining in Astronomy

1. THE US NATIONAL VIRTUAL OBSERVATORY Scientific Data Mining in Astronomy Kirk D. Borne George Mason University kborne@gmu.edu http://classweb.gmu.edu/kborne/ 1

2. OUTLINE • Scientific Databases • Some key astronomy problems • Astronomy Data Mining examples • Suggested Reading • Some Data Mining Software • Summary

3. OUTLINE • Scientific Databases • Some key astronomy problems • Astronomy Data Mining examples • Suggested Reading • Some Data Mining Software • Summary

4. 10 Unique Features of Scientific Data Each of these characteristics requires special handling beyond what you read in standard data mining textbooks: Scientific data depend on experimental equipment and conditions. Scientific data have noise. Scientific data have been (or need to be) calibrated. Scientific units on data values are imperative. Scientific databases often contain associated columns: { value, error }. Scientific data values are often non-linear (log values, magnitudes, asinh). History of scientific data creation, processing, and versioning is critical = Provenance. Metadata,Metadata,Metadata = tells us “who, what, when, where, how”. NOTE: SemanticMetadataare becoming more important = “why”. Context is critical (e.g., brightness in an optical catalog is expressed in mags, but expressed in counts/sec in an X-ray catalog, or milli-Jansky in a radio catalog). Scientific data have different levels of abstraction: raw, calibrated, reduced data products, derived information, extracted knowledge, published results. All of this makes the “Data Preparation” phase of any scientific data mining experiment even more critical and essential. 4

5. OUTLINE • Scientific Databases • Some key astronomy problems • Astronomy Data Mining examples • Suggested Reading • Some Data Mining Software • Summary

6. Some key astronomy problems Some key astronomy problems that can be addressed with data mining techniques: Cross-Match objects from different catalogues The distance problem (e.g., Photometric Redshift estimators) Star-Galaxy Separation Cosmic-Ray Detection in images Supernova Detection and Classification Morphological Classification (galaxies, AGN, gravitational lenses, ...) Class and Subclass Discovery (brown dwarfs, methane dwarfs, ...) Dimension Reduction = Correlation Discovery Learning Rules for improved classifiers Classification of massive data streams Real-time Classification of Astronomical Events Clustering of massive data collections Novelty, Anomaly, Outlier Detection in massive databases 6

7. OUTLINE • Scientific Databases • Some key astronomy problems • Astronomy Data Mining examples • Suggested Reading • Some Data Mining Software • Summary

8. Classification Methods:Decision Trees, Neural Networks, SVM (Support Vector Machines) • There are 2 Classes! • How do you ... • Separate them? • Distinguish them? • Learn the rules? • Classify them? Apply Kernel (SVM)

9. Decision Tree Classification Example: SKICAT Star-Galaxy Discrimination Reference: ftp://iraf.noao.edu/iraf/conf/web/adass_proc/adass_95/yooj/yooj.html

10. Decision Tree Classification Example: Classification of candidates for new supernova in galaxies Reference:http://spiff.rit.edu/richmond/sdss/sn_survey/scan_manual/sn_scan.html

11. Clustering is used to discover the different unique groupings (classes) of attribute values.The case shown below is not obvious: one or two groups?

12. This case is easier: there are two groups.(in fact, this is the same set of data elements as shown on the previous slide, but plotted here using a different attribute.)

13. Clustering in multiple dimensions: colors combined from SDSS & 2MASS magnitudes

14. Clustering: Class Discovery and Rule Learning • Reference: http://www.cs.princeton.edu/courses/archive/spr04/cos598B/bib/BrunnerDPS.pdf Good Not good Clusters and the separation of classes depend on which attributes (dimensions) are chosen to be projected, as in the following star-galaxy discrimination test:

15. Semisupervised Learning:Outlier Detection A demonstration of a generic machine-assisted discovery problem — data mapping and a search for outliers. This schematic illustration is of the clustering problem in a parameter space given by three object attributes: P1, P2, and P3. In this example, most of the data points are assumed to be contained in three, dominant clusters (DC1, DC2, and DC3). However, one may want to discover less populated clusters (e.g., small groups or even isolated points), some of which may be too sparsely populated, or lie too close to one of the major data clouds. In some cases, negative clusters (holes), may exist in one of the major data clusters. Reference: http://www.cs.princeton.edu/courses/archive/spr04/cos598B/bib/BrunnerDPS.pdf

16. Outlier Detection: Serendipitous Discovery of Rare or New Objects & Events

17. Principal Components Analysis &Independent Components Analysis Cepheid Variables: Cosmic Yardsticks -- One Correlation -- Two Classes! ... Class Discovery!

18. Example: SOM (Self-Organizing Map) The SOM (Self-Organizing Map) is one technique for organizing information in a database based upon links between concepts. It can be used to find hidden relationships and patterns in more complex data collections, usually based on links between keywords or metadata. 18

19. Astronomy Data Mining in Action Exploringthe Time Domain Mega-Flares on normal Sun-like stars = a star like our Sun increased in brightness 300X one night! … say what??

20. Example: The Thinking Telescope Sample Data Mining Applications:(credit: http://www.thinkingtelescopes.lanl.gov/ ) Automated Feature Extraction:Real-time identification of artifacts and transients in direct and difference images. Classifiers:Automated classification of celestial objects based on temporal and spectral properties. Anomaly Detection:Real-time recognition of important deviations from normal behavior for persistent sources.

21. From Sensors to Sense From Data to Knowledge: from sensors to sense Data  Information  Knowledge

22. VOEventNet Reference:http://voeventnet.caltech.edu/

23. Learning From Archived Temporal Data (Time Series):Classify New Data (Bayes Analysis or Markov Modeling)

24. Photometric-Redshift Estimation Photometric vs. Spectroscopic Redshift Estimates: Left panel: standard technique Right panel: Machine Learning (data mining) application Reference: http://arxiv.org/abs/0710.4482 24

25. Star-Galaxy Separation in Clustered Feature Space • * = star • = galaxy http://arxiv.org/abs/astro-ph/9508012 25

26. Bayesian Probabilistic Estimationfor Catalog Cross-Matching Reference: http://arxiv.org/abs/astro-ph/0605216 26

27. Fundamental Plane for 156,000 cross-matched Sloan+2MASS Elliptical Galaxies: plot shows variance captured by first 2 Principal Components as a function of local galaxy density. Slide Content Slide content Slide content Slide content Reference: Borne, Dutta, Giannella, Kargupta, & Griffin 2008 % of variance captured by PC1+PC2 27 low (Local Galaxy Density) high

28. OUTLINE • Scientific Databases • Some key astronomy problems • Astronomy Data Mining examples • Suggested Reading • Some Data Mining Software • Summary

29. Suggested Reading: Data Mining in Astronomy Djorgovski et al. 2000, Searches for Rare and New Types of Objects. http://arxiv.org/abs/astro-ph/0012453 Djorgovski et al. 2000, Exploration of Large Digital Sky Surveys. http://arxiv.org/abs/astro-ph/0012489 Djorgovski et al. 2001, Exploration of Parameter Spaces in a Virtual Observatory. http://arxiv.org/abs/astro-ph/0108346 Mining the Sky, 2001, published proceedings of ESO conference. Suchkov et al. 2003, Automated Object Classification with ClassX. astro-ph/0210407 Suchkov, Hanisch, & Margon 2005, A Census of Object Types and Redshift Estimates in the SDSS Photometric Catalog from a Trained Decision Tree Classifier. http://adsabs.harvard.edu/abs/2005AJ....130.2439S Giannella et al. 2006, Distributed Data Mining for Astronomy Catalogs. http://www.cs.umbc.edu/~hillol/PUBS/Papers/Astro.pdf Rohde et al. 2006, Matching of Catalogues by Probabilistic Pattern Classification. http://adsabs.harvard.edu/abs/2006MNRAS.369....2R Budavari & Szalay 2008, Probabilistic Cross-Identification of Astronomical Sources. http://adsabs.harvard.edu/abs/2008ApJ...679..301B 29

30. Suggested Reading, continued: Data Mining in Astronomy Odewahn et al. 1993, Star-Galaxy Separation with a Neural Network. 2: Multiple Schmidt Plate Fields. http://adsabs.harvard.edu/abs/1993PASP..105.1354O Borne 2000, Science User Scenarios for a Virtual Observatory Design Reference Mission: Science Requirements for Data Mining. astro-ph/0008307 Brunner et al. 2001, Massive Datasets in Astronomy. astro-ph/0106481 Gray et al. 2002, Data Mining the SDSS SkyServer Database. http://arxiv.org/abs/cs/0202014 Odewahn et al. 2004, The Digitized Second Palomar Observatory Sky Survey (DPOSS). III. Star-Galaxy Separation. http://adsabs.harvard.edu/abs/2004AJ....128.3092O Ball, Brunner, et al. 2006, Robust Machine Learning Applied to Astronomical Data Sets. I. Star-Galaxy Classification of the Sloan Digital Sky Survey DR3 Using Decision Trees. http://adsabs.harvard.edu/abs/2006ApJ...650..497B Ball, Brunner, et al. 2007, Robust Machine Learning Applied to Astronomical Data Sets. II. Quantifying Photometric Redshifts for Quasars Using Instance-based Learning. http://adsabs.harvard.edu/abs/2007ApJ...663..774B Ball, Brunner, et al. 2008, Robust Machine Learning Applied to Astronomical Data Sets. III. Probabilistic Photometric Redshifts for Galaxies and Quasars in the SDSS and GALEX. http://adsabs.harvard.edu/abs/2008ApJ...683...12B 30

31. Suggested Reading, continued: Data Mining in Astronomy Rogers & Riess 1994, Detection and Classification of CCD Defects with an Artificial Neural Network. http://adsabs.harvard.edu/abs/1994PASP..106..532R Feeney et al. 2005, Automated Detection of Classical Novae with Neural Networks. http://adsabs.harvard.edu/abs/2005AJ....130...84F Wadadekar 2005, Estimating Photometric Redshifts Using Support Vector Machines. http://adsabs.harvard.edu/abs/2005PASP..117...79W Bazell & Miller 2005, Class Discovery in Galaxy Classification. http://adsabs.harvard.edu/abs/2005ApJ...618..723B Bazell, Miller, & SubbaRao 2006, Objective Subclass Determination of Sloan Digital Sky Survey Spectroscopically Unclassified Objects. http://adsabs.harvard.edu/abs/2006ApJ...649..678B Ferreras et al. 2006, A Principal Component Analysis approach to the Star Formation History of Elliptical Galaxies in Compact Groups. http://adsabs.harvard.edu/abs/2006MNRAS.370..828F Way & Srivastava 2006, Novel Methods for Predicting Photometric Redshifts from Broadband Photometry Using Virtual Sensors. http://adsabs.harvard.edu/abs/2006ApJ...647..102W Carliles et al. 2007, Photometric Redshift Estimation on SDSS Data Using Random Forests. http://arxiv.org/abs/0711.2477 31

32. OUTLINE • Scientific Databases • Some key astronomy problems • Astronomy Data Mining examples • Suggested Reading • Some Data Mining Software • Summary

33. Some Data Mining Software & Projects General data mining software packages: Weka (Java): http://www.cs.waikato.ac.nz/ml/weka/ Weka4WS (Grid-enabled): http://grid.deis.unical.it/weka4ws/ RapidMiner: http://www.rapidminer.com/ Astronomy-specific software and/or user clients: VO-Neural: http://voneural.na.infn.it/ AstroWeka: http://astroweka.sourceforge.net/ OpenSkyQuery: http://www.openskyquery.net/ ALADIN: http://aladin.u-strasbg.fr/ MIRAGE: http://cm.bell-labs.com/who/tkh/mirage/ AstroBox: http://services.china-vo.org/ Astronomical and/or Scientific Data Mining Projects: GRIST: http://grist.caltech.edu/ ClassX: http://heasarc.gsfc.nasa.gov/classx/ LCDM: http://dposs.ncsa.uiuc.edu/ F-MASS: http://www.itsc.uah.edu/f-mass/ NCDM: http://www.ncdm.uic.edu/ 33

34. Weka:http://www.cs.waikato.ac.nz/ml/weka/ Weka is in your NVOSS software distribution. Weka is a collection of open source machine learning algorithms for data mining tasks. Weka algorithms can either be applied directly to a dataset or called from your own Java code. Weka comes with its own GUI. Weka contains tools for data pre-processing, classification, regression, clustering, association rules, and visualization. 34

35. AstroWeka:http://astroweka.sourceforge.net/ http://www.iterating.com/products/Weka http://weka.sourceforge.net/wekadoc/index.php/en:Knowledge_Flow_%283.4.10%29 35

36. ALADIN: http://aladin.u-strasbg.fr/

37. MIRAGE:http://cm.bell-labs.com/who/tkh/mirage/ Java Package for exploratory data analysis (EDA), correlation mining, and interactive pattern discovery.

38. OUTLINE • Scientific Databases • Some key astronomy problems • Astronomy Data Mining examples • Suggested Reading • Some Data Mining Software • Summary

39. Science is Knowledge Work Data  Information  Knowledge • Knowledge Discovery is the central theme of science. • Knowledge Discovery in Databases (KDD) is the killer app for large scientific databases. • Therefore, KDD (i.e., Data Mining) is an essential tool, since “big-data” science is here to stay (at petabytes and beyond).

40. Scientific Knowledge Discovery

41. Heliophysics Space Weather Example

42. Sun-Earth Space Environment – Rich Source of Heliophysical Phenomena

43. Multi-point Observations and Models of Space Plasmas Deliver a Deluge of Physical Measurements

45. Heliophysics Space Weather Example CME = Coronal Mass Ejection SEP = Solar Energetic Particle

46. Data Mining:It is more than just connecting the dots Reference: http://homepage.interaccess.com/~purcellm/lcas/Cartoons/cartoons.htm

47. Sample Astronomy Data Mining ApplicationIdeas for your Projects • Neural Network for Pixel Classification: Event Detection and Prediction (e.g., Supernova or Cosmic-ray hit?) • Bayesian Network for Object Classification (star or galaxy?) • PCA for finding Fundamental Planes of Galaxy Parameters • PCA (weakest component) for Outlier Detection: anomalies, novel discoveries, new objects • Link Analysis (Association Mining) for Causal Event Detection (e.g., linking optical transients with gamma-ray events) • Clustering analysis: Spatial, Temporal, or any scientific database parameters • Markov models: Temporal mining, classification, and prediction from time series data