Applications of change point detection in Gravitational Wave Data Analysis

1. Applications of change point detection in Gravitational Wave Data Analysis Soumya D. Mohanty AEI

2. Plan of the talk • Brief introduction to change point detection and its relevance to GW data analysis • Contrast with prevalent methods • Three applications in different areas UT Brownsville

3. What is a change point? UT Brownsville

4. Signals and Change points • The most elementary signature of a signal is to introduce a change in the distribution of data • Isolating a subset of given data that is significantly different from the rest is the most general signal detection method • This division is subject to statistical uncertainty UT Brownsville

5. Mathematical Statement • Data described by a joint probability density p(x). • CP detection: Can the data be divided into disjoint sets y, z (x = yz), such that p(y) is different from p(z)? Not required to know p(y) or p(z) themselves. • Adaptive detection: Somehow deduce or estimate a noise p(x). Then given new data y, test if it could have come from p(x). UT Brownsville

6. Change point detection Can go from full prior information to no prior Less sensitive Possible to tune away response to different types of inhomogeneity Post analysis definition required of what is a signal and what is noise Adaptive detection Needs prior information and assumption of stationarity More sensitive provided prior information is correct Tuning is a complicated process if at all possible Signal & noise pre-defined Pros and Cons UT Brownsville

7. Applications • Change point detection in the time-frequency plane – burst detection • Change point detection in a multivariate time series – Data/Detector Characterization Robot • Two sample comparison – GRB-GW association UT Brownsville

8. Bursts in time-frequency plane • Time frequency plane – arena for burst detection • Example: split time series into segments and FFT each one. • Basic signature of a burst: changes the distribution of samples in some region of the time-frequency plane. UT Brownsville

9. UT Brownsville

10. Most Burst detection algorithms try to look for this effect in different ways • Excess power: thresholds the average (=band limited rms) • Tfclusters: thresholds cluster size • PSDCD (Mohanty, PRD,’99): tests for difference in sample distributions of blocks in TF plane. • PSDCD is a change point detector, others are adaptive detectors. UT Brownsville

11. Non-parametric CP detection • Non-parametric detection: the false alarm rate is independent of noise distribution by construction. Sets it apart from other burst detectors. • A non-stationary time series can be thought of as a sequence of transitions from one noise model to another (e.g. 1 10...). A non-parametric detector should maintain a constant false alarm rate even for non-stationary noise. • CP detection can be tuned to prevent triggering on known technical features. UT Brownsville

12. KSCD • Power Spectral Density Change Detector [ DMT Monitor] • Kolmogorov-Smirnov test based Change Detector (KSCD) • KSCD: improvement in detection efficiency and implementation UT Brownsville

13. UT Brownsville

14. UT Brownsville

15. Trial run on GEO S1 data • Uncalibrated h(t). 3.47 days (some breaks). • Plagued by fast non-stationarity in the <1.5kHz band. • 90% - 95% of MTFC triggers could be attributed to this fast non-stationarity. • These false triggers skew the interpretation of histograms such as the time interval between triggers. • KSCD can be tuned to be insensitive to these features but still catch “genuine” glitches. UT Brownsville

16. Rejection of features UT Brownsville

17. Analysis goals • Disentangle fast low frequency non-stationarity from “genuine” triggers. • Study time dependent behavior of the triggers. • Study trigger rate vis a vis band limited rms trend. • Does KSCD trigger rate track band limited rms? • Tune KSCD to reject triggers but catch fast non-stationarity • Analyze the dependence of “genuine” trigger channel on fast non-stationarity channel. UT Brownsville

18. Trigger rate UT Brownsville

19. Future of KSCD • Test various aspects of non-parametric change point detection using real data (S1 GEO/LIGO, S2 LIGO) • Understand efficiency (very preliminary: 40% of matched filtering) • Build LDAS DSO • KSCD: Main engine of DCR UT Brownsville

20. Data/Detector Characterization Robot View data as a single multivariate time series DCR Detect change points Database All channels Transform the multivariate data Example: construct cross-correlation of two channels Design Data Mining UT Brownsville

21. Data CharacterizationWhat is the best analysis strategy given some data? • Quantify • non-stationarity of noise floor • Types and rates of transients • Drifting carrier frequencies • Simulate real data and do Monte Carlo studies • Hopefully, lead to more believable detection of GW signals. UT Brownsville

22. Detector Characterization • Hunt down sources of deviations from expected ideal behavior and fix them • To help, interferometers blindly record data from several other sensors • control system • environment monitors (e.g., temperature) • Seismometers, magnetometers UT Brownsville

23. Change PointsMathematical abstraction of the problem • Main interest in both data and detector characterization– change points • Example: transients, change in rate of transients, non-stationarity, change in coupling between two channels • Natural conclusion-- Build database of change points using automated algorithms and analyse the database UT Brownsville

24. Analysis of databases • Exploratory • Limited to small databases of high confidence detections • Data mining • Emerging field of synthesis between statistics and computing – aim is to detect new, informative patterns in huge databases • Requires reliable database quality UT Brownsville

25. DCR project • Overall Aim: enable data mining of multi-channel interferometric data • Elements: • Algorithms – few, well understood and complementary (not an arbitrary set of independent simple monitors) • Software/Hardware • Data mining UT Brownsville

26. Algorithms in DCR • Change point detector – KSCD • generalized to the case of cross-spectral density of two channels • Line removal – MBLT • no modeling required of line behavior • transient resistant • Robust noise floor tracking – MNFT UT Brownsville

27. Sample Power Spectral Density UT Brownsville

28. DCR implementation • Core Digital Signal Processing library in C++ • Template based Statistics and Signal Processing library (TSSP). Uses STL. • FFT, Filtering, Filter Design, Windows, PSD, Modulation, Demodulation, ... • Stand alone C++ main function for a given pipeline UT Brownsville

29. Stand alone code • Frame reading class • Multiple ADC channels • Database IO class (uses MySQL) • Database to be used for both job description and storing job outputs • Multiple jobs launched using Condor • At present: dedicated 10 node cluster (Linux-alpha) UT Brownsville

30. GRB-GW association • Finn, Mohanty, Romano, PRD, 1999 • Based on two sample comparison • on-source sample • off-source sample • Two sample tests also used in CP detection UT Brownsville

31. Introduction to Gamma-Ray Bursts http://online.itp.ucsb.edu/online/gamma_c99/piran/oh/06.html • High-energy, short-duration electromagnetic radiation from extra-galactic sources • Favored models point to exploding fireball • Involve large amounts of matter, • ejected at relativistic speeds, • producing a series of high-energy E/M shockwaves--- • initially gamma-rays (some redshift to lower-energy gamma-rays or X-rays, others are absorbed), • then X-rays (red-shifted to optical wavelengths), • then visible light (red-shifted to radio wavelengths) UT Brownsville

32. GRBs and Gravitational Waves • GRB progenitors thought to be new formed Black Holes • Black Hole formed as a result of massive stellar collapse or binary NS mergers • BH accretes debris rapidly • Leads to beams of ultra-relativistic ejecta • This violent scenario is a natural candidate for strong GW emission also UT Brownsville

33. Motivation for an FMR type search • GRBs occur at cosmological distances. Hence chance of detecting GWs from an individual GRB is small • However, GRB astronomy is very active • Relatively large number of events were detected (~O(1/day)) by BATSE • Several more missions coming up soon (e.g., SWIFT and GLAST) • FMR: Combine information from several triggers to build up signal to noise ratio UT Brownsville

34. Algorithm • Cross-correlate time series between two interferometers for each GRB trigger • time shift segments to align GW signal • Compare cross-correlation to times not associated with GRBs • Build an on-source and a off-source sample of cross-correlations • Test if the means values of the two samples are significantly different UT Brownsville

35. Implementation • External Triggers subgroup of Bursts Upper Limit group • S. Marka, R. Rahkola, S. Mohanty, S. Mukherjee, R. Frey • Could not apply FMR in toto forS1 because only one trigger received during double lock (LIGO tech note) • Already have 15 triggers for S2! UT Brownsville

36. Issues • Non-stationarity of data • Data conditioning – line removal • Noise floor tracking -- MNFT • Lack of directional accuracy • Use H1+H2 – but strong (non-stationary?) correlations • How to best use multiple interferometers • Systematic uncertainties • Rely on signal injection and Monte Carlo simulations • DCR – simulate real data? UT Brownsville

37. Summary • Applications of change point detection in GW data analysis • Exploration of such techniques has just only started • Offers better control on data analysis with real, complicated data • Improvements in efficiency possible. Can be combined with adaptive methods. UT Brownsville