Szapudi´s talk – F alse D etection R ate

1. Szapudi´s talk – False Detection Rate Simultaneous hypothesis testing - Setting the statistical significance Detections of Non-Gaussianity in CMB observations • Brief review of detections/methods used to set the statistical • significance. • Application of FDR to individual statistical methods. • Application of FDR to a combination of statistical methods?. • What is the appropriate method for assessing the statistical • significance of localized detections?.

2. CMB: Non-Gaussianity tests Blind tests – The alternative to the null hypothesis is not specified.

3. Non-Gaussianity tests: Real Space Based on the temperature fluctuation observed at each pixel i, ΔT(i) • Tests (detections): • Genus (Park 2003). • N-point correlation function (Eriksen et al. 2003, Eriksen et al. 2004). • Minkowski functionals and length of skeleton (Eriksen et al. 2004). • Extrema (Larson & Wandelt 2004) • 2-point correlation function of maxima and minima (Tojeiro et al. 2005, Larson & • Wandelt 2005) Courtesy of WMAP Science Team

4. Non-Gaussianity Tests: Spherical Harmonic Space Based on the complex coefficients • Tests (detections): • Power spectrum distribution (Eriksen • et al. 2003, Hansen et al. 2004). • Correlations between adjacent • multipoles (Prunet et al. 2004).

5. Non-Gaussianity Tests: Wavelet Space Based on wavelet coefficients calculated at each pixel i, at a given scale R, wv(i,R) . • Tests(detections): • Kurtosis – Spherical Mexican Hat • Wavelet (Vielva et al. 2003, Mukherjee & • Wang 2004, McEwen et al. 2004, Liu & • Zhang 2005, Cruz et al. 2006). • Skewness – Real Morlet Wavelet • (McEwen et al. 2004, Liu & Zhang 2005). • Number and area and volume of spots – • SMHW (Cruz et al. 2004, Cruz et al. 2006). • Higher Criticism – SMHW (Cayon et al. • 2005, Cruz et al. 2006).

6. Testing the null hypothesis through Monte Carlo simulations Simulating a CMB map: At each pixel (i – θ,ψ) 1) Bl – Antenna Cl- Cosmology • Add Noise - dispersion given by simulated • experiment. Mask – Galaxy plus point sources (zeros to the pixel in the mask). Monopole and dipole removal.

7. Testing the null hypothesis through Monte Carlo simulations

8. Testing the null hypothesis with a single configuration (confidence level). Simultaneous hypotheses testing : - Χ2Example (McEwen et al.2004) Statistical Hypothesis Testing SMHW Skewness and kurtosis – 24 statistics. Detection at the 99.9% significance level. - Conservative significance level Based on marginal distribution of all statistics. Ex. Above 95.3% significance level. - Hypothesis Test Larson & Wandelt, astro-ph/0505046 (maximum risk of false detection at the same level as the claimed significance) SMHW Kurtosis- scale 9 above 99% confidence level.

9. Statistical Hypothesis Testing(FDR) • Simultaneous Hypotheses testing – False Detection Rate - Controlofthe fraction of false discoveries (detections) over the total number of discoveries. - No assumption on the Gaussianity of the error distribution. - Correlations between statistics can be taken into account (?). αx100 % of discoveries may be mistakes.

10. False Discovery Rate – Ex. Wavelet Space 16 tests / statistical test No correlations, α=0.05, detection scales=9,8,7 Correlations, α=0.1, detection scales=9,8,7 No correlations, α=0.1, detection scales=9,8 Correlations, α=0.2, detection scales=9 Figs from Cruz et al. 2006

11. False Discovery Rate – Ex. Wavelet Space Acknowledgement – M. Cruz

12. Only considering detections based on SMHW: 112 statistical tests. - Different statistical methods. - Several scales. Is it possible to assess the statistical significance of all detections all together? The pixels behind some of the detections are localized. What is the appropriate method for setting the statistical significance?