1 / 37

Data Analysis

Learn how to analyze your data without being a mathematician or programmer. Discover various methods such as visual inspection, Fourier analysis, and wavelet analysis. Practice and improve your skills with freely available programs.

demetriusn
Download Presentation

Data Analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Data Analysis Do it yourself!

  2. What to do with your data? • Report it to professionals (e.g., AAVSO) • Excellent! A real service to science; don’t neglect this • Publish observations (e.g., JAAVSO) • Analyze it – yourself!

  3. But … • I’m not a mathematician • Let the computer do the math • I’m not a programmer • Get programs from the net (often free) • I don’t know how to use or interpret them • Neither do the pros! • Practice, practice, practice …

  4. Time Series Analysis • A time series is a set of data pairs • t is the time, x is the data value • Usually, times are assumed error-free • Data = Signal + Error • x can be anthing, e.g. brightness of variables star, time of eclipse, eggs/day from a laying hen

  5. Actual Mean = = expected value Standard deviation = expected rms difference from mean Estimated Average = estimated Sample standard deviation = estimated Basic properties of data x

  6. Average and sample standard deviation • Average • Sample standard deviation

  7. Method #1: world’s best • Eye + Brain: Look at the data! • Plot x as a function of t: Explore! • Scientific name: Visual Inspection • World’s best – but not infallible • Programs: • TS http://www.aavso.org • MAGPLOT http://www.aavso.org

  8. Method #2: Fourier Analysis • Period analysis and curve-fitting • Powerful, well-understood, popular • Programs • TS http://www.aavso.org • PerAnSo http://www.peranso.com

  9. Method #3: Wavelet Analysis • Time-frequency analysis • Old versions bad, new version good • Programs: • WWZ http://www.aavso.org • WinWWZ http://www.aavso.org

  10. Visual Inspection

  11. Let’s take a look

  12. Fourier Analysis

  13. Fourier analysis for period search • Match the data to sine/cosine waves • = frequency • Period = • Amplitude = A = size of fluctuation • Obvious choice is period; mathematically sound choice is frequency

  14. Null Hypothesis (important!) • Null hypothesis: no time variation at all • So = constant • So, • Quite important! Often neglected. Even the pros often forget this.

  15. Is it real? • Fit produces a test statistic under the null hypothesis • Is usually “ /degree of freedom” (d) • Linear: is significant (not just by accident) at 95% confidence • 95% confidence means 5% false-alarm probability

  16. Meaning of significance • Significance does not mean the signal is linear, sinusoidal, periodic, etc. • It only means the null hypothesis is incorrect, i.e., the signal is not constant • Important!!!

  17. Pre-whitening • If you find a significant fit, then subtract the estimated signal, leaving residuals • Analyze the residuals for more structure • This process is called pre-whitening

  18. How to choose frequency? • Test all reasonable values, get a “strength of fit” for each. Common is “chi-square per degree of freedom” (but there are many) • Plot frequency .vs. fit – the Fourier transform (aka periodogram, aka power spectrum)

  19. Fourier decomposition Any periodic function of period P (frequency ) can be expressed as a Fourier series:

  20. Fundamental + harmonics For a pure sinusoid, expect response at frequency For a general periodic signal at a given frequency, expect a fundamental component at , as well as harmonics at frequencies etc.

  21. Lots of Fourier methods • FFT: fast Fourier transform • Not just fast: it’s wicked fast • Requires even time spacing • Requires N=integer power of 2 • Beware! • DFT: discrete Fourier transform • Applies to any time sampling, but incorrect results for highly uneven (as in astronomy!) • Beware!

  22. Problems from uneventime sampling • Aliasing: false peaks, often from a periodic data density • Aliases at • Common in astronomy: data density have a period P = 1 yr = 365.2422 d, so • Solution: pre-whitening

  23. Aliasing

  24. Aliasing: UZ Hya

  25. Problems from uneventime sampling • Mis-calculation of frequency (slightly) and amplitude (greatly); sabotages prewhitening

  26. Solution: better Fourier methods (for astronomy) • Lomb-Scargle modified periodogram • Improvement over FFT, DFT • CLEAN spectrum • Bigger improvement • DCDFT: date-compensated discrete Fourier transform (this is the one you want) • CLEANEST spectrum: DCDFT-like for multiple frequencies

  27. DCDFT • Much better estimates of period, amplitude

  28. Let’s take a look • Peranso (uses DCDFT and CLEANEST) • Available from CBA Belgium • http://www.peranso.com

  29. Fourier transform (CLEANEST) of TU Cas

  30. Wavelet Analysis

  31. Wavelets • Fit sine/cosine-like functions of brief duration • Shift them through time • Gives a time-frequency analysis

  32. Problems • Same old same old: uneven time spacing, especially variable data density, invalidate the results • But: even worse than Fourier • Essentially useless for most astronomical data

  33. Wavelet methods • DWT: discrete wavelet transform • Just not right for unevenly sampled data (astronomy!) • Solution: WWZ = weighted wavelet Z-transform

  34. Let’s take a look

  35. Data Analysis • Do it yourself • Use your eyes and brain • Healthy skepticism • tamino_9@hotmail.com • Enjoy!

More Related