1 / 67

Perspective of SCMA IV from Particle Physics

Perspective of SCMA IV from Particle Physics. Louis Lyons Particle Physics, Oxford (CDF experiment, Fermilab) l.lyons@physics.ox.ac.uk SCMA IV Penn State 15 th June 2006. Topics.

vinnie
Download Presentation

Perspective of SCMA IV from Particle Physics

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Perspective of SCMA IV from Particle Physics Louis Lyons Particle Physics, Oxford (CDF experiment, Fermilab) l.lyons@physics.ox.ac.uk SCMA IV Penn State 15th June 2006

  2. Topics • Basic Particle Physics analyses • Similarities between Particles and Astrophysics issues • Differences • What Astrophysicists do particularly well • What Particle Physicists have learnt • Conclusions

  3. Particle Physics • What it is • Typical experiments • Typical data • Typical analysis

  4. Typical Experiments • Experiment Energy Beams # events Result • LEP 200 GeV e+ e- 107 Z N = 2.987± 0.008 • BaBar/Belle 10 GeV e+ e- 108 B anti-B CP-violation • Tevatron 2000 GeV p anti-p “1014” SUSY? • LHC 14000 GeV p p (2007…) Higgs? • KK ~3 GeV νμ 100 ν oscillations

  5. CDF at Fermilab

  6. TypicalAnalysis • Parameter determination: dn/dt = 1/τ * exp(-t / τ) Worry about backgrounds, t resolution, t-dependent efficiency • 1) Reconstruct tracks • 2) Select real events • 3) Select wanted events • 4) Extract t from L and v • 5) Model signal and background • 6) Likelihood fit for lifetime and statistical error • 7) Estimate systematic error τ± στ(stat)± στ(syst)

  7. TypicalAnalysis Hypothesis testing: Peak or statistical fluctuation?

  8. Similarities • Large data sets {ATLAS: event = Mbyte; total = 10 Pbytes} • Experimental resolution • Systematics • Separating signal from background • Parameter estimation • Testing models (versus alternative?) • Search for signals: Setting limits or Discovery • SCMA and PHYSTAT

  9. Differences • Bayes or Frequentism? • Background • Specific Astrophysics issues Time dependence Spatial structures Correlations Non-parametric methods Visualisation Cosmic variance • Blind analyses `

  10. Bayesian versus Frequentism BayesianFrequentist

  11. Bayesian versus Frequentism Bayesian Frequentist

  12. Bayesianism versus Frequentism “Bayesians address the question everyone is interested in, by using assumptions no-one believes” “Frequentists use impeccable logic to deal with an issue of no interest to anyone”

  13. Differences • Bayes or Frequentism? • Background • Specific Astrophysics issues Time dependence Spatial structures Correlations Non-parametric methods Visualisation Cosmic variance • Blind analyses `

  14. What Astrophysicists do well • Glorious pictures • Sharing data • Making data publicly available • Dealing with large data sets • Visualisation • Funding for Astrostatistics • Statistical software

  15. Whirlpool Galaxy Width of Z0 3 light neutrinos

  16. What Astrophysicists do well • Glorious pictures • Sharing data • Making data publicly available • Dealing with large data sets • Visualisation • Funding for Astrostatistics • Statistical software

  17. What Particle Physicists now know • (ln L) = 0.5 rule • Unbinned Lmax and Goodness of fit • Prob (data | hypothesis) ≠ Prob (hypothesis | data) • Comparing 2 hypotheses • Δ(c 2) ≠c 2 • Bounded parameters: Feldman and Cousins • Use correct L (Punzi effect) • Blind analyses

  18. ΔlnL = -1/2 rule If L(μ) is Gaussian, following definitions of σ are equivalent: 1) RMS of L(µ) 2) 1/√(-d2L/dµ2) 3) ln(L(μ±σ) = ln(L(μ0)) -1/2 If L(μ) is non-Gaussian, these are no longer the same “Procedure 3) above still gives interval that contains the true value of parameter μ with 68% probability” Heinrich: CDF note 6438 (see CDF Statistics Committee Web-page) Barlow: Phystat05

  19. COVERAGE How often does quoted range for parameter include param’s true value? N.B. Coverage is a property of METHOD, not of a particular exptl result Coverage can vary with Study coverage of different methods of Poisson parameter , from observation of number of events n Hope for: 100% Nominalvalue

  20. COVERAGE If true for all : “correct coverage” P< for some “undercoverage” (this is serious !) P> for some “overcoverage” Conservative Loss of rejection power

  21. Coverage : L approach (Not frequentist) P(n,μ) = e-μμn/n! (Joel Heinrich CDF note 6438) -2 lnλ< 1 λ = P(n,μ)/P(n,μbest) UNDERCOVERS

  22. Frequentist central intervals, NEVER undercover(Conservative at both ends)

  23. = (n-µ)2/µ Δ = 0.1 24.8% coverage? • NOT frequentist : Coverage = 0%  100%

  24. Unbinned Lmax and Goodness of Fit? Find params by maximising L So larger L better than smaller L So Lmax gives Goodness of Fit?? Bad Good? Great? Monte Carlo distribution of unbinned Lmax Frequency Lmax

  25. Not necessarily: pdf L (data,params) fixedvaryL Contrast pdf (data,params) param vary fixed data e.g. p(t|λ) = λ e -λt Max at t = 0 Max at λ =1/ t pL tλ Unbinned Lmax and Goodness of Fit?

  26. Lmax and Goodness of Fit? Example 1 Fit exponential to times t1, t2 ,t3 ……. [Joel Heinrich, CDF 5639] L = ln Lmax = -N(1 + ln tav) i.e. Depends only on AVERAGE t, but is INDEPENDENT OF DISTRIBUTION OF t (except for……..) (Average t is a sufficient statistic) Variation of Lmax in Monte Carlo is due to variations in samples’ average t , but NOT TO BETTER OR WORSE FIT pdf Same average t same Lmax t

  27. Example 2 Lmax and Goodness of Fit? 1 + α cos2θ L = cos θ pdf (and likelihood) depends only on cos2 θi Insensitive to sign of cos θi So data can be in very bad agreement with expected distribution e.g. All data with cos θ< 0 and Lmax does not know about it. Example of general principle

  28. Lmax and Goodness of Fit? Example 3 Fit to Gaussian with variable μ, fixed σ ln Lmax = N(-0.5 ln2π – lnσ) – 0.5 Σ(xi – xav)2 / σ2 constant ~variance(x) i.e. Lmax depends only on variance(x), which is not relevant for fitting μ (μest = xav) Smaller than expected variance(x) results in larger Lmax x x Worse fit, larger Lmax Better fit, lower Lmax

  29. Transformation properties of pdf and L Lifetime example: dn/dt = λ e –λt Change observable from t to y = √t dn/dy = (dn/dt) (dt/dy) = 2 y λ exp(–λy2) So (a) pdf CHANGES but (b) i.e. corresponding integrals of pdf are INVARIANT

  30. Now for Likelihood When parameter changes from λ to τ = 1/λ (a’) L does not change dn/dt = 1/τ exp{-t/τ} and so L(τ;t) = L(λ=1/τ;t) because identical numbers occur in evaluations of the two L’s BUT (b’) So it is NOT meaningful to integrate L (However,………)

  31. CONCLUSION: • NOT recognised statistical procedure • [Metric dependent: • τ range agrees with τpred • λ range inconsistent with 1/τpred] • BUT • Could regard as “black box” • Make respectable by L Bayes’ posterior • Posterior(λ) ~ L(λ)* Prior(λ) [and Prior(λ) can be constant]

  32. P (Data;Theory) P (Theory;Data) Theory = male or female Data = pregnant or not pregnant P (pregnant ; female) ~ 3%

  33. P (Data;Theory) P (Theory;Data) Theory = male or female Data = pregnant or not pregnant P (pregnant ; female) ~ 3% but P (female ; pregnant) >>>3%

  34. P (Data;Theory) P (Theory;Data) HIGGS SEARCH at CERN Is data consistent with Standard Model? or with Standard Model + Higgs? End of Sept 2000 Data not very consistent with S.M. Prob (Data ; S.M.) < 1% valid frequentist statement Turned by the press into: Prob (S.M. ; Data) < 1% and therefore Prob (Higgs ; Data) > 99% i.e. “It is almost certain that the Higgs has been seen”

  35. p-value ≠ Prob of hypothesis being correct Given data and H0 = null hypothesis, Construct statistic T (e.g. χ2) p-value = probability {T tobserved}, assuming H0 = true If p = 10-3, what is prob that H0 = true? e.g. Try to identify μ in beam (H0: particle = μ) with π contam. Prob (H0) depends on a) similarity of μ and π masses b) relative populations of μ and π If N(π) ~ N(μ), prob(H0)  0.5 If N(π)  N(μ), prob(H0) ~ 1 0 p 1 If N(π) ~10*N(μ), prob(H0) ~ 0.1 i.e. prob(H0) varies with p-value, but is not equal to it

  36. p-value ≠ Prob of hypothesis being correct After Conference Banquet speech: “Of those results that have been quoted as significant at the 99% level, about half have turned out to be wrong!” Supposed to be funny, but in fact is perfectly OK

  37. PARADOX Histogram with 100 bins Fit 1 parameter Smin: χ2 with NDF = 99 (Expected χ2 = 99 ± 14) For our data, Smin(p0) = 90 Is p1 acceptable if S(p1) = 115? • YES. Very acceptable χ2 probability • NO. σp from S(p0 +σp) = Smin +1 = 91 But S(p1) – S(p0) = 25 So p1 is 5σ away from best value

  38. Comparing data with different hypotheses

  39. χ2 with ν degrees of freedom? • ν = data – free parameters ? Why asymptotic (apart from Poisson  Gaussian) ? a) Fit flatish histogram with y = N {1 + 10-6 cos(x-x0)} x0 = free param b) Neutrino oscillations: almost degenerate parameters y ~ 1 – A sin2(1.27 Δm2 L/E) 2 parameters 1 – A (1.27 Δm2 L/E)2 1 parameter Small Δm2

  40. χ2 with ν degrees of freedom? 2) Is difference in χ2 distributed as χ2 ? H0 is true. Also fit with H1 with k extra params e. g. Look for Gaussian peak on top of smooth background y = C(x) + A exp{-0.5 ((x-x0)/σ)2} Is χ2H0 - χ2H1 distributed as χ2 withν = k = 3 ? Relevant for assessing whether enhancement in data is just a statistical fluctuation, or something more interesting N.B. Under H0 (y = C(x)) : A=0 (boundary of physical region) x0 and σ undefined

  41. Is difference in χ2 distributed as χ2 ? Demortier: H0 = quadratic bgd H1 =……… + Gaussian of fixed width • Protassov, van Dyk, Connors, …. • H0 = continuum • H1 = narrow emission line • H1 = wider emission line • H1 = absorption line • Nominal significance level = 5%

  42. Is difference in χ2 distributed as χ2 ? • So need to determine the Δχ2 distribution by Monte Carlo • N.B. • Determining Δχ2 for hypothesis H1 when data is generated according to H0 is not trivial, because there will be lots of local minima • If we are interested in5σ significance level, needs lots of MC simulations

More Related