1 / 15

Distinguishability of Hypotheses

Institute for High Energy Physics, Protvino, Russia. Distinguishability of Hypotheses. Institute for Nuclear Research RAS, Moscow, Russia. S.Bityukov (IHEP,Protvino; INR RAS, Moscow) N.Krasnikov (INR RAS, Moscow ).

michaeljlewis
Download Presentation

Distinguishability of Hypotheses

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Institute for High Energy Physics, Protvino, Russia Distinguishability of Hypotheses Institute for Nuclear Research RAS, Moscow, Russia S.Bityukov (IHEP,Protvino; INR RAS, Moscow) N.Krasnikov(INR RAS, Moscow) December 1, 2003 ACAT’2003 KEK, Japan S.Bityukov

  2. Concept Distinguishability of hypotheses: What is it ? Let us consider the planned experiment for searching for new phenomenon. Planned experiment will give information about : (a) existence of new phenomenon (yes or no), (b) magnitude and accuracy of measured value. We consider the case (a) from the frequentist point of view, namely, we will calculate the total amount of possible cases and, after that, the amount of cases in favor of one of the statements. This approach allows to estimate the probability of making a correct decision and, correspondingly, the quality of planned experiment by using the hypotheses testing. Also we can introduce the conception of the distinguishability of hypotheses. December 1, 2003 ACAT’2003 KEK, Japan S.Bityukov

  3. Hypotheses testing Hypothesis Hypothesis Hypothesis H0 : new physics is present in Nature Hypothesis H1 : new physics is absent Many researchers prefer to exchange the places of these hypotheses a = P(reject H0 | H0 is true) -- Type I error b = P(accept H0 | H0 is false) -- Type II error a is a significance of the test 1- b is a power of the test Note that a and b can be considered as random variable. If H0 is true then a takes place and b is absent. If H1 is true then a is absent and b takes place. December 1, 2003 ACAT’2003 KEK, Japan S.Bityukov

  4. What to test ? Suppose the probability of the realization of n events in the experiment is described by function f(n;  ) with parameter  Expected number of signal events in experiment is  s Expected number of background events is  b Hypothesis H0 corresponds to  = s + b, i.e. f(n; s + b) H1 corresponds to  = b, i.e. f(n; b) The Type I error a and the Type II error b allows to estimate the probability of making a correct decision when testing H0 versus H1 with an equal-tailed test and to estimate the distinguishability of these hypotheses H0 and H1 with an equal probability test. December 1, 2003 ACAT’2003 KEK, Japan S.Bityukov

  5. The probability of making a correct decision in hypotheses testing Let us consider the random variable k = a + b = where the estimator is a constant term and is a stochastic term. Here are the estimators of Type I (a) and Type II (b ) errors. In the case of applying the equal-tailed test the stochastic term is equal to 0 independently of whether H0 or H1 is true. Hence the estimator can be named the probability of making incorrect choice in favor of one of the hypotheses. Correspondingly, is the probability to make a correct decision in hypotheses testing. December 1, 2003 ACAT’2003 KEK, Japan S.Bityukov

  6. : advantages and disadvantages Advantages +1° This probability is independent of whether H0 or H1 is true +2° In the case of discrete distributions the error  of this estimator can be taken into account +3° This is an estimator of quality of planned experiment … However, the probability of making a correct decision has disadvantages to be the measure of distinguishability of hypotheses. Disadvantages -1° Non minimal estimation of possible error in hypotheses testing -2° The region of determination is [0, 0.5] (desirable area [0, 1]) -3° Difficulties in applying of equal-tailed test for complex distributions … Goal: disadvantages ---> advantages December 1, 2003 ACAT’2003 KEK, Japan S.Bityukov

  7. Distinguishability of hypotheses -1° --> + 4° The applying of the equal probability test gives the minimal half-sum of estimators of Type I error  and Type II  by the definition of this test. The critical value n0 is chosen by the condition f(n0;  b) = f(n0;  s +  b) . December 1, 2003 ACAT’2003 KEK, Japan S.Bityukov

  8. The relative number of incorrect decisions under equal probability test -2° --> + 5° The transformation makes a good candidate to be a measure of distinguishability It is a relative number of incorrect decisions. The region of determination is [0, 1] . -3° --> + 6° The applicability of equal probability test to the complex distributions is obviously. December 1, 2003 ACAT’2003 KEK, Japan S.Bityukov

  9. The equal probability test : the critical value for Poisson distribution The applying of the equal probability test for Poisson distribution gives the critical value where square brackets mean the the integer part of a number. Note that the critical value conserves the liner dependence on the time of measurements December 1, 2003 ACAT’2003 KEK, Japan S.Bityukov

  10. The estimation of the hypotheses distinguishability The relative number of correct decisions under equal probability test is a measure of distinguishability of hypotheses at given  s and  b. The magnitude of this value can be found from equations December 1, 2003 ACAT’2003 KEK, Japan S.Bityukov

  11. Dependence of relative number correct decisions on the measurement time December 1, 2003 ACAT’2003 KEK, Japan S.Bityukov

  12. Gamma- and Poisson distributions Let us consider the Gamma-distributions with probability density If to redefine the parameters and variable 1/,  and x via a, n+1, , correspondingly, the probability density will be If a = 1 then the probability density looks like Poisson probabilities December 1, 2003 ACAT’2003 KEK, Japan S.Bityukov

  13. Conditional probability density The probability density g( ;N) of the true value of the parameter of Poisson distribution to be  in the case of the single observation N has Gamma distribution, where It follows from identity We checked the statement about g(;N) by the Monte Carlo and these calculations give the confirmation of given supposition. As a result we can take into account the statistical uncertainties by the simple way (see, NIM A502(2003)795; JHEP09(2002)060). December 1, 2003 ACAT’2003 KEK, Japan S.Bityukov

  14. Statistical uncertainties in determination of s and b Suppose that the expected number of signal events s and expected background bare determined with statistical errors. If we can reconstruct the distribution g(; N) of the true value of the parameter  of the function f(n; ) in the case of the observed N events then we can determine the relative number of correct decisions: December 1, 2003 ACAT’2003 KEK, Japan S.Bityukov

  15. Conclusions The probability language in terms of a and bis more acceptable for estimation of the quality of planned experiments than the language of standard deviations. The proposed approach allows to use the relative number of correct decisions in the equal probability test as a measure of distinguishability of hypotheses. This approach gives the easy way for including of the systematics and statistical uncertainties into the estimation of distinguishability of hypotheses. December 1, 2003 ACAT’2003 KEK, Japan S.Bityukov

More Related