Uncertainty Analysis (I) Methods

1. Uncertainty Analysis (I) Methods PHYSTAT

2. a2 nearby points are also acceptable the standard fit, minimum c2 a1 We continue to use 2globalas figure of merit. Explore the variation of 2global in the neighborhood of the minimum. The Hessian method (m, n = 1 2 3 … d) PHYSTAT

3. diagonalize rescale The numerical computation of Hmnby finite differences is nontrivial: • the eigenvalues of Hmnvary over many orders of magnitude, • computation of 2global is subject to small numerical errors leading to discontinuities as a function of {am}. The finite step size must be direction dependent. We devised an iterative method which converges to a good complete set of eigenvectors of Hmn . PHYSTAT

4. “Master Formula” Classical error formula for a variable X(a) Obtain better convergence using eigenvectors of Hmn Sm(+) and Sm(-) denote PDF sets displaced from the standard set, along the  directions of the mth eigenvector, by distance T = (Dc2) in parameter space. (available in the LHAPDF format : 2d alternate sets) PHYSTAT

5. l : Lagrange multiplier controlled by the parameter l The Lagrange Multiplier Method … for analyzing the uncertainty of PDF-dependent predictions. The fitting function for constrained fits Minimization of F [w.r.t {am} and l] gives the best fit for the value X(a min,m ) of the variable X. Hence we obtain a curve of c2global versus X. PHYSTAT

6. The question of tolerance X : any variable that depends on PDF’s X0 : the prediction in the standard set 2(X) : curve of constrained fits For the specified tolerance ( c2 = T2 ) there is a corresponding range of uncertainty,  DX. What should we use for T? PHYSTAT

7. Estimation of parameters in Gaussian error analysis would have T = 1 We do not use this criterion. PHYSTAT

8. ( = s / N ) Aside: The familiar ideal example Consider N measurements {i} of a quantity q with normal errors {si} Estimate q by minimization of c2, The mean of qcombined is qtrue , the SD is and The proof of this theorem is straightforward. It does not apply to our problem because of systematic errors. PHYSTAT

9. Add a systematic error to the ideal model… (for simplicity suppose bi = b ) Estimate q by minimization of c 2 ( s : systematic shift, q : observable ) and ( = s 2/N + b 2 ) Then, letting , again PHYSTAT

10. Still we do not apply the criterion Dc2 = 1 ! • Reasons • We keep the normalization factors fixed as we vary the point in parameter space. The criterion Dc2= 1 requires that the systematic shifts be continually optimized versus {am}. • Systematic errors may be nongaussian. • The published “standard deviations” bij may be inaccurate. • We trust our physics judgement instead. PHYSTAT

11. Some groups do use the criterion of 2 = 1 for PDF error analysis. Often they are using limited data sets – e.g., an experimental group using only their own data. Then the 2 = 1 criterion may underestimate the uncertainty implied by systematic differences between experiments. An interesting compendium of methods by R. Thorne PHYSTAT

12. To judge the PDF uncertainty, we return to the individual experiments. Lumping all the data together in one variable – Dc2global – is too constraining. Global analysis is a compromise. All data sets should be fit reasonably well -- that is what we check. As we vary {am}, does any experiment rule out the displacement from the standard set? PHYSTAT

13. In testing the goodness of fit, we keep the normalization factors (i.e., optimized luminosity shifts) fixed as we vary the shape parameters. End result e.g., ~100 for ~2000 data points. This does not contradict the Dc2= 1 criterion used by other groups, because that refers to a differentc2 in which the normalization factors are continually optimized as the {am} vary. PHYSTAT