1 / 35

STATISTICAL INFERENCE PART III

STATISTICAL INFERENCE PART III. EXPONENTIAL FAMILY, LOCATION AND SCALE PARAMETERS. EXPONENTIAL CLASS OF PDFS. X is a continuous (discrete) rv with pdf f(x; ),  . If the pdf can be written in the following form.

rfredette
Download Presentation

STATISTICAL INFERENCE PART III

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. STATISTICAL INFERENCEPART III EXPONENTIAL FAMILY, LOCATION AND SCALE PARAMETERS

  2. EXPONENTIAL CLASS OF PDFS • X is a continuous (discrete) rv with pdf f(x;), . If the pdf can be written in the following form then, the pdf is a member of exponential class of pdfs of the continuous (discrete) type. (Here, k is the number of parameters)

  3. REGULAR CASE OF THE EXPONENTIAL CLASS OF PDFS • We have a regular case of the exponential class of pdfs of the continuous type if • Range of X does not depend on. • c() ≥ 0, w1(),…, wk() are real valued functions of  for . • h(x) ≥ 0, t1(x),…, tk(x) are real valued functions of x. If the range of X depends on , then it is called irregular exponential class or range-dependent exponential class.

  4. EXAMPLES X~Bin(n,p), where n is known. Is this pdf a member of exponential class of pdfs? Why? Binomial family is a member of exponential family of distributions.

  5. EXAMPLES X~Cauchy(1,). Is this pdf a member of exponential class of pdfs? Why? Cauchy is not a member of exponential family.

  6. REGULAR CASE OF THE EXPONENTIAL CLASS OF PDFS • Random Sample from Regular Exponential Class is a css for . If Y is an UE of , Yis the MVUE of .

  7. EXAMPLES Recall: X~Bin(n,p), where n is known. This family is a member of exponential family of distributions. is a CSS for p. is UE of p. is MVUE of p.

  8. EXAMPLES X~N(,2) where both  and 2 is unknown. Find a css for  and 2 .

  9. FISHER INFORMATION AND INFORMATION CRITERIA • X, f(x;), , xA (not depend on ). Definitions and notations:

  10. FISHER INFORMATION AND INFORMATION CRITERIA The Fisher Information in a random variable X: The Fisher Information in the random sample: Let’s prove the equalities above.

  11. FISHER INFORMATION AND INFORMATION CRITERIA

  12. FISHER INFORMATION AND INFORMATION CRITERIA

  13. FISHER INFORMATION AND INFORMATION CRITERIA The Fisher Information in a random variable X: The Fisher Information in the random sample: Proof of the last equality is available on Casella & Berger (1990), pg. 310-311.

  14. CRAMER-RAO LOWER BOUND (CRLB) • Let X1,X2,…,Xnbe sample random variables. • Range of X does not depend on . • Y=U(X1,X2,…,Xn): a statistic not containing . • Let E(Y)=m(). • Z=′(x1,x2,…,xn;) is a r.v. • E(Z)=0 and V(Z)=In() (from previous slides). • Let prove this!

  15. CRAMER-RAO LOWER BOUND (CRLB) • Cov(Y,Z)=E(YZ)-E(Y)E(Z)=E(YZ)

  16. CRAMER-RAO LOWER BOUND (CRLB) • E(Y.Z)=m’() • -1Corr(Y,Z)1 • 0 Corr(Y,Z)21  The Cramer-Rao Inequality (Information Inequality)

  17. CRAMER-RAO LOWER BOUND (CRLB) • CRLB is the lower bound for the variance of the unbiased estimator of m(). • When V(Y)=CRLB, Y is the MVUE of m(). • For a r.s., remember that In()=n I(), so,

  18. EFFICIENT ESTIMATOR • T is an efficient estimator (EE) of  if • T is UE of , and, • Var(T)=CRLB • Y is an efficient estimator (EE) of its expectation, m(), if its variance reaches the CRLB. • An EE of m() may not exist. • The EE of m(), if exists, is unique. • The EE of m() is the unique MVUE of m().

  19. ASYMPTOTIC EFFICIENT ESTIMATOR • Y is an asymptotic EE of m() if

  20. EXAMPLES A r.s. of size n from X~Poi(θ). • Find CRLB for any UE of θ. • Find UMVUE of θ. • Find an EE for θ. • Find CRLB for any UE of exp{-2θ}. Assume n=1, and show that is UMVUE of exp{-2θ}. Is this a reasonable estimator?

  21. EXAMPLE A r.s. of size n from X~Exp(). Find UMVUE of , if exists.

  22. LIMITING DISTRIBUTION OF MLEs • : MLE of  • X1,X2,…,Xnis a random sample.

  23. EE of m()= LIMITING DISTRIBUTION OF MLEs • Let be MLEs of 1, 2,…, m. • If Y is an EE of , then Z=a+bY is an EE of a+bm() where a and b are constants.

  24. LOCATION PARAMETER • Let f(x) be any pdf. The family of pdfs f(x) indexed by parameter  is called the location family with standard pdf f(x) and  is the location parameter for the family. • Equivalently,  is a location parameter for f(x) iff the distribution of X does not depend on .

  25. Example • If X~N(θ,1), then X-θ~N(0,1)  distribution is independent of θ.  θ is a location parameter. • If X~N(0,θ), then X-θ~N(-θ,θ)  distribution is NOT independent of θ.  θ is NOT a location parameter.

  26. LOCATION PARAMETER • Let X1,X2,…,Xnbe a r.s. of a distribution with pdf (or pmf); f(x; ); . An estimator t(x1,…,xn) is defined to be a location equivariantiff t(x1+c,…,xn+c)= t(x1,…,xn) +c for all values of x1,…,xnand a constant c. • t(x1,…,xn) is location invariantiff t(x1+c,…,xn+c)= t(x1,…,xn) for all values of x1,…,xnand a constant c. Invariant = does not change

  27. Example • Is location invariant or equivariant estimator? • Let t(x1,…,xn) = . Then, t(x1+c,…,xn+c)= (x1+c+…+xn+c)/n = (x1+…+xn+nc)/n = +c = t(x1,…,xn) +c  location equivariant

  28. Example • Is s² location invariant or equivariant estimator? • Let t(x1,…,xn) = s²= • Then, t(x1+c,…,xn+c)= =t(x1,…,xn) Location invariant (x1,…,xn) and (x1+c,…,xn+c) are located at different points on real line, but spread among the sample values is same for both samples.

  29. SCALE PARAMETER • Let f(x) be any pdf. The family of pdfs f(x/)/ for >0, indexed by parameter , is called the scale family with standard pdf f(x) and  is the scale parameter for the family. • Equivalently, is a scale parameter for f(x) iff the distribution of X/ does not depend on .

  30. Example • Let X~Exp(θ). Let Y=X/θ. • You can show that f(y)=exp(-y) for y>0 • Distribution is free of θ • θ is scale parameter.

  31. SCALE PARAMETER • Let X1,X2,…,Xnbe a r.s. of a distribution with pdf (or pmf); f(x; ); . An estimator t(x1,…,xn) is defined to be a scale equivariantiff t(cx1,…,cxn)= ct(x1,…,xn) for all values of x1,…,xnand a constant c>0. • t(x1,…,xn) is scale invariantiff t(cx1,…,cxn)= t(x1,…,xn) for all values of x1,…,xnand a constant c>0.

  32. Example • Is scale invariant or equivariant estimator? • Let t(x1,…,xn) = . Then, t(cx1,…,cxn)= c(x1+…+xn)/n = c = ct(x1,…,xn)  Scale equivariant

  33. LOATION-SCALE PARAMETER • Let f(x) be any pdf. The family of pdfs f((x) /)/ for >0, indexed by parameter (,), is called the location-scale family with standard pdf f(x) and  is a location parameter and  is the scale parameter for the family. • Equivalently,  is a location parameter and is a scale parameter for f(x) iff the distribution of (X)/ does not depend on  and.

  34. Example 1. X~N(μ,σ²). Then, Y=(X- μ)/σ ~ N(0,1) • Distribution is independent of μ and σ² • μ and σ² are location-scale paramaters 2. X~Cauchy(θ,β). You can show that the p.d.f. of Y=(X- β)/ θ is f(y) = 1/(π(1+y²))  β and θ are location-and-scale parameters.

  35. LOCATION-SCALE PARAMETER • Let X1,X2,…,Xnbe a r.s. of a distribution with pdf (or pmf); f(x; ); . An estimator t(x1,…,xn) is defined to be a location-scale equivariant iff t(cx1+d,…,cxn+d)= ct(x1,…,xn)+d for all values of x1,…,xnand a constant c>0. • t(x1,…,xn) is location-scale invariant iff t(cx1+d,…,cxn+d)= t(x1,…,xn) for all values of x1,…,xnand a constant c>0.

More Related