Download Asymptotic Distribution - Statistical Inference - Exam and more Exams Statistics in PDF only on Docsity!
LANCASTER UNIVERSITY
2011 EXAMINATIONS
PART II (Third year)
MATHEMATICS & STATISTICS 2 hours
MATH331: Statistical Inference
You should answer all Section A questions and TWO Section B questions. In Section A there are questions worth a total of 50 marks, but the maximum mark that you can gain there is capped at 40. Note there is a formula sheet at the back of this exam. Throughout the paper “MLE” will stand for “maximum likelihood estimator”.
The following table of quantiles of a χ^2 d may be required.
d 95% quantile 97.5% quantile 1 3.8 5. 2 6.0 7. 3 7.8 9.
SECTION A
A1. (a) State the asymptotic distribution of the maximum likelihood estimator for a model with a single parameter θ. [2] (b) Indicate how the result in Part (a) can be used to find an approximate 95% con- fidence interval (CI), mentioning a drawback of this method when the likelihood function is asymmetric. [3] (c) Define the deviance for the same parameter and state the distribution of its deviance under the null hypothesis as n → ∞. [3] (d) How would a deviance based 95% CI would be calculated? (Plot the deviance function together with the asocciated C.I.) [5]
please turn over
SECTION A continued
A2. Let X 1 ,... , Xn be independent with Xi ∼ N (α + βzi, 1) for known constants {zi}. (a) Calculate the log-likelihood (α, β). [3] (b) Show that the MLEs, (ˆα, βˆ), satisfy the equation ¯x = ˆα + βˆ z,¯ where ¯x and ¯z and the sample means of x and z respectively. [3] (c) Show that Fisher’s information matrix IE (α, β) can be expressed as
IE (α, β) =
n ∑ni=1 zi ∑n i=1 zi
∑n i=1 z^2 i
[4]
(d) Give a condition which leads to the orthogonality of α and β. (^) [3]
A3. An observation x is taken from a binomial distribution X ∼ Binomial (n, π). The probability of success, π, is allocated a beta prior π ∼ Beta (α, β). (a) Find and identify the posterior distribution of π. [3] (b) Define the marginal likelihood m(x) and show that for the likelihood and prior in (a) that m(x) =
n x
B(α + x, β + n − x) B(α, β) , where B(α, β) = Γ( Γ(αα)Γ(+ββ)). [5] (c) Why is the marginal likelihood a useful quantity for Bayesian statisticians? [2]
A4. (^) (a) Define Jeffreys’ prior and explain briefly why it is often preferred over other methods for specifying prior ignorance. [4] (b) Calculate the Jeffrey’s prior in the case where X 1 ,... , Xn are independently distributed with a Geometric (θ) distribution, and determine the corresponding posterior distribution for θ. Note that the probability mass function for the geometric distribution is given by f (x | θ) = (1 − θ)x−^1 θ x = 1, 2 ,... and E(X) = (^1) θ. [6] (c) This is an improper distribution. Explain what this means and why this is the case. [4]
please turn over
SECTION B continued
B2. (a) Give one advantage and one disadvantage of using a conjugate family of prior distributions in a Bayesian analysis. [4] (b) The variables Y 1 ,... , Yn are independently distributed as Yi ∼ Poisson (xiθ), where x 1 ,... , xn are known constants. The prior distribution for θ is chosen to be Gamma(p, q). (i) Show that the posterior distribution of θ|y, x is Gamma(P, Q) where P = p +
∑^ n i=
yi, Q = q +
∑^ n i=
xi,
and y = y 1 ,... , yn, x = x 1 ,... , xn.
[6]
(ii) Show that the posterior mean, E[θ|x, y], satisfies
E[θ|x, y] = γμ 0 + (1 − γ)
∑ (^) y ∑ i xi^ , where μ 0 is the prior mean and γ is a constant to be found. [4] (iii) Give an interpretation of the result in (ii) and find under what conditions the posterior mean will be greater than the prior mean. [5]
(c) We assume that the data, X 1 ,... , Xn are independent Normal variables such that Xi|θ ∼ Normal
θ, σ^2
, i = 1,... , n, (i) Assume θ is assumed known and the unknown precision, τ = (^) σ^12 , has the prior distribution τ ∼ Gamma (α, β). Show that the posterior distribution of τ can be written as τ |x 1 ,... , xn ∼ Gamma (A, D) and find the quantities A and D. [5] (ii) Use the same likelihood but this time assume σ^2 known and θ unknown with prior θ ∼ Normal
μ, σ^2
. The posterior distribution of θ can be shown to be θ | x 1 ,... , xn ∼ Normal
μp, ψ^2
where ψ^2 is the posterior variance and μp is the posterior mean. Let Y be a future observation from the same likelihood. Show that the predictive. distribution of Y can be written as Y |x 1 ,... , xn ∼ Normal
μp, σ^2 + ψ^2
State all necessary assumptions. [6] please turn over
SECTION B continued
B3. (a) (From Antleman, 1997). Suppose that a trucking company owns a large fleet of well-maintained trucks and assume that breakdowns appear to occur at random times. The president of the company is interested in learning about the daily rate R at which breakdowns occur. For a given value of the rate parameter R, it is known that the number of breakdowns y on a particular day has a Poisson distribution with mean R. (i) Suppose that one observes the number of truck breakdowns for n consec- utive days and denote them by y 1 ,... , yn. If one assumes that these are independent measurements, derive an expression for the likelihood of these observations y 1 ,... , yn. [2] (ii) Find the MLE of R and its associated asymptotic standard error. [5] (b) Suppose that an employee of the trucking firm wishes to learn about the break- down rate. She is uncertain about the value of R and so assigns the non infor- mative prior density: p(R) ∝ (^) R^1 , R > 0 (i) Show that the posterior distribution of R is from the gamma family and show that the posterior mean of R is the same as the MLE. [3] (ii) Explain in what way the interpretation of a classical confidence interval is different to the interpretation of a Bayesian credible interval. [4] (c) The president has some knowledge about the location of the Poisson rate pa- rameter R based on the observed number of breakdowns from previous years. His prior beliefs about R are represented by means of the gamma density
g(R) ∝ R^4 −^1 exp(− 2 R) R > 0.
(i) Find the mean, variance and mode of the president’s prior knowledge about the rate parameter, R. Draw a rough sketch of these beliefs. [4] (ii) Using this prior, find the posterior density of R. [2] (iii) The sample mean over n = 12 observations is ¯y = 2. Explain a difference and a similarity of this posterior distribution from the one derived in part (b) under a non-informative prior. Interpret this result. [5]
Question B3 is continued over the page
please turn over
Formula Sheet
You may use the following:
∗ A univariate Normal with mean μ ∈ IR and variance σ^2 , is denoted by Normal
μ, σ^2
and the corresponding density function is:
p (y | μ, σ) = √^1 2 πσ^2 exp
− (^21) σ 2 (y − μ)^2
for y ∈ R.
In terms of the precision τ = (^) σ^12 the univariate normal can be expressed as
p (y | μ, τ ) = τ^
(^12) √ 2 π exp
− τ 2 (y − μ)^2
for y ∈ R.
∗ A Gamma distribution with shape parameter α > 0 and rate parameter β > 0 is denoted by Gamma (α, β), and the corresponding density is:
p(y | α, β) = β
α Γ(α) y
α− (^1) exp(−βy) for y > 0.
The Gamma distribution, Gamma (α, β), has mean α β and variance (^) βα 2.
∗ An Exponential distribution with rate parameter λ > 0 is denoted by Exp (λ) and the corresponding density is:
p(y|λ) = λ exp(−λy) for y > 0.
∗ A Beta distribution with parameters α > 0 and β > 0 is denoted by Beta (α, β), and the corresponding density is:
p(y | α, β) = (^) B(α, β^1 ) yα−^1 (1 − y)β−^1 for 0 < y < 1.
where B(α, β) = Γ( Γ(αα)Γ(+ββ)) where Γ(n + 1) = n! for integer n. The Beta distribution, Beta (α, β), has mean (^) αα+β.
∗ A Binomial distribution with parameter 0 ≤ p ≤ 1 is denoted by Binomial (n, p), and the corresponding probability mass function is:
p(y | n, p) =
n y
py(1 − p)n−y^ for y = 0, 1 ,... , n.
∗ A Poisson distribution with parameter λ > 0 is denoted by Poisson (λ), and the corresponding probability mass function is:
p(y | λ) = e
−λ y! λ
y (^) for y = 0, 1 ,....
The mean and variance of a Poisson distribution is λ.
∗ A Negative binomial distribution with parameters 0 ≤ θ ≤ 1 and k ∈ { 1 ,.. .} is denoted by Negative-Binomial (θ, k), and the corresponding probability mass function is:
p(y|θ, k) =
y + k − 1 k − 1
θk(1 − θ)y, y = 0, 1 , 2 ,...
