Study with the several resources on Docsity
Earn points by helping other students or get them with a premium plan
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Community
Ask the community for help and clear up your study doubts
Discover the best universities in your country according to Docsity users
Free resources
Download our free guides on studying techniques, anxiety management strategies, and thesis advice from Docsity tutors
Material Type: Notes; Professor: Sickles; Class: APPLIED ECONOMETRICS; Subject: Economics; University: Rice University; Term: Spring 2009;
Typology: Study notes
1 / 28
This page cannot be seen from the preview
Don't miss anything!
Chapter 4 Properties of the Least Squares Estimators
Assumptions of the Simple Linear Regression Model SR1.
1
2
t^
t^
t
y
x
e
= β + β
) t
E e
1
2
) t
t
y
x
β + β
2
var(
var(
t^
t
e
y
= σ
cov(
cov(
i^
j^
i^
j
e e
y
y
t x
is not random and takes at least two values
t
2
t e
σ
t y
x
2
1
2
β
σ
( optional
Undergraduate Econometrics, 2
nd Edition –Chapter 4
The Least Squares Estimators as Random Variables
The least squares
estimator
b
2
of the slope parameter
β
2
, based on a sample of
observations, is
2
2
2 t^
t^
t^
t
t^
t
x y
x
y
b
x
x
(3.3.8a)
The least squares
estimator
b
1
of the intercept parameter
β
1
is
1
2
Slide 4.
Undergraduate Econometrics, 2
nd Edition –Chapter 4
b
y
b x −
(3.3.8b)
where
and
t^
t
y
y
x
x
are the sample means of the observations on
y
and
x
, respectively.
The Sampling Properties of the Least Squares Estimators
The Expected Values of
b
1
and
b
2
We begin by rewriting the formula in equation 3.3.8a into the following one that ismore convenient for theoretical
purposes, 2
2
t^
t
b
w e
Undergraduate Econometrics, 2
nd Edition –Chapter 4
β +
where
w
t^
is a constant (non-random) given by
2
t
t
t x
x
w
x
x −
The expected value of a sum is the sum of the expected values (see Chapter 2.5.1):
2
2
2
2
2
[since
t^
t^
t^
t
t^
t^
t
E b
w e
E w e
w E e
E e
β +
β
β +
= β
Undergraduate Econometrics, 2
nd Edition –Chapter 4
4.2.1a
The Repeated Sampling Context
Table 4.1 contains least squares estimates of the food expenditure model from 10 randomsamples of size
40 from the same population
Table 4.
Least Squares Estimates from
10 Random Samples of size
n
b
1
b
2
b
2
in deviation from the mean form is:
2
2
t^
t
t
x
x
y
y
b
x
x
Recall that
t x
x −
Then, the formula for
b
2
becomes
2
2
2
2
2
t^
t^
t^
t^
t
t^
t
t^
t
t
t^
t^
t
t^
t
x
x
y
y
x
x y
y
x
x
b
x
x
x
x
x
x y
x
x
y
w y
x
x
x
x
where
w
t^
is the constant given in equation 4.2.2.
Undergraduate Econometrics, 2
nd Edition –Chapter 4
To obtain equation 4.2.1, replace
y
t^
by
t^
t
1
2
t y
x
e
β + β
b
w y
w
x
e
w
w x
β + β
= β
and simplify:
2
1
2
1
2
t^
t^
t^
t^
t^
t^
t^
t^
t^
t
w e
(4.2.9a)
t w
, this eliminates the term
1
t w
β
t^
t
w x
, so
2
= β
, and (4.2.9a) simplifies to equation 4.2.
2
t^
t
w x
β
b
2
2
t^
t
w e
Undergraduate Econometrics, 2
nd Edition –Chapter 4
β +
(4.2.9b)
2
t^
t^
t^
t
t^
t
t^
t^
t
x
x x
x
x x
w x
x
x
x
x x
The Variances and Covariance of
b
1
and
b
2
2
2
2
2
var(
b
E b
E b
If the regression model assumptions SR1-SR5 are correct
(SR6 is not required),
then the
variances and covariance of
b
1
and
b
2
are:
Undergraduate Econometrics, 2
nd Edition –Chapter 4
2
2
1
2
2
2
2
2
1
2
2
var(
var(
cov(
t t
t
t
x
b
x
x
b
x
x
x
b b
x
x
= σ
σ
= σ
Undergraduate Econometrics, 2
nd Edition –Chapter 4
Deriving the variance of
b
The starting point is equation 4.2.1.
2
2
2
2
2
2
var(
var
var
[since
is a constant]
var(
[using cov(
t^
t^
t^
t
t^
t^
i^
j
t
b
w e
w e
w
e
e e
w
β +
β
σ
2
2
2
[using var(
t
t
e
x
x
= σ
σ
The very last step uses the fact that
2
2
2
2
2
t
t
t
t x
x
w
x
x
x
x
Undergraduate Econometrics, 2
nd Edition –Chapter 4
Linear Estimators
Slide 4.
Undergraduate Econometrics, 2
nd Edition –Chapter 4
b
w y
The least squares estimator
b
2
is a weighted sum of the observations
y
, t
t
2
t
Estimators like
b
, that are linear combinations of an observable random variable, 2
linear estimators 4.
The Gauss-Markov Theorem Gauss-Markov Theorem:
Under the assumptions SR1-SR5 of the linear
regression model the estimators
b
1
and
b
2
have the
smallest variance of all
linear and unbiased estimators
of
β
1
and
β
They are the
est
inear
nbiased
stimators (BLUE) of
β
1
and
β
2
Proof of the Gauss-Markov Theorem:
Undergraduate Econometrics, 2
nd Edition –Chapter 4
k
c
c x
w
c
e
β + β
β +
β
= β
β
β
β
= β
Let
y
(where the
k
t^
t
b
k
t^
are constants) be any other linear estimator of
β
Suppose that
t c
, where
c
t^
t^
t^
is another constant and
w
t^ is given in equation 4.2.2.
Into this new estimator substitute
y
t^
and simplify, using the properties of
w
t^
in equation
1
2
1
2
1
1
2
2
1
2
2 (^
t^
t^
t^
t^
t^
t^
t^
t^
t
t^
t^
t^
t^
t^
t^
t^
t
t^
t^
t^
t^
t^
t^
t^
t^
t
t^
t^
t^
t^
t^
t
b
k y
w
c
y
w
c
x
e
w
c
w
c
x
w
c
e
w
c
w x
c x
w
c
e
since
w
t^
= 0 and
w
t^
x
t^
1
2
2
1
2
2
t^
t^
t^
t^
t^
t
t^
t^
t
E b
c
c x
w
c E e
c
c x
= β
= β
In order for the linear estimator
y
t^
t
b
k
to be unbiased it must be true that
0 and
t^
t^
t x
c
c
Slide 4.
Undergraduate Econometrics, 2
nd Edition –Chapter 4
These conditions must hold in order for
y
t^
t
b
k
to be in the class of
linear
and
unbiased estimators.
Use the properties of variance to obtain:
2
2
2
2
2
2
2
2
2
2
2
2
2
var(
var
) var(
var(
var(
) since
t^
t^
t^
t^
t^
t
t^
t^
t^
t
t
t
b
w
c
e
w
c
e
w
c
w
c
b
c
b
c
β +
= σ
= σ
Undergraduate Econometrics, 2
nd Edition –Chapter 4
The Probability Distribution of the Least Squares Estimators
If
we make the normality assumption, assumption SR6 about the error term, then the least squares estimators are normally distributed.
2
2
1
1
2
2
2
2
2
t ) t t
x
b
x
x
b
x
x
σ
β ⎜
σ
β ⎜
If assumptions SR1-SR5 hold, and if the sample size
is
sufficiently large
, then
the least squares estimators have a distribution that approximates the normaldistributions shown in equation 4.4.
Undergraduate Econometrics, 2
nd Edition –Chapter 4
