Docsity
Docsity

Prepare for your exams
Prepare for your exams

Study with the several resources on Docsity


Earn points to download
Earn points to download

Earn points by helping other students or get them with a premium plan


Guidelines and tips
Guidelines and tips

The Classical Linear Regression Model (CLRM), Exams of Statistics

The Classical Linear Regression Model (CLRM). The CLRM makes the following assumptions: A-1: The regression model is linear in the parameters as in.

Typology: Exams

2021/2022

Uploaded on 09/12/2022

rechel--
rechel-- ๐Ÿ‡ฌ๐Ÿ‡ง

4.6

(10)

229 documents

1 / 3

Toggle sidebar

This page cannot be seen from the preview

Don't miss anything!

bg1
1 | Page
The Classical Linear Regression Model (CLRM)
The CLRM makes the following assumptions:
A-1: The regression model is linear in the parameters as in
๐‘Œ๐‘Œ
๐‘–๐‘–=๐ต๐ต1+๐ต๐ต2๐‘‹๐‘‹2๐‘–๐‘– +๐ต๐ต3๐‘‹๐‘‹3๐‘–๐‘– +โ‹ฏ+๐ต๐ต๐‘˜๐‘˜๐‘‹๐‘‹๐‘˜๐‘˜๐‘–๐‘– +๐‘ข๐‘ข๐‘–๐‘–
it may or may not be linear in the variables Y and the Xs.
A-2: The regressors are assumed to be fixed or nonstochastic in the sense that their
values are fixed in repeated sampling. This assumption may not be appropriate
for all economic data, but as we will show later, if X and u are independently
distributed the results based on the classical assumption discussed below hold true
provided our analysis is conditional on the particular X values drawn in the
sample. However, if X and u are uncorrelated, the classical results hold true
asymptotically (i.e.in large samples.)1
A-3: Given the values of the X variables, the expected, or mean, value of the error term
is zero. That is,2
๐ธ๐ธ(๐‘ข๐‘ข๐‘–๐‘–|X)= 0 (1.8)
where, for brevity of expression, X (the bold X) stands for all X variables in the
model. In words, the conditional expectation of the error term, given the values
of the X variables, is zero. Since the error term represents the influence of factors
that may be essentially random, it makes sense to assume that their mean or
average value is zero.
As a result of this critical assumption, we can write (1.2) as:
๐ธ๐ธ(๐‘Œ๐‘Œ
๐‘–๐‘–|X)=๐‘ฉ๐‘ฉ๐‘ฉ๐‘ฉ +๐ธ๐ธ(๐‘ข๐‘ข๐‘–๐‘–|X)
=๐‘ฉ๐‘ฉ๐‘ฉ๐‘ฉ (1.9)
which can be interpreted as the model for mean or average value of ๐‘Œ๐‘Œ
๐‘–๐‘– conditional
on the X values. This is the population (mean) regression function (PRF)
mentioned earlier. In regression analysis our main objective is to estimate this
function. If there is only one X variable, you can visualize it as the (population)
regression line. If there is more than one X variable, you will have to imagine it
to be a curve in a multi-dimensional graph. The estimated PRF, the sample
counterpart of Eq. (1.9), is denoted by ๐‘Œ๐‘Œ
๏ฟฝ๐‘–๐‘–=๐‘๐‘๐‘๐‘. That is, ๐‘Œ๐‘Œ
๏ฟฝ๐‘–๐‘–=๐‘๐‘๐‘๐‘ is an estimator of
๐ธ๐ธ(๐‘Œ๐‘Œ
๐‘–๐‘–|๐‘‹๐‘‹).
1 Note that independence implies no correlation, but no correlation does not necessarily imply
independence.
2 The vertical bar after ๐‘ข๐‘ข๐‘–๐‘– is to remind that the analysis is conditional on the given values of X.
pf3

Partial preview of the text

Download The Classical Linear Regression Model (CLRM) and more Exams Statistics in PDF only on Docsity!

The Classical Linear Regression Model (CLRM)

The CLRM makes the following assumptions:

A-1 : The regression model is linear in the parameters as in

it may or may not be linear in the variables Y and the X s.

A-2 : The regressors are assumed to be fixed or nonstochastic in the sense that their values are fixed in repeated sampling. This assumption may not be appropriate for all economic data, but as we will show later, if X and u are independently distributed the results based on the classical assumption discussed below hold true provided our analysis is conditional on the particular X values drawn in the sample. However, if X and u are uncorrelated , the classical results hold true asymptotically (i.e.in large samples.)^1

A-3 : Given the values of the X variables, the expected, or mean, value of the error term is zero. That is,^2

๐ธ๐ธ(๐‘ข๐‘ข (^) ๐‘–๐‘– |X) = 0 (1.8)

where, for brevity of expression, X (the bold X ) stands for all X variables in the model. In words, the conditional expectation of the error term, given the values of the X variables, is zero. Since the error term represents the influence of factors that may be essentially random, it makes sense to assume that their mean or average value is zero. As a result of this critical assumption, we can write (1.2) as:

๐ธ๐ธ(๐‘Œ๐‘Œ๐‘–๐‘– |X) = ๐‘ฉ๐‘ฉ๐‘ฉ๐‘ฉ + ๐ธ๐ธ(๐‘ข๐‘ข (^) ๐‘–๐‘–|X) = ๐‘ฉ๐‘ฉ๐‘ฉ๐‘ฉ (1.9)

which can be interpreted as the model for mean or average value of ๐‘Œ๐‘Œ๐‘–๐‘– conditional on the X values. This is the population (mean) regression function ( PRF ) mentioned earlier. In regression analysis our main objective is to estimate this function. If there is only one X variable, you can visualize it as the (population) regression line. If there is more than one X variable, you will have to imagine it to be a curve in a multi-dimensional graph. The estimated PRF, the sample counterpart of Eq. (1.9), is denoted by ๐‘Œ๐‘Œ๏ฟฝ๐‘–๐‘– = ๐‘๐‘๐‘๐‘. That is, ๐‘Œ๐‘Œ๏ฟฝ๐‘–๐‘– = ๐‘๐‘๐‘๐‘ is an estimator of ๐ธ๐ธ(๐‘Œ๐‘Œ๐‘–๐‘–|๐‘‹๐‘‹).

(^1) Note that independence implies no correlation, but no correlation does not necessarily imply

independence. (^2) The vertical bar after ๐‘ข๐‘ข๐‘–๐‘– is to remind that the analysis is conditional on the given values of X.

A-4 : The variance of each ๐‘ข๐‘ข (^) ๐‘–๐‘– given the values of X , is constant, or homoscedastic ( homo means equal and scedastic means variance). That is,

๐‘ฃ๐‘ฃ๐‘ฃ๐‘ฃ๐‘ฃ๐‘ฃ(๐‘ข๐‘ข (^) ๐‘–๐‘–|๐‘‹๐‘‹) = ฯƒ^2 (1.10)

Note : There is no subscript on ฯƒ^2.

A-5 : There is no correlation between two error terms. That is, there is no autocorrelation. Symbolically,

Cov (๐‘ข๐‘ข (^) ๐‘–๐‘– , ๐‘ข๐‘ข๐‘—๐‘— | X ) =0 i โ‰  j (1.11)

where Cov stands for covariance and i and j are two different error terms. Of course, if I = j , Eq. (1.11) will give the variance of ๐‘ข๐‘ข (^) ๐‘–๐‘– given in Eq. (1.10).

A-6 : There are no perfect linear relationships among the X variables. This is the assumption of no multicollinearity. For example, relationships like ๐‘‹๐‘‹ 5 = 2๐‘‹๐‘‹ 3 + 4 ๐‘‹๐‘‹ 4 are ruled out.

A-7 : The regression model is correctly specified. Alternatively, there is no specification bias or specification error in the model used in empirical analysis. It is implicitly assumed that the number of observations, n , is greater than the number of parameters estimated. Although it is not a part of the CLRM, it is assumed that the error term follows the normal distribution with zero mean and (constant) variance ๐œŽ๐œŽ 2. Symbolically,

A-8 : ๐‘ข๐‘ข (^) ๐‘–๐‘– ~ N (0, ๐œŽ๐œŽ 2 ) (1.12)

On the basis of Assumptions A-1 to A-7, it can be shown that the method of ordinary least squares ( OLS ), the method most popularly used in practice, provides estimators of the parameters of the PRF that have several desirable statistical properties , such as:

1- The estimators are linear, that is, they are linear functions of the dependent variable Y****. Linear estimators are easy to understand and deal with compared to nonlinear estimators.

2- The estimators are unbiased, that is, in repeated applications of the method, on average, the estimators are equal to their true values.

3- In the class of linear unbiased estimators, OLS estimators have minimum variance. As a result, the true parameter values can be estimated with least possible uncertainty; an unbiased estimator with the least variance is called an efficient estimator.

In short, under the assumed conditions, OLS estimators are BLUE : best linear unbiased estimators. This is the essence of the well-known Gaussโ€“Markov theorem , which provides a theoretical justification for the method of least squares.