Download Regression Analysis - Econometrics - Lecture Notes and more Study notes Econometrics and Mathematical Economics in PDF only on Docsity!
Lecture 01
The Regression Analysis
The figure shows a scatter diagram showing the relationship between two variables X & Y. It is obvious that the two variables have some relationship as the higher values of X are associated with higher values of Y, however it is also visible that the relationship cannot be modeled by a deterministic function of X.
Therefore we assume that the relationship between X and Y is sum of deterministic and stochastic relationship which can be written as follows: 𝑦𝑖 = 𝛼 + 𝛽𝑥𝑖 + 𝜀𝑖 Where 𝛼 + 𝛽𝑥𝑖 is deterministic and𝜀𝑖 is stochastic part. The deterministic part we hope to be able to predict, however the stochastic part cannot be predicted.
Obviously, it is desirable to have minimum of the stochastic i.e. unpredictable part. If we ignore the stochastic part we get: 𝛼 + 𝛽𝑥𝑖 = 𝑦� This is equation of straight line. This straight lines joins the values on X axis with corresponding predictions on Y axis. We want such a straight line which minimizes the unpredicted part. Instead of focusing on any one 𝜀𝑖 we want to minimize the aggregate error. This is a problem which should be solvable by calculus (differentiation]. Let us see how it can be done.
Minimizing aggregate error: We have 𝑦𝑖 − 𝛼 − 𝛽𝑥𝑖 = 𝜀𝑖 There are three options to minimize the aggregate error:
Option I: minimize ∑ 𝜺 (^) 𝒊 This option is not appropriate because 𝜀𝑖 = 𝑦𝑖 − 𝑦�𝑖 Here 𝑦�𝑖 is the prediction of 𝑦𝑖 lying on the straight line. The actual 𝑦𝑖 are on both sides of straight line. If 𝑦𝑖 is above straight line => actual value of 𝑦𝑖 is greater than 𝑦�𝑖 => 𝜀𝑖 is positive. On the other hand, if 𝑦𝑖 is below the straight line => actual value of 𝑦𝑖 is smaller than 𝑦�𝑖 => 𝜀𝑖 is negative. These negative and positive values cancel each other and total result we get is zero.
Option III: minimize ∑ 𝜺 (^) 𝒊𝟐 Here the problem posed by the sign of errors is solved by taking squares. The expression is now differentiable and can give us the analytical solution. Let us see how this can be done. Remember the term ∑ 𝜀𝑖^2 is called sum of squared residual (RSS). (Residual is sample counterpart of error, there is difference between residual which will be explained later, so far assume that residual and errors are the same thing). So
𝑅𝑆𝑆 = � 𝜀𝑖^2 = �(𝑦𝑖 − 𝛼 − 𝛽𝑥𝑖)^2
We want to minimize RSS by changing 𝛼 and 𝛽, therefore: Differentiating w.r.t. 𝛼, we get 𝜕𝑅𝑆𝑆 𝜕𝛼 =^ −^2 �(𝑦𝑖^ − 𝛼 − 𝛽𝑥𝑖) = 0 ∑ 𝑦𝑖 − 𝑛𝑎 − ∑ 𝛽𝑥𝑖 = 0 𝑛𝛼 = ∑ 𝑦𝑖 − ∑ 𝛽𝑥𝑖 Similarly, differentiating with respect to 𝛽 we get: 𝜕𝑅𝑆𝑆 𝜕𝛽 =^ −^2 �(𝑦𝑖^ − 𝛼 − 𝛽𝑥𝑖)𝑥𝑖^ = 0 ∑ 𝑦𝑖 𝑥𝑖 − 𝑎 ∑ 𝑥𝑖 − 𝛽 ∑ 𝑥𝑖^2 = 0 ∑ 𝑦𝑖 𝑥𝑖 − (^1) 𝑛 (∑ 𝑦𝑖 − ∑ 𝛽𝑥𝑖) ∑ 𝑥𝑖 − 𝛽 ∑ 𝑥𝑖^2 = 0 𝑛 ∑ 𝑦𝑖 𝑥𝑖 − ∑ 𝑥𝑖 ∑ 𝑦𝑖 + 𝛽(∑ 𝑥𝑖)^2 − 𝑛𝛽 ∑ 𝑥𝑖^2 = 0 𝛽̂ = 𝑛 ∑ 𝑦−(∑ 𝑥^ 𝑖^ 𝑥 (^) 𝑖^ 𝑖)^ −∑ 𝑥 (^2) +𝑛 ∑ 𝑥^ 𝑖^ ∑ 𝑦 (^) 𝑖 2 𝑖 Here 𝛽̂ is used to denote estimate from the sample. Using this value, one can find 𝛼 as well.
𝛼� =^1 𝑛 �� 𝑦𝑖 − 𝛽̂ � 𝑥𝑖 �
Example 1: Given the data X Y 10 8 12 7 14 13 16 14 18 15 20 19 Estimate the regression equation 𝑦𝑖 = 𝛼 + 𝛽𝑥𝑖 + 𝜀𝑖 Solution: X Y XY X^ 10 8 80 100 12 7 84 144 14 13 182 196 16 14 224 256 18 15 270 324 20 19 380 400 ∑ (^) 𝑥𝑖 = 90 ∑ (^) 𝑦𝑖 = 76 ∑ (^) 𝑥𝑖 𝑦𝑖 = 1220 ∑ (^) 𝑥𝑖^2 = 1420
𝛽̂ = 𝑛 ∑ 𝑦−(∑ 𝑥𝑖^ 𝑥𝑖^ − ∑ 𝑥𝑖^ ∑ 𝑦𝑖
𝑖)^2 +^ 𝑛 ∑ 𝑥𝑖^2
= 7 × 12207 × 1420^ − −^ 90 × 76 1420 =^17001840
Now
where actual value of y should lie, but we will learn this technique later. Exercise Three data sets are given to you. For the first data set:
- Enter the data in Excel spreadsheet
- Make a scatter diagram and comment what type of regression do you expect? What are rough guess about slope and intercept of the each of the data?
- Calculate the regression equation using the formulae given above and write results.
- Summarize the information present in the regression equation; take help of example 1 in this regard.
- Predict value of Y when X=
- Repeat the exercise for second and third data sets
Set 1 Set 2 Set 3
X Y X Y X Y
4 7 6 -3 7 2 1 4 7 -4 3 2 7 9 2 1 5 3 5 9 3 0 2 1 5 6 2 -1 1 4 1 3 7 -3 6 6 3 5 1 1 7 3 7 11 3 1 5 5 4 8 7 -6 6 1 1 2 2 2 2 7 7 8 1 2 4 1 1 3 2 0 2 5 2 6 3 -1 4 7 4 8 1 0 6 5 6 10 1 2 1 3
