



Study with the several resources on Docsity
Earn points by helping other students or get them with a premium plan
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Community
Ask the community for help and clear up your study doubts
Discover the best universities in your country according to Docsity users
Free resources
Download our free guides on studying techniques, anxiety management strategies, and thesis advice from Docsity tutors
Regression Analysis, Minimizing aggregate error, Estimated regression equation, Three data sets, Slope and intercept, Slope coefficient, Deterministic and stochastic relationship are points you can learn about Econometric in this lecture.
Typology: Study notes
1 / 7
This page cannot be seen from the preview
Don't miss anything!
The figure shows a scatter diagram showing the relationship between two variables X & Y. It is obvious that the two variables have some relationship as the higher values of X are associated with higher values of Y, however it is also visible that the relationship cannot be modeled by a deterministic function of X.
Therefore we assume that the relationship between X and Y is sum of deterministic and stochastic relationship which can be written as follows: 𝑦𝑖 = 𝛼 + 𝛽𝑥𝑖 + 𝜀𝑖 Where 𝛼 + 𝛽𝑥𝑖 is deterministic and𝜀𝑖 is stochastic part. The deterministic part we hope to be able to predict, however the stochastic part cannot be predicted.
Obviously, it is desirable to have minimum of the stochastic i.e. unpredictable part. If we ignore the stochastic part we get: 𝛼 + 𝛽𝑥𝑖 = 𝑦� This is equation of straight line. This straight lines joins the values on X axis with corresponding predictions on Y axis. We want such a straight line which minimizes the unpredicted part. Instead of focusing on any one 𝜀𝑖 we want to minimize the aggregate error. This is a problem which should be solvable by calculus (differentiation]. Let us see how it can be done.
Minimizing aggregate error: We have 𝑦𝑖 − 𝛼 − 𝛽𝑥𝑖 = 𝜀𝑖 There are three options to minimize the aggregate error:
Option I: minimize ∑ 𝜺 (^) 𝒊 This option is not appropriate because 𝜀𝑖 = 𝑦𝑖 − 𝑦�𝑖 Here 𝑦�𝑖 is the prediction of 𝑦𝑖 lying on the straight line. The actual 𝑦𝑖 are on both sides of straight line. If 𝑦𝑖 is above straight line => actual value of 𝑦𝑖 is greater than 𝑦�𝑖 => 𝜀𝑖 is positive. On the other hand, if 𝑦𝑖 is below the straight line => actual value of 𝑦𝑖 is smaller than 𝑦�𝑖 => 𝜀𝑖 is negative. These negative and positive values cancel each other and total result we get is zero.
Option III: minimize ∑ 𝜺 (^) 𝒊𝟐 Here the problem posed by the sign of errors is solved by taking squares. The expression is now differentiable and can give us the analytical solution. Let us see how this can be done. Remember the term ∑ 𝜀𝑖^2 is called sum of squared residual (RSS). (Residual is sample counterpart of error, there is difference between residual which will be explained later, so far assume that residual and errors are the same thing). So
𝑅𝑆𝑆 = � 𝜀𝑖^2 = �(𝑦𝑖 − 𝛼 − 𝛽𝑥𝑖)^2
We want to minimize RSS by changing 𝛼 and 𝛽, therefore: Differentiating w.r.t. 𝛼, we get 𝜕𝑅𝑆𝑆 𝜕𝛼 =^ −^2 �(𝑦𝑖^ − 𝛼 − 𝛽𝑥𝑖) = 0 ∑ 𝑦𝑖 − 𝑛𝑎 − ∑ 𝛽𝑥𝑖 = 0 𝑛𝛼 = ∑ 𝑦𝑖 − ∑ 𝛽𝑥𝑖 Similarly, differentiating with respect to 𝛽 we get: 𝜕𝑅𝑆𝑆 𝜕𝛽 =^ −^2 �(𝑦𝑖^ − 𝛼 − 𝛽𝑥𝑖)𝑥𝑖^ = 0 ∑ 𝑦𝑖 𝑥𝑖 − 𝑎 ∑ 𝑥𝑖 − 𝛽 ∑ 𝑥𝑖^2 = 0 ∑ 𝑦𝑖 𝑥𝑖 − (^1) 𝑛 (∑ 𝑦𝑖 − ∑ 𝛽𝑥𝑖) ∑ 𝑥𝑖 − 𝛽 ∑ 𝑥𝑖^2 = 0 𝑛 ∑ 𝑦𝑖 𝑥𝑖 − ∑ 𝑥𝑖 ∑ 𝑦𝑖 + 𝛽(∑ 𝑥𝑖)^2 − 𝑛𝛽 ∑ 𝑥𝑖^2 = 0 𝛽̂ = 𝑛 ∑ 𝑦−(∑ 𝑥^ 𝑖^ 𝑥 (^) 𝑖^ 𝑖)^ −∑ 𝑥 (^2) +𝑛 ∑ 𝑥^ 𝑖^ ∑ 𝑦 (^) 𝑖 2 𝑖 Here 𝛽̂ is used to denote estimate from the sample. Using this value, one can find 𝛼 as well.
Example 1: Given the data X Y 10 8 12 7 14 13 16 14 18 15 20 19 Estimate the regression equation 𝑦𝑖 = 𝛼 + 𝛽𝑥𝑖 + 𝜀𝑖 Solution: X Y XY X^ 10 8 80 100 12 7 84 144 14 13 182 196 16 14 224 256 18 15 270 324 20 19 380 400 ∑ (^) 𝑥𝑖 = 90 ∑ (^) 𝑦𝑖 = 76 ∑ (^) 𝑥𝑖 𝑦𝑖 = 1220 ∑ (^) 𝑥𝑖^2 = 1420
Now
where actual value of y should lie, but we will learn this technique later. Exercise Three data sets are given to you. For the first data set:
4 7 6 -3 7 2 1 4 7 -4 3 2 7 9 2 1 5 3 5 9 3 0 2 1 5 6 2 -1 1 4 1 3 7 -3 6 6 3 5 1 1 7 3 7 11 3 1 5 5 4 8 7 -6 6 1 1 2 2 2 2 7 7 8 1 2 4 1 1 3 2 0 2 5 2 6 3 -1 4 7 4 8 1 0 6 5 6 10 1 2 1 3