











































































Study with the several resources on Docsity
Earn points by helping other students or get them with a premium plan
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Community
Ask the community for help and clear up your study doubts
Discover the best universities in your country according to Docsity users
Free resources
Download our free guides on studying techniques, anxiety management strategies, and thesis advice from Docsity tutors
Nonlinear Models, Nonlinear Optimization, Estimator, Parameters, Conditional Mean Function, Asymptotic Normality of Estimators, Least Squares, Iterations, Exponential Model are points which describes this lecture importance in Econometric Analysis of Panel Data course.
Typology: Slides
1 / 83
This page cannot be seen from the preview
Don't miss anything!
14. Nonlinear Models
And Nonlinear Optimization
Agenda
Estimators
Properties
M Estimation
Nonlinear Least Squares
Maximum Likelihood Estimation
GMM Estimation
Minimum Distance Estimation
Minimum Chi-square Estimation
Nonlinear Least Squares
Newton-like Algorithms; Gradient Methods
What is a Nonlinear Model?
Learn about θ from y , X
Usually “estimate” θ
Not wrt m( x , θ ). E.g., y=exp( θ’x + ε )
Wrt estimator: Implicitly defined. h( y , X, )=0,
E.g., E[y|x]= exp( θ’x )
ˆ
θ
θ
ˆ
What is an Estimator?
Point and Interval
Classical and Bayesian
ˆ
f(data | mod el)
ˆ ˆ
I( ) sampling variability
θ =
θ = θ ±
ˆ
E[ | data,prior f( )] expectation from posterior
ˆ
I( ) narrowest interval from posterior density
containing the specified probability (mass)
θ = θ θ =
θ =
The Conditional Mean Function
y,x
m(x, ) E[y | x] for some in.
A property of the conditional mean:
E (y m(x, )) is minimized by E[y | x]
(Proof, pp. 343-344, JW)
θ = θ Θ
− θ
M Estimation
n
i
i=
n
2
i i i
i=
1
ˆ
arg min q( , )
n
Example : Nonlinear Least squares
1
ˆ
arg min [y -E(y | , )]
n
θ = θ
θ = θ
∑
∑
data
x
n
P
i
i 1
0
P
P
0
=
Identification
4
1 0 1 0
1 2 3
Uniqueness :
If , then m(x, ) m(x, )
Examples
(1) (Multicollinearity)
(2) (Need for normalization) E[y|x] = m( x/ )
(3) (Indeterminacy) m(x, )= x x
β
θ ≠ θ θ ≠ θ
′
β σ
θ β + β + β
Consistency
n
P
i
i 1
0
P
P
0
1
q= q(data , ) q*=E[q(data, )]
n
ˆ
Estimator minimizes q
True parameter minimizes q*
q q*
ˆ
Does this imply?
Yes. Consistency follows from identification
and continuity with the other a
=
θ → θ
θ
θ
→
θ → θ
∑
ssumptions
Asymptotic Normality of M
Estimators
N
i=1 i
N i
i=
N
i=1 i
Asymptotic Normality
1
0 0
1
0
ˆ
n ( ) [ ( )] n (data, )
[ ( )] converges to its expectation (a matrix)
n (data, ) converges to a normally distributed
vector (Lindberg-Feller)
ˆ
Implies limiting normal distribution of n (
−
−
θ − θ = θ θ
θ
θ
θ − θ
H g
H
g
0
).
Limiting mean is 0.
Limiting variance to be obtained.
Asymptotic distribution obtained by the usual means.
Asymptotic Variance
a 1
0 0
0
1 1
0 0 0
0
i 0 i 0
ˆ
[ ( )] (data, )
Asymptotically normal
Mean
ˆ
Asy.Var[ ] [ ( )] Var[ (data, )] [ ( )]
(A sandwich estimator, as usual)
What is Var[ (data, )]?
1
E[ (data , ) (data , ) ']
n
Not known
−
− −
θ → θ + θ θ
= θ
θ = θ θ θ
θ
θ θ
H g
H g H
g
g g
n
i 1 i i
what it is, but it is easy to estimate.
1 1
ˆ ˆ
(data , ) (data , ) '
n n
=
× Σ g θ g θ
Nonlinear Least Squares
i
i
0 i
i i
(k+1) (k) 1
Gauss-Marquardt Algorithm
q the conditional mean function
= m(x , )
m(x , )
x 'pseudo regressors '
Algorithm - iteration
ˆ ˆ
[ ]
−
=
θ
∂ θ
= = = −
∂θ
θ = θ +
0 0 0 0
g
X 'X X 'e
Application - Income
German Health Care Usage Data, 7,293 Individuals, Varying Numbers of Periods
Variables in the file are
Data downloaded from Journal of Applied Econometrics Archive. This is an unbalanced
panel with 7,293 individuals. They can be used for regression, count models, binary
choice, ordered choice, and bivariate binary choice. This is a large data set. There are
altogether 27,326 observations. The number of observations ranges from 1 to
the variable NUMOBS below tells how many observations there are for each
person. This variable is repeated in each row of the data for the person. (Downlo0aded
from the JAE Archive)
HHNINC = household nominal monthly net income in German marks / 10000.
(4 observations with income=0 were dropped)
HHKIDS = children under age 16 in the household = 1; otherwise = 0
EDUC = years of schooling
AGE = age in years