Download Exploiting Symmetry in High-Dimensional Dynamic Programming and more Exercises Dynamics in PDF only on Docsity!
Exploiting Symmetry in
High-Dimensional Dynamic Programming
Mahdi Ebrahimi Kahou^1 Jes´us Fern´andez-Villaverde^2 Jesse Perla^1 Arnav Sood^3 September 1, 2022 (^1) University of British Columbia, Vancouver School of Economics
(^2) University of Pennsylvania
(^3) Carnegie Mellon University
Motivation
- Most models in macro (and other fields) deal with either:
- Representative agent (canonical RBC and New Keynesian models). Sometimes two agents (models of financial frictions, international business cycles).
- A continuum of agents (canonical Krusell-Smith model).
- However, many models of interest deal with a finite (but large) number of agents and aggregate uncertainty: 1. Models with many locations (countries, regions, metropolitan areas, industries). 2. Models with many households (e.g., overlapping generations, different types) instead of a continuum. 3. Models of industry dynamics with many firms. 4. Models of networks.
- Models with finite agents are increasingly popular as we accumulate more micro data.
Curse of dimensionality
Two components (Bellman, 1958, p. IX):
- The cardinality of the state space is enormous: memory requirements, update of coefficients, ...
- With 266 state variables, a grid of 2 points per state (low and high) or, equivalently, a tensor of 2 polynomials per state (a level and a slope) have more elements than the Eddington number, the estimated number of protons (≈ 1080 ) in the universe.
- It is difficult to evaluate highly-dimensional conditional expectations: continuation value function, Euler equations, ... - Computing integrals is an exponentially-hard function of their dimensions.
Symmetry
- Symmetry is one of the most powerful ideas in math.
Hermann Weyl, 1952 Symmetry, as wide or as narrow as you may define its meaning, is one idea by which man through the ages has tried to comprehend and create order, beauty, and perfection...
Symmetry is a vast subject, significant in art and nature. Mathematics lies at its root, and it would be hard to find a better one on which to demonstrate the working of the mathematical intellect.
- Also, old tradition in economics that goes back decades (Samuelson, 1990, Mertens and Judd, 2018); let me skip the literature review.
- Intuition: we search in “better” function spaces.
What do we do?
- We introduce permutation-invariant dynamic programming and the associated concept of permutation-invariant economies.
- Common feature of many (most?) models of interest: the policy functions of the agents are the same, they are just evaluated at different points.
- In a multi-location model of the U.S.: if (the representative agent in) California had the same capital and productivity as Texas, it would behave as Texas and vice versa.
- Many forms of ex-ante heterogeneity are encompassed as pseudo-states (more on this later on).
- The solution of the model is invariant under all the permutations of other agents’ states. In equilibrium, the Walrasian auctioneer removes indexes!
A deep learning approach
- We show how to train a neural network that implements the permutation-invariant dynamic programming problem as dictated by our representation theorem.
- Strictly speaking, neural networks are not required. You only need a flexible functional basis to implement a projection.
- Neural networks have the numerical advantages, though, that we have described in previous slide decks.
How do we pick our application to show how all this works?
- In terms of application, there are two routes:
- I can introduce a sophisticated application where our method “shines.”
- Or, I can show you how our ideas work in a well-known example.
- Today, I do not have the time to tell you about the methods and the application.
- Besides, if I tell you about a sophisticated application, how do you know our “solution” works?
- So, let me present a well-known example (with a twist)...
- ...and leave for another day the more sophisticated applications.
A ‘big X , little x’ dynamic programming problem
Consider:
v (x, X ) = max u
r
x, u, X
- βE [v (x′, X ′)] s.t. x′^ = g (x, u) + σw + ηω X ′^ = G (X ) + ΩW + ηω (^1) N
where:
- x is the individual state of the agent.
- X is a vector stacking the individual states of all of the N agents in the economy.
- u is the control.
- w is random innovation to the individual state, stacked in W ∼ N ( (^0) N , IN ) and where, w.l.o.g., w = W 1.
- ω ∼ N (0, 1) is a random aggregate innovation to all the individual states.
Some preliminaries
- A permutation matrix is a square matrix with a single 1 in each row and column and zeros everywhere else. - These matrices are called “permutation” because, when they premultiply (postmultiply) a conformable matrix A, they permute the rows (columns) of A.
- Let SN be the set of all n! permutation matrices of size N × N. For example:
S 2 =
- (If you know about this): SN is the symmetric group under matrix multiplication.
- The algebraic properties of the symmetric group will be doing a lot of the heavy lifting in the proof of our theorems, but you do not need to worry about it.
Permutation invariance of the optimal solution
Proposition The optimal solution of a permutation-invariant dynamic programming problem is permutation invariant. That is, for all π ∈ SN : u(x, πX ) = u(x, X )
and: v (x, πX ) = v (x, X )
Main result I: Representation of permutation-invariant functions
Proposition (based on Wagstaff et al., 2019) Let f : RN+1^ → R be a continuous permutation-invariant function under SN , i.e., for all (x, X ) ∈ RN+ and all π ∈ SN :
f (x, πX ) = f (x, X )
Then, there exist a latent dimension L ≤ N and continuous functions ρ : RL+1^ → R and ϕ : R → RL such that: f (x, X ) = ρ x,
N
X^ N
i=
ϕ(Xi )
This proposition should remind you of Krusell-Smith!
Intuition: Take f : 2N^ → R. If f is permutation invariant, it has at most N + 1 outputs. Thus, ϕ(·) is the identity function (i.e., L = 1) and ρ maps the sum of 1’s in the string into the N + 1 outputs. Using some results from the symmetric group, you can generalize the idea to the RN+1^ domain. (^16)
A permutation-invariant economy
- Industry consisting of N > 1 firms, each producing the same good.
- A firm i produces output x with x units of capital.
- Thus, the vector X ≡ [x 1 ,... xN ]⊤^ is the production (or capital) of the whole industry.
- The inverse demand function for the industry is, for some ν ≥ 1 (this is our twist!):
p(X ) = 1 − 1 N
X^ N
i=
x iν
- The firm does not consider the impact of its individual decisions on p(X ).
- Due to adjustment frictions, investing u has a cost γ 2 u^2.
- Law of motion for capital x′^ = (1 − δ)x + u + σw + ηω where w ∼ N (0, 1) an i.i.d. idiosyncratic shock, and ω ∼ N (0, 1) an i.i.d. aggregate shock, common to all firms.
- The firm chooses u to maximize E
P∞
t=0 βt^
p(X )x − γ 2 u^2
Recursive problem
- The recursive problem of the firm taking the exogenous policy ˆu(·, X ) for all other firms as given is:
v (x, X ) = max u
n p(X )x − γ 2 u^2 + βE [v (x′, X ′)]
o
s.t. x′^ = (1 − δ)x + u + σw + ηω X (^) i′ = (1 − δ)Xi + ˆu(Xi , X ) + σWi + ηω, for i ∈ { 1 , ..., N}
