Docsity
Docsity

Prepare for your exams
Prepare for your exams

Study with the several resources on Docsity


Earn points to download
Earn points to download

Earn points by helping other students or get them with a premium plan


Guidelines and tips
Guidelines and tips

Exploiting Symmetry in High-Dimensional Dynamic Programming, Exercises of Dynamics

The use of symmetry in high-dimensional dynamic programming, which is increasingly popular as we accumulate more micro data. It explains how to train a neural network that implements the permutation-invariant dynamic programming problem as dictated by the representation theorem. The document also presents a well-known example of a variation of the Lucas and Prescott model of investment under uncertainty with N firms. The main result is the representation of permutation-invariant functions. The document ends with the training and calibration algorithm.

Typology: Exercises

2022/2023

Uploaded on 03/14/2023

salim
salim 🇺🇸

4.4

(24)

243 documents

1 / 41

Toggle sidebar

This page cannot be seen from the preview

Don't miss anything!

bg1
Exploiting Symmetry in
High-Dimensional Dynamic Programming
Mahdi Ebrahimi Kahou1Jes´us Fern´andez-Villaverde2Jesse Perla1Arnav Sood3
September 1, 2022
1University of British Columbia, Vancouver School of Economics
2University of Pennsylvania
3Carnegie Mellon University
pf3
pf4
pf5
pf8
pf9
pfa
pfd
pfe
pff
pf12
pf13
pf14
pf15
pf16
pf17
pf18
pf19
pf1a
pf1b
pf1c
pf1d
pf1e
pf1f
pf20
pf21
pf22
pf23
pf24
pf25
pf26
pf27
pf28
pf29

Partial preview of the text

Download Exploiting Symmetry in High-Dimensional Dynamic Programming and more Exercises Dynamics in PDF only on Docsity!

Exploiting Symmetry in

High-Dimensional Dynamic Programming

Mahdi Ebrahimi Kahou^1 Jes´us Fern´andez-Villaverde^2 Jesse Perla^1 Arnav Sood^3 September 1, 2022 (^1) University of British Columbia, Vancouver School of Economics

(^2) University of Pennsylvania

(^3) Carnegie Mellon University

Motivation

  • Most models in macro (and other fields) deal with either:
    1. Representative agent (canonical RBC and New Keynesian models). Sometimes two agents (models of financial frictions, international business cycles).
    2. A continuum of agents (canonical Krusell-Smith model).
  • However, many models of interest deal with a finite (but large) number of agents and aggregate uncertainty: 1. Models with many locations (countries, regions, metropolitan areas, industries). 2. Models with many households (e.g., overlapping generations, different types) instead of a continuum. 3. Models of industry dynamics with many firms. 4. Models of networks.
  • Models with finite agents are increasingly popular as we accumulate more micro data.

Curse of dimensionality

Two components (Bellman, 1958, p. IX):

  1. The cardinality of the state space is enormous: memory requirements, update of coefficients, ...
    • With 266 state variables, a grid of 2 points per state (low and high) or, equivalently, a tensor of 2 polynomials per state (a level and a slope) have more elements than the Eddington number, the estimated number of protons (≈ 1080 ) in the universe.
  2. It is difficult to evaluate highly-dimensional conditional expectations: continuation value function, Euler equations, ... - Computing integrals is an exponentially-hard function of their dimensions.

Symmetry

  • Symmetry is one of the most powerful ideas in math.

Hermann Weyl, 1952 Symmetry, as wide or as narrow as you may define its meaning, is one idea by which man through the ages has tried to comprehend and create order, beauty, and perfection...

Symmetry is a vast subject, significant in art and nature. Mathematics lies at its root, and it would be hard to find a better one on which to demonstrate the working of the mathematical intellect.

  • Also, old tradition in economics that goes back decades (Samuelson, 1990, Mertens and Judd, 2018); let me skip the literature review.
  • Intuition: we search in “better” function spaces.

What do we do?

  • We introduce permutation-invariant dynamic programming and the associated concept of permutation-invariant economies.
  • Common feature of many (most?) models of interest: the policy functions of the agents are the same, they are just evaluated at different points.
  • In a multi-location model of the U.S.: if (the representative agent in) California had the same capital and productivity as Texas, it would behave as Texas and vice versa.
  • Many forms of ex-ante heterogeneity are encompassed as pseudo-states (more on this later on).
  • The solution of the model is invariant under all the permutations of other agents’ states. In equilibrium, the Walrasian auctioneer removes indexes!

A deep learning approach

  • We show how to train a neural network that implements the permutation-invariant dynamic programming problem as dictated by our representation theorem.
  • Strictly speaking, neural networks are not required. You only need a flexible functional basis to implement a projection.
  • Neural networks have the numerical advantages, though, that we have described in previous slide decks.

How do we pick our application to show how all this works?

  • In terms of application, there are two routes:
    1. I can introduce a sophisticated application where our method “shines.”
    2. Or, I can show you how our ideas work in a well-known example.
  • Today, I do not have the time to tell you about the methods and the application.
  • Besides, if I tell you about a sophisticated application, how do you know our “solution” works?
  • So, let me present a well-known example (with a twist)...
  • ...and leave for another day the more sophisticated applications.

A ‘big X , little x’ dynamic programming problem

Consider:

v (x, X ) = max u

r

x, u, X

  • βE [v (x′, X ′)] s.t. x′^ = g (x, u) + σw + ηω X ′^ = G (X ) + ΩW + ηω (^1) N

where:

  1. x is the individual state of the agent.
  2. X is a vector stacking the individual states of all of the N agents in the economy.
  3. u is the control.
  4. w is random innovation to the individual state, stacked in W ∼ N ( (^0) N , IN ) and where, w.l.o.g., w = W 1.
  5. ω ∼ N (0, 1) is a random aggregate innovation to all the individual states.

Some preliminaries

  • A permutation matrix is a square matrix with a single 1 in each row and column and zeros everywhere else. - These matrices are called “permutation” because, when they premultiply (postmultiply) a conformable matrix A, they permute the rows (columns) of A.
  • Let SN be the set of all n! permutation matrices of size N × N. For example:

S 2 =

  • (If you know about this): SN is the symmetric group under matrix multiplication.
    • The algebraic properties of the symmetric group will be doing a lot of the heavy lifting in the proof of our theorems, but you do not need to worry about it.

Permutation invariance of the optimal solution

Proposition The optimal solution of a permutation-invariant dynamic programming problem is permutation invariant. That is, for all π ∈ SN : u(x, πX ) = u(x, X )

and: v (x, πX ) = v (x, X )

Main result I: Representation of permutation-invariant functions

Proposition (based on Wagstaff et al., 2019) Let f : RN+1^ → R be a continuous permutation-invariant function under SN , i.e., for all (x, X ) ∈ RN+ and all π ∈ SN :

f (x, πX ) = f (x, X )

Then, there exist a latent dimension L ≤ N and continuous functions ρ : RL+1^ → R and ϕ : R → RL such that: f (x, X ) = ρ x,

N

X^ N

i=

ϕ(Xi )

This proposition should remind you of Krusell-Smith!

Intuition: Take f : 2N^ → R. If f is permutation invariant, it has at most N + 1 outputs. Thus, ϕ(·) is the identity function (i.e., L = 1) and ρ maps the sum of 1’s in the string into the N + 1 outputs. Using some results from the symmetric group, you can generalize the idea to the RN+1^ domain. (^16)

A permutation-invariant economy

  • Industry consisting of N > 1 firms, each producing the same good.
  • A firm i produces output x with x units of capital.
  • Thus, the vector X ≡ [x 1 ,... xN ]⊤^ is the production (or capital) of the whole industry.
  • The inverse demand function for the industry is, for some ν ≥ 1 (this is our twist!):

p(X ) = 1 − 1 N

X^ N

i=

x iν

  • The firm does not consider the impact of its individual decisions on p(X ).
  • Due to adjustment frictions, investing u has a cost γ 2 u^2.
  • Law of motion for capital x′^ = (1 − δ)x + u + σw + ηω where w ∼ N (0, 1) an i.i.d. idiosyncratic shock, and ω ∼ N (0, 1) an i.i.d. aggregate shock, common to all firms.
  • The firm chooses u to maximize E

P∞

t=0 βt^

p(X )x − γ 2 u^2

Recursive problem

  • The recursive problem of the firm taking the exogenous policy ˆu(·, X ) for all other firms as given is:

v (x, X ) = max u

n p(X )x − γ 2 u^2 + βE [v (x′, X ′)]

o

s.t. x′^ = (1 − δ)x + u + σw + ηω X (^) i′ = (1 − δ)Xi + ˆu(Xi , X ) + σWi + ηω, for i ∈ { 1 , ..., N}