Assignment 3 - Solving the Dynamic Program

Source code for the model

Since you only have a week, I’m not going to make you code every piece of the model solution. The folder /children-cash-transfers/src contains:

functions to calculate utility and net income from each choice
an indexing rule that maps choices to hours and participation
the structure for the nested logit and code to calculate nested logit probabilities and inclusive values
functions to calculate transition probabilities
code to define a default set of parameters and hyperparameters (the number of wage shocks, number of types, etc)

If you have cloned the course git repo, you can load all of this source code as follows:

include("../children-cash-transfers/src/model.jl")

plain_logit (generic function with 1 method)

It will help to quickly explain some of this, but I recommend you read the code carefully since I won’t explain the functions in depth.

Setting exogenous state variables

Recall that in the model there are only three state variables to track: (1) the individual’s type (\(k\)); (2) the wage shock (\(\varepsilon\)); and (3) cumulative welfare use (\(\omega\)). Below we define a struct called model_data that contains all of the exogenous state variables that are taken as given for each individual in the data. The struct is defined in /children-cash-transfers/src/model.jl, but we repeat the definition here:

struct model_data
    T::Int64 #<- length of problem

    y0::Int64 #<- year to begin problem
    age0::Int64 # <- mother's age at start of problem
    SOI::Vector{Int64} #<- state SOI in each year
    num_kids::Vector{Int64} #<- number of kids in household that are between age 0 and 17
    TotKids::Int64 #<- indicares the total number of children that the mother will have over the available panel
    age_kid::Matrix{Int64} #< age_kid[f,t] is the age of child f at time t. Will be negative if child not born yet.
    cpi::Vector{Float64} #<- cpi

    R::Vector{Int64} #<- indicates if work requirement in time t
    Kω::Int64 #<- indicates length of time limit once introduced
    TL::Vector{Bool} #<- indicating that time limit is in place
end

Eventually we will have one of these objects for every mother we observe in the data, and we’ll solve the resulting dynamic program for each of them. To test our functions below we can create a test version of this struct as well as some default parameters:

md = test_model()
p = pars(2,5) #<- set Kτ = 2 and Kε = 5

Nested Logit Probabilities

The function nested_logit takes the value of each choice vj and fills log-choice probabilities into a pre-allocated vector logP. It also returns the inclusive value (the “emax” or continuation value). The input \(B=(B_1,B_2,...,B_L)\) specifies the partitions in each layer of the tree, while the input \(C\) reports the final choices that are ultimately contained in each node in each layer. By definition, at the highest layer, the partition takes the trivial form \(B_L=\{\{1,2,...K_L\}\}\) where \(K_L\) is the number of partitions in layer \(L-1\). Similarly at the lowest layer, \(C_{1}\) must take the form \(C_{1} = \{\{1\},\{2\},...,\{J\}\}\). For this model with three participation choices that lead into an extensive marginal labor supply choice, the structure is:

B₁ = [[1,2],[3,4],[5,6]]
C₁ = [[1,],[2,],[3,],[4,],[5,],[6,]]

B₂ = [[1,2,3]]
C₂ = [[1,2],[3,4],[5,6]]

B = (B₁,B₂)
C = (C₁,C₂)

This is defined in src/model/choices.jl. This is for your understanding, but these inputs are all given to you so you can use the function naively if that is all your time allows for.

Indexing the State

You may find the following objects useful for iterating over the state variables. Let \(k_{\tau}\in\{1,...,K_{\tau}\}\) index latent types, let \(k_{\varepsilon}\in\{1,...,K_{\varepsilon}\}\) index wage shocks, and let \(k_{\omega} = \omega+1\) index cumulative time use. The total size of the state space is \(K=K_\tau\times K_{\varepsilon} \times K_{\omega}\).

One way to do simple indexing is to just work with multi-dimensional arrays and build this into every function. However if you want to add state variables later on it will make it cumbersome to change. Another option is to use LinearIndices objects and their converse, CartesianIndices.

Here’s a demonstration:

# Hypothetical state space dimensions:
Kε = 5
Kτ = 5
Kω = 6

k_idx = LinearIndices((Kτ,Kε,Kω))
k_inv = CartesianIndices(k_idx)

# To get the aggregate index k, call:
k = k_idx[2,3,2]
@show k
# Then if we have k we can work back with:
k_tuple = Tuple(k_inv[k])
@show k_tuple

k = 37
k_tuple = (2, 3, 2)

(2, 3, 2)

So when iterating, you could think about passing around state indices along with instances of these linear and cartesian indices that allow you to convert back and forth.

Transition Probabilities

In the paper, wage shocks are parameterized with a single parameter \(\pi_{W}\) that dictates the probability that an individual remains in the same place on the grid space, with symmetric probabilities of moving up or down. The function fε in states_transitions.jl takes the current wage shock kε and the total number of shocks Kε, along with πW and returns two tuples. The first tuple is the set of grid points that are possible and the second is the probability of being in each of those points.

For example:

fε(3,5,0.9)

((2, 3, 4), (0.04999999999999999, 0.9, 0.04999999999999999))

So the function is telling me that when I’m in state 3 I can move to states 3, 4, or 5 next period with probabilities (0.05,0.9,0.05). When we are at the bottom or the top of the grid space, the probabilities are slightly different:

fε(1,5,0.9)

((1, 2), (0.04999999999999999, 0.95))

We could have alternatively just written the transition probabilities into a matrix, but this approach essentially limits us to points with positive probabilities and will simplify iteration.

Part 1

Write a function calc_vj that calculates the choice-specific value (i.e. the deterministic value of the choice) of a particular choice \(j\) in a particular time period \(t\) given the state and other exogenous variables. If you are confident you can code this however you like, but given the existing setup you might like to write the function in a way that it takes the following inputs:

j: the discrete choice
t: the time period in the model
state: a tuple that contains the state \((k_\tau,k_\varepsilon,k_\omega)\) as well as a linear indexing rule
V: a vector that contains the continuation value for each state at time \(t+1\)
pars: the parameters of the model
md: an instance of the model_data object that holds all relevant state variables

Verify that your function works by testing it on the model_data instance created by test_model. Use the @time macro to look at evaluation time and memory allocations.

Part 2

Write a function called iterate! that iterates over all states at time period \(t\) and fills in choice probabilities and continuation values for period \(t\) in pre-allocated arrays. Again, you can do this however you like but here is a suggested set of inputs:

t: the time period
logP: a \(J \times K \times T\) array of choice probabilities where the function will fill in logP[:,:,t]
V: a \(K \times T\) array of continuation values
state_idx: a named tuple that contains the size of the overall state space, a linear indexing rule that maps \((k_\tau,k_{\varepsilon},k_{\omega}\)) to an overall state \(k\), and a Cartesian Indexing rule that inverts this mapping
vj: a \(J\)-dimensional vector that, for each state, can be used as a container for the choice-specific values
pars: model parameters
md: model_data for the problem

Verify that your function works by testing it on the model_data instance created by test_model. Use the @time macro to look at evaluation time and memory allocations.

Part 3

Write a function called solve! that performs backward induction to calculate continuation values and choice probabilities (storing them in pre-allocated arrays) in every period of the data across the whole state space. As before, some suggested inputs:

logP: a \(J\times K\times T\) array for choice probabilities
V: a \(K \times T\) array for continuation values
vj: a container or buffer for choice-specific values in each iteration
pars: model parameters
md: an instance of model_data

Verify that your function works by testing it on the model_data instance created by test_model. Use the @time macro to look at evaluation time and memory allocations.