{
"cells": [
{
"cell_type": "markdown",
"id": "34a8d902",
"metadata": {},
"source": [
"\n",
""
]
},
{
"cell_type": "markdown",
"id": "174ccd2b",
"metadata": {},
"source": [
"# Job Search I: The McCall Search Model"
]
},
{
"cell_type": "markdown",
"id": "db4818f7",
"metadata": {},
"source": [
"## Contents\n",
"\n",
"- [Job Search I: The McCall Search Model](#Job-Search-I:-The-McCall-Search-Model) \n",
" - [Overview](#Overview) \n",
" - [The McCall Model](#The-McCall-Model) \n",
" - [Computing an Optimal Policy: Take 1](#Computing-an-Optimal-Policy:-Take-1) \n",
" - [Computing an Optimal Policy: Take 2](#Computing-an-Optimal-Policy:-Take-2) \n",
" - [Exercises](#Exercises) "
]
},
{
"cell_type": "markdown",
"id": "930e0092",
"metadata": {},
"source": [
"> “Questioning a McCall worker is like having a conversation with an out-of-work friend:\n",
"> ‘Maybe you are setting your sights too high’, or ‘Why did you quit your old job before you\n",
"> had a new one lined up?’ This is real social science: an attempt to model, to understand,\n",
"> human behavior by visualizing the situation people find themselves in, the options they face\n",
"> and the pros and cons as they themselves see them.” – Robert E. Lucas, Jr.\n",
"\n",
"\n",
"In addition to what’s in Anaconda, this lecture will need the following libraries:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d2adb519",
"metadata": {
"hide-output": false
},
"outputs": [],
"source": [
"!pip install quantecon"
]
},
{
"cell_type": "markdown",
"id": "006329ff",
"metadata": {},
"source": [
"## Overview\n",
"\n",
"The McCall search model [[McC70](https://python.quantecon.org/zreferences.html#id191)] helped transform economists’ way of thinking about labor markets.\n",
"\n",
"To clarify notions such as “involuntary” unemployment, McCall modeled the decision problem of an unemployed worker in terms of factors including\n",
"\n",
"- current and likely future wages \n",
"- impatience \n",
"- unemployment compensation \n",
"\n",
"\n",
"To solve the decision problem McCall used dynamic programming.\n",
"\n",
"Here we set up McCall’s model and use dynamic programming to analyze it.\n",
"\n",
"As we’ll see, McCall’s model is not only interesting in its own right but also an excellent vehicle for learning dynamic programming.\n",
"\n",
"Let’s start with some imports:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d9f4c326",
"metadata": {
"hide-output": false
},
"outputs": [],
"source": [
"%matplotlib inline\n",
"import matplotlib.pyplot as plt\n",
"plt.rcParams[\"figure.figsize\"] = (11, 5) #set default figure size\n",
"import numpy as np\n",
"from numba import jit, float64\n",
"from numba.experimental import jitclass\n",
"import quantecon as qe\n",
"from quantecon.distributions import BetaBinomial"
]
},
{
"cell_type": "markdown",
"id": "c2c6f14e",
"metadata": {},
"source": [
"## The McCall Model\n",
"\n",
"\n",
"\n",
"At the beginning of each period, a worker who was unemployed last period receives one job offer to work this period at all subsequent periods at a wage $ w_t $.\n",
"\n",
"The wage offer $ w_t $ is a nonnegative function of some underlying state $ s_t $:\n",
"\n",
"$$\n",
"w_t = w(s_t) \\quad \\text{where } \\; s_t \\in \\mathbb{S}\n",
"$$\n",
"\n",
"Here you should think of state process $ \\{s_t\\} $ as some underlying, unspecified\n",
"random factor that determines wages.\n",
"\n",
"(Introducing an exogenous stochastic state process is a standard way for\n",
"economists to inject randomness into their models.)\n",
"\n",
"In this lecture, we adopt the following simple environment:\n",
"\n",
"- $ \\{s_t\\} $ is IID, with $ q(s) $ being the probability of observing state $ s $ in $ \\mathbb{S} $ at each point in time, \n",
"- the agent observes $ s_t $ at the start of $ t $ and hence knows\n",
" $ w_t = w(s_t) $, \n",
"- the set $ \\mathbb S $ is finite. \n",
"\n",
"\n",
"(In later lectures, we shall relax some of these assumptions.)\n",
"\n",
"At time $ t $, our agent has two choices:\n",
"\n",
"1. Accept the offer and work permanently at constant wage $ w_t $. \n",
"1. Reject the offer, receive unemployment compensation $ c $, and reconsider next period. \n",
"\n",
"\n",
"The agent is infinitely lived and aims to maximize the expected discounted\n",
"sum of earnings\n",
"\n",
"$$\n",
"\\mathbb{E} \\sum_{t=0}^{\\infty} \\beta^t y_t\n",
"$$\n",
"\n",
"The constant $ \\beta $ lies in $ (0, 1) $ and is called a **discount factor**.\n",
"\n",
"The smaller is $ \\beta $, the more the agent discounts future utilities relative to current utility.\n",
"\n",
"The variable $ y_t $ is income, equal to\n",
"\n",
"- his/her wage $ w_t $ when employed \n",
"- unemployment compensation $ c $ when unemployed \n",
"\n",
"\n",
"The worker knows that $ \\{s_t\\} $ is IID with common\n",
"distribution $ q $ and uses knowledge when he or she computes mathematical expectations of various random variables that are functions of\n",
"$ s_t $."
]
},
{
"cell_type": "markdown",
"id": "d7b2492c",
"metadata": {},
"source": [
"### A Trade-Off\n",
"\n",
"The worker faces a trade-off:\n",
"\n",
"- Waiting too long for a good offer is costly, since the future is discounted. \n",
"- Accepting too early is costly, since better offers might arrive in the future. \n",
"\n",
"\n",
"To decide optimally in the face of this trade-off, we use dynamic programming.\n",
"\n",
"Dynamic programming can be thought of as a two-step procedure that\n",
"\n",
"1. first assigns values to “states” \n",
"1. then deduces optimal actions given those values \n",
"\n",
"\n",
"We’ll go through these steps in turn."
]
},
{
"cell_type": "markdown",
"id": "df880a8b",
"metadata": {},
"source": [
"### The Value Function\n",
"\n",
"In order optimally to trade-off current and future rewards, we think about two things:\n",
"\n",
"1. current payoffs that arise from making alternative choices \n",
"1. different states that those choices take us to next period (in this case, either employment or unemployment) \n",
"\n",
"\n",
"To assess these two aspects of the decision problem, we assign expected discounted *values*\n",
"to states.\n",
"\n",
"This leads us to construct an instance of the celebrated **value function** of dynamic programming.\n",
"\n",
"Definitions of value functions typically begin with the word ``let’’.\n",
"\n",
"Thus,\n",
"\n",
"Let $ v^*(s) $ be the optimal value of the problem when $ s \\in \\mathbb{S} $ for a previously unemployed worker who, starting this period, has just received an offer to work forever at wage $ w(s) $ and who is yet to decide whether to accept or reject the offer.\n",
"\n",
"Thus, the function $ v^*(s) $ is the maximum value of objective\n",
"[(35.1)](https://python.quantecon.org/mccall_model_with_separation.html#equation-objective) for a previously unemployed worker who has offer $ w(s) $ in hand and has yet to choose whether to accept it.\n",
"\n",
"Notice that $ v^*(s) $ is part of the **solution** of the problem, so it isn’t obvious that it is a good idea to start working on the problem by focusing on $ v^*(s) $.\n",
"\n",
"There is a chicken and egg problem: we don’t know how to compute $ v^*(s) $ because we don’t yet know\n",
"what decisions are optimal and what aren’t!\n",
"\n",
"But it turns out to be a really good idea by asking what properties the optimal value function $ v^*(s) $ must have in order it\n",
"to qualify as an optimal value function.\n",
"\n",
"Think of $ v^* $ as a function that assigns to each possible state\n",
"$ s $ the maximal expected discounted income stream that can be obtained with that offer in\n",
"hand.\n",
"\n",
"A crucial observation is that the optimal value function $ v^* $ must satisfy functional equation\n",
"\n",
"\n",
"\n",
"$$\n",
"v^*(s)\n",
"= \\max \\left\\{\n",
" \\frac{w(s)}{1 - \\beta}, \\, c + \\beta\n",
" \\sum_{s' \\in \\mathbb{S}} v^*(s') q (s')\n",
" \\right\\} \\tag{34.1}\n",
"$$\n",
"\n",
"for every possible $ s $ in $ \\mathbb S $.\n",
"\n",
"Notice how the function $ v^*(s) $ appears on both the right and left sides of equation [(34.1)](#equation-odu-pv) – that is why it is called\n",
"a **functional equation**, i.e., an equation that restricts a **function**.\n",
"\n",
"This important equation is a version of a **Bellman equation**, an equation that is\n",
"ubiquitous in economic dynamics and other fields involving planning over time.\n",
"\n",
"The intuition behind it is as follows:\n",
"\n",
"- the first term inside the max operation is the lifetime payoff from accepting current offer, since \n",
"\n",
"\n",
"$$\n",
"\\frac{w(s)}{1 - \\beta} = w(s) + \\beta w(s) + \\beta^2 w(s) + \\cdots\n",
"$$\n",
"\n",
"- the second term inside the max operation is the lifetime payoff from rejecting the current offer and then behaving optimally in all subsequent periods \n",
"\n",
"\n",
"If we optimize and pick the best of these two options, we obtain maximal lifetime value from today, given current state $ s $.\n",
"\n",
"But this is precisely $ v^*(s) $, which is the l.h.s. of [(34.1)](#equation-odu-pv)."
]
},
{
"cell_type": "markdown",
"id": "faa0f444",
"metadata": {},
"source": [
"### The Optimal Policy\n",
"\n",
"Suppose for now that we are able to solve [(34.1)](#equation-odu-pv) for the unknown\n",
"function $ v^* $.\n",
"\n",
"Once we have this function in hand we can figure out how behave optimally (i.e., to choose whether to accept and reject $ w(s) $).\n",
"\n",
"All we have to do is select the maximal choice on the r.h.s. of [(34.1)](#equation-odu-pv).\n",
"\n",
"The optimal action in state $ s $ can be thought of as a part of a **policy** that maps a\n",
"state into an action.\n",
"\n",
"Given *any* $ s $, we can read off the corresponding best choice (accept or\n",
"reject) by picking the max on the r.h.s. of [(34.1)](#equation-odu-pv).\n",
"\n",
"Thus, we have a map from $ \\mathbb R $ to $ \\{0, 1\\} $, with 1 meaning accept and 0 meaning reject.\n",
"\n",
"We can write the policy as follows\n",
"\n",
"$$\n",
"\\sigma(s) := \\mathbf{1}\n",
" \\left\\{\n",
" \\frac{w(s)}{1 - \\beta} \\geq c + \\beta \\sum_{s' \\in \\mathbb S}\n",
" v^*(s') q (s')\n",
" \\right\\}\n",
"$$\n",
"\n",
"Here $ \\mathbf{1}\\{ P \\} = 1 $ if statement $ P $ is true and equals 0 otherwise.\n",
"\n",
"We can also write this as\n",
"\n",
"$$\n",
"\\sigma(s) := \\mathbf{1} \\{ w(s) \\geq \\bar w \\}\n",
"$$\n",
"\n",
"where\n",
"\n",
"\n",
"\n",
"$$\n",
"\\bar w := (1 - \\beta) \\left\\{ c + \\beta \\sum_{s'} v^*(s') q (s') \\right\\} \\tag{34.2}\n",
"$$\n",
"\n",
"Here $ \\bar w $ (called the *reservation wage*) is a constant that depends on $ \\beta, c $, and the wage probability distribution induced by $ q(s) $ and $ w(s) $.\n",
"\n",
"The agent should accept offer $ w(s) $ if and only if it exceeds the reservation wage.\n",
"\n",
"In view of [(34.2)](#equation-reswage), we can compute this reservation wage if we can compute the value function."
]
},
{
"cell_type": "markdown",
"id": "aa42247d",
"metadata": {},
"source": [
"## Computing an Optimal Policy: Take 1\n",
"\n",
"To put the above ideas into action, we need to compute the value function at\n",
"each possible state $ s \\in \\mathbb S $.\n",
"\n",
"Let’s suppose that $ \\mathbb S = \\{1, \\ldots, n\\} $.\n",
"\n",
"The value function is then represented by the vector $ v^* =\n",
"(v^*(i))_{i=1}^n $.\n",
"\n",
"In view of [(34.1)](#equation-odu-pv), this vector satisfies the nonlinear system of equations\n",
"\n",
"\n",
"\n",
"$$\n",
"v^*(i)\n",
"= \\max \\left\\{\n",
" \\frac{w(i)}{1 - \\beta}, \\, c + \\beta \\sum_{1 \\leq j \\leq n}\n",
" v^*(j) q (j)\n",
" \\right\\}\n",
"\\quad\n",
"\\text{for } i = 1, \\ldots, n \\tag{34.3}\n",
"$$"
]
},
{
"cell_type": "markdown",
"id": "ccfdee41",
"metadata": {},
"source": [
"### The Algorithm\n",
"\n",
"To compute th vector $ v^*(i), i = 1, \\ldots, n $, we use successive approximations:\n",
"\n",
"Step 1: pick an arbitrary initial guess $ v \\in \\mathbb R^n $.\n",
"\n",
"Step 2: compute a new vector $ v' \\in \\mathbb R^n $ via\n",
"\n",
"\n",
"\n",
"$$\n",
"v'(i)\n",
"= \\max \\left\\{\n",
" \\frac{w(i)}{1 - \\beta}, \\, c + \\beta \\sum_{1 \\leq j \\leq n}\n",
" v(j) q (j)\n",
" \\right\\}\n",
"\\quad\n",
"\\text{for } i = 1, \\ldots, n \\tag{34.4}\n",
"$$\n",
"\n",
"Step 3: calculate a measure of a discrepancy between $ v $ and $ v' $, such as $ \\max_i |v(i)- v'(i)| $.\n",
"\n",
"Step 4: if the deviation is larger than some fixed tolerance, set $ v = v' $ and go to step 2, else continue.\n",
"\n",
"Step 5: return $ v $.\n",
"\n",
"For a small tolerance, the returned function $ v $ is a close approximation to the value function $ v^* $.\n",
"\n",
"The theory below elaborates on this point."
]
},
{
"cell_type": "markdown",
"id": "0d77bc36",
"metadata": {},
"source": [
"### Fixed Point Theory\n",
"\n",
"What’s the mathematics behind these ideas?\n",
"\n",
"First, one defines a mapping $ T $ from $ \\mathbb R^n $ to\n",
"itself via\n",
"\n",
"\n",
"\n",
"$$\n",
"(Tv)(i)\n",
"= \\max \\left\\{\n",
" \\frac{w(i)}{1 - \\beta}, \\, c + \\beta \\sum_{1 \\leq j \\leq n}\n",
" v(j) q (j)\n",
" \\right\\}\n",
"\\quad\n",
"\\text{for } i = 1, \\ldots, n \\tag{34.5}\n",
"$$\n",
"\n",
"(A new vector $ Tv $ is obtained a given vector $ v $ by evaluating\n",
"the r.h.s. at each $ i $.)\n",
"\n",
"The element $ v_k $ in the sequence $ \\{v_k\\} $ of successive\n",
"approximations corresponds to $ T^k v $.\n",
"\n",
"- This is $ T $ applied $ k $ times, starting at the initial guess\n",
" $ v $ \n",
"\n",
"\n",
"One can show that the conditions of the [Banach fixed point theorem](https://en.wikipedia.org/wiki/Banach_fixed-point_theorem) are\n",
"satisfied by $ T $ on $ \\mathbb R^n $.\n",
"\n",
"One implication is that $ T $ has a unique fixed point $ \\bar v \\in \\mathbb R^n $.\n",
"\n",
"- The fixed point is a unique vector $ \\bar v $ that satisfies $ T \\bar v = \\bar v $. \n",
"\n",
"\n",
"Moreover, it’s immediate from the definition of $ T $ that this fixed\n",
"point is $ v^* $.\n",
"\n",
"A second implication of the Banach contraction mapping theorem is that\n",
"$ \\{ T^k v \\} $ converges to the fixed point $ v^* $ regardless of the initial\n",
"$ v \\in \\mathbb R^n $."
]
},
{
"cell_type": "markdown",
"id": "b6a9f18a",
"metadata": {},
"source": [
"### Implementation\n",
"\n",
"Our default for the probability distribution $ q $ of the state process is a\n",
"[Beta-binomial](https://en.wikipedia.org/wiki/Beta-binomial_distribution)."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "024ba697",
"metadata": {
"hide-output": false
},
"outputs": [],
"source": [
"n, a, b = 50, 200, 100 # default parameters\n",
"q_default = BetaBinomial(n, a, b).pdf() # default choice of q"
]
},
{
"cell_type": "markdown",
"id": "f9004b01",
"metadata": {},
"source": [
"Our default set of values for wages will be"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "fee72ae9",
"metadata": {
"hide-output": false
},
"outputs": [],
"source": [
"w_min, w_max = 10, 60\n",
"w_default = np.linspace(w_min, w_max, n+1)"
]
},
{
"cell_type": "markdown",
"id": "f1a7eefa",
"metadata": {},
"source": [
"Here’s a plot of probabilities of different wage outcomes:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3f6dca27",
"metadata": {
"hide-output": false
},
"outputs": [],
"source": [
"fig, ax = plt.subplots()\n",
"ax.plot(w_default, q_default, '-o', label='$q(w(i))$')\n",
"ax.set_xlabel('wages')\n",
"ax.set_ylabel('probabilities')\n",
"\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "a5ec256d",
"metadata": {},
"source": [
"We are going to use Numba to accelerate our code.\n",
"\n",
"- See, in particular, the discussion of `@jitclass` in [our lecture on Numba](https://python-programming.quantecon.org/numba.html). \n",
"\n",
"\n",
"The following helps Numba by providing some information about types"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "cb9db3ae",
"metadata": {
"hide-output": false
},
"outputs": [],
"source": [
"mccall_data = [\n",
" ('c', float64), # unemployment compensation\n",
" ('β', float64), # discount factor\n",
" ('w', float64[:]), # array of wage values, w[i] = wage at state i\n",
" ('q', float64[:]) # array of probabilities\n",
"]"
]
},
{
"cell_type": "markdown",
"id": "92c851f3",
"metadata": {},
"source": [
"Here’s a class that stores the data and computes values of state-action pairs,\n",
"i.e., values associated with pairs consisting of the current state and alternative feasible actions that occur inside the maximum bracket on the right hand side of Bellman equation [(34.4)](#equation-odu-pv2p).\n",
"\n",
"Default parameter values are embedded in the class."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "afcc9677",
"metadata": {
"hide-output": false
},
"outputs": [],
"source": [
"@jitclass(mccall_data)\n",
"class McCallModel:\n",
"\n",
" def __init__(self, c=25, β=0.99, w=w_default, q=q_default):\n",
"\n",
" self.c, self.β = c, β\n",
" self.w, self.q = w_default, q_default\n",
"\n",
" def state_action_values(self, i, v):\n",
" \"\"\"\n",
" The values of state-action pairs.\n",
" \"\"\"\n",
" # Simplify names\n",
" c, β, w, q = self.c, self.β, self.w, self.q\n",
" # Evaluate value for each state-action pair\n",
" # Consider action = accept or reject the current offer\n",
" accept = w[i] / (1 - β)\n",
" reject = c + β * np.sum(v * q)\n",
"\n",
" return np.array([accept, reject])"
]
},
{
"cell_type": "markdown",
"id": "0d8fc965",
"metadata": {},
"source": [
"Based on these defaults, let’s try plotting the first few approximate value functions\n",
"in the sequence $ \\{ T^k v \\} $.\n",
"\n",
"We will start from guess $ v $ given by $ v(i) = w(i) / (1 - β) $, which is the value of accepting $ w(i) $.\n",
"\n",
"Here’s a function to implement this:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4b9ff02f",
"metadata": {
"hide-output": false
},
"outputs": [],
"source": [
"def plot_value_function_seq(mcm, ax, num_plots=6):\n",
" \"\"\"\n",
" Plot a sequence of value functions.\n",
"\n",
" * mcm is an instance of McCallModel\n",
" * ax is an axes object that implements a plot method.\n",
"\n",
" \"\"\"\n",
"\n",
" n = len(mcm.w)\n",
" v = mcm.w / (1 - mcm.β)\n",
" v_next = np.empty_like(v)\n",
" for i in range(num_plots):\n",
" ax.plot(mcm.w, v, '-', alpha=0.4, label=f\"iterate {i}\")\n",
" # Update guess\n",
" for i in range(n):\n",
" v_next[i] = np.max(mcm.state_action_values(i, v))\n",
" v[:] = v_next # copy contents into v\n",
"\n",
" ax.legend(loc='lower right')"
]
},
{
"cell_type": "markdown",
"id": "11c3c1d2",
"metadata": {},
"source": [
"Now let’s create an instance of `McCallModel` and watch iterations $ T^k v $ converge from below:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "706f0d6f",
"metadata": {
"hide-output": false
},
"outputs": [],
"source": [
"mcm = McCallModel()\n",
"\n",
"fig, ax = plt.subplots()\n",
"ax.set_xlabel('wage')\n",
"ax.set_ylabel('value')\n",
"plot_value_function_seq(mcm, ax)\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "6e5893db",
"metadata": {},
"source": [
"You can see that convergence is occurring: successive iterates are getting closer together.\n",
"\n",
"Here’s a more serious iteration effort to compute the limit, which continues until a discrepancy between successive iterates is below tol.\n",
"\n",
"Once we obtain a good approximation to the limit, we will use it to calculate\n",
"the reservation wage.\n",
"\n",
"We’ll be using JIT compilation via Numba to turbocharge our loops."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4882a70d",
"metadata": {
"hide-output": false
},
"outputs": [],
"source": [
"@jit(nopython=True)\n",
"def compute_reservation_wage(mcm,\n",
" max_iter=500,\n",
" tol=1e-6):\n",
"\n",
" # Simplify names\n",
" c, β, w, q = mcm.c, mcm.β, mcm.w, mcm.q\n",
"\n",
" # == First compute the value function == #\n",
"\n",
" n = len(w)\n",
" v = w / (1 - β) # initial guess\n",
" v_next = np.empty_like(v)\n",
" j = 0\n",
" error = tol + 1\n",
" while j < max_iter and error > tol:\n",
"\n",
" for i in range(n):\n",
" v_next[i] = np.max(mcm.state_action_values(i, v))\n",
"\n",
" error = np.max(np.abs(v_next - v))\n",
" j += 1\n",
"\n",
" v[:] = v_next # copy contents into v\n",
"\n",
" # == Now compute the reservation wage == #\n",
"\n",
" return (1 - β) * (c + β * np.sum(v * q))"
]
},
{
"cell_type": "markdown",
"id": "66bef247",
"metadata": {},
"source": [
"The next line computes the reservation wage at default parameters"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5784dbc3",
"metadata": {
"hide-output": false
},
"outputs": [],
"source": [
"compute_reservation_wage(mcm)"
]
},
{
"cell_type": "markdown",
"id": "c25a25a6",
"metadata": {},
"source": [
"### Comparative Statics\n",
"\n",
"Now that we know how to compute the reservation wage, let’s see how it varies with\n",
"parameters.\n",
"\n",
"In particular, let’s look at what happens when we change $ \\beta $ and\n",
"$ c $."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "809d94e5",
"metadata": {
"hide-output": false
},
"outputs": [],
"source": [
"grid_size = 25\n",
"R = np.empty((grid_size, grid_size))\n",
"\n",
"c_vals = np.linspace(10.0, 30.0, grid_size)\n",
"β_vals = np.linspace(0.9, 0.99, grid_size)\n",
"\n",
"for i, c in enumerate(c_vals):\n",
" for j, β in enumerate(β_vals):\n",
" mcm = McCallModel(c=c, β=β)\n",
" R[i, j] = compute_reservation_wage(mcm)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "983c8419",
"metadata": {
"hide-output": false
},
"outputs": [],
"source": [
"fig, ax = plt.subplots()\n",
"\n",
"cs1 = ax.contourf(c_vals, β_vals, R.T, alpha=0.75)\n",
"ctr1 = ax.contour(c_vals, β_vals, R.T)\n",
"\n",
"plt.clabel(ctr1, inline=1, fontsize=13)\n",
"plt.colorbar(cs1, ax=ax)\n",
"\n",
"\n",
"ax.set_title(\"reservation wage\")\n",
"ax.set_xlabel(\"$c$\", fontsize=16)\n",
"ax.set_ylabel(\"$β$\", fontsize=16)\n",
"\n",
"ax.ticklabel_format(useOffset=False)\n",
"\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "a3a924d8",
"metadata": {},
"source": [
"As expected, the reservation wage increases both with patience and with\n",
"unemployment compensation.\n",
"\n",
"\n",
""
]
},
{
"cell_type": "markdown",
"id": "475368f3",
"metadata": {},
"source": [
"## Computing an Optimal Policy: Take 2\n",
"\n",
"The approach to dynamic programming just described is standard and\n",
"broadly applicable.\n",
"\n",
"But for our McCall search model there’s also an easier way that circumvents the\n",
"need to compute the value function.\n",
"\n",
"Let $ h $ denote the continuation value:\n",
"\n",
"\n",
"\n",
"$$\n",
"h\n",
"= c + \\beta\n",
" \\sum_{s'} v^*(s') q (s')\n",
"\\quad \\tag{34.6}\n",
"$$\n",
"\n",
"The Bellman equation can now be written as\n",
"\n",
"$$\n",
"v^*(s')\n",
"= \\max \\left\\{ \\frac{w(s')}{1 - \\beta}, \\, h \\right\\}\n",
"$$\n",
"\n",
"Substituting this last equation into [(34.6)](#equation-j1) gives\n",
"\n",
"\n",
"\n",
"$$\n",
"h\n",
"= c + \\beta\n",
" \\sum_{s' \\in \\mathbb S}\n",
" \\max \\left\\{\n",
" \\frac{w(s')}{1 - \\beta}, h\n",
" \\right\\} q (s')\n",
"\\quad \\tag{34.7}\n",
"$$\n",
"\n",
"This is a nonlinear equation that we can solve for $ h $.\n",
"\n",
"As before, we will use successive approximations:\n",
"\n",
"Step 1: pick an initial guess $ h $.\n",
"\n",
"Step 2: compute the update $ h' $ via\n",
"\n",
"\n",
"\n",
"$$\n",
"h'\n",
"= c + \\beta\n",
" \\sum_{s' \\in \\mathbb S}\n",
" \\max \\left\\{\n",
" \\frac{w(s')}{1 - \\beta}, h\n",
" \\right\\} q (s')\n",
"\\quad \\tag{34.8}\n",
"$$\n",
"\n",
"Step 3: calculate the deviation $ |h - h'| $.\n",
"\n",
"Step 4: if the deviation is larger than some fixed tolerance, set $ h = h' $ and go to step 2, else return $ h $.\n",
"\n",
"One can again use the Banach contraction mapping theorem to show that this process always converges.\n",
"\n",
"The big difference here, however, is that we’re iterating on a scalar $ h $, rather than an $ n $-vector, $ v(i), i = 1, \\ldots, n $.\n",
"\n",
"Here’s an implementation:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "47afa1c5",
"metadata": {
"hide-output": false
},
"outputs": [],
"source": [
"@jit(nopython=True)\n",
"def compute_reservation_wage_two(mcm,\n",
" max_iter=500,\n",
" tol=1e-5):\n",
"\n",
" # Simplify names\n",
" c, β, w, q = mcm.c, mcm.β, mcm.w, mcm.q\n",
"\n",
" # == First compute h == #\n",
"\n",
" h = np.sum(w * q) / (1 - β)\n",
" i = 0\n",
" error = tol + 1\n",
" while i < max_iter and error > tol:\n",
"\n",
" s = np.maximum(w / (1 - β), h)\n",
" h_next = c + β * np.sum(s * q)\n",
"\n",
" error = np.abs(h_next - h)\n",
" i += 1\n",
"\n",
" h = h_next\n",
"\n",
" # == Now compute the reservation wage == #\n",
"\n",
" return (1 - β) * h"
]
},
{
"cell_type": "markdown",
"id": "14885412",
"metadata": {},
"source": [
"You can use this code to solve the exercise below."
]
},
{
"cell_type": "markdown",
"id": "59333a32",
"metadata": {},
"source": [
"## Exercises"
]
},
{
"cell_type": "markdown",
"id": "7c8d5c45",
"metadata": {},
"source": [
"## Exercise 34.1\n",
"\n",
"Compute the average duration of unemployment when $ \\beta=0.99 $ and\n",
"$ c $ takes the following values\n",
"\n",
"> `c_vals = np.linspace(10, 40, 25)`\n",
"\n",
"\n",
"That is, start a worker off as unemployed, compute a reservation wage\n",
"given the parameters, and then simulate to see how long it takes the worker to accept.\n",
"\n",
"Repeat a large number of times and take the average.\n",
"\n",
"Plot mean unemployment duration as a function of $ c $ in `c_vals`."
]
},
{
"cell_type": "markdown",
"id": "6652a592",
"metadata": {},
"source": [
"## Solution to[ Exercise 34.1](https://python.quantecon.org/#mm_ex1)\n",
"\n",
"Here’s one solution"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "224d46ad",
"metadata": {
"hide-output": false
},
"outputs": [],
"source": [
"cdf = np.cumsum(q_default)\n",
"\n",
"@jit(nopython=True)\n",
"def compute_stopping_time(w_bar, seed=1234):\n",
"\n",
" np.random.seed(seed)\n",
" t = 1\n",
" while True:\n",
" # Generate a wage draw\n",
" w = w_default[qe.random.draw(cdf)]\n",
" # Stop when the draw is above the reservation wage\n",
" if w >= w_bar:\n",
" stopping_time = t\n",
" break\n",
" else:\n",
" t += 1\n",
" return stopping_time\n",
"\n",
"@jit(nopython=True)\n",
"def compute_mean_stopping_time(w_bar, num_reps=100000):\n",
" obs = np.empty(num_reps)\n",
" for i in range(num_reps):\n",
" obs[i] = compute_stopping_time(w_bar, seed=i)\n",
" return obs.mean()\n",
"\n",
"c_vals = np.linspace(10, 40, 25)\n",
"stop_times = np.empty_like(c_vals)\n",
"for i, c in enumerate(c_vals):\n",
" mcm = McCallModel(c=c)\n",
" w_bar = compute_reservation_wage_two(mcm)\n",
" stop_times[i] = compute_mean_stopping_time(w_bar)\n",
"\n",
"fig, ax = plt.subplots()\n",
"\n",
"ax.plot(c_vals, stop_times, label=\"mean unemployment duration\")\n",
"ax.set(xlabel=\"unemployment compensation\", ylabel=\"months\")\n",
"ax.legend()\n",
"\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "d18d8773",
"metadata": {},
"source": [
"## Exercise 34.2\n",
"\n",
"The purpose of this exercise is to show how to replace the discrete wage\n",
"offer distribution used above with a continuous distribution.\n",
"\n",
"This is a significant topic because many convenient distributions are\n",
"continuous (i.e., have a density).\n",
"\n",
"Fortunately, the theory changes little in our simple model.\n",
"\n",
"Recall that $ h $ in [(34.6)](#equation-j1) denotes the value of not accepting a job in this period but\n",
"then behaving optimally in all subsequent periods:\n",
"\n",
"To shift to a continuous offer distribution, we can replace [(34.6)](#equation-j1) by\n",
"\n",
"\n",
"\n",
"$$\n",
"h\n",
"= c + \\beta\n",
" \\int v^*(s') q (s') ds'.\n",
"\\quad \\tag{34.9}\n",
"$$\n",
"\n",
"Equation [(34.7)](#equation-j2) becomes\n",
"\n",
"\n",
"\n",
"$$\n",
"h\n",
"= c + \\beta\n",
" \\int\n",
" \\max \\left\\{\n",
" \\frac{w(s')}{1 - \\beta}, h\n",
" \\right\\} q (s') d s'\n",
"\\quad \\tag{34.10}\n",
"$$\n",
"\n",
"The aim is to solve this nonlinear equation by iteration, and from it obtain\n",
"the reservation wage.\n",
"\n",
"Try to carry this out, setting\n",
"\n",
"- the state sequence $ \\{ s_t \\} $ to be IID and standard normal and \n",
"- the wage function to be $ w(s) = \\exp(\\mu + \\sigma s) $. \n",
"\n",
"\n",
"You will need to implement a new version of the `McCallModel` class that\n",
"assumes a lognormal wage distribution.\n",
"\n",
"Calculate the integral by Monte Carlo, by averaging over a large number of wage draws.\n",
"\n",
"For default parameters, use `c=25, β=0.99, σ=0.5, μ=2.5`.\n",
"\n",
"Once your code is working, investigate how the reservation wage changes with $ c $ and $ \\beta $."
]
},
{
"cell_type": "markdown",
"id": "7e86e0fc",
"metadata": {},
"source": [
"## Solution to[ Exercise 34.2](https://python.quantecon.org/#mm_ex2)\n",
"\n",
"Here is one solution:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "076af766",
"metadata": {
"hide-output": false
},
"outputs": [],
"source": [
"mccall_data_continuous = [\n",
" ('c', float64), # unemployment compensation\n",
" ('β', float64), # discount factor\n",
" ('σ', float64), # scale parameter in lognormal distribution\n",
" ('μ', float64), # location parameter in lognormal distribution\n",
" ('w_draws', float64[:]) # draws of wages for Monte Carlo\n",
"]\n",
"\n",
"@jitclass(mccall_data_continuous)\n",
"class McCallModelContinuous:\n",
"\n",
" def __init__(self, c=25, β=0.99, σ=0.5, μ=2.5, mc_size=1000):\n",
"\n",
" self.c, self.β, self.σ, self.μ = c, β, σ, μ\n",
"\n",
" # Draw and store shocks\n",
" np.random.seed(1234)\n",
" s = np.random.randn(mc_size)\n",
" self.w_draws = np.exp(μ+ σ * s)\n",
"\n",
"\n",
"@jit(nopython=True)\n",
"def compute_reservation_wage_continuous(mcmc, max_iter=500, tol=1e-5):\n",
"\n",
" c, β, σ, μ, w_draws = mcmc.c, mcmc.β, mcmc.σ, mcmc.μ, mcmc.w_draws\n",
"\n",
" h = np.mean(w_draws) / (1 - β) # initial guess\n",
" i = 0\n",
" error = tol + 1\n",
" while i < max_iter and error > tol:\n",
"\n",
" integral = np.mean(np.maximum(w_draws / (1 - β), h))\n",
" h_next = c + β * integral\n",
"\n",
" error = np.abs(h_next - h)\n",
" i += 1\n",
"\n",
" h = h_next\n",
"\n",
" # == Now compute the reservation wage == #\n",
"\n",
" return (1 - β) * h"
]
},
{
"cell_type": "markdown",
"id": "87cc3fc6",
"metadata": {},
"source": [
"Now we investigate how the reservation wage changes with $ c $ and\n",
"$ \\beta $.\n",
"\n",
"We will do this using a contour plot."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "13b819e3",
"metadata": {
"hide-output": false
},
"outputs": [],
"source": [
"grid_size = 25\n",
"R = np.empty((grid_size, grid_size))\n",
"\n",
"c_vals = np.linspace(10.0, 30.0, grid_size)\n",
"β_vals = np.linspace(0.9, 0.99, grid_size)\n",
"\n",
"for i, c in enumerate(c_vals):\n",
" for j, β in enumerate(β_vals):\n",
" mcmc = McCallModelContinuous(c=c, β=β)\n",
" R[i, j] = compute_reservation_wage_continuous(mcmc)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6dd82370",
"metadata": {
"hide-output": false
},
"outputs": [],
"source": [
"fig, ax = plt.subplots()\n",
"\n",
"cs1 = ax.contourf(c_vals, β_vals, R.T, alpha=0.75)\n",
"ctr1 = ax.contour(c_vals, β_vals, R.T)\n",
"\n",
"plt.clabel(ctr1, inline=1, fontsize=13)\n",
"plt.colorbar(cs1, ax=ax)\n",
"\n",
"\n",
"ax.set_title(\"reservation wage\")\n",
"ax.set_xlabel(\"$c$\", fontsize=16)\n",
"ax.set_ylabel(\"$β$\", fontsize=16)\n",
"\n",
"ax.ticklabel_format(useOffset=False)\n",
"\n",
"plt.show()"
]
}
],
"metadata": {
"date": 1680677903.4880998,
"filename": "mccall_model.md",
"kernelspec": {
"display_name": "Python",
"language": "python3",
"name": "python3"
},
"title": "Job Search I: The McCall Search Model"
},
"nbformat": 4,
"nbformat_minor": 5
}