{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Likelihood Ratio Processes and Bayesian Learning" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Contents\n", "\n", "- [Likelihood Ratio Processes and Bayesian Learning](#Likelihood-Ratio-Processes-and-Bayesian-Learning) \n", " - [Overview](#Overview) \n", " - [The Setting](#The-Setting) \n", " - [Likelihood Ratio Process and Bayes’ Law](#Likelihood-Ratio-Process-and-Bayes’-Law) \n", " - [Sequels](#Sequels) " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide-output": false }, "outputs": [], "source": [ "%matplotlib inline\n", "import matplotlib.pyplot as plt\n", "plt.rcParams[\"figure.figsize\"] = (11, 5) #set default figure size\n", "import numpy as np\n", "from numba import vectorize, njit\n", "from math import gamma" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Overview\n", "\n", "This lecture describes the role that **likelihood ratio processes** play in **Bayesian learning**.\n", "\n", "As in [this lecture](https://python.quantecon.org/likelihood_ratio_process.html), we’ll use a simple statistical setting from [this lecture](https://python.quantecon.org/exchangeable.html).\n", "\n", "We’ll focus on how a likelihood ratio process and a **prior** probability determine a **posterior** probability.\n", "\n", "We’ll derive a convenient recursion for today’s posterior as a function of yesterday’s posterior and\n", "today’s multiplicative increment to a likelihood process.\n", "\n", "We’ll also present a useful generalization of that formula that represents today’s posterior in terms of an initial prior and\n", "today’s realization of the likelihood ratio process.\n", "\n", "We’ll study how, at least in our setting, a Bayesian eventually learns the probability distribution that generates the data, an outcome that\n", "rests on the asymptotic behavior of likelihood ratio processes studied in [this lecture](https://python.quantecon.org/likelihood_ratio_process.html).\n", "\n", "This lecture provides technical results that underly outcomes to be studied in [this lecture](https://python.quantecon.org/odu.html)\n", "and [this lecture](https://python.quantecon.org/wald_friedman.html) and [this lecture](https://python.quantecon.org/navy_captain.html)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## The Setting\n", "\n", "We begin by reviewing the setting in [this lecture](https://python.quantecon.org/likelihood_ratio_process.html), which we adopt here too.\n", "\n", "A nonnegative random variable $W$ has one of two probability density functions, either\n", "$f$ or $g$.\n", "\n", "Before the beginning of time, nature once and for all decides whether she will draw a sequence of IID draws from $f$ or from $g$.\n", "\n", "We will sometimes let $q$ be the density that nature chose once and for all, so\n", "that $q$ is either $f$ or $g$, permanently.\n", "\n", "Nature knows which density it permanently draws from, but we the observers do not.\n", "\n", "We do know both $f$ and $g$, but we don’t know which density nature\n", "chose.\n", "\n", "But we want to know.\n", "\n", "To do that, we use observations.\n", "\n", "We observe a sequence $\\{w_t\\}_{t=1}^T$ of $T$ IID draws\n", "from either $f$ or $g$.\n", "\n", "We want to use these observations to infer whether nature chose $f$ or\n", "$g$.\n", "\n", "A **likelihood ratio process** is a useful tool for this task.\n", "\n", "To begin, we define the key component of a likelihood ratio process, namely, the time $t$ likelihood ratio as the random variable\n", "\n", "$$\n", "\\ell (w_t)=\\frac{f\\left(w_t\\right)}{g\\left(w_t\\right)},\\quad t\\geq1.\n", "$$\n", "\n", "We assume that $f$ and $g$ both put positive probabilities on the\n", "same intervals of possible realizations of the random variable $W$.\n", "\n", "That means that under the $g$ density, $\\ell (w_t)=\n", "\\frac{f\\left(w_{t}\\right)}{g\\left(w_{t}\\right)}$\n", "is evidently a nonnegative random variable with mean $1$.\n", "\n", "A **likelihood ratio process** for sequence\n", "$\\left\\{ w_{t}\\right\\} _{t=1}^{\\infty}$ is defined as\n", "\n", "$$\n", "L\\left(w^{t}\\right)=\\prod_{i=1}^{t} \\ell (w_i),\n", "$$\n", "\n", "where $w^t=\\{ w_1,\\dots,w_t\\}$ is a history of\n", "observations up to and including time $t$.\n", "\n", "Sometimes for shorthand we’ll write $L_t = L(w^t)$.\n", "\n", "Notice that the likelihood process satisfies the *recursion* or\n", "*multiplicative decomposition*\n", "\n", "$$\n", "L(w^t) = \\ell (w_t) L (w^{t-1}) .\n", "$$\n", "\n", "The likelihood ratio and its logarithm are key tools for making\n", "inferences using a classic frequentist approach due to Neyman and\n", "Pearson [[NP33](https://python.quantecon.org/zreferences.html#id212)].\n", "\n", "We’ll again deploy the following Python code from [this lecture](https://python.quantecon.org/likelihood_ratio_process.html) that\n", "evaluates $f$ and $g$ as two different\n", "beta distributions, then computes and simulates an associated likelihood\n", "ratio process by generating a sequence $w^t$ from *some*\n", "probability distribution, for example, a sequence of IID draws from $g$." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide-output": false }, "outputs": [], "source": [ "# Parameters in the two beta distributions.\n", "F_a, F_b = 1, 1\n", "G_a, G_b = 3, 1.2\n", "\n", "@vectorize\n", "def p(x, a, b):\n", " r = gamma(a + b) / (gamma(a) * gamma(b))\n", " return r * x** (a-1) * (1 - x) ** (b-1)\n", "\n", "# The two density functions.\n", "f = njit(lambda x: p(x, F_a, F_b))\n", "g = njit(lambda x: p(x, G_a, G_b))" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide-output": false }, "outputs": [], "source": [ "@njit\n", "def simulate(a, b, T=50, N=500):\n", " '''\n", " Generate N sets of T observations of the likelihood ratio,\n", " return as N x T matrix.\n", "\n", " '''\n", "\n", " l_arr = np.empty((N, T))\n", "\n", " for i in range(N):\n", "\n", " for j in range(T):\n", " w = np.random.beta(a, b)\n", " l_arr[i, j] = f(w) / g(w)\n", "\n", " return l_arr" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We’ll also use the following Python code to prepare some informative simulations" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide-output": false }, "outputs": [], "source": [ "l_arr_g = simulate(G_a, G_b, N=50000)\n", "l_seq_g = np.cumprod(l_arr_g, axis=1)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide-output": false }, "outputs": [], "source": [ "l_arr_f = simulate(F_a, F_b, N=50000)\n", "l_seq_f = np.cumprod(l_arr_f, axis=1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Likelihood Ratio Process and Bayes’ Law\n", "\n", "Let $\\pi_t$ be a Bayesian posterior defined as\n", "\n", "$$\n", "\\pi_t = {\\rm Prob}(q=f|w^t)\n", "$$\n", "\n", "The likelihood ratio process is a principal actor in the formula that governs the evolution\n", "of the posterior probability $\\pi_t$, an instance of **Bayes’ Law**.\n", "\n", "Bayes’ law implies that $\\{\\pi_t\\}$ obeys the recursion\n", "\n", "\n", "\n", "$$\n", "\\pi_t=\\frac{\\pi_{t-1} l_t(w_t)}{\\pi_{t-1} l_t(w_t)+1-\\pi_{t-1}} \\tag{1}\n", "$$\n", "\n", "with $\\pi_{0}$ being a Bayesian prior probability that $q = f$,\n", "i.e., a personal or subjective belief about $q$ based on our having seen no data.\n", "\n", "Below we define a Python function that updates belief $\\pi$ using\n", "likelihood ratio $\\ell$ according to recursion [(40.1)](#equation-eq-recur1)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide-output": false }, "outputs": [], "source": [ "@njit\n", "def update(π, l):\n", " \"Update π using likelihood l\"\n", "\n", " # Update belief\n", " π = π * l / (π * l + 1 - π)\n", "\n", " return π" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Formula [(40.1)](#equation-eq-recur1) can be generalized by iterating on it and thereby deriving an\n", "expression for the time $t$ posterior $\\pi_{t+1}$ as a function\n", "of the time $0$ prior $\\pi_0$ and the likelihood ratio process\n", "$L(w^{t+1})$ at time $t$.\n", "\n", "To begin, notice that the updating rule\n", "\n", "$$\n", "\\pi_{t+1}\n", "=\\frac{\\pi_{t}\\ell \\left(w_{t+1}\\right)}\n", "{\\pi_{t}\\ell \\left(w_{t+1}\\right)+\\left(1-\\pi_{t}\\right)}\n", "$$\n", "\n", "implies\n", "\n", "\n", "\\begin{aligned}\n", "\\frac{1}{\\pi_{t+1}}\n", " &=\\frac{\\pi_{t}\\ell \\left(w_{t+1}\\right)\n", " +\\left(1-\\pi_{t}\\right)}{\\pi_{t}\\ell \\left(w_{t+1}\\right)} \\\\\n", " &=1-\\frac{1}{\\ell \\left(w_{t+1}\\right)}\n", " +\\frac{1}{\\ell \\left(w_{t+1}\\right)}\\frac{1}{\\pi_{t}}.\n", "\\end{aligned}\n", "\n", "\n", "$$\n", "\\Rightarrow\n", "\\frac{1}{\\pi_{t+1}}-1\n", "=\\frac{1}{\\ell \\left(w_{t+1}\\right)}\\left(\\frac{1}{\\pi_{t}}-1\\right).\n", "$$\n", "\n", "Therefore\n", "\n", "\n", "\\begin{aligned}\n", " \\frac{1}{\\pi_{t+1}}-1\n", " =\\frac{1}{\\prod_{i=1}^{t+1}\\ell \\left(w_{i}\\right)}\n", " \\left(\\frac{1}{\\pi_{0}}-1\\right)\n", " =\\frac{1}{L\\left(w^{t+1}\\right)}\\left(\\frac{1}{\\pi_{0}}-1\\right).\n", "\\end{aligned}\n", "\n", "\n", "Since $\\pi_{0}\\in\\left(0,1\\right)$ and\n", "$L\\left(w^{t+1}\\right)>0$, we can verify that\n", "$\\pi_{t+1}\\in\\left(0,1\\right)$.\n", "\n", "After rearranging the preceding equation, we can express $\\pi_{t+1}$ as a\n", "function of $L\\left(w^{t+1}\\right)$, the likelihood ratio process at $t+1$,\n", "and the initial prior $\\pi_{0}$\n", "\n", "\n", "\n", "$$\n", "\\pi_{t+1}=\\frac{\\pi_{0}L\\left(w^{t+1}\\right)}{\\pi_{0}L\\left(w^{t+1}\\right)+1-\\pi_{0}} . \\tag{2}\n", "$$\n", "\n", "Formula [(40.2)](#equation-eq-bayeslaw103) generalizes formula [(40.1)](#equation-eq-recur1).\n", "\n", "Formula [(40.2)](#equation-eq-bayeslaw103) can be regarded as a one step revision of prior probability $\\pi_0$ after seeing\n", "the batch of data $\\left\\{ w_{i}\\right\\} _{i=1}^{t+1}$.\n", "\n", "Formula [(40.2)](#equation-eq-bayeslaw103) shows the key role that the likelihood ratio process $L\\left(w^{t+1}\\right)$ plays in determining\n", "the posterior probability $\\pi_{t+1}$.\n", "\n", "Formula [(40.2)](#equation-eq-bayeslaw103) is the foundation for the insight that, because of how the likelihood ratio process behaves\n", "as $t \\rightarrow + \\infty$, the likelihood ratio process dominates the initial prior $\\pi_0$ in determining the\n", "limiting behavior of $\\pi_t$.\n", "\n", "To illustrate this insight, below we will plot graphs showing **one** simulated\n", "path of the likelihood ratio process $L_t$ along with two paths of\n", "$\\pi_t$ that are associated with the *same* realization of the likelihood ratio process but *different* initial prior probabilities $\\pi_{0}$.\n", "\n", "First, we tell Python two values of $\\pi_0$." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide-output": false }, "outputs": [], "source": [ "π1, π2 = 0.2, 0.8" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Next we generate paths of the likelihood ratio process $L_t$ and the posterior $\\pi_t$ for a\n", "history of IID draws from density $f$." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide-output": false }, "outputs": [], "source": [ "T = l_arr_f.shape\n", "π_seq_f = np.empty((2, T+1))\n", "π_seq_f[:, 0] = π1, π2\n", "\n", "for t in range(T):\n", " for i in range(2):\n", " π_seq_f[i, t+1] = update(π_seq_f[i, t], l_arr_f[0, t])" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide-output": false }, "outputs": [], "source": [ "fig, ax1 = plt.subplots()\n", "\n", "for i in range(2):\n", " ax1.plot(range(T+1), π_seq_f[i, :], label=f\"$\\pi_0$={π_seq_f[i, 0]}\")\n", "\n", "ax1.set_ylabel(\"$\\pi_t$\")\n", "ax1.set_xlabel(\"t\")\n", "ax1.legend()\n", "ax1.set_title(\"when f governs data\")\n", "\n", "ax2 = ax1.twinx()\n", "ax2.plot(range(1, T+1), np.log(l_seq_f[0, :]), '--', color='b')\n", "ax2.set_ylabel(\"$log(L(w^{t}))$\")\n", "\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The dotted line in the graph above records the logarithm of the likelihood ratio process $\\log L(w^t)$.\n", "\n", "Please note that there are two different scales on the $y$ axis.\n", "\n", "Now let’s study what happens when the history consists of IID draws from density $g$" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide-output": false }, "outputs": [], "source": [ "T = l_arr_g.shape\n", "π_seq_g = np.empty((2, T+1))\n", "π_seq_g[:, 0] = π1, π2\n", "\n", "for t in range(T):\n", " for i in range(2):\n", " π_seq_g[i, t+1] = update(π_seq_g[i, t], l_arr_g[0, t])" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide-output": false }, "outputs": [], "source": [ "fig, ax1 = plt.subplots()\n", "\n", "for i in range(2):\n", " ax1.plot(range(T+1), π_seq_g[i, :], label=f\"$\\pi_0$={π_seq_g[i, 0]}\")\n", "\n", "ax1.set_ylabel(\"$\\pi_t$\")\n", "ax1.set_xlabel(\"t\")\n", "ax1.legend()\n", "ax1.set_title(\"when g governs data\")\n", "\n", "ax2 = ax1.twinx()\n", "ax2.plot(range(1, T+1), np.log(l_seq_g[0, :]), '--', color='b')\n", "ax2.set_ylabel(\"$log(L(w^{t}))$\")\n", "\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Below we offer Python code that verifies that nature chose permanently to draw from density $f$." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide-output": false }, "outputs": [], "source": [ "π_seq = np.empty((2, T+1))\n", "π_seq[:, 0] = π1, π2\n", "\n", "for i in range(2):\n", " πL = π_seq[i, 0] * l_seq_f[0, :]\n", " π_seq[i, 1:] = πL / (πL + 1 - π_seq[i, 0])" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide-output": false }, "outputs": [], "source": [ "np.abs(π_seq - π_seq_f).max() < 1e-10" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We thus conclude that the likelihood ratio process is a key ingredient of the formula [(40.2)](#equation-eq-bayeslaw103) for\n", "a Bayesian’s posteior probabilty that nature has drawn history $w^t$ as repeated draws from density\n", "$g$." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Sequels\n", "\n", "This lecture has been devoted to building some useful infrastructure.\n", "\n", "We’ll build on results highlighted in this lectures to understand inferences that are the foundations of\n", "results described in [this lecture](https://python.quantecon.org/odu.html) and [this lecture](https://python.quantecon.org/wald_friedman.html) and [this lecture](https://python.quantecon.org/navy_captain.html)." ] } ], "metadata": { "date": 1623047193.3532643, "filename": "likelihood_bayes.md", "kernelspec": { "display_name": "Python", "language": "python3", "name": "python3" }, "title": "Likelihood Ratio Processes and Bayesian Learning" }, "nbformat": 4, "nbformat_minor": 4 }