Commit a7105df5 authored by gmontavon's avatar gmontavon
Browse files

...

parent 119fe86f
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<h1 align='center'>Tutorial: Implementing Layer-Wise Relevance Propagation</h1>\n",
"<br>\n",
"<p><em>\n",
"first version: Jul 14, 2016<br>\n",
"last update: Sep 17, 2019\n",
"</em></p>\n",
"\n",
"<p>This tutorial explains how to implement layer-wise relevance propagation (LRP) easily and efficiently, as described in the overview paper:\n",
"\n",
"<blockquote style='background-color:#EEEEEE; padding: 3px; border: 1px dashed #999999'>\n",
"G. Montavon, A. Binder, S. Lapuschkin, W. Samek, K.-R. Müller<br><a href=\"https://doi.org/10.1007/978-3-030-28954-6_10\">Layer-wise Relevance Propagation: An Overview</a><br><font color=\"#008800\">in Explainable AI: Interpreting, Explaining and Visualizing Deep Learning, Springer LNCS, vol. 11700,\n",
"2019</font>\n",
"</blockquote>\n",
"\n",
"We consider two models: (1) a simple plain deep rectifier network trained on the MNIST handwritten digits data, (2) the VGG-16 network trained on ImageNet and applicable to general image classification.</p>\n",
"\n",
"<p><em>Note:</em> If you are instead looking for ready to use software, have a look at the <a href=\"http://www.heatmapping.org/index.html#software\">software section</a> of this website. If you want to try relevance propagation without installing software, check our <a href=\"http://www.heatmapping.org/index.html#demos\">interactive demos</a>. For the original paper on LRP, see instead:</p>\n",
"\n",
"<blockquote style='background-color:#EEEEEE; padding: 3px; border: 1px dashed #999999'>\n",
"S. Bach, A. Binder, G. Montavon, F. Klauschen, K.-R. Müller, W. Samek<br><a href=\"https://doi.org/10.1371/journal.pone.0130140\">On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation</a><br><font color=\"#008800\">PloS ONE 10 (7), e0130140,\n",
"2015</font>\n",
"</blockquote>\n",
"\n",
"<h3>Prerequisites</h3>\n",
"\n",
"<p>For this tutorial, you need to install Python, Numpy, PyTorch, Matplotlib, and OpenCV. Then, you should download <a href=\"tutorial.zip\">tutorial.zip</a> and extract it in your working directory. It contains data, model parameters, and some additional functions (in the file <code>utils.py</code>)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<h2>1 &nbsp; Numpy Implementation for a Fully-Connected Network</h2>\n",
"\n",
"<p>We first load 12 examplary MNIST test digits.</p>"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAq0AAABYCAYAAADfnM4oAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi40LCBodHRwOi8vbWF0cGxvdGxpYi5vcmcv7US4rQAAGcZJREFUeJzt3Xmc1HUdx/EXmWVhh2VlalmJV5JkmnikiFqmViLZpYKYpUailUGmSYZ5FKR4ZKZlqVRSQh7gmXdaHolWloZd8MCiMimxu+gPHq/57fx2h92FOX4L7+c/q7uzw292Zn7z/X2+n2PQsmXLiIiIiIiosmd0+gAiIiIiInqTRWtEREREVF4WrRERERFReVm0RkRERETlZdEaEREREZWXRWtEREREVF4WrRERERFReVm0RkRERETlZdEaEREREZX3zHb+Y4MGDcr4rYiIiIioWbZs2aC+3C6R1oiIiIiovCxaIyIiIqLysmiNiIiIiMrLojUiIiIiKi+L1oiIiIiovLZ2D+ikY489FoDp06cD8OSTTwIwZcqUuu9HtMIzn7n8rXb66acDcNxxxwGw3377AXDdddd15sAiIiIGiERaIyIiIqLyBi1b1r7Wqe3o03rwwQcDsNFGGwEwZswYALbaaiuPocffmzFjBoceemirDy/WUK95zWsAmD9/PgA33XQTAEcffTQAjz32WGcObCU95znPAWDzzTcH4Igjjqj7/7322guAOXPmAMsjzD/72c8AWLJkSVuPtS9e/OIXA/C+970PgC222AIoHtcTTzwBwCmnnALAzJkzAfjzn//c1uNc05x88skAfPrTn677/m233QbA7bff3uPtI6J3w4cPB2DfffcF4KSTTurxdq6bFi5cCMCb3/xmAB599NGmHUv6tEZERETEamO1ibR6xb3LLrv4b/Xr959++ml22203AB588MHmHlwLjRgxAoDPfOYzAJx55pkAXH311R07ppVhZOt73/seABtvvHGPt5s3bx6wPIJ37733AjB37lwA2vla7q/LL78cgHe9610A/PCHPwSK5+8///lPZw6sn7beemsAvvnNbwIwdOjQPv/uL37xCwDGjx8PwK233trko+u/cePGAXDiiScCsOmmm/bp9zxHjB8/vvZcVsHs2bMBGDVqFFBE9j0fbrbZZkDj94q3u/3229l9991beagrtLLvZSOwng/9/4Hqec97HgA77bRT3fcPOeQQALbddlugeF8CXHHFFUCxy/jPf/6z5ccZ/XfggQcCMHnyZAB+9atfAcV7tx38PN1mm2369XsPPPAAAIcddhgAP/3pT1f5WBJpjYiIiIjVxoDvHmCu08477ww0jrCak/ad73wHgD333BMoIg+DBw8eUJHWffbZB1ieiwvwwhe+EIB11lkHgB/84AcA/PGPf+zA0fVurbXWAmDLLbcE4NprrwVgww03BOB///tfj783bNiwuq8Ao0ePBuCqq65qzcGuBB/XF77wBaB4vowg3XfffcDAibC+7W1vA4ooztprrw0UOU5Gcx5++GGA2ntpvfXWq92H+a6nnnoqAO95z3vq7qMdPD8YhfrSl74EwLOf/ex+3c/rX/96AG655ZbafxtJbifPexMmTABg//33B4rX2ZAhQ+pu3+h9JX9vxx13rOWtmX/dDqsa3fX3/drfHbd2MzfcSNc73/lOAF7+8pcD8Na3vhWAF73oRXW/t3TpUgD+/ve/A/WRaSN47j5+8YtfbMmx94U5xj4uTZs2DYBLLrmk3Ye0Svx8dedMRx55JACLFy/u0/1ssMEGfPnLXwbgBS94AdDeXYGPfOQjQPfzg3w9+fp61rOeBRRdcN7whjcAxedcMyKtfTXgF62bbLIJAM94Rs9B4xtuuAGA0047DYDvf//7AFx22WVAsWgdaCzg8QWv7bffHoC9994bKBa1VeF216RJkwA44YQTerzdv/71r7qv6667bsP73HHHHYFqLFp32GEHAKZOnQrAm970prqf33zzzUD3wpKq8u9+0UUXAcVJ65hjjgGK99Ff//rXut9zIXfQQQcBcMcdd9QWiCb/P/TQQwC8+93vBorUkFZ4/vOfX/dvXXjhhSu8/T333APA17/+daBYRPi69cNrnXXW4b3vfS9QFGm1I03F97dpGl60lv3pT38CaJjC4PnP9BytvfbatYuLRx55BGjvxUVZebvfry5OG6WauGjqdIGWF+lvfOMbgeJ19NrXvhbo/jnkYtvXkkEXL3J9XXp+71pE7DmnncWdpiccddRRQLFI3WCDDYDu74mLL74YKBbj5557LlD9i3iDJV7Ey4uP3rz0pS8Fln9W+dlt+83zzz+/WYfZq1e84hUAPPe5z637/k9+8hMAfvSjHwFw+OGHA/DRj34UKFo0Wlj8u9/9rvUHW5L0gIiIiIiovAFfiLXddtsBxfAAfe1rXwOKCINhbhkhMhIExdXEOeec0+zDbBqvwG0lZMSlfGVuAYatvjrNbf877rgDgFe/+tU93s6IwsSJEwG46667gKLFhtuhXZ830zmMcv73v/9t6rH3hZHF66+/HoCXvOQldT83smdk2avrqnMbyTQHt8G+8pWv9Pu+jHb6t7Ao7YILLgDgwx/+8Kod7Ar+TVtU+X5pxOfFSFF5y86iiZ4id0bNjEy2gv+Gx2WrrjIjWWeffTbQePvula98JVCkahjR7JouYbTF6F4V9fY51uk0AVsJ+bopn6/lboMFMu7M+Dn21FNPtfxY+8NdriuvvBLoft5r9DjLPzetyNftWWed1fyDXQXurMyaNQsoziOmS/l51Fv6jWsMd+KgeH+1M1XC87nnd33oQx8CGu9Eme7l510zi2lTiBURERERq40Bn9Nq7sXYsWM7fCTtYWJ+o4jRb37zGwAOOOCAdh3SCq2//vpAkevUKMJqArtNjsvFcOZn/fvf/wbqI62///3vgc60vPKK09xpH69X3BbEeQU70JhrZwTSx7kyzHu1CM3ouUMHWsHc4UbvF58nc+qMpJj7XmaEwuEJXXOWLbZrVaR18ODBtfG/jSKsDke45pprgO47TGULFiwAisiPkfRG79OqKue4Vo2ReyOLDqWwyM3XnZG7gcKdjHKEVYsWLQKKwivf6x/4wAeAYrfFHQRzYKvCGgwjkuXziMVUvUVY3cmwYBKKArl2RlgtqDJy3F9+DnSyXWEirRERERFReQM+0tpfXjmVm9cvWbKkVolbZZ/61KdW+POf//znQGvz6vrCiKO5WOXIjS2S7rzzTqCIlPfWMuQvf/kLsPzK1o4RVkK2M2/N6klzWH288urZiMJAZeTbqtJVqSB/1ateBRS5q3YPaEVLHqu1rXYt+8c//gEU+WTf+ta3+nS/Tz/9NFDkjJe7Q7TSueee23DUtMNEjNh1Iq+7k3qLsHa6i4C7QH69//77gSIyvrKsFTjppJOYMmUK0N6KbrtqlM+97oDaLaHMv4Pvn0b302m+Xso1Mw4C8HH25rOf/SxQ5JDfdNNNtV2TdnrLW94CFDufA1EirRERERFReWtMpNUIq1VxNj/XI488UutpWEU2DX/d617X48+XLFkCwHnnnde2Y1oRo2qNcuNuvPFGoP8j6xwOcfbZZ/Oyl70MgF/+8pdAe3Na7TVazuW65ZZbgKL7QV9ZlbnrrrvWRjPaaaGT+UPmoJX7+fXXyJEja7sE9lZuZa9a+7E60EDmnvU3wir7MzaKeLaSr7mu3LEw13Z1j7CubKS00+NcP/e5zwFFdbxjWVd2gIP50/Y+vvHGGzvSM3OPPfYAinOvw2wcp96IXRL86ljaqozi9txUrkVwYIO5uOX+1GXTp08His/Bxx9/HFgeua16T9qqSqQ1IiIiIipvjYm0eqVjBEZGV8tj5qrGXJhGES+7BRid6xQnnHz729+u+74RIftArmzFpLmjTmaCIn+3twrOZrBq3KpRedVsP8a+9mH1/uwrbI4aFJ0S/FtZ2d1O5rL6+jIi3Nvjc0KTkcwRI0bURr9+/OMfB4o8zFbo+nfsykjkb3/7W6B4POYeN5psY9WwOx1dJ/DZIcKpYc1mZXV5lCdQy4tr1O2gGcxr7kSf1t4mXvWmPEGrUxz76Xn8jDPOAIqdP8di9va+8nxhn2T76brD02meL5xkWGauuVPkypOlrLI3gmw3nHZYd911a8fzyU9+Eig+Z+69914APvGJTwBF94cV3RcUEXHZFacTI5+hyOW336870PLzuVG+reukdo5tLUukNSIiIiIqb42JtB522GE9ft/+n/b6rCqrKhtVV3Y6wiqr6s1blJEOc7tWljmwjfpUtor9CL0S9Upab3/724HGM9493qFDhwJFF4g999yz7naLFi2qVdIaRTH/zVnVdlBoB6vkfbxOoDMXTUaQjEx84xvfAIpI5uabb177WTuiJ426UBjtbVZkcv78+YwZMwaApUuXNuU+y8xldW5513+rHV1C7EXcCaua9+z7tSo+//nPA0XUfNKkSUDxOWQOtpMB5U6Aj8dzxMEHHwwUkdxOe+CBB3r8vueBE088ESgmQ5UnZtkJxv7Q7Yy0Dh8+nBkzZtR9z8fjcdv1oTdGkjfddFOgiDz7/HeK5+2vfvWrQPeJWJ5jup5rurJPtxHXRp93rZRIa0RERERU3mofaR05ciTQvR+eOZbmFlWdV3pVqa7sydChQ5k9e3bd9+6++26gcaS7GZzq02xGDxcsWMDHPvYxoJi17fNghNuq0kb3YU6eXRU0d+5coMhtmz9/fu1vZmTVnoDtjLDK3sXHH388AHPmzAGKiSpGvo02GhmxStYJbu0+do979OjRQP+7VPTG/sKjRo3qc/7yyhoyZEi37/3hD38A2tNZwnzrdrJLwKpOuPLvU7X+n0aQjSwamXv00UeBYgfGCOsRRxwBFDsZZ555JgAzZ85s0xH3zHxu3/92E7AfqBFhc1QbTWLy+XHHs7/dFJqhp44g/p0bvc/cWXSnzXN2uXuMXQRamXveDk4s85xwzz33AEXvbftYt1IirRERERFReQM20mp1spEseWXq942Mlftp2jXAHp9VZZ84q+aN8JkjYxV0J1kRevLJJ7PRRhsB3bsF9Dbpqq+cnAVFhLVVVc3mly1YsKAWsZP9CL2i9vHq6KOPBooqVCMO9qe97rrrgGLmvc/vnDlzalfrsiK3E5xdf9pppwFw/vnnA8XjtarWx+vtfW6sVm03O0kYkWwWJ87Zp7HVUVboeaqXM8R93bSyx3Qn+n+W+7H2ltvqeabR7To9EavM9095Yp4R13nz5gHdJ2mdcsopwPI+1dCejikrYjW8nU7sguD5rZyz6lffN+b2+v2+TphqJqPdBxxwQLeIvLn5fu2NXUXKz8uECROAoq/r9OnTO3ZuhOJ84evQ+gn5fJpbbYRVfjb61RzlRFojIiIiIhgAkVarDp1+5Irefok9TYnpCyOCVgPOnDmzlrfYjuhJb4wMf/CDH+zx507AsS9oJw0bNgxYfqXqlZs5LuUq85V1zDHHAPWzrK3ENLrXbCs6dvNMff14he4VtblQft9JXuYmWy1slaZdFbbbbjv+9re/AUWFZyejQ+UeofajtV+wOa4PPfRQB46uMSMe5QhBI+ZmlafUuGNjNGarrbYC4IQTTgC6V9+2glE1e3MCbLzxxkBRxVvuG7y66G+EdMSIEUD3XFgjsLfddlvHe7Z25bnLvphGWsucXV+VSLGsHrf38rRp04D6PtpQRO58nXq78k6hE/jaaeHChQCMGzeu1gvbWphGudBGFD1veH4w8liuPbH/s8/jrbfeWvvdTjj99NOBom/9NttsU/dzc4t9PZrDWp4wqHe84x3Ayvdf749EWiMiIiKi8ioZaTVfdZ999uHYY48F6iNszWQux9ixY9l6660BePjhh4HWVrz3xrxGo5jlXJm77rqrMwfWhcdkz1EocmCaVXFsRMlJJObyQecrZ7tyGpKvGaP1TgYzim/PSyNBvr6NaD7xxBNMnjwZgAsuuKANR96z7bffHoCpU6cCRSTBCEN5Mls5d63TnGVuBKDMiTZWPZtr7MQsWR1sNbOT9YxwtiPSakRm8eLFtR0nGdEyr7x8/H1lhNIcWSiey5W9z06wi0ejrgO77757pSKtTiS67LLLgO6RPf+/CrULK2JuvpOSrMQ3YnfllVcCnenr2VezZs1i1qxZQHE+7jr5rivzUd0xM4JcjkR6nrCbgrmkP/7xj5t56E1nX3H7cnueNFe5PJ3PncVly5Zx6aWXtvTYEmmNiIiIiMqrVKTVXmdG7oz29MSqYCMNPc3l7sqcGqu+jUx0jVx4VWEFpNEX56e3k1d6RjuMsBolaNQXtJ2cLrT//vsDyzsaOOFlVRlhveqqq4DuuYmTJ0/uSN9SOdnKPoW+ZmSEZNttt627XTl3SEaoR48e3dFefr4HzFU1t9qIgjPOnRFuVwEjFN6u0xrlenseMPLRaIKPzF2+7777gCLS6k7I0KFDWz6H2zy/J598sluk9ZxzzgGK96CzzvsbHb366quB+klvdkowChjN5/vN/qae7+3S4Weiec3usHVqdn1v7GfaW/9g+7bK82Vfq/Rbra+5tXZzOPzww4Hic3rKlCkAXHPNNUDxOVY1vo6c+GgfcXfS3GkzJ9zXY3mn193xPfbYoxZVt1NCsyXSGhERERGVV6lIa08RVvtAGvmR0Q2r/fbee+8e79MrJq98rOL2ysLfnzhxYi3KYJ6rU4raGWkdPHgwUHRJKDvrrLOA1lXMr4qlS5eu8t/KSJFVluW55+Z7nnHGGW3NtTNSat6zz9MOO+zQ4+298vTnRhI8ZrsO+Lo2stLuzhXOxnaC1JZbbgkUER8nuZib6/vR99EVV1wBFM+buW2dVq5eln//RrlqjZjLZbWt54qRI0e2PNKqQw89lO9+97sAbLjhhnU/c4LSLrvsAhST2hpxp8n3a9cIKyyfaDZu3LhVPub+Mhe1VXmnVclnNaJV7jFt5xC7kLgTYOeILbbYAqhupLWv3LHQokWLgM70BF4ZdgNwx0bm7BqBrTp3zOw7ffnll9f93LWQk0P9nGpkzJgxtVoTz5nNlkhrRERERFRepSKtdgjoWoFspKpRRKvMnBIjPuYClasvzbcwZ2Pq1Km1qyajD0aP2snI1q677trjz80zW104KcqJSlbZd+0SAPURVmh/RbMReXMKze1p5KmnngKK152Rr2uvvRZo3VVofxxyyCG1Lg9GHj0+Z52XIx9G6OwTXFX2US3nyDn73OfDiXgXX3wxAHPnzgW6R7LK3RI0atSotkWX77///trzUt55krmpdjsoR07cYXJXy4hf2WOPPdbW6URGWMu5kEZGnXhVjpSWuwTYBaHM36tKpPWoo44Ciq4cRljtdyp3MuzvbDW6uZIDjTtUm2yySd333cGx5qSqPP8Z+TYn2fOFEcuBxjoKz//77rtvj7cbP358246pkUotWv0gNym9vHBZEQssLMBwMdofFl7Z7qETPOmWW584yrOTxUe9WW+99WqFIQ4DkCkfFvY43tRm7eVCul//+tdAsTgyVaPT7Xdsh+ai1QEA5bQIC6uq2NrEhtEXXnhh7eLNMbWevMp/Zz9cLbiy6b7jhKs2XMDFmkNIbE3lotWvpnv4OvO8UW7d1SidoN1Fc3feeSdQLLLf//731/3cFko+n+Xxw72xJY/Nx9ulUeGOi9JGLaz6qgqFq13Zws/X2fXXXw90HxLhmFeH7DRqdj9Q+Post4ay4LHqPH631WU6gC2+BhqDE55fTDPyImLIkCF9up/HH3+85WuUpAdEREREROVVKtK633771f3/pEmT2HnnnYEismUT2yVLltTd1sjDQEnkbsQr73Kkx20tIyFVYJTN7dEJEybUtg8OOuigutua0G2bLL+WGWE99dRTgeYNKWi2Bx98ECgKJhxfOxDstttuwPLnxKR5I6W26PL95vaxaRG29po4cSLQeDu903z/mCZgQaZNzy1cKutrgZaR9HLrnlZbunQpUESOLTgbO3ZsU+7f4it3ndrF89uqRlTLHMdZlbQAlc/z7mQ4mtpm7u5MGWHtrUVb1R144IFA8Xj8WvUIpYW1tnOSLQAbpesMNL7+jLCabun6qpHFixcDy8fO33333S08wkRaIyIiImIAGNTOsYuDBg2qxozHCvJKzitpcwZ13nnnAe0ZG9lf5vncfPPNtcT0vlq4cCFQ5NBdcsklQDEmL5rPtiVHHnlkbZypeUgWSDSKOBqRLQ9TGCh8XDvttBNQ5KZZBDp8+PAef8+cQwtgbrjhBqDzuXjm/buz4Q6FkfHeciAtADruuOOAIjfbQsJ2K+ewmuPf1whsVSOrZQ6raPQ+Ko9FdqTz8ccfD3Tu+VlV8+bNA4ohKz5Om/NXdWfNQjgLAt3xcNeqirULzeBQH2tPHPpTbtW21157Ab0PlViRZcuW9SlhO5HWiIiIiKi8RForYtiwYUDR6mr99dcHitY0dlTorWl4J6211lq1yvRLL7207mezZ88Gihygiy66CCgiCZ3uChCxOrGlkkMDrELfbLPNgGKspDUCnRwdvCayQ427HuZ6yrxAW16ZUzhQI6wqR1ptCejnX1Ufn8+HrTdnzJgB0JEBHKurRFojIiIiYrWRSGtERES03LRp04BiTLn9uq1ajzVXIq0RERERsdpIpDUiIiIiOiaR1oiIiIhYbWTRGhERERGVl0VrRERERFReW3NaIyIiIiJWRiKtEREREVF5WbRGREREROVl0RoRERERlZdFa0RERERUXhatEREREVF5WbRGREREROVl0RoRERERlZdFa0RERERUXhatEREREVF5WbRGREREROVl0RoRERERlZdFa0RERERUXhatEREREVF5WbRGREREROVl0RoRERERlZdFa0RERERUXhatEREREVF5WbRGREREROVl0RoRERERlZdFa0RERERUXhatEREREVF5WbRGREREROVl0RoRERERlfd/kwOcE7Pwr8gAAAAASUVORK5CYII=\n",
"text/plain": [
"<Figure size 648x54 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"import utils\n",
"X,T = utils.loaddata()\n",
"%matplotlib inline\n",
"utils.digit(X.reshape(1,12,28,28).transpose(0,2,1,3).reshape(28,12*28),9,0.75)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Each digit is stored as a 784-dimensional vector of pixel values, where \"-1.0\" corresponds to black and \"+1.0\" corresponds to white. \n",
"\n",
"<h3>1.1 &nbsp; Predicting the class of MNIST digits</h3>\n",
"\n",
"<p>These digits are fed to a fully connected neural network with layer sizes 784-300-100-10 with ReLU activations for each hidden layer. The architecture is depicted in the figure below.</p>\n",
"<center>\n",
"<img src=\"mlp.png\" width=350>\n",
"</center>\n",
"<p>The network we consider achieves an error of 1.6% which is a typical performance for a neural network without particular structure or regularization. The function <code>utils.loadparams()</code> retrieves its parameters for us.</p>"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"W,B = utils.loadparams()\n",
"L = len(W)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<p>From these parameters, the forward pass can be computed as a sequence of matrix multiplications and nonlinearities.</p>"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"import numpy\n",
"A = [X]+[None]*L\n",
"for l in range(L):\n",
" A[l+1] = numpy.maximum(0,A[l].dot(W[l])+B[l])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Note that this code has added an additional top-layer ReLU activation compared to the original neural network. This however doesn't affect computations when looking at positive output scores. The top layer activations are scores measuring the evidence the network has found for each class. In the following, we show the first three digits and the scores produced for each class at the output:"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAFsAAABYCAYAAACaoos3AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi40LCBodHRwOi8vbWF0cGxvdGxpYi5vcmcv7US4rQAAA7hJREFUeJzt2s8rdFEcx/H3IBZ+hJIFsbKwwIaIUiiahT9AQkpZimJFykJKiWRjwYJItiwslQVKLJAFCyvyIzZIyjyL5zmuqxlm7tN8nZn5vpan++P49JnrzJ3jCwQCKBlJvz2BRKJhC9KwBWnYgjRsQRq2IA1bkIYtSMMWlCJ5M5/PF5dfVwOBgC+c47TZgjRsQRq2IA1bkIYtSHQ18pv6+voAmJ6eBuDh4QGAsbEx13g0abMF+SR/qZFYZ7e3twNQUFAAQEdHBwClpaVmDkHPW15epqury9M9dZ1tobhp9vb2NgB1dXXmXhGd//T0RH19PQBHR0cRnavNtlDMr0ZGR0cBqK2tBUI3+v7+HoD19XUAmpqaACgpKQEgPT3dc7PDFfNhFxcXA5CUFPxDurW1BcD4+DgAOzs7ACwtLQFO2BL0MSIo5ps9NzcHQEqK+09ZXFwEYHd3F4CXlxfZiQWhzRYU880+ODgAoLOz85dn8jNttqCYb3akMjMzASgsLHSNPz4+srKyEtV7a7MFJUyzTaPn5+cBPr7AGGdnZ9zd3UV1DtpsQXHzIuon5eXlABweHrrGTZsrKiq4vr72dG19EWWhhHlmd3d3Bx0/Pz8H8NzqSGizBcV9sxsaGgBoa2tzjb++vgIwMTEhNhdttqCYbXZ2djYARUVFrvGenh7XeE1NDQB5eXmu48wq5OLiIqrz/EybLcj6dXZOTg4A+fn5APT39wNQVlYGQHV19X/NyTy719bWGBgYAJwNPOHSdbaFrGy2eR77/f6PbWNVVVXRm9g/5t34yckJEHpt/pU220JWNbu1tRWA4eFhACorK0Mee3NzAzi/Pebm5n5777e3NwBub28BSE5OBpz/BZ+ZTMzWtdXV1W+vrc22kFXN3tvbA9yN3t/fB2BjY8N17PHxMQC9vb0AtLS0BL3m6ekp4GwNNpt0srKyXOcPDg6SkZEBQFpaGgDPz8+A8y48FG22haxq9vv7O+A8M70w15idnQVgZmYGgMvLyx/PbW5uBpxntN/vB5xPVyjhNtuqsDc3NwFobGwEIDU1Nexrm3+AIyMjAExOTnqaoxf6GLGQVc3+amho6GN3qlkWTk1NAX+3Hny2sLAAwNXV1X/PM1LabAtZ3exYoc22kIYtSMMWpGEL0rAFadiCNGxBGrYgDVuQ6DfIRKfNFqRhC9KwBWnYgjRsQRq2IA1bkIYtSMMWpGEL0rAFadiCNGxBGrYgDVuQhi1IwxakYQvSsAVp2II0bEEatiANW9AfLUQkj3WeSKkAAAAASUVORK5CYII=\n",
"text/plain": [
"<Figure size 54x54 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"[0] 0.0 [1] 3.6 [2] 49.1 [3] 8.9 [4] 0.0 [5] 0.0 [6] 0.0 [7] 1.4 [8] 1.6 [9] 0.0\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAFsAAABYCAYAAACaoos3AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi40LCBodHRwOi8vbWF0cGxvdGxpYi5vcmcv7US4rQAAArNJREFUeJzt27FOIlEYxfH/GE2MFJioLfQmhMbWWKKV8SEsTGioxYIHACx4BGoLn0BNiA9goaWURAuoTCiYLTY3uGZc2GQ944znV3pN/HI8TO7MXKI4jjGNlbQH+EkctpDDFnLYQg5byGELOWwhhy3ksIVWlX8siqJc3q7GcRwt83tutpDDFnLYQg5byGELOWwhhy3ksIUctpD0DjJNBwcHALRaLQA6nQ4A19fXshncbKHcN/vo6AiAfr8PwObmJgDr6+sA3N/fA/Dy8vLls+Q+7Hq9DkCxWPzj53t7ewDUajVg/s/4Sr6MCOW22VG01FNPzs/PATc7d3Lb7MPDQ2B+Tf7o+fkZgJOTE9VIbrZSbpvdbDb/uv74+AjA09OTYhzAzZbKXbOPj48BqFQqievj8RiAXq8nmylws4Vy1+xSqQTAxsZG4nrYfdzd3clmCtxsodw1O9w5fnYHmUajAzdbKFJ+gUlx/Gw0GgGwtbWVuL66+v8/zD5+9g3l5pp9dnYGwPb2NgDhEzudTgEYDofpDPaOmy2U+Wbv7OwAcHp6mrjebrcBuLi4kM30GTdbKPPNDu8Wq9UqACsrv/szm80AGAwG6QyWwM0Wynyzd3d3gfnuIzT65uYGgNvb21TmSuJmC2W22YVCAYBGo5G43u12AXh7e5PNtIibLZTZZl9eXgKwv7+fuB7eMX4nmQ07nEr9+Cj14eEBgMlkIp9pEV9GhDLb7LDV+/iIOGz5Xl9f1SMt5GYLZa7Z4Xz12tpaypP8OzdbKHPNLpfLwPw7MeFlQThudnV1lc5gS3CzhXL3wjcNfuH7DTlsIYct5LCFHLaQdDfy07nZQg5byGELOWwhhy3ksIUctpDDFnLYQg5byGELOWwhhy3ksIUctpDDFnLYQg5byGELOWwhhy3ksIUcttAvgveLaNCsEEcAAAAASUVORK5CYII=\n",
"text/plain": [
"<Figure size 54x54 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"[0] 0.0 [1] 27.0 [2] 0.0 [3] 0.0 [4] 5.3 [5] 0.0 [6] 0.0 [7] 13.0 [8] 8.1 [9] 2.3\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAFsAAABYCAYAAACaoos3AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi40LCBodHRwOi8vbWF0cGxvdGxpYi5vcmcv7US4rQAABG5JREFUeJztmksoNX8Yxz/HcUmxcMslctmgFOWSrFmwcVnJQtkpl4WiWChFLiWxZCGWSCkshHRE2LC0oFOUWBCb49Yx/4XGec/7xztuz3/O/30+NZ2a+fnN+PTtmd/MMw7DMFBkCPqvL+BvQmULorIFUdmCqGxBVLYgKlsQlS2IyhYkWPJkDofjf/m4ahiGw8o4TbYgKlsQlS2IyhZEZQuisgVR2YKobEFUtiCiT5CSZGZmArC6ugpAcnLyq+P29/cBWFxcZG9vD4ClpSUAvrs/q8kWxCHZXf/JdyNOpxOArKwsAJaXl4G3E/0eNTU1ACwsLFgab/XdSMCXkcjISAA6OjoA6OrqenXcw8OD329ERMSbcxYXFwPWZVtFy4ggAVtGkpKSAHC5XACkp6e/Ou7y8hKA9vZ2ALa2tgAoKysDoKSkBIC6urqXvzk4OACgqKgIAK/X++616CtWGxJwNTs2NhaAxsZG4O1EX1xcAFBRUQH40mpydHQEwOPjI+Cf7PPzc0CXfgFNwCTbTPTOzg7w70Tf398DsLm5CUB9fT3gS/hb3NzcAPD09ERQ0HP2UlJSAHA4LJViy2iyBQmYZKelpQFv1+iVlRUAqqqqPjTv7OwsAKOjo8THxwNwfHwMaM0OaGyf7OjoaABmZmb89ps1uqenB4CpqalPzW/eC4KDfSoODw+B5zr+nWiyBbF9sjMyMgBITU3127+xsQHA4ODgl+Y3a3xMTMyX5rGCJlsQWyc7JyeH+fl5v33b29sANDQ0/Nh5T05OfmReTbYkhmGIbYBhZXM6nYbT6TTm5uYMr9dreL1ew+PxGB6PxygtLTVKS0stzWNlc7lchsvlMrxer+F2uw23222Eh4cb4eHhluew+v9rsgWxZc3Ozc0FoLq6+mU93dTUBPgauF+ltbUVgMLCwpd9Q0NDANze3n7LOX5Hky2IrTo15lu3ubk5ACorKzk7OwN8b+K+itkA3t3dBSAhIeHlWFxcHABXV1cfmlM7NTbEVjU7JCQEeE40PHfCza75VzETbXbMf000QHd398u77Z9Cky2IrWp2WFgYAB6PB3iunWYd/Szl5eUA9Pb2ApCXl+d3vLu7G4CBgYE/dtHfQmu2DbFVzf4OEhMTAWhubgagra0NgNDQUL9xvyYa/vxtyHdga9lRUVGMjY0BvocQk4KCAsC3XOvs7AQgOzsb8DUdTNxuNwDDw8MAjI+PAzKSTbSMCGKrG6T56cDIyAgALS0tL03X6+trv7HmzdRcLpq/v2Mmuq+vD4DJyclPXft76A3Shtgq2SbmZ8Bra2vk5+d/6Bynp6cA9Pf3A75G8N3d3Yfm+QiabBtiy2SbOJ1OamtrAZienvY7ZrbL1tfXAZiYmAB8H9ZIrjI02TbE1skOFDTZNkRlC6KyBVHZgqhsQURXI387mmxBVLYgKlsQlS2IyhZEZQuisgVR2YKobEFUtiAqWxCVLYjKFkRlC6KyBVHZgqhsQVS2ICpbEJUtiMoWRGULorIF+QelTP5Z8g9KVgAAAABJRU5ErkJggg==\n",
"text/plain": [
"<Figure size 54x54 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"[0] 49.1 [1] 0.0 [2] 10.6 [3] 0.0 [4] 0.0 [5] 0.2 [6] 0.0 [7] 3.0 [8] 0.0 [9] 9.2\n"
]
}
],
"source": [
"for i in range(3):\n",
" utils.digit(X[i].reshape(28,28),0.75,0.75)\n",
" p = A[L][i]\n",
" print(\" \".join(['[%1d] %.1f'%(d,p[d]) for d in range(10)]))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<p>As expected, the highest score systematically corresponds to the correct digit.</p>\n",
"\n",
"<h3>1.2 &nbsp; Explaining the predictions with LRP</h3>\n",
"\n",
"<p>We now implement the layer-wise relevance propagation (LRP) procedure from the top to the bottom of the network. As a first step, we create a list to store relevance scores at each layer. The top layer relevance scores are set to the top-layer activations, which we multiply by a label indicator in order to retain only the evidence for the actual class.</p>"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
"R = [None]*L + [A[L]*(T[:,None]==numpy.arange(10))]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<p> The LRP-0, LRP-ϵ, and LRP-γ rules described in the <a href=\"https://link.springer.com/chapter/10.1007/978-3-030-28954-6_10\">LRP overview paper</a> (Section 10.2.1) for propagating relevance on the lower layers are special cases of the more general propagation rule</p>\n",
"\n",
"<img src=\"http://latex.codecogs.com/svg.latex?R_j = \\sum_k \\frac{a_j \\rho(w_{jk})}{\\epsilon + \\sum_{0,j} a_j \\rho(w_{jk})} R_k\">\n",
"\n",
"<p>(cf. Section 10.2.2), where ρ is a function that transform the weights, and ϵ is a small positive increment. We define below two helper functions that perform the weight transformation and the incrementation. In practice, we would like to apply different rules at different layers (cf. Section 10.3). Therefore, we also give the layer index \"<code>l</code>\" as argument to these functions.</p>"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [],
"source": [
"def rho(w,l): return w + [None,0.1,0.0,0.0][l] * numpy.maximum(0,w)\n",
"def incr(z,l): return z + [None,0.0,0.1,0.0][l] * (z**2).mean()**.5+1e-9"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<p>In particular, these functions and the layer they receive as a parameter let us reduce the general rule to LRP-0 for the top-layer, to LRP-ϵ with ϵ = 0.1std for the layer just below, and to LRP-γ with γ=0.1 for the layer before. We now come to the practical implementation of this general rule. It can be decomposed as a sequence of four computations:</p>\n",
"\n",
"<p>\n",
"<img src=\"http://latex.codecogs.com/svg.latex?\n",
"\\begin{align*}\n",
"\\forall_k:~z_k = {\\textstyle \\epsilon + \\sum_{0,j}} a_j \\rho(w_{jk}) & (\\text{step }1)\\\\\n",
"\\forall_k:~s_k = R_k / z_k \\qquad & (\\text{step }2)\\\\\n",
"\\forall_j:~c_j = {\\textstyle \\sum_k} \\rho(w_{jk}) s_k \\qquad & (\\text{step }3)\\\\\n",
"\\forall_j:~R_j = a_j \\cdot c_j \\qquad & (\\text{step }4)\n",
"\\end{align*}\n",
"\">\n",
"</p>\n",
"\n",
"<p>The layer-wise relevance propagation procedure then consists of iterating over the layers in reverse order, starting from the top layer towards the first layers, and at each layer, applying this sequence of computations.</p>"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
"for l in range(1,L)[::-1]:\n",
"\n",
" w = rho(W[l],l)\n",
" b = rho(B[l],l)\n",
" \n",
" z = incr(A[l].dot(w)+b,l) # step 1\n",
" s = R[l+1] / z # step 2\n",
" c = s.dot(w.T) # step 3\n",
" R[l] = A[l]*c # step 4"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<p>Note that the loop above stops one layer before reaching the pixels. To propagate relevance scores until the pixels, we need to apply an alternate propagation rule that properly handles pixel values received as input (cf. Section 10.3.2). In particular, we apply for this layer the zB-rule given by:</p>\n",
"\n",
"<img src=\"http://latex.codecogs.com/svg.latex?R_i = \\sum_j \\frac{a_i w_{ij} - l_i w_{ij}^+ - h_i w_{ij}^-}{\\sum_{i} a_i w_{ij} - l_i w_{ij}^+ - h_i w_{ij}^-} R_j\">\n",
"\n",
"<p>In this rule, <i>l<sub>i</sub></i> and <i>h<sub>i</sub></i> are the lower and upper bounds of pixel values, i.e. \"-1\" and \"+1\", and (·)<sup>+</sup> and (·)<sup>–</sup> are shortcut notations for max(0,·) and min(0,·). The zB-rule can again be implemented with a four-step procedure similar to the one used in the layers above. Here, we need to create two copies of the weights, and also create arrays of pixel values set to <i>l<sub>i</sub></i> and <i>h<sub>i</sub></i> respectively:</p>"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [],
"source": [
"w = W[0]\n",
"wp = numpy.maximum(0,w)\n",
"wm = numpy.minimum(0,w)\n",
"lb = A[0]*0-1\n",
"hb = A[0]*0+1\n",
"\n",
"z = A[0].dot(w)-lb.dot(wp)-hb.dot(wm)+1e-9 # step 1\n",
"s = R[1]/z # step 2\n",
"c,cp,cm = s.dot(w.T),s.dot(wp.T),s.dot(wm.T) # step 3\n",
"R[0] = A[0]*c-lb*cp-hb*cm # step 4"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<p>We have now reached the bottom layer. The obtained pixel-wise relevance scores can be rendered as a heatmap.</p>"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAq0AAABYCAYAAADfnM4oAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi40LCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcv7US4rQAAGcZJREFUeJzt3Xmc1HUdx/EXmWVhh2VlalmJV5Jk\nmnikiFqmViLZpYKYpUailUGmSYZ5FKR4ZKZlqVRSQh7gmXdaHolWloZd8MCiMimxu+gPHq/57fx2\nh92FOX4L7+c/q7uzw292Zn7z/X2+n2PQsmXLiIiIiIiosmd0+gAiIiIiInqTRWtEREREVF4WrRER\nERFReVm0RkRERETlZdEaEREREZWXRWtEREREVF4WrRERERFReVm0RkRERETlZdEaEREREZX3zHb+\nY4MGDcr4rYiIiIioWbZs2aC+3C6R1oiIiIiovCxaIyIiIqLysmiNiIiIiMrLojUiIiIiKi+L1oiI\niIiovLZ2D+ikY489FoDp06cD8OSTTwIwZcqUuu9HtMIzn7n8rXb66acDcNxxxwGw3377AXDdddd1\n5sAiIiIGiERaIyIiIqLyBi1b1r7Wqe3o03rwwQcDsNFGGwEwZswYALbaaiuPocffmzFjBoceemir\nDy/WUK95zWsAmD9/PgA33XQTAEcffTQAjz32WGcObCU95znPAWDzzTcH4Igjjqj7/7322guAOXPm\nAMsjzD/72c8AWLJkSVuPtS9e/OIXA/C+970PgC222AIoHtcTTzwBwCmnnALAzJkzAfjzn//c1uNc\n05x88skAfPrTn677/m233QbA7bff3uPtI6J3w4cPB2DfffcF4KSTTurxdq6bFi5cCMCb3/xmAB59\n9NGmHUv6tEZERETEamO1ibR6xb3LLrv4b/Xr959++ml22203AB588MHmHlwLjRgxAoDPfOYzAJx5\n5pkAXH311R07ppVhZOt73/seABtvvHGPt5s3bx6wPIJ37733AjB37lwA2vla7q/LL78cgHe9610A\n/PCHPwSK5+8///lPZw6sn7beemsAvvnNbwIwdOjQPv/uL37xCwDGjx8PwK233trko+u/cePGAXDi\niScCsOmmm/bp9zxHjB8/vvZcVsHs2bMBGDVqFFBE9j0fbrbZZkDj94q3u/3229l9991beagrtLLv\nZSOwng/9/4Hqec97HgA77bRT3fcPOeQQALbddlugeF8CXHHFFUCxy/jPf/6z5ccZ/XfggQcCMHny\nZAB+9atfAcV7tx38PN1mm2369XsPPPAAAIcddhgAP/3pT1f5WBJpjYiIiIjVxoDvHmCu08477ww0\njrCak/ad73wHgD333BMoIg+DBw8eUJHWffbZB1ieiwvwwhe+EIB11lkHgB/84AcA/PGPf+zA0fVu\nrbXWAmDLLbcE4NprrwVgww03BOB///tfj783bNiwuq8Ao0ePBuCqq65qzcGuBB/XF77wBaB4vowg\n3XfffcDAibC+7W1vA4ooztprrw0UOU5Gcx5++GGA2ntpvfXWq92H+a6nnnoqAO95z3vq7qMdPD8Y\nhfrSl74EwLOf/ex+3c/rX/96AG655ZbafxtJbifPexMmTABg//33B4rX2ZAhQ+pu3+h9JX9vxx13\nrOWtmX/dDqsa3fX3/drfHbd2MzfcSNc73/lOAF7+8pcD8Na3vhWAF73oRXW/t3TpUgD+/ve/A/WR\naSN47j5+8YtfbMmx94U5xj4uTZs2DYBLLrmk3Ye0Svx8dedMRx55JACLFy/u0/1ssMEGfPnLXwbg\nBS94AdDeXYGPfOQjQPfzg3w9+fp61rOeBRRdcN7whjcAxedcMyKtfTXgF62bbLIJAM94Rs9B4xtu\nuAGA0047DYDvf//7AFx22WVAsWgdaCzg8QWv7bffHoC9994bKBa1VeF216RJkwA44YQTerzdv/71\nr7qv6667bsP73HHHHYFqLFp32GEHAKZOnQrAm970prqf33zzzUD3wpKq8u9+0UUXAcVJ65hjjgGK\n99Ff//rXut9zIXfQQQcBcMcdd9QWiCb/P/TQQwC8+93vBorUkFZ4/vOfX/dvXXjhhSu8/T333APA\n17/+daBYRPi69cNrnXXW4b3vfS9QFGm1I03F97dpGl60lv3pT38CaJjC4PnP9BytvfbatYuLRx55\nBGjvxUVZebvfry5OG6WauGjqdIGWF+lvfOMbgeJ19NrXvhbo/jnkYtvXkkEXL3J9XXp+71pE7Dmn\nncWdpiccddRRQLFI3WCDDYDu74mLL74YKBbj5557LlD9i3iDJV7Ey4uP3rz0pS8Fln9W+dlt+83z\nzz+/WYfZq1e84hUAPPe5z637/k9+8hMAfvSjHwFw+OGHA/DRj34UKFo0Wlj8u9/9rvUHW5L0gIiI\niIiovAFfiLXddtsBxfAAfe1rXwOKCINhbhkhMhIExdXEOeec0+zDbBqvwG0lZMSlfGVuAYatvjrN\nbf877rgDgFe/+tU93s6IwsSJEwG46667gKLFhtuhXZ830zmMcv73v/9t6rH3hZHF66+/HoCXvOQl\ndT83smdk2avrqnMbyTQHt8G+8pWv9Pu+jHb6t7Ao7YILLgDgwx/+8Kod7Ar+TVtU+X5pxOfFSFF5\ny86iiZ4id0bNjEy2gv+Gx2WrrjIjWWeffTbQePvula98JVCkahjR7JouYbTF6F4V9fY51uk0AVsJ\n+bopn6/lboMFMu7M+Dn21FNPtfxY+8NdriuvvBLoft5r9DjLPzetyNftWWed1fyDXQXurMyaNQso\nziOmS/l51Fv6jWsMd+KgeH+1M1XC87nnd33oQx8CGu9Eme7l510zi2lTiBURERERq40Bn9Nq7sXY\nsWM7fCTtYWJ+o4jRb37zGwAOOOCAdh3SCq2//vpAkevUKMJqArtNjsvFcOZn/fvf/wbqI62///3v\ngc60vPKK09xpH69X3BbEeQU70JhrZwTSx7kyzHu1CM3ouUMHWsHc4UbvF58nc+qMpJj7XmaEwuEJ\nXXOWLbZrVaR18ODBtfG/jSKsDke45pprgO47TGULFiwAisiPkfRG79OqKue4Vo2ReyOLDqWwyM3X\nnZG7gcKdjHKEVYsWLQKKwivf6x/4wAeAYrfFHQRzYKvCGgwjkuXziMVUvUVY3cmwYBKKArl2Rlgt\nqDJy3F9+DnSyXWEirRERERFReQM+0tpfXjmVm9cvWbKkVolbZZ/61KdW+POf//znQGvz6vrCiKO5\nWOXIjS2S7rzzTqCIlPfWMuQvf/kLsPzK1o4RVkK2M2/N6klzWH288urZiMJAZeTbqtJVqSB/1ate\nBRS5q3YPaEVLHqu1rXYt+8c//gEU+WTf+ta3+nS/Tz/9NFDkjJe7Q7TSueee23DUtMNEjNh1Iq+7\nk3qLsHa6i4C7QH69//77gSIyvrKsFTjppJOYMmUK0N6KbrtqlM+97oDaLaHMv4Pvn0b302m+Xso1\nMw4C8HH25rOf/SxQ5JDfdNNNtV2TdnrLW94CFDufA1EirRERERFReWtMpNUIq1VxNj/XI488Uutp\nWEU2DX/d617X48+XLFkCwHnnnde2Y1oRo2qNcuNuvPFGoP8j6xwOcfbZZ/Oyl70MgF/+8pdAe3Na\n7TVazuW65ZZbgKL7QV9ZlbnrrrvWRjPaaaGT+UPmoJX7+fXXyJEja7sE9lZuZa9a+7E60EDmnvU3\nwir7MzaKeLaSr7mu3LEw13Z1j7CubKS00+NcP/e5zwFFdbxjWVd2gIP50/Y+vvHGGzvSM3OPPfYA\ninOvw2wcp96IXRL86ljaqozi9txUrkVwYIO5uOX+1GXTp08His/Bxx9/HFgeua16T9qqSqQ1IiIi\nIipvjYm0eqVjBEZGV8tj5qrGXJhGES+7BRid6xQnnHz729+u+74RIftArmzFpLmjTmaCIn+3twrO\nZrBq3KpRedVsP8a+9mH1/uwrbI4aFJ0S/FtZ2d1O5rL6+jIi3Nvjc0KTkcwRI0bURr9+/OMfB4o8\nzFbo+nfsykjkb3/7W6B4POYeN5psY9WwOx1dJ/DZIcKpYc1mZXV5lCdQy4tr1O2gGcxr7kSf1t4m\nXvWmPEGrUxz76Xn8jDPOAIqdP8di9va+8nxhn2T76brD02meL5xkWGauuVPkypOlrLI3gmw3nHZY\nd911a8fzyU9+Eig+Z+69914APvGJTwBF94cV3RcUEXHZFacTI5+hyOW336870PLzuVG+reukdo5t\nLUukNSIiIiIqb42JtB522GE9ft/+n/b6rCqrKhtVV3Y6wiqr6s1blJEOc7tWljmwjfpUtor9CL0S\n9Upab3/724HGM9493qFDhwJFF4g999yz7naLFi2qVdIaRTH/zVnVdlBoB6vkfbxOoDMXTUaQjEx8\n4xvfAIpI5uabb177WTuiJ426UBjtbVZkcv78+YwZMwaApUuXNuU+y8xldW5513+rHV1C7EXcCaua\n9+z7tSo+//nPA0XUfNKkSUDxOWQOtpMB5U6Aj8dzxMEHHwwUkdxOe+CBB3r8vueBE088ESgmQ5Un\nZtkJxv7Q7Yy0Dh8+nBkzZtR9z8fjcdv1oTdGkjfddFOgiDz7/HeK5+2vfvWrQPeJWJ5jup5rurJP\ntxHXRp93rZRIa0RERERU3mofaR05ciTQvR+eOZbmFlWdV3pVqa7sydChQ5k9e3bd9+6++26gcaS7\nGZzq02xGDxcsWMDHPvYxoJi17fNghNuq0kb3YU6eXRU0d+5coMhtmz9/fu1vZmTVnoDtjLDK3sXH\nH388AHPmzAGKiSpGvo02GhmxStYJbu0+do979OjRQP+7VPTG/sKjRo3qc/7yyhoyZEi37/3hD38A\n2tNZwnzrdrJLwKpOuPLvU7X+n0aQjSwamXv00UeBYgfGCOsRRxwBFDsZZ555JgAzZ85s0xH3zHxu\n3/92E7AfqBFhc1QbTWLy+XHHs7/dFJqhp44g/p0bvc/cWXSnzXN2uXuMXQRamXveDk4s85xwzz33\nAEXvbftYt1IirRERERFReQM20mp1spEseWXq942Mlftp2jXAHp9VZZ84q+aN8JkjYxV0J1kRevLJ\nJ7PRRhsB3bsF9Dbpqq+cnAVFhLVVVc3mly1YsKAWsZP9CL2i9vHq6KOPBooqVCMO9qe97rrrgGLm\nvc/vnDlzalfrsiK3E5xdf9pppwFw/vnnA8XjtarWx+vtfW6sVm03O0kYkWwWJ87Zp7HVUVboeaqX\nM8R93bSyx3Qn+n+W+7H2ltvqeabR7To9EavM9095Yp4R13nz5gHdJ2mdcsopwPI+1dCejikrYjW8\nnU7sguD5rZyz6lffN+b2+v2+TphqJqPdBxxwQLeIvLn5fu2NXUXKz8uECROAoq/r9OnTO3ZuhOJ8\n4evQ+gn5fJpbbYRVfjb61RzlRFojIiIiIhgAkVarDp1+5Irefok9TYnpCyOCVgPOnDmzlrfYjuhJ\nb4wMf/CDH+zx507AsS9oJw0bNgxYfqXqlZs5LuUq85V1zDHHAPWzrK3ENLrXbCs6dvNMff14he4V\ntblQft9JXuYmWy1slaZdFbbbbjv+9re/AUWFZyejQ+UeofajtV+wOa4PPfRQB46uMSMe5QhBI+Zm\nlafUuGNjNGarrbYC4IQTTgC6V9+2glE1e3MCbLzxxkBRxVvuG7y66G+EdMSIEUD3XFgjsLfddlvH\ne7Z25bnLvphGWsucXV+VSLGsHrf38rRp04D6PtpQRO58nXq78k6hE/jaaeHChQCMGzeu1gvbWphG\nudBGFD1veH4w8liuPbH/s8/jrbfeWvvdTjj99NOBom/9NttsU/dzc4t9PZrDWp4wqHe84x3Ayvdf\n749EWiMiIiKi8ioZaTVfdZ999uHYY48F6iNszWQux9ixY9l6660BePjhh4HWVrz3xrxGo5jlXJm7\n7rqrMwfWhcdkz1EocmCaVXFsRMlJJObyQecrZ7tyGpKvGaP1TgYzim/PSyNBvr6NaD7xxBNMnjwZ\ngAsuuKANR96z7bffHoCpU6cCRSTBCEN5Mls5d63TnGVuBKDMiTZWPZtr7MQsWR1sNbOT9YxwtiPS\nakRm8eLFtR0nGdEyr7x8/H1lhNIcWSiey5W9z06wi0ejrgO77757pSKtTiS67LLLgO6RPf+/CrUL\nK2JuvpOSrMQ3YnfllVcCnenr2VezZs1i1qxZQHE+7jr5rivzUd0xM4JcjkR6nrCbgrmkP/7xj5t5\n6E1nX3H7cnueNFe5PJ3PncVly5Zx6aWXtvTYEmmNiIiIiMqrVKTVXmdG7oz29MSqYCMNPc3l7sqc\nGqu+jUx0jVx4VWEFpNEX56e3k1d6RjuMsBolaNQXtJ2cLrT//vsDyzsaOOFlVRlhveqqq4DuuYmT\nJ0/uSN9SOdnKPoW+ZmSEZNttt627XTl3SEaoR48e3dFefr4HzFU1t9qIgjPOnRFuVwEjFN6u0xrl\nenseMPLRaIKPzF2+7777gCLS6k7I0KFDWz6H2zy/J598sluk9ZxzzgGK96CzzvsbHb366quB+klv\ndkowChjN5/vN/qae7+3S4Weiec3usHVqdn1v7GfaW/9g+7bK82Vfq/Rbra+5tXZzOPzww4Hic3rK\nlCkAXHPNNUDxOVY1vo6c+GgfcXfS3GkzJ9zXY3mn193xPfbYoxZVt1NCsyXSGhERERGVV6lIa08R\nVvtAGvmR0Q2r/fbee+8e79MrJq98rOL2ysLfnzhxYi3KYJ6rU4raGWkdPHgwUHRJKDvrrLOA1lXM\nr4qlS5eu8t/KSJFVluW55+Z7nnHGGW3NtTNSat6zz9MOO+zQ4+298vTnRhI8ZrsO+Lo2stLuzhXO\nxnaC1JZbbgkUER8nuZib6/vR99EVV1wBFM+buW2dVq5eln//RrlqjZjLZbWt54qRI0e2PNKqQw89\nlO9+97sAbLjhhnU/c4LSLrvsAhST2hpxp8n3a9cIKyyfaDZu3LhVPub+Mhe1VXmnVclnNaJV7jFt\n5xC7kLgTYOeILbbYAqhupLWv3LHQokWLgM70BF4ZdgNwx0bm7BqBrTp3zOw7ffnll9f93LWQk0P9\nnGpkzJgxtVoTz5nNlkhrRERERFRepSKtdgjoWoFspKpRRKvMnBIjPuYClasvzbcwZ2Pq1Km1qyaj\nD0aP2snI1q677trjz80zW104KcqJSlbZd+0SAPURVmh/RbMReXMKze1p5KmnngKK152Rr2uvvRZo\n3VVofxxyyCG1Lg9GHj0+Z52XIx9G6OwTXFX2US3nyDn73OfDiXgXX3wxAHPnzgW6R7LK3RI0atSo\ntkWX77///trzUt55krmpdjsoR07cYXJXy4hf2WOPPdbW6URGWMu5kEZGnXhVjpSWuwTYBaHM36tK\npPWoo44Ciq4cRljtdyp3MuzvbDW6uZIDjTtUm2yySd333cGx5qSqPP8Z+TYn2fOFEcuBxjoKz//7\n7rtvj7cbP358246pkUotWv0gNym9vHBZEQssLMBwMdofFl7Z7qETPOmWW584yrOTxUe9WW+99WqF\nIQ4DkCkfFvY43tRm7eVCul//+tdAsTgyVaPT7Xdsh+ai1QEA5bQIC6uq2NrEhtEXXnhh7eLNMbWe\nvMp/Zz9cLbiy6b7jhKs2XMDFmkNIbE3lotWvpnv4OvO8UW7d1SidoN1Fc3feeSdQLLLf//731/3c\nFko+n+Xxw72xJY/Nx9ulUeGOi9JGLaz6qgqFq13Zws/X2fXXXw90HxLhmFeH7DRqdj9Q+Post4ay\n4LHqPH631WU6gC2+BhqDE55fTDPyImLIkCF9up/HH3+85WuUpAdEREREROVVKtK633771f3/pEmT\n2HnnnYEismUT2yVLltTd1sjDQEnkbsQr73Kkx20tIyFVYJTN7dEJEybUtg8OOuigutua0G2bLL+W\nGWE99dRTgeYNKWi2Bx98ECgKJhxfOxDstttuwPLnxKR5I6W26PL95vaxaRG29po4cSLQeDu903z/\nmCZgQaZNzy1cKutrgZaR9HLrnlZbunQpUESOLTgbO3ZsU+7f4it3ndrF89uqRlTLHMdZlbQAlc/z\n7mQ4mtpm7u5MGWHtrUVb1R144IFA8Xj8WvUIpYW1tnOSLQAbpesMNL7+jLCabun6qpHFixcDy8fO\n33333S08wkRaIyIiImIAGNTOsYuDBg2qxozHCvJKzitpcwZ13nnnAe0ZG9lf5vncfPPNtcT0vlq4\ncCFQ5NBdcsklQDEmL5rPtiVHHnlkbZypeUgWSDSKOBqRLQ9TGCh8XDvttBNQ5KZZBDp8+PAef8+c\nQwtgbrjhBqDzuXjm/buz4Q6FkfHeciAtADruuOOAIjfbQsJ2K+ewmuPf1whsVSOrZQ6raPQ+Ko9F\ndqTz8ccfD3Tu+VlV8+bNA4ohKz5Om/NXdWfNQjgLAt3xcNeqirULzeBQH2tPHPpTbtW21157Ab0P\nlViRZcuW9SlhO5HWiIiIiKi8RForYtiwYUDR6mr99dcHitY0dlTorWl4J6211lq1yvRLL7207mez\nZ88Gihygiy66CCgiCZ3uChCxOrGlkkMDrELfbLPNgGKspDUCnRwdvCayQ427HuZ6yrxAW16ZUzhQ\nI6wqR1ptCejnX1Ufn8+HrTdnzJgB0JEBHKurRFojIiIiYrWRSGtERES03LRp04BiTLn9uq1ajzVX\nIq0RERERsdpIpDUiIiIiOiaR1oiIiIhYbWTRGhERERGVl0VrRERERFReW3NaIyIiIiJWRiKtERER\nEVF5WbRGREREROVl0RoRERERlZdFa0RERERUXhatEREREVF5WbRGREREROVl0RoRERERlZdFa0RE\nRERUXhatEREREVF5WbRGREREROVl0RoRERERlZdFa0RERERUXhatEREREVF5WbRGREREROVl0RoR\nERERlZdFa0RERERUXhatEREREVF5WbRGREREROVl0RoRERERlZdFa0RERERUXhatEREREVF5WbRG\nREREROVl0RoRERERlfd/kwOcE7Pwr8gAAAAASUVORK5CYII=\n",
"text/plain": [
"<Figure size 900x75 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAq0AAABYCAYAAADfnM4oAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi40LCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcv7US4rQAAIABJREFUeJztnW2LpUe1hqtPd6ZlxvQ5c5jRAcWA\n4BtEEggoCIqCoOi/8Bf5S/ys3wKCYEAwOGA0EIjHQMTAcHqcxj120+fD3vd+1r6q7q49gdOzkfv6\nsrv3Sz31/jx111qrjq6vr1sIIYQQQgiHzH+87AyEEEIIIYQwIw+tIYQQQgjh4MlDawghhBBCOHjy\n0BpCCCGEEA6ePLSGEEIIIYSDJw+tIYQQQgjh4MlDawghhBBCOHjy0BpCCCGEEA6ePLSGEEIIIYSD\n5+Q2L/and95ZH791erp+4/ISudlk5/jYJ/L06fr17t316/n57m/v3Fm/Xl3tpqVrXlzsfH59/79b\na60dPfvHkobypTT0W7J5/7odrdO4/NduXgTTG6Xp6sLB75PZ7/dJe7XaTUv/uzq9uFjaRa9C3xEs\nv/5Xe/J9XrvmS/nV/7qW+gLTqu0wQtfQ9y4vl9/ymoJpKg3lW5+fna1fVU7VU+2vrHfmh3UyG09i\n1Gf2+U5lNiZcHi4v+zHp+pWrY9en2f41Tdd/3Of37o3TZvvWOme5BOuK+dQ12Uf0PdbxxcVyDfUj\nfvfJk/Fv3fhindb5U9fSeFJf1f+cc137KE+cE8Rq1dr9++PfKM1XXx2/zzrn2BGqp7Ozfn5g++l/\nveraws07o/7Kfsb8KC/sI26+cX2qNX8t1z/5uXD9uI4dN/5nabn2cXOYu35ry1h1daPvvei9dZTG\nTc8k9dr6Pvu6+m3t56wL9QX2N16DeZnNl3WO2nfeZhr83gvcN65PXmmt3fB8tOFrX//6kU2kEKU1\nhBBCCCEcPLeqtG5xqxaudEeqk1YwdeVc4fvPnq1ftYrRNTbpHD37x/r/09N+NULlw6wutDzYrija\n9e739Lv6e6b5osooVlfdaqawVYI3+eL/HW61zNUwV6F37/ZtS2WcKA1ey6lO9XOpRmprtK291kyl\nEbr2vXv9Z1ztz1bBVPCUZ6Z7deXVPkGV1qk2+6zMnTrjVu9KS+Nqpuq4sVM/Y/54TZaL37upvngN\njjeWQ/+7PjIqF8cLr+FUXqdg3tR+TuVjO7o51rWrVMd6bTe+nj/f/S2VSda1ysexXXdlON7dGFb+\nnBL28OHu+66P1XwQpzRSCXPlHuHGgcrJ9mPfcGqjGClobmwLpzCT0c6OK7Mbo/o+y8/x5PI4ugbT\nZt3of76vVz4jnJz0/cUpp4L3N/2O70thre3G5w31K6fAsi+wXcSovWdKscMpzfuo19vno+vd77Bf\nqv73JEprCCGEEEI4eG5XaXV2fm4lOPotV15OidPTu1QYrrK4EqqrFaeCTla52xXFPraFM5uQmf0e\nvne0+udOXiq0JZkqrlz9UqWW0kLbm2q3yhWqsw0UVHW4KtP/1T6OaqXyxzbnSpV5JFxVPnvW20/x\nf2ePifa4brtmO0f83Sh/bA+nbjpVRoxUNqd0UE1nnTK/XD2znauNJJmlxXIJp7BW9dMpv+wTygPn\nJCqxIzWUc5NTk2Z1p7yxnWt7s1xuPGmMsl05BlQ+5fHjj5c8Sh2iHazeV12wXE4BYrmrgunm99Hu\nWy0Xd4H+/vc2pM5D/I1w84GuRbVsplCO0mN7ONvqmc/DSAFz9tmce908Mpsv6uec31kHs77OuZv3\n3Hotly/WgVMkBecwju3Vahk37MtKS/efBw/Wr9qhGO06Vpi31cr3H2dnLmb3g9HzC+cW93yB97vd\nY+fLMdpFdji/gj2J0hpCCCGEEA6el2PTKqiOUv28ulq+o8+k7nGVwVWkVkxUHJxXX10daEXNa1Bl\n2eTbKpab1Uf3eV35UOERLC9VQ9YdVp/Xp59ZX3Ng4yqsN5+zpRFcdYqHD/uVs9Jg/atdtIpUnXN1\n79TR09O+v+i3tPkUTkF3ylC1iaJa4RR/5WlTHkaWOGKdUVmuKjzVDGfH5+wuHc6usbWxQtraUqf6\nn7af+9rEjr4rqOi4KBbOdq1e09m9OXWC5XMqAFWd42Ov2Lk6YVrC9WP1mS98oS+HPpOKxD5BVZF1\npv7HefbRo0W1VNoaq598sn6l8ugUIcExUtWq2TzN3Tn9T/tSoXpR+f72tyVPtE2lpz6jIehabl4R\nIyWZapqzEVSeXHSHmZ16a72HuruW26nZN6LNs2d9f+J9iXWsNhf6Xe3bNa+1n3PMTaL5TOeLmyIC\ncO5Rv1GfqJFyWlvaS3W+r93p6DMXZcTZgs7mndp+7JvOjhT53O4Lsh9yHh1dy/m+7GMDfgNRWkMI\nIYQQwsFzu0qr88gd2Xu0tn6K52+cMknlgSsLXeOPf1y/Sj348peXa1Glpf0rVYtNHo6MUtkprCN7\nEucZ7myEnFcjVjxbFfXqytuTzjy83aqRdjtVjXOquXDtxxU780BbqCdPepVIypDeZ7s5RYHq8ChK\nglPcqAAhzSPWPeNosk5r+V0kBaq9sxW7s++sOxlUTqn4MG3ah0mJ0NgZ1fXMs1jM7Bsdo/TYD6lY\n0R6Y/U/1wPaqY8W1k7O3n0GlSCrO06dLPVdFtLWlX1HtdEo421eM7FidwsoY2fqcdcW65bxxctLH\nmVU/+utf169vv71+lTr72mvrVyl5urbmc8Gx8PSpjw4wU8Kdcj5TUVvzcZnZ1pq79LnmNCmR3AWr\n9yTuEM488mc7Fu57z5/3+VB/1P8qhxtP7JdMp95jZjb7VCI1N9FeW7jxyKgYFeaBzyXqt05xHtlR\nc7eD77vIScTtHlXF0+1aufi43GHjfZhx8kd29592J3BClNYQQgghhHDwvByb1omH9c5Kg7YgWtG4\nVS3VN31PK1bG8fvwwyUdrd65UqAN1p5YhbWeUEGFUf9LOaBCJNVDq0n9jjZhI/s+p3TPYj/eZJ9Y\n81a9L9k+ajfWP1eg1Xu5/o42bNV+VjCepLNRdeWhelhRe9CmWmmp3FrdM784ic3uLtRrcwUu3Op4\nX1uvqgjyuvvaT/HkM6bD9jw99ba5I+WtwnZjZBAqrKuV9zZn9AelRSWCeaW9oNK/c6dXFGjP52xc\nnfLs1JCLiz5t1a/e5/+CKg3bR+XRHFf7nsrO8rFcHH9UnjkWRtEemP+//GX9qvlb9f+b36xfv/vd\n3bw8frx+ff313Wsq3dVqbndIZZjt67zqlZ7uF3fuLNel8jhS7Ovngqpn7XetLfPO06dL/lVHVA7d\n/E9PflcvdVeF/UbjSPM7d26YBvs4+0Sdi1lHVPD5vpjZhLK8x8f9Tpjqme3HaBtUWtlOahPtjJyc\nLHUku1n11br7W9Okyitc9Io6Hzo7bsaxpwrNfsfnFar7dfeO+XA7HC9IlNYQQgghhHDw3K7SSttB\nrQSp1tQVOleeVO5m6hq8ubeMVmtU0/RbrqhdbM1ZDLS6GnHKlGxEPv/59atWnM5WV8j2S+UceS3y\nzG0qVQ6qoswTV5cVKsZsJyk7+p5WokIrQXpSn5wsq1ypKlRtGaFAUNFyqpyodsHvv7/7GeP20faJ\nK3LlTX1M6VUbIbVxVZXr/+yHXAULp8jWclO1nJ1OQpXJ2Z3SxvL4uFfeZjZazC9t1ljeqgCqv9AW\nUPlTOV0UC847upbasXoLsw6cDafgLgLzprQ1JqpCobT0W9qRUq1Wvj/4YP36jW+Mr808PXrU2xc6\n2zqOG8au1P9OWanzvPq6dsDUTsq35nuVR8oq7do5v9S+ofekcFFdns2Ho/ierfXK5pMnXiVzY1g4\nu0vaItdYxU4hdgrr6H5br017RqqhrS19Va/O019paX6nj4K+p3at12A9cx6c4eaJ0eesO6H+qPyO\nbDlba+3Xv16/ahz99Kfr148+Wr9Wm2t9R2lxnNB3hDtNwkVQqvcFZ4sqnBrNHSflSX2dzxQXF/21\nOMcQ977hdh9aOUAZeJ6fX1z0htusVDc4lDYfQHkNdYCrKx9yhtsteBjYBuB1D7HCTW41X8JtL7iQ\nUo7TUz/A6ndGuCD97ND1oaJujbXW2le+spsmb8T83223aDKog0cPJAyb4pxrGMpGuK2P2sd4g9Pk\nqkmYDzsMkaLfqU7/8IfddMTpaW9CoTrVw7ngWHCOE8I5/Y3ScqYhfBDRJM6HcrbN6elSF7zpu21l\ntzhknvn9OiY41lgnbgHHm5faWTefGsKHW+uuXcTIwWqUx5HZFEOm8WbPsUmzKOfEp/6o7fjnz1v7\n0pfWf7vDDrgoYjg7t7XN7fknT5a+yBveV786Lu/PfrZ+pYOtfqf++O6769ebFhk0f+DiV9dwN18u\nfPT5xx/3fZR1wvbU92WqpnLpYcdtU69Wfd+lUCN0bfZ5Z76i9NSX6phyZkRcRDizL8GFusbZo0f+\nns/fOlMgF3aKplqjtGneoTTee2/9qnlO9zEtoiQ68R5UzeSUpsqqNnfPAq7dWKecA87P+/sQy86F\nAMcIHcS5eK6/U7nolDtzpt+TmAeEEEIIIYSD5+WEvJptMdI4uLXlSV+KgdDKh8eAatUrhYxG6qMg\nyPytVlGjbZGSbymsw0MERr+r29WzYxa1CqMyou9RgWWoiouL+fGlbqXK7Qan4FVFSfnRCoyhWwRD\nozCQuepe/0uxrfWhz7jVwS17F57DOZzp86oCu9A6LlQXVTat2KUYS13jNvYodAidf9gXuHJ1IWu4\niq4qhtueFNyaE2o3qm50vnn+vA8Ez0DdOpBBOxc8GINb+65cn/98r7C6HQm3rewOeKCaX9OehSBj\nmqojKs/OMfX58+UzjQP1HzqJsj0Zpkrl0P9vvbX7u+PjPlwYd0M4BuisxrywXmp4K9bNF7+4fpXZ\nE+fB3/9+938pW0J5U56rIqZr6DvcbnXznuCuHs2Pah7otDYzNdO1VV4qx/qfobHOznrzEs5NVNNG\nDqf1ffZDzbf37/dHedfjtVtb2k2/4X2bO6JS9RnK7fx8+Zt9Wv87pds5cFHVr3Og2kf1rXGlfLk6\nU9o/+cn6VfcJ1b3Kp3o4PV3aQTto3F10h0SwP6r/Ug1Wns/O/K4B79ec77g7zN1nmnmMQgE606Uc\nLhBCCCGEEP5duV2l1Tkq0Xmj2mZwJUpFRCsAPfnTJlIrQaqedBqoac6UEqOqdce4CioNV1feeYmK\nA1ddDFXjPq/2fS74slvp6PvumFS3Gjs/7+1LtTLjUZCsQykrMnjnyk+rbV3z3Xf74MxUqngNhpBy\nDhUj5z/+1hnsK02q23S4EjwU4803feB31rc7NGF2VOnIhpnqulvlM7QX1V9BG6rLy97RwKi3Owdj\n1GuznII2ocfHfUggKkEuvxw/qhfZeo6cpVTGTf1eP/zcuhybpLfK8eqfu2k4ZZIOqlWtU35o66/y\n0Ia/HsahNFpbbO+UF9pRP368KEC0+9UrVXbnBMp+O7JpU1r67Je/3C3Hn/+8fuUBB1KIlX/aX2oe\nqja/7nhSjh83P1Jd0veZt9aWsnP3sCqkrS3zHudz/Z52jCN7dRfCz4WCcu3hbP5rmDvu0NDhjz4l\nnKup9mp+1FiqfYpjk3OVU/QEy+ns8D/6qLezp/Oh0pZyyjxp7ErF//rX168cMzWPrEs5HfL5wx0A\n4Gx26/dobz7zd1B7uLBvdEitdcznvM0YvD77z9ba4DnpBW1bo7SGEEIIIYSD5+UcLsAncdpZ1BUT\nV0/6rpQGrRz05M8VnF5Htmj182rfx/BRhOqFYWvjyhV7XdVw5ew82OltShsnZ4NYr+GCYjsv7dGx\nuqP3ZTd8fDxWh1pblBB9V2gl/rvf3Xxt8sYbiwqjutisfq/vfba1Vup95r3obEDpDd5ar7jSXpYK\nnVassrnj76mqPniwrOLpmUllzoWDo5LCvI5WtlTh+T5faePEumP0gBpA3tlwMiA5x43zbB3ZFNKj\nVn1Ein21mWttUYgY5YH2jvpd9Zh3tuuyeafSIxhsn6qO8lZ3PDjnSIGU4kibV5VbKqDaRZ7KtIWV\nylOvxZ0ZvVLNdkeMUgnX95Wn1pY21Xs6PIA7aFTCeciFlCwpXayv+/d7+0RGP3D9jv/v83vuREhR\nVf1zvqyHILS27EBxrKt8+n69Bu3OBcewixrj0qs7JezT2onQ3KV207jiNTUPqvy6P3Dn6f79JW3l\nwx176uZ5ji8XVeD8fEmbh8gI7lZpnP3qV7vlYblqO6kM6qvc0eRBDYxIQzivMMzk1dUy5lQ3DOGl\neU3XVv3re7R5FwwvWfOCaBZHm+9en35m/b87dGBClNYQQgghhHDw3K7SStsYF/B25M3nlB19zniE\nDGKs1YfstEYKkq5Rj1prrV/h0M5PYFV21EBdLbt4i7THVJpSRFROruTomS115OSkVyVcPM9ZnFkq\ntDw6dmQfTAWB9m9qF0Yb4DVUzmqDR+VRtsWyiaSCyogFLL9b8VU1je2i/LNd6m9b69Ud5UHlqqo+\nY4MKKgUu7izVQcbbreV0dsG0G2NMQR4HyLRHCjl3DzZpbm0++1/sQvs5Z+Nb7arUPowZOIov2JqP\nTsJ2r3lB5BL1v62icIrjnFkep3hRXbu87FUUtS2Dnut7DMZPFYqRAKQA1jSE0mL/op2tOzSCc7dU\n4kePlh0JtYt2ZBgH+ZvfXL8qmgjbR99XefU7la+qhIxYwrFJ2Ldn0TlquzIiCZVG5U+qIm0gGf1G\n80m1daV9oXA2qs5mnPXAXaR6UAhVdcaRFcwvvfLVD9WXqpe92ppxmDkPOHhv5E6U0n3zzd5OW/nT\n+7q2lMff/nY3De7sar4cqbsqI+uMOxyzqAg8IMZFIWmtf8bSNaWoaqdFfPvb61ce1+piNV9e+t3I\nDdv78ywOtyFKawghhBBCOHhe7olYVBV5VGf9LmNv8ihRKkVanWilpBUEPQ2rlyBtahnfjitTczzj\n1pbVeUGPThThaoreo6wHllfoGrLLffTIKweMikDbV3fSEr9f80J735Fnd02TURDoda7v01bt8nJR\nVdhe7iQsqvNUmKkO1NjA7pQSF+WANqKqB63AnY3RxcXyHappzsaYq116GrtjKevugvsO1XXXJ3ia\nC1W1p0+X/MNDeOtN6rxI2U+dbWv9n1EQnLrOOJFUX9SeLu5wjXCCMWy9ZFnXo3mhonQ/+qj3WNfO\nkfoMx5PUTCmXPApSeVI0ge9+d/k9Y7uy3/EoWOdRzt0V9fVql8ndrB/9aP3K+My6pmxeNa4UH1P1\noJO0eCLTvXu9SshdOO7guPuWOzWoKtKcz7/1rd1rq32UX0Zdobc6FS/OKzW/7kRGFwWG9sI3xezk\nHMuTy6QY8/hWtTNVfJ485WKS1vw73Lwwq48nT/r7l+79Uu7VR1R+lUvjS31a/Zana9byudjzrt/N\nyu12u+r9n/lW/+GRsjp+lja8PNmM6nedD3lNx+zocBClNYQQQgghHDy3q7TSdohec1SlTk+Xp3Ct\nPujdzJOhGKdQK1TaFtLLb7XqFQSh3zDmK1Zq3YlYYnTCFCMTzE5rElTX3MkVdYXPFSVsPZc4kmYl\nTpzN0507vRewoAKnOqaNHhVX1b1WuNUbU2kwhittkXl6i4sWQFW7Kisqs/ojVbE331y/ss6UR8Xr\nm3n41vyp7OzLVCGokDMP/F6143aqOu2k1B6KguBOtaPCTAW64s6zZr6FU4OpSNTYxLS55dxDW0Ge\nsqZ6YN+p6TOmI+1l6VVOr3pn6686k1p4dtaXQ/mSUvruu+tXKUMaA1KAqmrb2qKcfO97yzVUfp4K\nNFN+aOcsRmpMzePFxaLUURmnEiTbV+6o6X1527voI3U+ZNtzPoD3s41dzL5flWlGXmAajCOseY73\ng1kc5OPjvj1Udy66weykREb1qHOz+qbbnVK+1O/YHup/qg99nx7zNcaoe25wO4QuWsIo1rI+10lr\nUk6lmEoh53OEXtUPNUerXbn7V08KZD9i/gQ/d/F4uStW52j2DZa97oi1JdZ0U2xVxZhWO9GuWO9X\nPyRdg7s7nB9mUSxAlNYQQgghhHDw3K7SytUjT0fS/1WVcbHzqE7Qe1HQS472ItVjr64WatrCnaq1\nSeuI3xOjEyu4OuRqkqtD/i/FS7/TionRBi4u+rqDre5WGXYrHq72GaGg2gHp+lTkeDKWYJpa4Wol\nLlstqThsz/qZO52J9l/O654KrV4vL3tlWFCd5cpTOM9crT6rTahUI5WLK1bilFVChbmezMZ8ur7P\neJ9UWpknemq31tsg046NzJRkqhknJz62sPoVvXwZE5W2ydyFqHEN+R3CujE2sLYuq8LFWJX6jOOF\nOxtSsPQ9KbJvvLH7uxo7UnXH07IElXt3UhEjUKgPVdVa46tGL2itj/c5ip3c2qJQ8nMprzWaAFV/\n9jvOd1TJHFS8Tk7GJyDVNJVfngGvuuJuEU/cUznr/cudEy9cfEzaxJMaKYCquVR5pa35XzAmqYsE\nwPvFa6/1O59u98O1D+9rtE1W3//ww/4eoTqlwqg8SFlVNAHlm9EGuFM4OvWQzzAuRqw7oVLQ3vTk\npP3rcv108oralr4MiE3e+eNwjnO7mnXu5W4kn2W4G7cnUVpDCCGEEMLB83KiB3BF5xSv09N+lTWz\nw+EKhifbzM60b61XgGe2TMj/1rbVrQBHp0bQ81arP9rO0G6Mq2aesvP0aX8m+Oy0KeaTsfiY99qu\n9KKk3aKLR8t2Ux5VD8zL+fmyenURJlwcOKVBD36n5q9WfXw6QeVe/c3Z9NJOjKrPO+8s1xr1zZqG\nU+ZmSsrIm5Zn2TsPVikqrGuqA1TE62lpTs11Ch0/Jy76Q02LaiA949k/OZ6oatS4w1R694WqEt8X\n1bOaXsy0y3R9mHUjJeymGLGy56NCqjHJOcjtDlFVU5+ppyUpGoA+U7QX+iooTSlWzCPnUSGb2fPz\n3m+AMVNdGs4LnfOgxkb9Ht/jSUSCcTNVfinGjPcqVqu+L3Dn0J00x50znkxHpfz8fCmP7pW6tk42\n5AltUtClQNKWUmNAfasqz1T3uAvCnRjBXS22G/0oTk6W62pH4u23168///n6VdE11P9UTvVljmWV\nU9EUxNOnfcQMRtfQNfS90Y5txcXIvrpqr+g0Kj2bKE1ny+92sJ1dao2DzfmAY9fZvO9JlNYQQggh\nhHDw3K7Suq9XX1VaaE9EmxKm7U4pETw5q67iaFvnzioGjMs6VVjrCp4ngFARUTmpDDuVgwrY3bte\nuXI2ki6+pzvBp3r4alVFRZsKAW2zBG31qBpU9YaenC7+KlfitIElN3mjM1qFoNc9lWTmiepTvZZU\nNAfHAhVMtM+2f1KZqGlQhWcduJNQaENOZa+2F717WR5Bmydnq+ZszC8uenVGaqUUAeaBtqK0Fa/j\nqb625pUsZ8sqWKf6PhUY7Vp88EHvVU/FmHGRpaiq3LNoJHWOZtuyr87iBbudDo6hjz9e0patrRS7\n73xn/ar5RP9LYdX7PKlN5ZUCpugdl5e9XSGjPQi1A2Ns0haPp1eJ1WpR2DhfOPVZ5eCOB/PASAU1\nKgLvIW6c0HbV3VOFyldjjNLuXq8qt4uIIVtQfU/vMw5y7Tu8fzKyibON5/MGVcMa0YX+NrL55s6h\n+kw91a3+Pzt98cGDpa642+juLbNxxu+rvv7+9+17R1T2NRbcvZ5+SOqfqg/OXaOdJ+Zn3+cQQ5TW\nEEIIIYRw8Nyu0soYh1TGuLKrtoSylXRebFpZUxHRikif0/arrly5qtBnXClAIenisoqb4v65eKzu\nzHSq1IwzSS9h/e7evV7JdjaPs5OxeA4zTwu6uFjaSStpnqcsex3lW2eJC31fv2fMUrXrN77Rq4TO\n/tCVn3VKJWKkmLuzw6uNY4WfM6qA3le//MpX+vIIp5IJE4+1i2pRV7r0WuYKm/+rPOpnVJgZZ7cq\nkzOFjuVy5VX5qGpXz2SqFc4jWuXgzgwVMMHoAVIean6p1lARcQq/oF14tUlnfEsqcDxZSPnUGenK\ni2xGuZOlPL7+ej8/KD9OyRJsP9ow04P87t2lPDrJb6PAffLHP7bWWvuvTdLHv/hFa621ox//eP2G\nbCB/8IPda6keNI+oTquKzV0Ezie8R7g4mbwX1Tjj6j+MCsB4zVKWlW+eAU/Vm5Ed7t7txyzHIG0j\nZ8rqaAeD+an9v7XFxpN+EByPbs7W9zXP3r/f369cf3O2kXyfsbbrPKJ4rFLm+dygcimeq9sZVHlk\nA6v2r3Me7zucD5lvFy+eUOE8O+vVXP6W+dM9XuXh8xX70GgXmvcjqrKzqA+GKK0hhBBCCOHguV2l\nld6ktAnSKqDajVCV4BM9vRK18pGix5UFz6HXSuHhwz62Gu38nPf1LL7pKG4ZFR13CgYVY0GbHyp/\nVTnhyhReptsTsTaK8fL/BuftrZVf9dZXe7BOaOPKFTptothX1E5qv6osUG0ntGVy8SadzXX9vjvZ\ni0qR+jTt95jHkU0v7YJpT0sl2Cnno52L+v7paW9HRQ9o1RmVHubBnY5UFRaeAOXy6aIIuJNgeGpe\nPQGGKpgbV4ybyVNcBNO9f7+PkcxdFBflQVDB06v6kur+q19d6lDzG6ON0HZcNp16pdqhk9x4/nmN\nxcnTj1y70EubETOopsmj/+qqt/3b/P9A5dj0p+uN8tr0KkVS7aHvj6JXtLbuM7zPsP/Rppj9zrWv\n0pMd8cXF0l5SqVVnjOjCnRrNo/RpYB6q4s57Gq9BXDkI7z0XF8v8rfqlgs/TErnL4mJ4Kj315/qe\nu1fO4tCOoh+01kdJqPck7jChH27zrzQ1vjSOVA+07RcPH/Z+KLRjdifMCddeN/n1OLtY9c/33ttN\niye6CfpojCIaMDrPvvmfEKU1hBBCCCEcPLertG6erK9PP9Naa+2IK3Wu6FrrV5RUeGhfScWVq32t\n4OgJ++GHvS2IVvPf/vbutV1cuH1tNKpHv1MkVV7arjovZ8YBrOmynl08yc3nR84mVDDvNd4i40fq\nWsq3Vm4//OFu3pxNEJWJ6jFKWzMXeYHqrVMgaT9bVTaughknkVEpGKHA2WPxVKTVqj9/nbbf7n16\na7uTo2qZWDcsl4shSvVCOG81rU3tAAANmElEQVTZq6u+rLOzw1k+F6OZqs3VlY/dypOxxEZh1S6D\nOKKtF6lRLDi+OA9Q+aFtF+1lRa1DzV/Vjry13tZaachWkuOMJ0hxB+r8vI9UQg/90U7LqLyc01gP\nX/jCUi5FBXjnnfUrYmke0WaXqqiLM6myPH3a737QE56q4Cw2uHt/tepPKGN/pGLMvk910Nm7189o\nR0p13eWb88fo1CZ930Wvefx493/Vne4RUpB5ohLHTvUFcHG4qbAKt+NE1VfU0w7Zt6nKKzqAyqFy\n8oQspcn7d42CQ/+UUZuOyuF21sRoR2R2z9McKsWVcxf74U0qL09x4z1CvGDUABGlNYQQQgghHDy3\nq7RuVLKj8/9d/0/FjjERz856Lz2tcLSakLJFRdadIc44hlUJrKpra729lFNYiVO66oqWq2KqEVq5\naWXDc6XdSogxBK+u+rPOzUle3f9uhU41oNoF0mNY+VG7yY6NXutu5ab2uknFrt669dpUbV38SJaH\nStlqtayC9Z6UVRfflHZKtKWkHXftKyNb2pqGi6lH3Eq82ndTOaWS7/JEWzwX37V69us7zobV9Uen\nyrjyjWIF6rsaP/TOVqSFZ//YzSNVUKq6d+70fcC1W81fLYfbARh53dPeTV7xgja42tGQ7aqLYECv\n9uPj3hNfc5CLJ8kIH06RZMzY09OlLqTmSrHSNZQXKcTKr8aZxqPmF83hoo5HFyeS8wgV1tl44nxS\nz2GnIulia9bYta0tyteofSrn5/3OE23I9/W94DVG0T2ownIel7IqVC7uxLnY6KOYs7qXU113cLfH\njbuqTOs9+ibofbUHx5Oi4Oh3GpfudMO6q+yiqHBudjbzhO1W24p1x9jQnN+UlspF+3YXoaL+7Z6b\nPqVN69H1tQnX9P/Anx4/Xl8MDljb4Od//Z/1+3W7VxWuylTl6H11InZybv/LOJ4TS32Q26S9dUTa\nHBZgHcb0Pp0YhAuNUuV6bhexY9Ihhg5NnLxUP/WhYxJ83uJuOrymFht1IKpz03FJ0EjbOb+5wyQe\nPeq3OHRTZUgd95BKOJmJJ096RyoGJHfhw9hOzqmtOvYwcLwLHSRcqBrmYZQXTlrsly5sGKFpD0NM\n1Zvq7LjTWWg2wfavY4cTvQtzJvggwwUMF4t1MVKdfGraLtSL4DaeC8Y/Mt3guKFDDKFjGduHh3hc\nXPTOQQxjx/Jwa5cCAfuY0r+6au3Xv94tlx5u6OCiaystmjCQ0UMC5203NlnHs+D17K/1kAs+lAq1\nKU1nON6c42Cdj2j24BYobkyz33E7XemXYPXWCVGLBpp90VHXhZpTu9+/3wfA5zxCcyk39woeRqOy\n/PnPvVkRhR7ln+HQ9FCn+lDfFuynDx70B5gwbUFBQbh5QowcQF1dcOyyXYVzKK71tLnG9b3P7nxl\n+zxl2udrr79+1PYg5gEhhBBCCOHgeTkhr2joffaf69cvfnH3++fnXiXjal4rHbdFzO1arVq4+m/l\nsABK31zJOEcSMloRcYvTbWVo9cUVtlOKRgGK3SEH7jAB4b6nlZQUadXlRx/51Z6g2YZg+ZUm20ll\nqSoOj/NzWyBOuWPdjcId0SGHjoH6nP1Mn8shgyFUGKrob39b8n+TglPL6cozCy11ctIr3DMnE+c4\nSMWW2001SDhh23N7ywWwduOtOoowbe5sCKq/7Lfu4JFReC3hHCc4/3FM06Sk7prQzEGKjnPqUHn1\nPY2VaoJV060ho6oqNCqX6zPKg/qA8kClqKq93//+7m85FqXcySlW5aH5kND4lGJXd+S4K+CcipyS\n53ZVRDV/4La+CxXEvkEzI9YxFcEHD/rx5UI/zdpPcDtd17p3bxmTnNc4P9Kxk/dj1j3no0eP+oM+\n3NgWzsyI9xaOmfr8ob7r5nM68anNdb8WMnehiVNV3HUvVJ3SNIv3PrebwvYSV1f9swyfo7gby90G\nHbfLHdLRoRGbz7bKqjCHM70oUVpDCCGEEMLB81IcsajaHK3+ufu9UUBeZ0TNlStXFLQ91KpKoVW0\nEr93b1k1Uqmi7ZlTRvZVLGva+xonO9tVrpBuClf1KUNMuN937fbgwbLypKLFkFCCCpdzOFN6ta5Z\nB05xZN9wTl1UPeoRulIMmJZWxTzkgUGy9b8M9LWqHjl7UP0f2VGO8q3xNVOUq/JChc3ZcnEsMOyM\nC+lTr81yUUl2YVlULhdCSlQbLs4DVIIF5xp+jzbldC6osL1mNrmzEHm0Gzw93V5D9mJHcgihQxKP\nn3ROUwwLV6H6xzBntDNn33BtMHJeZN94//31q+pGNq+0b6x2lq0ttrAsl+b542NvB+scHcW+/bDC\ne4RzJuSOBe9zKrfsikfOYi5wvOuHM4dGMnLW4a6IC8XIUG2cA1xbHB/PlX536I9QX6ASzV3O6iDN\nV41B2oByntf9T2kq7+z7n3zSP29QBeX4cT4ZM4emq6te3eT/9AMR6vN6XtrH6XfznW1oU3fM/ack\nSmsIIYQQQjh4XsrhAt3/9JSvCgsDWjOsh578tXpW8F8qB0pbK9VRGAyt1rW60MqSK2wXQsMxUj+d\ndzOOGt2uVugdTGWLK976Pld5s/zuG9qL33v0aKl3tZte1T5aXdIzXjCQOZXMqoayXVgu/oYqmfOq\nVV1WW0pGrWA4EafAcVWs/kekpKxWva20C4XkFNeZF3pdwTN8EY8OdLsIqhuq0lSgq4pAr3ra2jmv\nZmdbLUZjgIoOd0dc2Bh6xrOPjQ6NcMdLCjdfUM1muUc29Qqyz7rRd2S7SVs85UHB7DUOufNR+5A7\noIWRQNxxqCyPykkb5doGm/r+1/3PtdZae+Vko9K89dZu2iofA5hrJ2N0aEdr6/pRfvgd5lu4Me28\n1atdrmxvqRArDb3PI3tZ51Ls2K5VfXO238LZuLooA06ZvXOnVyn5W3rPizrPjdIWCl1WbXVv2lmp\n/7txyDrlTlwNecUdPu4eME8aA/TZkC0ony2+9CUfWoy7XbMdKc6LI18IN3fSV2a2K+Tu14P0O4WV\n96d9nzNAlNYQQgghhHDw3G6c1vfeW1+MK3PnCX/nTm/bWONZtubjsjIYMG3tqBbcveuVAmf/sVkJ\nbe3LuLK4SSGa2R0yDX7u1B2uWmrwc/edWX5dvM+bFAmuCqka6btaeWolyoDyLlbi+fnc+5AqAK85\nynfNQ13J628GRqedc7U/rJ+7vs7/793r1Tx60dO2dWY76QKbn5zsv9qleutUXWdHN7KrYv5mNrh8\nn21SlTP2UTfHUDESztN1ND5n/dApWqwrF/e0fs+plYzl6OJa025Ov9P4q/MqbemEU1ucIunKTy/9\n+p6L5MH2pOLK3bFthJqiYLL+aVfOewXV+pniXwPnc552nvqz2MuO2mdexKei5knMYkvXPLE/MfYp\n7dFpD6t20/tSYPV+PfZ7ZvM9u4cKjuVRJAdGutB8zoOPOK+72OD0FRCvvrpEK3D9kPM843e7HUNR\n64X9zsVadvFzXQxql9f6nouVLTafJ05rCCGEEEL4t+F2ldZ3311fbKa4VNs72tg5mxKutqjsMf7b\n6PczFYk45c59rzJJszuVS1BxdIrLTWrazPZz5pVJFaf+jvY4bC+lTTXRqbdULxjTrv5mpkLPTmJy\n9VNVQmcD6Y4WZF3dpES2thPnbupVP1NgyajP7eN5OkpzFgNylK77jlPyPq3N+D7e0MzL7DQZ5zF/\ncmKvtx3DjLLh8kf7MlFVRH1H40Dqi1P+BZUgd/pRPTbV9Vl3RO6+7ejiovLv1nobT31OZXXfWMz1\nuG4XB3lf5djN/05Nrb9x3vNuDmMUgZF65eYF2mW7vs//3e7KTb9lOcUmDZ1+KaxneZ0b3PifKcPC\nKcrcbT09Xd6jwk8bebWDPpctNeNuawfDHVfbWh+/WFCtftHnkhHs6y42tNudE24s3LRDMMlvlNYQ\nQgghhPBvw+0qrY8f717M2W9WxWsWy9DFMuNKwikq9fuzkzUE09zX/migLGnlOYtltv0ez+/9lB54\ne+WP9iuzlW1Vq2kb5GzVaJs2U2JF9UZ36opWqFoN8xzsm5TVyqjvMW2+7zw5nfJf++u+/cmlPRsz\notp+Cqfq7ms/K5ytVGteAXL2i46bFFanhjmbO+aJfd/tZFRm3tnumiyPs/mv9cHxxZNqaGvINJyK\nKs7O+hjXVGNpN0oVWDAWKetytAMyU81v2u0ZlavW3adV7t3cRUZ5doqkm2vctWa216M0XZQRF3fW\npVPTc+Ng5g/BPO0zn3BemNnm7qu0ci44Oel3zmZe87NdTF7zpn7NZxfeI3j/ErNngJon9zzk+psb\nTy/SD50yjLqK0hpCCCGEEP5tuFWlNYQQQgghhE9DlNYQQgghhHDw5KE1hBBCCCEcPHloDSGEEEII\nB08eWkMIIYQQwsGTh9YQQgghhHDw5KE1hBBCCCEcPHloDSGEEEIIB08eWkMIIYQQwsGTh9YQQggh\nhHDw5KE1hBBCCCEcPHloDSGEEEIIB08eWkMIIYQQwsGTh9YQQgghhHDw5KE1hBBCCCEcPHloDSGE\nEEIIB08eWkMIIYQQwsGTh9YQQgghhHDw5KE1hBBCCCEcPHloDSGEEEIIB08eWkMIIYQQwsGTh9YQ\nQgghhHDw5KE1hBBCCCEcPHloDSGEEEIIB8//AamlCfTyEBBRAAAAAElFTkSuQmCC\n",
"text/plain": [
"<Figure size 900x75 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"utils.digit(X.reshape(1,12,28,28).transpose(0,2,1,3).reshape(28,12*28),9,0.75)\n",
"utils.heatmap(R[0].reshape(1,12,28,28).transpose(0,2,1,3).reshape(28,12*28),9,0.75)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Relevant pixels are highlighted in red. Pixels that contribute negatively to the prediction, if any, are shown in blue. On most digits, we find that the digit itself is highlighted, as well as some parts of the background. For example, we observe two red horizontal bars next to the digit \"3\", highlighting the fact that if those pixels would be different, the digit 3 would likely turn into a \"8\". Same for the vertical bar above the digit \"4\" that supports the class \"4\" instead of the class \"9\"."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<h2>2 &nbsp; PyTorch Implementation for the VGG-16 Network</h2>\n",
"\n",
"<p>In the example above, LRP rules could be easily expressed in terms of matrix-vector operations. In practice, state-of-the-art neural networks such as VGG-16 make use of more complex layers such as convolutions and pooling. In this case, LRP rules are more conveniently implemented by casting the operations of the four-step procedure above as <em>forward</em> and <em>gradient</em> evaluations on these layers. These operations are readily available in neural network frameworks such as PyTorch and TensorFlow, and can therefore be reused for the purpose of implementing LRP. Here, we take the VGG-16 pretrained network for image classification. For this network, we consider the task of explaining the evidence for the class \"castle\" it has found in the following image:</p>\n",
"<center>\n",
"<img src=\"castle.jpg\" width=224 style='margin: 0.5em'>\n",
"</center>\n",
"<p>The image is first loaded in the notebook.</p>"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [],
"source": [
"import cv2\n",
"img = numpy.array(cv2.imread('castle.jpg'))[...,::-1]/255.0"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<p>It is then converted to a torch tensor of appropriate dimensions and normalized to be given as input to the VGG-16 network.</p>"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [],
"source": [
"import torch\n",
"\n",
"mean = torch.Tensor([0.485, 0.456, 0.406]).reshape(1,-1,1,1)\n",
"std = torch.Tensor([0.229, 0.224, 0.225]).reshape(1,-1,1,1)\n",
"\n",
"X = (torch.FloatTensor(img[numpy.newaxis].transpose([0,3,1,2])*1) - mean) / std"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<p>The VGG-16 network is then loaded and its top-level dense layers are converted into equivalent 1x1 convolutions.</p>"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [],
"source": [
"import torchvision\n",
"\n",
"model = torchvision.models.vgg16(pretrained=True); model.eval()\n",
"layers = list(model._modules['features']) + utils.toconv(list(model._modules['classifier']))\n",
"L = len(layers)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<h3>2.1 &nbsp; Predicting the class of an image</h3>\n",
"\n",
"<p>The input can then be propagated in the network and the activations at each layer are collected:</p>"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [],
"source": [
"A = [X]+[None]*L\n",
"for l in range(L): A[l+1] = layers[l].forward(A[l])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<p>Activations in the top layer are the scores the neural network predicts for each class. We show below the 10 classes with highest score:</p>"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" castle (483): 11.029\n",
"church, church build (497): 9.522\n",
" monastery (663): 9.401\n",
" bell cote, bell cot (442): 9.047\n",
"cinema, movie theate (498): 8.016\n",
" analog clock (409): 7.108\n",
" street sign (919): 7.102\n",
"traffic light, traff (920): 7.058\n",
"thatch, thatched roo (853): 6.978\n",
" alp (970): 6.812\n"
]
}
],
"source": [
"scores = numpy.array(A[-1].data.view(-1))\n",
"ind = numpy.argsort(-scores)\n",
"for i in ind[:10]:\n",
" print('%20s (%3d): %6.3f'%(utils.imgclasses[i][:20],i,scores[i]))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<p>We observe that the neuron castle (index 483) has the highest score. This is expected due to the presence of a castle in the image. Note that other building-related classes are also assigned a high score, as well as classes corresponding to other objects present in the image (e.g. street sign and traffic light).</p>\n",
"\n",
"<h3>2.2 &nbsp; Explaining the prediction with LRP</h3>\n",
"\n",
"<p>The following code iterates from the top layer to the first layer in reverse order and applies propagation rules at each layer. Top-layer activations are first multiplied by the mask to retain only the predicted evidence for the class \"castle\".</p>"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {},
"outputs": [],
"source": [
"T = torch.FloatTensor((1.0*(numpy.arange(1000)==483).reshape([1,1000,1,1])))\n",
"\n",
"R = [None]*L + [(A[-1]*T).data]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<p>This evidence can then be propagated backward in the network by applying propagation rules at each layer.</p>\n",
"\n",
"<p><b>Convolution layers:</b> Observing that convolutions are special types of linear layers, we can use the same propagation rules as in the MNIST example, and a similar four-step procedure for applying these rules. Steps 2 and 4 are simple element-wise computations. Step 1 can be implemented as a forward computation in the layer, where we have preliminary transformed the layer parameters, and where we apply the increment function afterwards. As shown in the LRP overview paper, Step 3 can instead be computed as a gradient in the space of input activations:</p>\n",
"\n",
"<img src=\"http://latex.codecogs.com/svg.latex?c_j = \\big[\\nabla~\\big({\\textstyle \\sum_k}~z_k(\\boldsymbol{a}) \\cdot s_k\\big)\\big]_j\">\n",
"\n",
"<p>where <i>s<sub>k</sub></i> is treated as constant.</p>\n",
"\n",
"<p><b>Pooling layers:</b> It is suggested in Section 10.3.2 of the paper to treat max-pooling layers as average pooling layers in the backward pass. Observing that average pooling is also a special linear layer, the same propagation rules as for the convolutional layers become applicable.</p>\n",
"\n",
"<p>In the following code, we iterate the propagation procedure from the top-layer towards the lower layers. Whenever we meet a max-pooling layer, we convert it into an average pooling layer. The function <code>rho</code> and <code>incr</code> are set differently at each layer, following the strategy of Section 10.3.</p>"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [],
"source": [
"for l in range(1,L)[::-1]:\n",
" \n",
" A[l] = (A[l].data).requires_grad_(True)\n",
"\n",
" if isinstance(layers[l],torch.nn.MaxPool2d): layers[l] = torch.nn.AvgPool2d(2)\n",
"\n",
" if isinstance(layers[l],torch.nn.Conv2d) or isinstance(layers[l],torch.nn.AvgPool2d):\n",
"\n",
" if l <= 16: rho = lambda p: p + 0.25*p.clamp(min=0); incr = lambda z: z+1e-9\n",
" if 17 <= l <= 30: rho = lambda p: p; incr = lambda z: z+1e-9+0.25*((z**2).mean()**.5).data\n",
" if l >= 31: rho = lambda p: p; incr = lambda z: z+1e-9\n",
"\n",
" z = incr(utils.newlayer(layers[l],rho).forward(A[l])) # step 1\n",
" s = (R[l+1]/z).data # step 2\n",
" (z*s).sum().backward(); c = A[l].grad # step 3\n",
" R[l] = (A[l]*c).data # step 4\n",
" \n",
" else:\n",
" \n",
" R[l] = R[l+1]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"As each layer is composed of a collection of two-dimensional feature maps, relevance scores at each layer can be visualized as a two-dimensional map. Here, relevance scores are pooled over all feature maps at a given layer. The two-dimensional maps are shown for a selection of VGG-16 layers."
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAIsAAACOCAYAAAAB1PtMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi40LCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcv7US4rQAAAoNJREFUeJzt3cFKlFEYgOGZNEc0iBGFiQiDwDbe\nh7fgnbZ1FSR0AS3aCy4kJVEUpm3Ll/8wVPQ8a7/5lHk5Czn8/3y9Xs+gePGnfwH+HWIhEwuZWMjE\nQiYWMrGQiYVMLGTbm/jQbxcXk/8tvF69Gdo9f36aPnx3N7R7trU1fXaxGNt9dTV59OPZ2bz8nJOF\nTCxkYiETC5lYyMRCJhYysZCJhUwsZGIhEwuZWMjEQraRKwoPy+nXDJ5/ju1+9fXz9OH9/bHlp6fT\nZ5+fx3YfHo7NB04WMrGQiYVMLGRiIRMLmVjIxEImFjKxkImFTCxkYiETC5lYyDZyn+X+fvrswc33\nseWXl9Nnl8ux3Ts702dXq7Hdo4/sCJwsZGIhEwuZWMjEQiYWMrGQiYVMLGRiIRMLmVjIxEImFrKN\nXFE4+PJp+vDx8djy8/Ox+REjdzO2B7+Kvb2x+cDJQiYWMrGQiYVMLGRiIRMLmVjIxEImFjKxkImF\nTCxkYiETC9lG7rPMTk4mjz68/TC0+vp6+uy71dPQ7tnj4/TZ0UdmjOyOnCxkYiETC5lYyMRCJhYy\nsZCJhUwsZGIhEwuZWMjEQiYWss1cURh4u8buYj20+uhoPnl2vf1yaPdsdH7A3BUF/iZiIRMLmVjI\nxEImFjKxkImFTCxkYiETC5lYyMRCJhYysZBt5j7L7e302cFXoeyODN8MvAJmNhv6u38s3w+tfr3Y\nzFf5OycLmVjIxEImFjKxkImFTCxkYiETC5lYyMRCJhYysZCJhWy+Xo894oL/h5OFTCxkYiETC5lY\nyMRCJhYysZCJhUwsZGIhEwuZWMjEQiYWMrGQiYVMLGRiIRMLmVjIxEImFrJfVi83k7oil0oAAAAA\nSUVORK5CYII=\n",
"text/plain": [
"<Figure size 150x150 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAALUAAACyCAYAAAAEVo4RAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi40LCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcv7US4rQAACXBJREFUeJztncFqZMcVhqujbnXT8kiMkEFhBmsY\niA3BYAcvvI2DgxcJgXkJb/wAfiDvvTAYDDHYu3kAC4JjMMhgIrCxGGk0qC01nWXCnK+grm7rSjn5\nvuXh9q3q2z9F//ecOjVarVZFJBO/ue0JiKwbRS3pUNSSDkUt6VDUkg5FLelQ1JIORS3pUNSSjvGQ\ng317eBjTl+N+U1iVUYiNri7jhYtFjNHYJycxdnUVY8slT+j+/RibTvna615316DnU4Oe24sXMXZ8\nHEJvPHkSf2zAlVrSoaglHYpa0qGoJR2DGsWhuCyTEJtswVf96acYI0P5448xtrfHg29vx/lcRX8z\nKWBmWw1XT3O9blbj+Lyr0LXTWQiN7t279nxcqSUdilrSoaglHYpa0jGs4wCDQxnBGxiGM4+U/aPY\n99/HGGXBSkGzNwZztAIzi5nQO2YKB6PH93allnQoakmHopZ0KGpJx627kFHp10wHDSDck66jzOPZ\nWRzj6P6fQuwPj5/hfH45i/ecz+N1sw0whVSWSbGNjRi7AUPZ+my7/Iat9+yDK7WkQ1FLOhS1pENR\nSzoUtaTj1t9+9HXDrddSqTK9MDg9jbGDgxh7vrHDA1X248brGi+8xc24634rcVP3fBlXakmHopZ0\nKGpJh6KWdNy6UWxNaXcyGLB5dgybO8k8Ptp7HoO//gqDbPLYFD6CjbvUCYpquR8/DiHa6DqEAftf\nwZVa0qGoJR2KWtKhqCUdt24UiU6mh9wexEbTeM8XL6Ihnc9fCbHJ07/HMT7/nOfz4Ycx9uBBjH39\ndYx98EGMgekd/b9uxm3ElVrSoaglHYpa0qGoJR2DOg5sabuADF7fcksq6wTzOJ/HzBx18p18+WUM\nfvEFj/3kSQhd/u738Z6ffho/+/77MUZthCnzSLt7S+m3IXegDb7rxpVa0qGoJR2KWtKhqCUdg/7r\np464+/sxg7cNs8I2t6W0dzCC6ybTONDkDEpC6WyY11/n+Tx8GELU9XeHTOE338TYe+/FGBjpy41Y\nWltKKZPlRQzChC7mu3GYHuqoZYXt0CRyDRS1pENRSzoUtaRjUKP40Ucx9vHHMfbnPza2uS2FTSFw\nUcBIQfZwRqbwnXdi7PCQBzo/D6ETmOLO22/H4GuvhdAvy9g0ZxfKaCen3Fq4HB/HGOyPnJHxPYJn\nAXsmqSXyeLz+s3xacaWWdChqSYeilnQoaknHoEbxk09ijLKMdG7K7r0OA0GZ6XQrXkb9ZGavvhqD\nVG5JJaGllPL0aQg9+hvcEwwXxaDIlCf+1Vc8H9ofSTG6J31HysyCwV2NY6a4FN5SOiEV0oWNuFJL\nOhS1pENRSzoUtaRDUUs6RqvVcC1gDw+/bRpsUnqmyWn37FZ8/fHsNKZydzZgI/Bnn8XY9jZOZ/WX\nv4bYaAE1zeTuYY44BtQk18A6dHqW9PaDCsH39mKMNkpXNk/T3PGNCNSBv/HWW01f3JVa0qGoJR2K\nWtKhqCUdt95uZzIG7/iP72LsXiVPTt2KyPRAqvvkJNZY72zD+S5Ua0zjllJGx/9qu5aMFBlcauVL\nA9fqymkcuhYMIJ4tU9sA/RLUjasUfgkQRynNdfKEK7WkQ1FLOhS1pENRSzoGNYrYthe8EaaYvgPz\nWEoptIGVDvOE7Nj2NmzG3YTTPSmLVskoEis4mBS7ErW2ye3b6pieL2QZm8+Wgesmta5L1y+TbsaV\nWtKhqCUdilrSoaglHcNmFH/4IcbIhFHJ48EB35POOqllH19iE0zqsyW0Fj7gTaQEZdwwy0hmj0xq\nzzNWKLO3XMYc3nQK2cPWFrtkPCvzxiwljUPZ1UZcqSUdilrSoaglHYpa0jGsUdzfjzEqy6Q9dGQo\na5DRhHsuN+I5J/TR1u17pZSyWFAh5W9DhPwtVbiiiepgzAj6OFV6YuckGAfPcamUqFKWks+BuT6u\n1JIORS3pUNSSDkUt6RjWKFLGjDJHlf1/BGWoynY8J4UM1/E/40dbzxqtQaby6CjG3n03xppNYQfI\nP1KszzC9ymgrYy/pjJ5GXKklHYpa0qGoJR2KWtJx681sWkFDWNikXCyos2aMUfUnZQrJt1JGsHbt\nm2/GGB0t0zdT2AcaprlDKTSo6fJ7UYdT/nQbrtSSDkUt6VDUkg5FLelQ1JKOYd9+UC0tdC/ClG0l\njUs1v61HkJDjpzQ3ffbsjOdDWX8qBW9ticu1xvENQpdzYPqArZcX0N2pdkZP64/jxluR/6CoJR2K\nWtKhqCUdwxrFHi1oq9liOhOFnCbkv2fQjnd/v62Ot1ZjTWlySr1fQBp5VmK6uLWdbnM3pdJuKnt1\naKrtTIb6ADTDPbTiSi3pUNSSDkUt6VDUko47WU+NWaubANweZvogO/bKFhuZ1hrkdVMzf2T21t6i\nd2srzmeL2x+3P4vrZ0hdqSUdilrSoaglHYpa0jGoUWwto+xEaykjxC4bt3dOKHtYcTyUAdzaajU9\njVk0GLuTreqzmZdKQjtk/+glwLp14Uot6VDUkg5FLelQ1JKOQY1ir0xWjdYzRLB/bdPtCj6mDnvo\nmk1ca4/d2v4/AkwcdbCirzOfRyM9KXHsLkaPDivFZ94jDetKLelQ1JIORS3pUNSSjmFLT1v//Pds\nX9tqSCm7RUZmQo6ytkmxde70LFr3+vU8mObqKu7DpGGwXfHpaQiN5nHetdJTnk+M2cpX5L9Q1JIO\nRS3pUNSSjmGNYrMbaWfdZYvNGUUwTKWUUqBBDs6R2qvCZ9EA0tj02VIwo0jbK3/+Ocaos+tuY1lv\n7Tcg000lwLUzY1pwpZZ0KGpJh6KWdChqSYeilnTcyQ5NfbmRDb4v0yEtTbXKM+j5S47/CoYZP4wp\n6NEitgEupfAhNvd3Q+jRg7bOVGUKb1laU/4V1n2oryu1pENRSzoUtaRDUUs67qRRvImDLvuYx+aN\nvJVrZxtgwiAHPQLzOaG0dId9t1SGQKYSD2mF9DV9a/y1OhwOiu2Te7hHV2pJh6KWdChqSYeilnQM\naxSpaBcya6PNzXhdF+PQelYJgIdSdsiOjc6fx2CXjkotY9P9ah2jqF59zSk8zIRWts5yuXq89vz8\n+vNxpZZ0KGpJh6KWdChqScdotRroIE6RgXCllnQoakmHopZ0KGpJh6KWdChqSYeilnQoakmHopZ0\nKGpJh6KWdChqSYeilnQoakmHopZ0KGpJh6KWdChqSYeilnQoakmHopZ0KGpJh6KWdPwbsgE+WSSH\nEW4AAAAASUVORK5CYII=\n",
"text/plain": [
"<Figure size 200x200 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAANkAAADWCAYAAABc6CejAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi40LCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcv7US4rQAAEvhJREFUeJztnV2LbEcVhqudbmfMJBOOnJAjCQYC\nfgQSI0QMEQLilSjkDygI/gJB7wTBG3+BPyDeBXLlhXeCgiB4IfgRUBMMRCIkJhDM8YQzOXNoL0K6\n3lq9V2XN3r12d855nquamb1rV++eRb21atVai/V6XQAgj0/sewAAdzoYGUAyGBlAMhgZQDIYGUAy\nGBlAMhgZQDIYGUAyGBlAMss5H/aPl16KhZcsZx3WB1xcDP/++vXafvfdWF/n57X93nu1fft2bT/4\nYHvP2VltHx/HnnNZjo6m3b+P7yWK9/310O9D0e+vlFL+/e/afu21TfMLP/jBIvIYZjKAZDAygGQO\nY/4/BBmiY1Dpoe0rV2rbkxqllPLGG7WtEu2ee2pb5WEprUSMyjpvzN41dzJjPqf3/qxcv3q1tq2U\nDMBMBpAMRgaQzLxaYofSZV18x86iTDsjt16ual8PPLBp3zyvzzwpN9ubVEaorFSPpH5+9TqW0kqU\nqdIHLo++P/su5X+gkfxBmMkAksHIAJLByACSQcgPoN7c5bKuwxrP7oVxs+sfJSqgWXtduzb8EPtz\nVsQHTGfEd8NMBpAMRgaQDHJxAM8b3srIVfO3hbrwNRpEO9MoDysXVYZEg11x209Ct4HC2z4j3jkz\nGUAyGBlAMh9bvTE1qmMMXaWgEvHRR2tbZaRGC5gg4Ju3q/xU5bi4uOUPoNWvncHdvWRGBkVhJgNI\nBiMDSAaNMYDKCM8DpcHCpZTy3u37N21VhSdnw5Lk1kV7v5fZ4MqVKiNXVt5wnuwj2ceywsJMBpAM\nRgaQzN2tJQJ4csNmCFCJqMrt1lGVhatSPYV2M1uPLKlD8i5Xe3cEzGQAyWBkAMlgZADJoPg/Csc1\nvjKLJffcmbAuq+E/GHZ6nIyokL3DTAaQDEYGkAz6YQiVWBr46/2+lLK4pMbrBa42/eoWAufMPpYw\nkwEkg5EBJIOuGCKSf6D3N5WSmq1KwkQWNmREnnnr6GTTXi2Hr/nI8cDBwEwGkAxGBpAMcnGA5gyZ\nSjTP01hKeyBMJaJTCnWr4IQUqVg995z8QZ7/zjvtPb//fW1/9rOb5vqJL5UhDuFs1d0IMxlAMhgZ\nQDIYGUAyd/6aLJj7wovAaHN8CLZmtK6Xfvvb2n7wwdrWqBB7/y9+MTzORx6p7d/8pv3br35V288+\nW8cp99w8rrlHTqJBKeQL2SnMZADJYGQAydyZ8/+ISAh1b2u6NlV1J9qvJuIopXXVq3T81rdq++9/\nr+2HHmrvFxf+ez/+8aZ9z09+Uq+xEvOnP63tX/968PlHD91fQnjvzD5TQT6GYCYDSAYjA0hm1vne\n8+B1IxG8wFsbYKtMlDG9wI4Nb7/d/vzSS7X9yU/W9vvvD98v8rCUUsr3vrdp3qORIdrv1avtPeqt\nPD2t7Vde2TRXWkJ3adyLl5XVO5aO0TN1EQ45moWZDCAZjAwgmVnl4o0bta1KRzM9lWKm/oh260jH\nURLVoyeJVNZ9/eu1fd99ta11y159tb3/qadq+/nnh/t9+eX2nn/9a3gsKjeffLK2NU1xKc3nGVXa\nVQlKz/UylrHrToKZDCAZjAwgmVnloob0fe5zta3heaV0Yuy8jFA79myp+mxUkP5gJapKue9/v7b/\n+Mfa/vKXa1tfgL3fO4+mcrOUUs7Oalu1+PXrw31Z5PMsvFpnPY+iXqde1DfeqG0dfyll8cwzm/at\n43s37UipNXudMlnuJsJMBpAMRgaQDEYGkMysa7If/ajq9ieeqFERP/xhe93TT4u+nlh9wav/bPGW\nIc0aQNcKNt+Gro/+/Ofa1rWSBhV/+9vt/brGk8/836e+sWnf/9pf2nt0vaWD1udo7hEb1Ozh9WXX\nd/rSNAJG77fvSf6mRRHLX/9Wfy9bDetrnwkN+ZBhJgNIBiMDSGbmA0FVKnzxi/W3L7xgrhK18dWv\nSs3lpeOaDfp8e9LRcw03Hmx1U1vXthaN1rNib71V2xrEq27uUkp58806tqe+smnf/+o/6zUqSW1/\nOh6ViNLv1v06Zn0Bnty0ESYqcfX5Gvz8+OPtPfq59Tp9vkTDLDRippTmM+u5P719VDGPKCPOKjKT\nASSDkQEkM6tcPDv71KatTifrQPzDH2pb1cKXnnA63sExeFUBKkubgg+vifT73e+a+2/86U+b9qlK\nJJVEmrnq5z9vByAyavHd79bfq3STLMGllFaiad967qx37s4rjKFt/aJUXtrn6Jdor1M0AkSlo557\n0y9d5W4pzedZybtdn9U0C9EKvjsNHu/ATAaQDEYGkMysclH3XyUX55a6UBWhauHWY+Jp1I3MHk7k\n6cJszK68QFi9TqXTL3/ZXHb6zW/WHx57bPgelWHWa6YaWZ+p3kkrFxUdv3oRe5v5XsCzjk2/nF5f\n6oXU51tvnCdfdfx67s1er8HPEny9kODr5ZVPb9p2/127G7XK6AVMOzCTASSDkQEks1iv5zt787Of\n/WPzMFUeVgXpHqs65z59ZeLmoUoNPaJvHxTxlNnc9V/7Wm2rJlG90pNBet3SqUlmtY+OU/+mn/nh\nh2vbZsjSrFoqg7yN7Z7X0MskZjWZt9Gt1+m4ehJVv08Zm8Y7jjmb1vUuyvi/8OSToV1vZjKAZDAy\ngGQwMoBkZnXhq9te9bBdKjSauFlrBP2vXmEIXQPYNZG6nUXf/+/iZPCRJ5r2rZR2vaUucF1TyTO0\nblgppZyc/3d4zM45s62/eWsa72yYvd/7va7D7JpMx6PbE70oE2+NrNEjGohtg5q1b/2bfDb9/1mN\niPjYYkRQsMJMBpAMRgaQzKxy0Ts+tOUy9aZnlTuhqhDFD3a1kkZ/lva9x8N1y5qoDtu3BsHqmEUG\nnRyZY/m2gMWHqESzBSc8PNe+lXv6JXhnw7SvngzUvnuFQbS/yFaJXRZ4SwG9v1dHrkldPSKbMefJ\nAA4PjAwgmVnlYhPU25t1dYpX6fW3mtGoGwmgmXa9DE22+IInl0SuNOO3mZvUU6bFJHT8mkHYSj/1\nSKpE0mxXGgrTQyVN7z15HlovnW80ojbq+dUojaB0a/yBkZwRtj6c/G3hpV/o9UeAMMDhgZEBJDNv\ntir1wHnZkUppN41ffLG2dfNR63mZogYNKrF0k9ae51JZ4dXt0jFbGab9qedR254kLcU/tyWZr/7z\nVrt56h4bE+nVU3u7PGYfPcq/dsZ229l/38pQFilp7Hkw9wQzGUAyGBlAMhgZQDLzrsm8CINeJMFz\nzw3fr+EjmrG3FF+T956j6zXnupvndd1xfHzS/vGB+nOzDvEqWdgtAOeg4q1S1zB212FxUbcU1mXY\nBR5NiTZ1fRa9v72uPt+LdbZrvakFSNy1o7zLrfVytCiiAzMZQDIYGUAyMxecELxzQRaVVZ5UsAfS\nFL1nRPSCypujoyo1bJyouppbGSboTXYLQdEaXsedrQ55NwtHbqrLfMudHjlPtYPszB7+O5smXUcV\nnIh+ThtBEoCZDCAZjAwgmXnloga7qryxeF5E74i7lZHWc/chPc+QPtO5binSSz/KBwxLFL1n4USS\nbPUk1+kZti3/YSAb76Ijg+aI+Mh8Zhr2nen/Q+//1oGZDCAZjAwgmXnlogby9tJVeYG0GjisZ8as\nvPM8d9GNTOeI+Y3zKtisIvWOHLUfxZdU7en7YS/m8rTdAJ+6MTuVtTPOHsvlsBcxLCMDHuJe2eJL\nP8MywtvKTAaQDEYGkMy8clFlYa94gRLZQO7FJEav8zI0OekLtIaavd1LpOU9opQ2G4GXMWH7nsXg\n3/R+/f3q9s22g8nFuoZvj0pHdwO610EkdlAG05Wh3v+WfRdjPpzATAaQDEYGkAxGBpDMvGsybx1m\n/eHb4RTb90dyPfQwutvLPbFa1uecdlJHaBY4XXrqMHUHQ3cjSvFjpL3seKWU8p3v1LaeNRvl2R+T\n+s25fcx1rWtf6K3BJv4P9IKnXXDhAxweGBlAMvPKRc2y23OTe1NytKhAQFZqKgHbXfP4oxploWnL\nei58HY6Xxa5Xh8HLcGcDY7RGhT5H282xestEuaUSS+XeZMZsJwTv8SJAwufZcOEDHB4YGUAy88rF\n6A5776zYEPb+yLkz47TyahR4ERvvdMqLaXzytWvDfVm8Ulv6nMcfb+/xXk0jd6Z6YYOMOScWkm69\n+mQTo1S8rrYyZF10Co0EYCYDSAYjA0hmfwHCTnalLTxJED/ANPjrXjVb7zEq3R5+uL1OZeHnP1/b\nTjXb7nP0dWjgsH1N3pgbuXPsJF3tDcBhS0btUCKG+51aO83pqvv8ibKUmQwgGYwMIJl55WJTeb7j\nXXToep0UL8++yIulKZ/qnQfT/XOVbvp7y+uv17YOU+Xmyy+392ilW1XV6qm0oZ9euTT3bFbPUxdg\nS0bt0NOndDN5jUlIGqB7ng25CHDYYGQAyWBkAMnsb002Ec2su1ULuYnsGI54sIGzx8d1jaZud10T\n6e/tkTcNBHjkkeEx6/JQ13d2mEpv6equw7wOekQGMLbvCRxExuERQcEKMxlAMhgZQDLzysVgCdup\ntaraY+UxVD6enQ0XllAXukZ4lOKnYVOCJcncYGHb72QpNWIbZQ4mf67Il2GJSsIRQdbMZADJYGQA\nyexPIwTrZnnSMXzcfUSAsUrHe0+H77dRCVqa1ZMrq86ZJY0G8YZssywcH++uBOxceOOcXBhCX466\nem0aMAmYbgNWRmSuCsJMBpAMRgaQzEG4lHrnlMZM3ZH6VN0AY9URbjZOU1xWr3OCknv3RxJx2f1i\nve709PLS8bISzfY71QvsEQ4Q1heiElHb5nvWQAWtF9fGLLTPb7qw33sAZjKAZDAygGQwMoBkDiJA\nuKfnPd0fXQOEi8tdku7zvTRsIu7tUCJDsy58Ly+IV5e5t9aJnL/s3e+ufUd8T+G1YiR1oDnp6vXd\nW4ZH6g72YCYDSAYjA0hmXrm4jyDUianj3L56/TpBpD0Z5D3eOxpXShu8rFIymuosIp20r+j9s+El\nOVE6AcLR2OGpiZeZyQCSwcgAkvn4RnxMTXU2Vbpa7SbaI+IRXZnH25+HuO++9j2p4+zk2PGidlxj\nC9FBTRxDMEvvHEHJ4Wccd2oNB/rTYOvueHr13hyYyQCSwcgAkjkIubjr8qG7LEqQRm/H03FnrYz0\nWzXXyWfzchn03GRBidkw8X1GAozDRS6i37Pz/xQOGB+xM81MBpAMRgaQzGGUs+2h07MXrNdjlxJx\n6UiyDq4kGjMuK/d0DNGcBYreo/XiNBdCL11WoF8r99zjdU46iW6RizFE0lHYa0Z87wozGUAyGBlA\nMhgZQDLzrsm89UE0YsMrIrgP7Pqo0fSXzwPRMGYLwnu3uqay92suDF1r9FIdK3qPs16zw/KyI7uu\neVvZQxlRwGRUzheFDMIAhwdGBpDMvJrLi0SwblGvMMVMEnHqUfpIv1tns7wMth13+KLnnv+Q09Pa\ntjLce89e9Ek0SkXG3FNX7Rm4oIx7993B5zTSccRWUa/eXXc8AZjJAJLByACS2V99st65+oDXai7C\nEvGSMiKa+UnPL21JFf05UpOrVw+3d12ESPRJ8QMmVAW2R8NOmusWpn7EhmggtODF/VoVrh/HZheO\nwEwGkAxGBpDMYRywGrHBN7nk7dQiF1b3ZHk+Mz2q+t7ffru2vR1jW+ur1VGD959ftHJPY4+9oWj7\nypX2utPT2t/i/OZwZ72UCzLmaI07z/EahZkMIBmMDCCZ/W1GN6PY3TH2UvZU2nVi7KKOWTdGtYaW\nVdUrbwN2zFk7lYXRM2SezJex3Hvafhe3rw7XBNPjbJ4K7T7f2xgfIR17/45j/lWZyQCSwcgAksHI\nAJLZnwt/4jpsTF3kqbWoo4yptaXrsFH0AoEj6DosmlLOWzB11nGmXNggq2WnptwNGc/77390x2Py\nonQY83/DTAaQDEYGkMz+5OJMmX1nc+dP/AxeHbBuIYqILJxai9XbJiiln27Oef7qsgHfvfGr3z+K\nI3Ez/0+YyQCSwcgAkjmMiI99YKXPZeVeMKg5LENkPE0qAq2HZd+fJ7m9c3tRVAb2zqk5QcHuNaX0\nUgjH7p+YZkCZqxwvMxlAMhgZQDLzykVNVNlLujmRqWfNQkwdc8dTF8pCZRlxJi/UV08Sjkk228m+\nNXx9J9h6ahLZmWAmA0gGIwNIBiMDSGbeNdkrrwz//tq1y/fVWQOE6gr30tDtMmux90y7Jrt+vbY9\nt3svujZQczpMdKslUixwrlwoBwwzGUAyGBlAMvPO3Zom9s03a1vPQpXSSo9IvoZofbOoa7xXZOGy\nnDvnn2zdLa0VpvQkYiTiwXPH9+7RfnXMPXT8wSiTywZ/WHaRCvCyjPl3YCYDSAYjA0hmsV7vIX0a\nwF0EMxlAMhgZQDIYGUAyGBlAMhgZQDIYGUAyGBlAMhgZQDIYGUAyGBlAMhgZQDIYGUAyGBlAMhgZ\nQDIYGUAyGBlAMhgZQDIYGUAyGBlAMhgZQDIYGUAyGBlAMhgZQDL/BwR49orKWdTxAAAAAElFTkSu\nQmCC\n",
"text/plain": [
"<Figure size 250x250 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAQQAAAD6CAYAAABH5znXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi40LCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcv7US4rQAAIABJREFUeJztfX+IZNd15ulUTZe3ertx7fYwHXqY\nDr2MrCETNIsWKZ5ljLUI7FhgY4H+MBZrEy2CeOMQk0BMzDrEYIiJQ0y8yGDWIewaYmJQiMCgwBBp\nI5AjwYCMBkbRQLM9qKFnu5fKdqcLql21vX+c/up+79S979336lXV65n3QdPdVe/dd9+995x7ft+F\nk5MTqVGjRg0RkV+Ydwdq1KhRHdQMoUaNGiPUDKFGjRoj1AyhRo0aI9QMoUaNGiPUDKFGjRoj1Ayh\nRo0aI9QMoUaNGiPUDKFGjRojNOfdARGRf7x9+0SGQ/dBoxF3Y3NO3R8M8t+D92s09O/BQH/3+/oe\n/b5It6u/Ox2RVkuvaTb178VF/ds+m8cN7YrotTyOjcb4eOF7bmNaiJ3TNMxyvovMcV5kjXtoru36\n4Xnv9UQODnQt7e+LtNvykT/8w4XYLlWCIYjI+IKZF7HHIE/fMFFgBMOhm7ReT79fWdHv+339fDgU\nWV7W53S7Iu+9J7K5KXLpkjIHjFWzmezLYCDSbjtGwkgjyDKINYQqz2MaptVvJvKscbcMG30aDHSD\nGA7dxgFgA2m1RM6fF9nZydW9as3WWV08abC7eq8n8sEH+neno1y83xdZW9MJZM5/757I3p4yg+1t\n/Xn8cZHV1XEJwD5TJPl9WWMbs3M+iPNYFkJjkzauafM8GCS/Hw6VWcS062sy19XTxIO8iKAS9Hoi\nu7siS0tuF79yRa/BpOJ3u60/ly6J3LqlE93piLz8ssizzypT8C0USCI+FaGsd7E7Uo3JkWcc+Vq7\nBlotp4YOBipx5kBtVJwFMDF7e0rki4si6+uOqEG8VgVotVSdePxxkUcfFdnaUinijTdUlQB86sE0\nCdXX1xrzAc8FS4ZYA+12vuam0MUaFpAOMGFra/o7RFD283ZbZGND/97ZEblzR+T27aTxUSRpW6gx\nU5zIgixIBUoJWPtSzrVQSwizwPGxEnWnoz8i8ROFSW00VDoYDkWuXVPVA1JCzQDmghNZGP1UCpAQ\n8HcO1CtpVgBRt9v5CZhFwLU1ZQR376rUcP58sr3YtkPGppq5RKNMiaB0CaOgtFhLCLMA7AZl6PaX\nLunvy5dVFRFJxh7E4CFlBpXd0cuENTiyxyHm9pK7UwwP+EJMiHBFwOOzsqLGyAsXnPcip+HogR9v\nD6rOBEqVDppNF7vQauW7tbxeVBtzNfrkJMDUvjYaqjbknOgaSYBBVMIQOE3kXHsPBUOo+u7ACPYV\nAUsiTuKYRchxjbOJ2qiYjTO7K0DnRzBTr6fMYG9PvRZ5VYaHDMxkQ3NvGfGZWyMSkCxrhjCOBTlJ\nTPgsJj9mEfqQuBaMYHdXf1PCily5ol6GnR1lDohsbLWCYmJlfOVTQojhx7zzvMeF58YnJWb1b+we\nrIGcUmTtZZgBYlQW7zVIhtraEvnRj5QZ3LrlkqEaDZfwdOeO5j7A8+BpF4zxLKlQeQAfwlkAezww\nH2lzkzVnqTanHHgoJASRcSmBP58FQlIJdobUfty7J/LEE0r4zz6rHgag2dTMyGvXlHkY6cC3W2Lh\nzfLdzwqhVhWzGr+HgiGEFmQVFmlqH8Dd19edatDpqMTA4c+opWA8EFmEOCtCrcI4n3XEzJX3+9rL\nEMY8FqaNASoUAnD5sjIC5LZfvOhqHyACcjBQRkAGxpoQq4kqq2wPBUOoAmH4ih1F37S0pMS+uqqh\nypzQBA6DHAmP2sDGNrsYz6znpcZU8FAwhHmiSJqBiCjBI69dROTNN5UZwKDIDTYa6oo8OnIp06Q2\nTIRSRJyHE1WWBEKoZ7eqQKm1w0MlyscfV4aAGARbDYkzKft9kcFATprnEk2WskB95dkK4kGXTtLc\niEXamQVqt2OVsbKiHgVWA2AvSENEEpVdZMFF62unpAKkDyojKBOzHqNaQpgSiuwOiR0TuQqtlnoO\nYCxEkdU0gm82c+9KIQaxICcFDSDFnvsgIvYdqyAx1QxhzrCEq3S3IM0mGQE7/0r005OgyA63FNrj\n1AcRPw8JxWaMnsXgir81poIqMMdaZZgy8kbPsdnAh5PmOW+EG/8WcfR7cKBhC71e8f4lmEZdS/GB\nRs0Q5gifdNBoqG2w2fTnQ0Tr/qLaBcouHhyMRTVn3o/vq7Bz1ZgNalY/I4QiJe2ufnjocpV4I2b9\nMk3UB+C1PDxUJoNQBtuXQtFvNR5Y1BJCxdBuKyGfeg7HEGMstPetrGi6wyxwFn3vNRxqCWGWMAZB\nH/HgBK4QQjs2ezXwCJzqhb8L9TfnzbVEcbZRM4RZg5hCHtdkTI6/r53C9r/am/BQolYZpg0mrECx\nimRmvP8nBlnXFq4X4GMONcN4IFEzhAcMadJGafo9l3TLuqbGmULNEGYJGy0UwCRVjSqjw9exCmcS\n9axVDHn9/nOJE6iJ/YFFLSFMGywRREgHeYk77/WhegjBIp11ZOJDhZohzAKsT09Jt449pixXReKa\nETx0qBnCLDHFg1UmLTVeGdtDjbmiZghnAN6dfzAIShtnqRx5jWqhlglniQIHvuYpXBJSB+pw4hqx\nqBnCtDHJqc+nKGu3r0IBjhrVRs0QKgSfC3GMeLPyC/A96i42m2PZkTVjqBFCzRBmAa6HGEC0WM+M\nwNoQYLT0VErORfx8f+1peKhQz/ZZBUsCoc8ySq0FpQ+u6lxCleW6yMrZQc0Q5gEPkQWJ0yLEAFg6\nQGkkPsnJZFh6+zQc6u/jY/f54qI7HyKimjNQGzLPJmqGMG0UOaklK0OScyJAxCJKuFtbStDLy+78\nR5z0RM8fO3Z+ONT70R4/d3HR283Qzl9LBGcXNUPIixIPKhlrl4lbJBzIBOJF5VRIBNvbIq+8Ir3X\nXpP2xYsiv/3bIlevJs57HOs7SjP1evr38bH7DNVVRNxpUO12wljpQ80Mzi5qhlAV2J3ZMgMQabOp\nxLu35xgCjnJ7/XU5fO01We50RD7xCT0cFmc62JwEnLUAFQEM4e5dkdu3XbXXtTX9wb3Ly97zI2s8\nGKhnNS/KIgQQt60twLaAbleJ9P59Pbex2VSCbLVE7txRQm639XToZlPk05+W5Ucf1d387l2RH/9Y\nxf3VVSVwiP7s7ej19OfgQOS990ReflnknXdkX0SGItIQkdVr10S+8AVlDI1G3GExNc4k6hmdJ0Ke\ngl5Pd/e33hJ55BG1C6ytaQllnOH4+uvyX996S37z2jWRP/ojJyW02yL7+8oELlxQZnJw4A6BFUky\nBBgkj49FvvtdOdrbk6Vnn5XVTkfb+Na35GfvvCOPbW2JfPnL+pmIPmda6lONuaHSs1mpAJoci993\nnoKIKCH67AJ8XuNwqLv7q6+KrK+LfOc7Ip//vNoCul13JHy/L18QcYcvNBr6+cGBXnNwoER+cKD3\nQf9vtZLqA1SRblf+aW9PPry2JnLtmrb5xBMiTz8tj/V6It/7nqoph4eunyVEYdaoFirNECrBCKaB\nNGKCqnB4qAT+O7+jhAxiPjjQe3//92V5f9+d+Ix7Wy1ncGy39XoQMdyJPpdmoyEfbrWU8Vy65JgM\nJIvLl1VS2d11XguRWkLw4Cx7WSqV7egrLxq6buZIWfhpffb2FczAtgnR/ehIiXlzU6WEpSUl8NVV\nkY0NvfZnP5PjL31J5O23ndRxfOyIGAR9fKyE/N57ymjAbNggeXqc/GG/r+oGF0eBd+Hxx1Vt6Xan\nV98BRs60nwoiZs3Oqh+TohIMIe9AVo37ZqUbj70bW+mt9R82hE5HDYgrK7pjX7rkJAURkZ0dWXzx\nRWUeu7tKqGAG7bYyj9/7PZGPflReeeklke9/X6UFeBPsuW6tlvxLEZUE2m1n0AQ2NvSzgwNndyib\nGZxB+NbtPNZnWYyoEgwhDbFSQ+nIsTNF9yvjCPeR5HB46FyFrPNDEmg0VHK4eFEZxm/9lsgbb7jA\nIpwFd+eOyL178ssiMjg6Erl1S+TePWUgcFviuY2GLDz9tLa5u6v37+6KvP++Xtdo6P337zv3aJkF\nX2JKtc1QPckujO/mvEjJ/LJR1rMrzxDmAk9y0NTaxmccg3BKoF5mBKbRaKgXYXNT5LHHlHiBVkvV\njLU1kVZL1kWk+fzzIq+8IvI3f6OEzUdMNxr6/5Ur2tbly3o/iBSMBpGPjLIlBR/quo4zw8PHEGJ1\nUSzCjIWY+6zFNALCrruy4o6Axg5uMxDPn3e2gk9+UuQzn1HCRYwA7n/sMfnQSy/p55cuyX+7dUv+\n73e/q3YFVgn6fRfE1GqputJuazsYM0Qp4qV8eRWTgse9ZgQzx8PFEKwxLLSIS16ECck6JGrzZ52O\nMyRC32fPRKOhksHGhnMPwjgIQl1cVAJeW9PIw81NkU5H/tPSkvwPEZFvfzv5/FZLbQT7++P94/Tt\nabsaZ2xLmFQNrZo9a1I8POw3tNAmCK7Js5BGrqjBILnLijgmAeK3yUTdbiKHQBoNFyDETGIwUIJG\neHGno7aC555zBse1Nfn3P/zheAcXFzUXotVS1QEZjpA48D/6Wnb4ss9zkdZ+wXkLxYgUZQpn2cXo\nQyUYQt65DQb+pD1gjrB5QqMPRcbTlodDvXh/3zGIfl91fHgQWNSH5NDpOJclApGWl517EdGLTz4p\n/3Z3V4neotXSayBlLC05ZoB2bKo1MClz4DBuO0YhTBgpWZaB+kFiCpVQGaBWF6HbiSY1p45airfD\npy6AEfT7SszNpnoRYOx7+22Rb3xD9f7DQycVdLu6q7/3nn6Oe/kZEPM7HRfbcP26i2fAOMCbwXEI\n7LWA5yKEaTBdX0IWf56TGdiN5EEh4jJRCQmBkYfp21qBwQarBKuDc9pzvy/y1a+K/Pqvi9y4oTv+\npUsqzl+/7iQEEPzqqtoIsHu/8YZTDa5cccwBUYsiSekC6gfEf3zX7+vnkEYODpShIFLy6CiZlFWW\nXSE08UVqSngwrSrUDxJjqYSEEHkG6hgmlg7mBU5x5sIkW1uaqHTjhnoROh39/JFHlBhhW2i13I6O\nzxsNkY9+VD0Ur7yibTUaKjWAYYCh3L2rhI3nAgh8Wl3VdhoN/b2xoc+6csUlUdn3Ye9JVZhwithZ\nZkzLvCMUy0QlJITjY7eZwTZWGr36dNOcmGSyx5gdjAm24hFqEbRaGlOwvKzhwiDAblfkBz8QeeEF\nzTeAixFtIvdhbU0ZCnb7vT3nxmy3nbtxZ8dJAuLpn01vRjDU1pb2hcus4Xu88Lzhs0V4StbZeQ19\nBmStg0raEnLORyUYAtbl1LxaIYNVyVJCSIVJvBN27aWlpDcBu+xf/qX8Xbcr/1pEHvvGN9Sl2G5r\nKvT163ptr+c4DZKZ2m05aX1IFo7+WdtGAhLciAcHyhguXhR56imX9ARpA33zBUJh/DhxajAY739V\nsh9tsRnulylLn3W4jY1G5CK1FiGVpHJMIgWVUBms5JoH0YMdqYcWlQZC/YDta0RnIGImJvRnc1Ok\n2ZT/8MIL8thTTzniGw6VsLtdkddeUwmCxfbTBywMfp58GNSRwUDvGQxUHbh8WZnDnTvh06l9sQhQ\nTWw9BZFkwtY81TGO+mR1LOcCyyoPF6ol6Vs/Z0mlqISEsL+vv1G/g43JWcglpmUwhUlEPr7X7iAJ\n6RuiPj5AhqKIMoTnn9ff29tqUMR3a2v62d27LigJ3gNc0++rdCDimMKNG8o8Vlac5NBqKVN4+WVt\n65QRecHxBq2W+591vKqAGUG3K/LOO/r5xoa+L0mF3nk2blR7jV0fUUbtlPtDn80TlZIQjo6UPkqV\n5iOTlMqYGCtmemFfjAmq39f8hHffdeHJuB5W/vV1JW5kHXI7YAZ4TqslJ498RI2Bq6suuAnGxaee\nUhsDwLUToB74wDkQoe9mDeutuX1b5OZN+duvflVtL91ueti4r3qVTD/VvkrMQKQiEsILL/x3EdmW\njY3/Il/+strMer2kly2thF+wslKOJKVKTMxgoAT5zDPKAED4zabI00+LfOpTet1wKPKjH+n3qHNI\n72b13tE9TMjttnol7t939z/1lO6mIArYFjhdem/PBS/5GMKs1AWeW3ueBIywzz0nn/j615U5HB7q\neIXCxns93ZHAEE/TzxdImvOtkTR7Qkh98F1bFVRCQvjKV/6jiPw72d7+Dfnd370nb7/tVGcOkMva\nfLzcvKQkmZj01tzBLtYFgeAf1nlhA8D3+H1w4OwQHgv6KExaxBEuEwyiH9mWsbqqRAEGgbRnrpzE\nbszQmRFlFTSxbXHkJpeiR/n4oyMnOa2uOrctCsn6KlpjYcFzcvu21o54+WX1xEQsvJA9IRZpqdWz\nRiUkhNVVkT/901+TN9/8NVlZEfn61/+fvPjiL8hTT+kGCAmBVe+oqNmSdqsYIo9mBOwutGi11E6w\nvq5uRxFdqJ2OE5kaDV20KJEuMr5g7YJvNJRYWi3nIdjedvUQYBfAMxCwxOKZiAtigrQQYz8oGt7s\nC+22sIyz13Ml3qBa9Xo6TkdHLk/EV6kKXpSPflR//vqvNU0cklpG3/PaE6qKSkgIKBC0tKRM+WMf\n+wXZ3xf54Q9F3nxT5xgBcxwkB/D/VZgUyxwSG2nIXw8C3N93OxMIFucmYDfc39dFjuAhBsRl9kIg\nLwHEff++3gvxGH3hrErOcGQrLwKavC83IaxkwRLB8fH4D747OlJVBqdW/cmfaLg3+tnpOC8NA4wA\n77i/r2MzHKoxFlGgZUg780LO+amEhPDNb/6TiPwfEdmQL36xKY89pnO7saFModPR+bl6Vd3oLFn7\nNqlZWG7TwmBTmRImyGY0YldeWdGFCc8ASqlB3MdZi1evjhtWDg6UmayuOhchnglRmgu2rq+77wcD\nd7CL7S8/Y2nJGS+tCzIUBBYjGVibAH5bkZ3dnPju8NBld7bbaph94w397FOfcgZSfj+7cLjWAyeM\n8TMNylhnaZLFPDwQlWAIn/zkh+XVVxsi8hXZ2vquPPecrvf9fbWvffObh/Laa9vy9NNX5YUXRoWA\nUiXWaQ5mVrs+t+MIPms2+8sXF9WACIbBdgaIvSacc/SuBwda8mxry+UhgKmsrTk9HN9BFcHzd3ac\n7zeExcWk2G0ZRl7i57HwpYOD4LkoLMBl7SEttdsizz6rn1+6lPSobG05G4kNXuJgGDCCo6Px7E56\nvzLWV9WiHyvBEK5eFXn11X8Wkf8sb76pBw599rM6Lx//uMjm5rJ89atX5ebNQ3nyyeVRER+R6TIF\nmx2X1l7ad80mLR4mJJGkmNzraR4CdrGNDd3tm00Vd+/ccYVMhsPkYgLxb26qjWFvT68DoVy/roSB\nwit7e3r+AtSQvT3t02lSlL7PKeAP3t6W4a1b0tjcdM/lCYjxF/t2e8skQZyw/L/3nsgHH2jbly+r\nOxbSEVK02Z167ZpeD1etiI4jDKlLS37GjBL3KC5zdJTsa4bNZFp2hKAXbQqoBEO4dk3kBz/4Rbl1\n6xflpZf+QX78418dRfg+9ZRKzM88I9LvL8vm5nhko2+eyhg8O8F52vS6QDmt2OdHbbdFPv1pkc1N\n+fsvflF+UUQuf/vbSqS7u2rk2t3VAWs0ZGHwczlpntMIRbSJkmogYlRQXlnRPrz7rl5z65a2A0Z0\neKjffeELGvXI1lucJDUcSuPZZ8ddncydQ0yBd2MfU8DfsPgjuOjgQJkVqkOtrelvGJV6PVV90CYS\nvjY2nKoAwymyNZG9yVICjx+7e9FfjoBMcbdOK6NyVqgEQ+h2VdIdDES+9rVflc1N3Si3t3Vu/uIv\n/kFE/reI/LJcvvxvZG0taazn+amktdfG1YcWFDIKh0P52A9+IPIHf6DEsLmp+tPTT2voMizqzaYM\nBiLnRNxih6fi4EDklVfk3Z/8RH7luedEvvQlZ5e4fdulUzebrkDrN7/pvBvWOwAd7c03tR++d+Ds\nNNzHO3FaWitqQcAgirZWVrSvjYaTnHC2xPvvq63g059W5sY5G2wwBJPiehNgkBzMxMfdLS250vb8\nbvZvnuMpx2DMQlKoBEM4ONCI3KtXdU52dvTw4vV1XX/PPPOr8sYbui6efFLvAVPnEINms3ydK09b\nwWdbazbH14d2qL09dX9BP1peVkLHbn+6+zVbHxIZiKtstLSk1y0tyT/95CfyP0Wk9eMfyyPPP+8K\nr4q436urqluvryshorISH/qCw1nW1kQ+9zmRRx9NvpuIM3qC4XGhFg4p5v/xN+5H3Qbs7Ai8AtD2\nyopjYk8+qX/v7KgxEZGXYHYY67t3nbS0tOTsA/h+ackdist2ieHQ2TCgkviYgYhbkI2Gk7CkPGkh\n97ou4BmpBEPY2tJ5PDjQ8b5+3Ult167pponEvFu3dN6bTXfuKTwO88ipiZ4klhA4LJgDbLDLYDE/\n/rirRYCflRUdkNOdLBGJiEV78aKIiHx4bU3Wdnfll0ScbxeVlLlgynPPOUJsNJQpDAZ6HcTr8+dF\nbt5MHh2H+9kTwOHT/N5QAUSSTJDFvJUVZXzwuPjUEvbGgFFdvere5/x51x4YAgj4/HnX17t33QJC\nrAfagKSB7/A/xhCMFZIFmCEqVZ8+DxtEJaXWACrBEK5cGSdqeMngUfjjP9bP9/dVUvzOd/S+559X\nxs6SaNUSRhKiJu9KIk5E3tnR/3FY6+XLaiWHfn/hgjKIp592u+DBgSMAEbfDrq7qrvnss/LoSy/J\n4pNPOt378DAZitxs6rMPDnRAOV4csQ8i+tnjj6u6we8yHLoYgI0NVW+sfQTZltweciv29/U9OSmL\nMyeRxs07t4gjdJSjp9054YnhOQCTe+stV/gFlamt92JlxXlkLKNAhCdUnONjbfPaNWVOp8bMhbR4\n+5woLB3klBIqwRDW13X9N5vuWEMwdwSfwWu2uKjrstdzdgcfMpnCpBF0ee+18QcHB/p7a0vktddk\ncPOmNJ95ZmRDGBm9trf1HuzeWMBvvKGDhh0bbkOc1dBqiXz843L1s591xkUr3uN9UIpta0sJ7MIF\nF90o4iYAzOL99/UEqCtXtH8iIn/7tzoxeDZ2eBARCr9iouH2xPWQFGA/YLcrxpldj/2+9tWmZLOq\nwlLI2pr77uJFbRe7uS0Uw4wHi1HEvT8Cx86fT564ffeuMsxLl1zgWJ41UgFUoqd37oi8/rpT4TY3\nncog4ooIYRMQ0boh6+uueHCjke0aTMDuYPYzH0IxBOiArw1mIPgbhHaqs+/fvCk/EhH5yU/kN2FA\ng4pABVDGxGArnl++7Bbt+rqzrJ8/n9wxIOoiEhISA2IWOLR6OHS69Cmh/69vfUt+aWXFeSnW11XP\n+/jHnUtzaSl5/Dza5ngBEUdwGD/ESNg+YCwsfOMNiQLjBUMjpITFRSfJsFgKqQgMmJ/HDGN11Y3d\n0ZHrG4ytkKKo1N2kasOspN5KMIR+X5nC5qZTcUV0E1pe1vWMPBtc/7nPuTkVcXNaaNACjGDMqstW\n81gmAoCIuexZtyuyuCirn/ykXH71VflnEbcLon1O4OCFu7bm9G0RtxuxQQ8Lkhc2jDGHh44xdbu6\n68F9xxZ+cGWg05Ff+qu/0ja2tjTrcmvLiW2tlquvwAwMDMCGQuMzhq3uDEJmtYEDl2wbPCdWbEfW\nIxgR2yg4nsHOAZ6J9lmEPR2XEaO7ckXjINLyViqKSjCEa9dE/uzP3NxhkxsMlBn4Kir51lcIeVJO\ng1ycLckwNAHosG/yfZZ1tNNo6OK5fl0+8Ru/oYSF8GIRR1RM+Fi06+uj540Yl6+yMhMM2kYYqIgu\n6Lt3lSHAUou+sbUdbYi4MyDW1pzO/LGPJY2FHPzDhkK0CUaXpeNaoyKAw2gwF5AAYHNgNyBUGBgU\nV1e13JxvDWAMUe6e+2n7AJXHMu/lZX1/MKwAU8izp8zKJlYJhgADL4cjsxSZFqZcRDLIzQwAuKBu\n31YdZmNDf+/uqvtrczNJvL6OijhigSiLrDzkFvikAkaoeAkWro2G5MGDjQGW8m5XifraNe3P8bGT\nCpC3gM+xqw6HLl7h2jU9IHZx0S1+iP3MJNmgagnExyitgRDvx3kGsLHgWaCwdjtZaZpFy9PzK4Pr\nhfVSjB17ExCotbqaLE/vY1jsgUmh+hmEMESjEt2ABMfrhO0BzeZkRSWKctfEfSDAnR0lIvhG19ZU\nZ8ZBqdAtfcky9odLqTPRYWdhrsg6MdKYRU6J9Fyy/gE/k8ViEUe4EHmh21uxHm0PBvp+1vgHlQJq\nho1D4HGz3N2WkQOsuO7bBaw3gJmoDRiy7TOjtJKHDWqxKgqYEXRaeHSYmTU15Fua51RaC1ScwvDn\nYQIPlQ2B1xAGK1SObJqDEmzbBtJgQWGnWltLnpnAFnYsct6pWPQ+PHSRhazzWxWBrefGGDZiBnan\ntUE0TBBY3GACICjUTVhcdIYbywzQPmC/E3GSBcR6jg/AWLA6A8MmGIYlROtGtOBYCDArBnZ5nseQ\nOsJzZ3VS/I9IRx5XMWsIKeenzAabXCwjmEf8QiUYQpaEzJhlokcCWCRIR+aFzVF4IDS7BeBa+4Ko\nX7C5qbvdvXvO8MXFSgCoBWzBx+cMdnmdLv4TWdAIOvTZitVMSKg3gOdYgrd6HNsDWDVoNBxzsTsq\nrkV7YI7ttktg4l2fYydExiMguY8geF/YOO45Ph4vmMLPhNRjpICx9n3AmHW7LioyK5PUYB7BTJVg\nCD5kJYnMLfgIbjxefIgyhPjtWyQ+vRkLDXkFIi46EPo4iAEEhbahZwmNBdQAkXFD1+A0YQkL+fBw\nPFnn6CgZ3uyrLsRqEMO6V62bEEY6ZiLMQPD5xoaLCvRJWT7iZWnFR6SQQERcLgNHwoXUDDBeO558\nnW8H4+djPE8LuSy0WnLSPDf6OmYNh9b6NGigsgwhCzNnBnbiWbyFOyQUi5DW1vq60z1FnO4Zcqtg\nwaMcGHkaFuRkXO9looNbEMwG1ZcQvwBDIcPaMtBWGtgOAFEezI2NjD6pQ2Q8PBrw2QSsV4bfmwkT\nKtHSkroEESdhx5WvR3UoZkT6zyKjAAAgAElEQVRpFkBWSzigCe2djmGwGHCg3awzIspEJRiCz3ZU\nmdhvG3fA4uvRkS6s0ALOapNEyIS+aC3rvn6wuI8dh79jowzrwyIqCqNeAs53SPPjA6xmsMrkA4do\nh3TxSQHDXx7r3PKynpWJyE5IdmynsVIQ/w9bh49JcpEbH7KkiTJhPU6xt02nN5PDqgxzUQ9sZCJE\nTFjlkYxjrdtZSNsJsCux6oGFagNmKIBnTGWw1nruI7LIWi1lDL2eyxhj4x1H7uF/MCvfyVPW+Io+\n2rFhD8gkSLs/ZDAEMSOACuj11H7DeQ0INmKjZmiuOeYi1DfbF25rGn5HNkTH3lJ+L4rDWlXT7Ae4\nfuqAyAudH+Lv6mrSBYdrrQ4qMvosqPMxJ+eDUuziA7E3Gq5wyJUrqmbgfi6dbttGcVIYuAYD50Kz\nB2HgeWAoeFcO2/XtbqweFFnwZTvlff1kgyCe1W5rDgLu8alIrO743s9KGKG+5CTSWaISVZd5/vls\nAyYeW7loqswA+iczg17PxQpgd/OF2DLSnM1WBQA4CpLPEmDD2HCowVA27wDAfeg3MiZRLWhx0VUE\nQsQi69N4Hqz9fOITrkXsgSUc6zL1jYH9jMdiWjsl/832DPR/d9cdQ25h7TLsPuZ39dkCrC0D99uz\nIqahOhRoszISwiTWVhHxv7xv0dnPQ2C/Npc1b7WcqpADQZuI7QtXS4ZYzjs1gNRdMAq7q6NWwP37\nLk+fJZpm0z2LvwsRsAWyFwGrBuQh7IL6bm6khBGPaleGbDdp7wZVDfdzNKZPYoDaeXTkJDrrvbD9\nDn2XhrOqMmSdhuRTIxKReTZoKMs1Zo1FPh0YOQv7+/o3TkzGdZj00ICbRZ54j9Dih+2g2XQHjiCM\n+M4d/XxtTRfvzZuaUISMx/v39TfrvYeHLpKQXW3YtdhAmmYg9AFJJnjXadgDbAzBJM8I9RFMIsQo\n0p7Jm0yIYZh2E5sal7QXSZZ4Y9tLkfcuKHFUgiFkIah38y6Oo7wA9oNbBgEdmHVtZiqwwqM0140b\nycw6iMrWsgx3E8T107bHpIMsYxjcgcvLmjfR7Yp897sy7Pel8aUvaZqxiBPdOeyYPRSo+INAKpYk\nwAD4sBeygUT5uMswDIZgiQ2AZ6csePqPd08tsGttAZHMNHVMEePCUZt5vTETqh7VYAhFxUU29nW7\nzpXGpcJEHJENh1rcA+IdFxjBdcfHLmGp21VmADWBr2M9EjkMNrS4CFD3b309+Q43bkjj9m3t83vv\nqQEMpaKs3s7P9lWkZUMbFyKh3WxBTuTnAw2zDS7iaTADrIWQeD7FdOI0g3bQkB3yOEwyNpBYsMHF\ntMUbpDWK5kA1GIJIMT0JzGBvT3fSrS2Rt9+Ww3ffleXnntMa7iLORdjr6RFfr74qb3e78sQXvyjy\n+c8nrfCQDFDfHzuwb4AR6CIyfkgrERjvNpknVQ+HLlOOd6FORyMaUXwV5xGIOImHXYHs+4drFJ8z\nrMHsFIi51/W14FS0aev5aUbYMlQTg4niXSxzFSk0RsE1wWXlRNLHxs5hXlf4KarBEAp2foT9fdWx\nr14VuXVL/oWIil7b2xqEAo6L3fdzn5MnRPTvu3fVH40MQpQLw866va3tijhJg6UDgCcCYcaxYO7u\nC5Flaz0CiYZDlxDlC/yxln4rsUCdaZ4TBDb5kmm4yRO6zoIX9VTcwlViBCJhw2sBAynH3HjHDG1m\nGcatx6eAlFoJt+MYh82rB7XbStSnhNx88klVDYZDFas3N12ba2sqbkMVeO01rU+4s6Mi2sqKK1IK\n5mAtzoBvUcD9RJMTdf4jmAIIH5ZnVOXBEWo86UtLrp4CjFBgAPgbjIYt2ad9PWmeS7xSFpGkuXun\nWl24yJqYJWK8MhFIZaDYjFAS3o6Hz55TYJNdODmZf3Xif/zpT09kMBh3f6VhMHCuQOj7qNe/vq4q\nxPq6K+clogP65ptapQg1B9tt3Wn//M+1UOPjj+u1+/v6HdyL0NM5OChy4kNEEjSWMlgSsCfcBnb9\nkbGT8xYCxrNQlS8bAxLrFs58v4oglnFZaX2a78TGzNTn+IyOdtOidfKRq1ejuXQ1VIaDA92pr19X\nkTgNHO2FHRDW2eVlF333xBOOiFjkun7duebwebutpwSL6OdIi0UpLD70s2Q9emzyfW2zJ4CJPuSW\n4oSfQJsgCN5EQgxqtJpS3jskBVWZKfhgGQC/8rSZQfRzwMG5IGwoDienZFUNhtDpaMVakaR/v9Fw\nobxW5BUZt+pDdYCoLeLagm8X19sZ39hQqYCj72zYMphFYMedGkLSiNUt8T9T+STuMM9u40PabltV\nphBSb9KmdR4nigefCenAnrHBRuJAxaY0VIMhQP+9d0//R9nwfl9VgJs3Xd0/m5+OUF+uWITflmGE\n1JFm0+ni0Nk5/55zBTJ84L4JDC0+X1Wo0b2hMFj7fyyMQcoX8JXoN0thGL+CTPCsMYXQtfNAqgoB\nexOH14s4V7hIbsNiNYyKIs49hpOFUK1mMFDxH4c2ANCtt7dF/v7vnevNNwB5/LIoH1YwUixt4Uxs\nF/NZm/k7+7/PTWf+H9ONwYitqxVRdLTrnJyaEi3SvBBnEdEH/qR9Nq3ni7ggPKjM+/vJuJgcqIaE\nIKKLr9NRKQHGxcHAFTKFWxASQbvtLK6PPOLShWODONKuKzMSjpDLI8VGIp+NICujzidhxDwYi4jL\nqIt4y59lST6FMYt4h1NkSQlRxMgwzMBKf1OVlppNla5XVtRrtriYu1ZHdRgCRHUcgQUXIMRViEM4\nB/HCBX15PsrMl3tvMWnMQ14MBomSWbnXeeiGmPfwBBxlgkOyfX5v2Cgy2vRlqhY6VWsGKNVdavru\nC0qzodGlM4hWS13rZ1pCQOeRjAPrKXRX6P/Ly3rN/r47bYhP6Y15hsi4Ic4uQj4foAiofVvspWjy\nmoj4Iw7T1IJACHAmAdjEIm4vZ8envjOWgMzgoJIRTNgrC7aGQySqwxBsCii72XzuPrgFOSCHd8SQ\nHgcdGRIHXIsWRWPmmZA8bWRJw5mLw5dKG0IE4Y71BwQf+f5F4hXKRpkEVSWmVXg8OVbmTLodAU5M\nskQNomUvArwB+D9GFRgOVbpAevGlS+6AFGu0RAhyLHNgy3wKMaYxhUz1eQJxOpdYXECMyUpjnxZ8\n6kmVCDsWhfvOi4ZppsDZktVgCMhEXF11/4sExV0RUaKzoby+e3yqQLerx4+JuDgHLlQBZsNtxxq6\nUphBmtBiNRff7uBzT8aEGxfWj9PedyK9ZzoIZSgyqs4orHoVJSWw61HEZUoWUO+qMZvNpu7aiCy0\nB27wdfY+IEY6gFoBgwtqIwLY3dlwNkmRCg/4Ecyz4OnjkIesgJ8YnMhCMKQB/REROcevlyXC2Mbm\njDwMr+q2DBHHtHIxc7b3YP4KqL3VmVWcR8Civ118vv8Rq59ldecoPhuXEFrcthx6VnJTBnyXo0lE\nUGdtypaoOd6K2/NJkPYaRkL6aDbTCacizKAow5wqU/BsYDGEXfjsBZ/NilLv86IaM8spvyByxByg\nyCeuswE3vDi5glJIXGKbg73ftoWB9nkv+PsJvBFsL+XHhAISLTHbMHbbVU75sNf4GEyzmXGuZkWY\nwaQozBSyUpA9XqupBWWl2awKxnJUY3aPj112ISQEGPRQNQZbKLgfb60iyfJoMDaeHs3ujDXi2s8a\nLK6zZ48KA6ydgRlIxDOYd/lqbTB8xXA43wqSAoifGQC6ybYK2zaQ5m2tqrhtPaSRw1/MiGdFr1Dg\nGF2bZgvK/Xz7HPui8KJhh8kZc1MNhoBSYUyssCeIuAE+Pk4ehMErHC/e7Yp873tad/DGjcl3NGZA\nTE3NZrKsGiQbUCNqLAZWpk/oYELl6lnM81iSGAzcEY18rguIms9XxRo5Pk6qJyx0saQZ2gjn6VoM\nPdcKeXiX2E2y9HdiSbcEicrbP17zDDwvliva2wv2sXwghwBbIOf+w5vACUtWbUAm4va2yIsvJkJt\nE4Np7RO43wYqWcDylzbInAXJOeunUgpLeACrC3zyGeepiDihh6t9s4aEJE0QPNrEGatgJtvb+v/F\ni/o/PL3oC6sY1muV5gGZFmwlplCQk8/eXHJKwfgDCn5v3aRZ4xks8JoWvh7TR9+tue+YFsDx+DAQ\niP+QHnAWQIgzDodqc0DFo9Ot8KR5LjmooQQoZgq+I9G4r1ZXtIMPpkafs0aBW/m4BxAh4qw4s1Vk\n/OhAnNiOv/nRrM00mypYHR1pZDhLu0tL4+eTYK2BAdkEU7YzMKbBJMAA8ujhvqmaKnwPiXxwoZgD\nkaTUmnVtDlSHIQBsMcVpQiA+FtEPD5PFQ0UcA+n3deWfiu8LdruIcWtiwNnTYC14aZ4Hz0TZHRhN\ncuAkJE0eCpYEECJhNwg+rnE4dJqVtY2iNis/h/kfey2Y99rzTa2HY1oSQ0xsgUVa7Zi052RWKeJ1\nEyq8K+IGxxgW09qPkrosA0h7OY9xMwbVYAgQw7FFwngYUg5RKQZp0iiD1utpgRRMXrfr1A1f1BYm\nDt/Z732eiqz/0W7oO0nyMCY46w0QGedBOLLB8inuKldv5/uGQxWcGg1tA2Unez1NCdnYcMWkUEoS\nQ26ju1kzGvNwTeieZYRUhDTGUDR3LUiUkF75vaw7mifEI4FmEXtUvscMPDzVYAjDobOi8VkKDGtK\nFnGK88GBrnCEPfOJzNbkzmAKsvaBSQJw2KpFbdjTvRInujeT6gF2+sHAVYdHcemdHXdtu62EvL6u\nr838D0OFV+C6GTi2Ea+9u+te++rVZMhGaEj483O+YbISVEH53Uckab79kP1gIvXB579lJsFGnQIV\ntabimiwy1pUosnr79skoWxHKq6+MOcu1bMmHnUHEnVQE2Vok2RbLvtZdEzLU5BjYrInFZsJN++IB\nQo+GgGTdkPjt26SgSYmoeQVaGFQV8OKVFTU6djr6w2qNjePyGvl8htkp7WpFCCi2S1HFby1McFCe\nGok2VDnmnjz4yKOPnrEiq6yMWl8++9hEdDbv39ctEf+z4Q/SwXDoiqhYpZIVYd7uQgFI+J5gJ863\nQEMBQ/w3vAJMtCgchURMDmvmurEMqARI1WAb7Pvv6987OyLPPKOHP4EhvPOOFqK+c0fka19LGjpF\nkqoC07l3wU/dtO9QNEejMH/ycWb+PPDueQjbV9KukEdnAlGoGgwBW9VgoKcvbWyoEsumcvanHRzo\nVhcKz2TmYmMZWGnH9yxT+0AGopCRy66PLBcjyj1gh8bp7CBiViPefFPvx5ERKyvu/nbbmVEGA23r\n7l1XaR6nxl+5otf89Kc6tGA0KytaiBqV7HEMJIaRk0Cj19mcmUJMP0P3RhNfygPK2uVzF5aJ6FsW\nqlNTUUS3sIsXkz43bG27uy4Kx65UANICy872N6QHG4UYMmL6jI0yrhrjFsBGPXPzcBuiK4eHSqi9\nnshbb7nShfBAPPWU8kg4WuCFFXEV49F+t6u5W3t7+n23q0OKfiwu6neDgTKOxUVtGw4cCFQAD1tq\nEV/LBEJ2m5KZhY9QYuihVJ2dgpBAvHkIOEbaCYWTZ/YrJ6ohIYi4sLpOx1EMdu+dHfddu53MUORY\nVQvLDESSRiDfQRf2b0KaEcuCXYeNxvgxfVATcFqciMiXvyzyla/o3+x1hUsRhkPYsqy9FIIMnsEn\nyu/s6POWl11x6Zs3tc21NXe2LJw1MMMwwKC8sVkZonPiupkEB/gfPRXQBlNUKohVgXK1bwNZIlAN\nhtBoKJFDDeBtqdl0ZytiO/VlN6YZBDk+NytpIHLVBEwLIuI8A3g17gLWDqz4mLO9PdXh9/aUKOEt\nwPGTXK6BY7YaDVe1HgINjqUUcUTc6biq9v2+amYQtq5cUbvC+ro+C+qKNev4vLCpCF1cMmXG1oYo\nHSF3dQGGF+p7Id7JiyEkqQVQDYYwHKpPTcQZBTHI8CTAqoaIG6Y0XypzyFQfMiJmmKCtrcAHcHnm\nN+geCytgGDCdwIh35447tQ1WfhAzzp/hYE4meqwBDA+rJb2eMgCEZZw/r3bZ8+f15LoLFxyvXFtz\nbfKw2jWfuqPl5hzloIxiqdH6Orip77603SLwzBCibTYho2dOVIMhYKuDLAsMBo4hgBP7iDgNPmsf\n/mdrWa+n1AgF3Yi2TOwi40WdRJJMA91kmybo5OhIiROGQFyzuan6PLrBFeLsZsRMBv2wagrug7ml\n3Vb7wvq6CmPr6yoZMJhx+aIX7bumEVDCHTkjBjEzpmDeZ25ZoLzzMHJKBkA1GAJve6urrqwZdnGW\nArISOkIMwLcoORoIDCnClsBmiNB37AbsdJwbUSQZaNlq6bESzaYS6MGB/gZvhLTAtk8u5cDrwH6O\nkIt2Wz0JAIa70xn3iuB+MBU+2S5E1yECChJJzhiFvFZ2X43FvBh7ZlpwiEXO4iSllbkLRWPlaa5Y\nL0oGVh9W3vJyMmDfF0jkg2+h+QbMF5Bkt/IJgTUBJsDBQOwuxCtz0hEs/RxE5JMIfeqrlWR82Yoc\nhYj/ud/MfDgisjSAY0VKDpPsvqXZFzz9DDGqtKCkopmiuc6zwAQXWMvVYAgsB4u4bQ1bHD5jFImE\n8xkTsR3mrFCb9VgwAgC2A+j7nLVo24I6kVXw2aoIusMvJL73RW6zUCQyXjOGtSU+H8d3bSqyoj9n\nFK8gkn8XjokuLPKstFJpee8JIkQzMbfmvmMagJ1AJLl6+cV8MQcWMQNgr4FkgtUfkVYa8xg+CY2z\nFe39li74MCoLS1++frDtwHc/ew/4NW3sAasr7G7MhTkZF2eFmBoGMRIBxy74mMJE9SfOpMoA8JaE\nLZZdhtN6JielAB6mkMbFfcTF/Mzu9OxZZdXh4GC81mzIMSLiPBW2/IJVB2ygJl/ju47TQGz/7Ror\nvFhnwCxidt0QEdrrvPfzYHjyZGLGpgxDaNB2lhPVYAggeGutK5rHGgOmYMjzXLLIlwpt/rVjzoa5\n0KMsul11NyLuqtlUF+HBgR56DTcg2oFDBLEIjYYeft3tap4CGwq521m0x8wMzCVkN8C1uRhBERWv\nBIQIv/SYBbbElvyOun4iUqOxQEQeAC8Dgo5EkoFHuQPpCyCHBMLcPO0WfHd05DQRm5Q0HCohr6/r\n5wcHIrduaaDQ4mIyyYj51WCg0Y3DoQYyIZz59de1jOTqqrZXxKCG4V5ZKVnFr4DqUGQntuL6mOQw\n5feKUtVCxvTQDpWC+c+SiOOup1WSvQpwgYHPk4JaBthWAD38/n2NDtzcdIZEBCXBVgA/P4yInY4y\nBwQVYUH0esozP/hA5L339Fnf/74ykOeeE/n61w/l+eeX5cUXISlMtgtmDfk0UnWnjSJMMlqvz+lu\nzIL1Bo0+FBn3F4eenVPKrgZDAKbIbccmMOS1yOgTLw5rN4A4z2XR+n2343e7yhx6Pd3hb9xw7aIm\n7JUrycejPdSfvXBBJYBf+RWVPm7c0CDPgwORz3xmWe7dc4VTS9FNM3CWmEFR+MaxzPeOyZEJrt2Q\ndFtQoq4WQ7C2g5wvxDnkhWBdZCnMwAefHr66qkxhcVE9D0g7Xl5WokWlIhGXiYgwZUgNkBTY5AGz\nR6ejgU2QKFZWnAoyTWYQLCISmrM8gT0GVTgPYlY1I0tDQeNiNRgCFGSu2VXAoIhJs9V8omBdAxED\nyeva59NHBDRqHHCxEQQiQQJATsPVq8k0DnhBEeLMNIeUZXx244Z+xtKjHQN+La66xN/Z+okWQWaQ\nFgjDVljfEXmB+yYhmDSVJmZtzErCyoppyISdVKCAgbMaDIEt+zkTQxhZOm3qTmPr5Xmkg5gFYoUc\nZB/6+EuzqXEHlvdxJji+Qz6CT1XkPC+oKe477a8vPghBSzBeQhX1uUmjFqbPNxkKlggFY+Rw9fpg\n5yltLcS0lQsFPSm5GY9loNblxeGlZ1JCEEluU3DMF7QppBFvkClkPMtKHyHElmXEnK6vJ0ukYfOE\nesFrDF4GEce/8P3+vsvPOjx0JzmByKGyoB4Cx1+lBRQCuQ2IbBy2IZCh6wPhzJk5EfSMmHmKZe6F\nIwRzIq0fY+s1bYFhIdEBQXlRDYZg6xrAglYQdofwlT2zrqQFOckUX9OehVtDob127drrsbmikpyI\nm9dzzVPLeCsp+rNHFrVj4KVArkS/7wgetRa4RJvtX8hYnTf7T0SyE9FiP8/zTEJan/NICNO0XxRS\nSUK7CyaWF9uZlRCAgpJBKOgktm5egknkzGvAfXDz+cwQPsbOGyffAw8FbAhZwFrg0upoy+YybG+r\nTcOqH75wZn63QvBJBHnm1ho4su6NuGbmRVQCsKXWovsVYrI+NYJ/R6J6DCHPAiDk0cNSOT9iIgq4\nbRbkZMz3jzgCSHQwIkIt4H5zzQMOQrJtcjAaB1cCw6GrudDpqDqxu6sGSzbXWAnBx7SiLPwTeBC8\nbfkWc0y7VWYGZj2VIXWMSbZWMsi5sYlUkSEUdDkWQVB14BxgTz+899B3vPB8gZacp2CvzbPuIRnw\nOkAdRT6v8fJlLbwikizQCljel9s2FhOyGYsKRDSWCkuk2BlKCHEesy3Y5/pSXTNQzdFPIca8yJIc\ngvYFS2k+DAZ6bmTgmRA2OJEJ3gLfq8VsiKEoVXzHpdeWl124NPrBn+GetL7M2///QMAOsEfsj5Zw\nT9sak3KZy0NsLBA5WR2GwIkhE9oQYj6PQh4jY0AkbJEhEOEVk/C5rM0YoQDsoRgM3PkMdnh9r8h/\nVyEo6Mwiy6MyLSAqrgCqwxBKxKRVchLcN5Z6A9ctDH6e+O6cDATDHtO/2Fx6tIVNwYYE4FwHn+0g\nyyNyFnMWfJhFoNEIbJTBWXk4bAi7N3J3qH8i4+HxIgtjEltm5mNBVOeglglTnX2lqspsb8xow5/7\nrsHfoXzoEuGLs+f+8/rg2IW0DQzSBtyWDwrs2MRcnws+lx8bqqE3RmYiwgyApk5GbxD+mQQPlISA\nHWCSQQkugCyl3/4fcvuQUaFQ2awCrjXELLDh2do3LNhAjbofs8genZWKEist5H5n3xrgsFVbQz8D\n7A6OxSTz9EAxhKLgxTH1BUlGBN+zophZDpcob0TwSsDQnWagtIzCPjLPOBVROWJDjyeZq1DxlLQ+\nRblgQ0FaMCKJjB98kYGiWkDezbFmCISZ6MgZvuhCEYGBNuDp8N0Cz2rImMh/hwyheZlnLBEX8gpN\niEntTiOEDIbWolxy7YSyUD2GUCBBhBdF2sSmLZxCi6pgMgswLWkkZlFz6PM0kZfAeP6yshStilgW\nY8itxjFskQz+nMU1ZLBVLO6iYr0h53gOxKa35ibALNEcsnfBSZ2mRAKVAGqCyHgQoLUh+FKhRSZK\nQPUii+B86pvPcJqWrzIJ0hLjuJ9eZBkKc3Bh335TZlCoD9ViCEBJgUl2MZWevTarbRbPsqWTm03v\n4rVJg7ywOMMxlFtRMOp1hKKSTxoR+mDfnV2wkzLbvPaFESyHRVoquC9i2XGabwBIZbch6fyYGAE1\nr4eoGgzBvhGfnZ5igIvFROqAr39lIG+uRETAFru+UZmJ06S57oEvdoWzLPl33lixtFDuspAnmW1S\n5Fo/1sfLVHx4qAkmCAw5hW9tN5uuAI6Iy4NBLIlIMkemLJNENRhCCCUng4wQw1pjosnKYhT2OVmH\nKBBAAGiCb7HRrLZ4CxKt0DRrP8fH7oyITid5eEze106z6/iyK2N0+CzCr0SEJb8ctvRez2WdnQ6+\n7acdLwga9gAfSHKIF4HdMqu2RRqqzRCmhSwZaxaGnqzEBcQgR8p8Vs+09gJIA9xkWonvfl+zJLEj\nodaCJd5YjEffOcmD+xpDyD57gu/zSjAFi8VFLbSZ89hAltas8ApBBIL1RKHxxW+dL0oNkgmpB6xU\nT4NJcNYbh7SKuB2lxOdyxSWuVgdY4yGYSKoxPLKPvPOnXR4zlz7bge/zuTMDWxJwacnpbRyTEIHQ\n2ZvsxQx5O/PgTDKE0ic9ZtSOjgqXpQqCc6N99RwLxgynBVXCUIWaCPwIX0lEMAMbi1Bk3EM7uY3T\nj23LJxFM0r/S4TO+rK8XOlUpLSgW8+YzIud+Tv5b5gubzDMTsXDaKkSo/ZKfCxckJAVer1AzsKmt\nrOiP3cQmGuvTbc26DIvizIVP80lkIpnzW9QgW4TBju7NfcecMfWMtawCARVD3vFoNJLWa4A3LdZa\nUo8rtBJMaGviBIpTNclXR2KsrTmN+dQ2mBm8z6R9r062YwR4V+GfVGSl9s0TeVWDwHV5FgEKtvru\nsTqp7ab3OTZ4wZce6fOihLJHS8LM0pyn3IdYm0reLM4QzhRDyB1lyMwgljvPWxLIYmA5iMiXHIvP\nffDZuMAYUpkB/20ZxGCgPkx+yAyCuSYhDpsxW5Swp6nK8nyWyfyqKQdPiglzDLztzcKOkMXAUvqQ\ne/ENXPm3E1nwesFy66JohE/uRll926iBV2+fYB5jMkl934PQyqgtMIbIdWT7nhZxWzbTeTAZwiSR\nGT7M0qiY0vdcxi4br2yfQ8gVpmvaSvQJz7L6RoTPfey9csRgxKCssOjCyLGphJ4zC8/JmVIZolBF\nW0EsrNhNKLxbxeaFnI6bVS28hJqFAvYB7/ulxEznSZGepB+l6efzVkUjUY1elimSl52aN2t4dtNC\nC3zC908lgJiQ8mbTGzCUZ/dLuyeGQK2LOi8qEcswY1RHQihjZ/cF9M8aeXfHFMK1uf6FM/CKulHx\nLhnvk9C3aVf37awxujmuGbu/wBpJe1aouTSDYhW8F9NEdbZRG9ReFHaWi4Yfc/RgkVSy2OdlXMcG\nLk7tze1xyTuunAONYHk+IdbTz8S9vZ5zL3Y6smAkhhGOjpwX4vRE2rHAs8HPXQYPUjm5jyKFo0hD\nQ5MmnZRZkKVqqI6EUBY4dY/L1eYO6s4Xa564L0X3zYtpp/amAkSW4SocYwYY+8VFkZ/9TA+FICTe\no9sVuXVL77l/f5QenJ3YPYAAAA/7SURBVHCRgjGvrIwbSblwaQ7YaPEzj5LiOqohIZQlHYg4V1cZ\nbZVI2KnPACKZV7TFOw8z9AXLZy0yTpu0JwUtL+vvy5cTKsQYUxsO9dDJTsfdYw+s5FBJGzZp+h3L\nNJvNMEPw+fdDXorKSAklrdNqSAhg12VwOd7RSjg/b6YocBZfKiZlaCmEN0KvJ3L3rhZPwE5vn+uR\nLkYEtrY2XvH1+Fjk/fddZmAOZAVgsVkkzYaQ5m0obMs5A6gGQwAz4PzcvPBF+Pl23ypjRlF8ueBj\nKkxNw6Fm8C0t6W8Rf92v0Pi3WqpaDIfKVET0f5twYYs8TIgYIdIbKHX6E80UfJynwmuxGttnr6eL\ngPXDPHphTCL4WZAUTB/TXGZTE1cN0Xmfw/2EmM9/8/eXLun8Hh6KtNuyQPM6antpKVnzTUQlB99J\n4DzXgcwrNkj6bovFWDr14aHaQ3o9kc1NWeh0xsZnbLzQR1uxJuO5bEDmvtjPy0Y1qGR5ebwkTB5x\nn20QFea+mfBEAaZhFkwh2D5LCFy4EZ4ItgG0WiLvvKNEvr7uZzjcHrwUyL/G2vAxCA9ANGw4jFke\nmcsNdo7hUPvX78tCqxV0DY/WcVTjSfiYmu/7BzN0mQcL1SR932W1MceU2YkwARObqWHLmuat4Y9h\n/9/YcMR0SuALIs6tibnDiUa+kGdfdqgnEA3EwtoOeFZI8My1bELliSZsOMYekRUXMelaqAb12B1l\nZSXpYz4rmYpFUVXJxhIeyi3B1YdioTFSHZ90fHzsKlAdHrr7OO6DGbxlBAXdyRA2rAaSy/aKC6k8\nWlq0ZuJBOftbBFbNyWv0rIZR0ZfcM4nF3aY+P8BIC5yJgjW7M/HZOeD67KcBRyMQVSWe3+8r4zg8\ndAwFFVuxZa+sKMNAEcejo/G+8DPAlAogZI7IWibeMaWbvBGYvAZLOgU8dIvPc1LEA1KNLZXfBNZl\nSAu+WoNZiHGXVQ2B6L+sjLy0z20sf2ZNA/4MOz6fR86SXMa8jCIM+31/lVYuA83qBevcvtXPfQBT\niTgWzb6SiJ+npHkPvAZD370hqi0hzyZ0a7PpAksn0ZyrQy1LS8mB5Nx6/u2T9yqCifR5j0jp0w/z\nGJMmSuGNGdeAi3T0XLYDWOoL1RYE1bIh0UotIu5QCc+JMyFGanmMz+wUtauy9HR0NAq3TiCUxs7r\nOIdXKQZ5TBshVIOaeKZYqWMrLdflqzAKM4WIiLuslN88GYBRYALkYoxs+Atg9Ax7tJlIklj4mDOA\njz1jewKebdsV8fbFCjQ+ZsDdymhORE7Hz4Y5Wq7C6xeNW6kqgDxMIUsQKYJqUNfRkdtNYllcRW0D\n8wplncpzLVWEiDEEn5JuJT5851Mb2u2kB4Kvh4zMMSwB+BwjVn1IEza9n+N8xtBNvgC5CONxHgkh\nq79FSKQaDMF3PDEAyQC7ErP9ikkLUUQZcRBf1qKYlPhz5UJkVZ9iB78vGMyqQvgM7sdTf76srCgD\nsH77EMVCpeBrzXU2DiitTBzfmrW0RlJCCGnjlsEUQuXSChkICzivquFlEEkarxjsSMZ1ZxUcuDNh\n7MGkiKlLMKIOih0YUUuIOH1tYGX2++7ASH5/qCJImRYJrwP7N85I5L6c3pvWLcZgoN26d09/py0x\nJtCT5rlxjsLp2LavuCZiIyulSpPn8ZnXT/zEssBioGXt+DtvBGMK5pKp5nsvD0JeAkZM//MEumT6\n0m1GI78LuyfT5FifcRAMEvNuj6T2wdoTLIZDWWg2pdVaGF3K42gjGNE98Bbm1yGJAW18qGXeOeYg\nFnuoZsZ6mGUCVXUYgj1HLMQUfNfkQGWz0zzvE8uwfERd6ntafd/OVVZMgI0bZolDRGMU+n09BNV6\nFXzU6NnCRwySYiH29lyYA8YDR0fAQcDhEHBuIM8KggvHVPG4jmjaqkU2poORJl1lrGefWcJ+zt8X\nIZHqMAQgVgIoKCWMVeOZkxHQC88MxuQz8N+Tuq6CCDnv88wVfrfbSUkgdNY8W+otsZ32Y6yC1KmK\nsyAn0m4veLvXainBdzrjgkqz6dIorOdbxDHdZjMwxrz7W0YY6kwkIERBoLL9w6MLxmxpF4vfWjKy\nypVNIeagUszAg6ygpLyuyYlwWslINjby3+uT+GLn06oZZjv0ZmKSUbHVSl4DNaLV0ghqEBZv1IuL\n45nog4E79SoxxtaTEkpmYpXKqkyRQOU4q21YM1uWSzUN1WEI8DtbkbGo7JOBuTKDNHEyEmUTfqYt\nYX29uP2Giw/wCoW7mc37aUhzA3h2YdAaS4ILciLN5oK020nbJhOT7Qa6byXKRHm3tI2MDbL8wIJr\nOtY9WsT+Xg2GgEXBrNrqWjHIYcGdKyLfa2ri/yl8kqw3e27Soi0+AxrbHrLahwuPi6taeMaUl4NV\np2AL5Qhq+zhf98cYp6++g+2XT+HPclvS90XWQRHyEakKQ2D4ohFj32xK0kSpiFkQhGkzhdAzE/Dt\n7gzfmLN0Z8sTsfhvt2Tr4sQmUSBK9VwzPHbWi2B309DumtaFqI1mivaxMlA9yjH5qbkNf1VmBiJJ\nBW/OyDVUacxAxIn/vt0+JAEcHycTkzhg6d49rbYENwBb0XJ0PBE3QMwBO2iWBzOtTRHR90YDee0j\nOZAnQGmiYKbcd0wDWAgIQz1FIS9A1SUEkXD/TN+n5QVJE1K8dQRjRV6rkOO6UJAR5HV7faulNgtI\nB5AU+Nn4PNIVHSKSvEvFe/4EGsrrFs8aS8/naRKjnbuzXQ+h0VBf0PHxaKDmoetbN17pYPMvfuD8\nDsQhFB2H0H2+UhPByDgfQcOZj+/xm0qLjYD34/dlxmEjUNmtyDYGn/Ue7dq+8DMjxyQG3ozGw8Ow\nvmHfNw9SmImtAJ0W1Zj3fauzlTabGphSRjsFUXY5Ki+s+83+nQHbp7FCoOJnZFxnMLeNMGS+5l2M\nHeDshxeJk8fteRq+ceI2EUXU7Y4f1mIDqUw7eXdO7zoA0+GoJWsTSVMPIRX7JmOK3rUsVIMhQPSa\nJKKiBMzDgFcWxtxh4nclYp2lGdwyF6LNSg0RL7eX9j2uMcFG3jYYKLXnu8YEMBUlrmA/uNoTxxXE\nvmsassZziqgGQxBJFuuMKZo/JczdJRlAbN6CZQw+CeJcM9CmJai0Ochj60hjFJ5dEv1OtOmzY7C6\nYK/znQ1hpATue0g69PbDhlRDXeLvLdHz/1kZpAHE2g4mQTVsCCJJUauo3nVWUeK7FpZwuA+ofxjZ\nr1IWZKgcGWAJyh7ymlbYICWeNy3i01sKHUzM2g3SCBwGR/zwmZmRG19WcZyo7NUIVENCgKW21RqP\nVnyQkWdHDiB2p8t8PgOenrxVr7ndWH8769lWDWFRnD8PPVOk8DkIIXgZRihpICDqByWonLajPNmr\nfF8eVENCsPnsvspJRYOzp4lJJJmcAUpZiPJIxPTV1vyDi29asIRsd3oYDzk+hQus2u/yPleyz4Mc\nQ2QGUZ76l7FtTfu+amzDKJS5v++q5oSiFXOEfE4dkzzLehmmxeisqJ1WyQe/7WLPa+zNMy7WNsAn\nP7FaEIptsBZ+FufTpIWAj59/MxLp1YH2rL0mKx09jVHMy7hdDQlBxGXTQb+yu1IMwZxlNWOaffcR\nSQpKWYwsPcUyO8uUwIiwFixzg52DdfJQXzLg9+wn9fLQdz79PdRG3memxRhMA9WgICxWuHC4WoWP\ny5+FaMQYlP0evohA1sEhecE6bwudnMJbL3CSvsbeZ5/BKotPsuGaCr7SbpGYhAGeVTd1CNWQELpd\nVRc4y4QXQlqY7FlG2czAMgBfRGBMsJBvbPPOQVGd3hfdh7Rhjj6EAVpk3PCYY208aAQ9KaqxzQ4G\nLkoRWXCcm2rxIEgH04Jv14fRDVJBxJkGpaBIu74TofC713MBbK1WotJSwpLPdoVAP2bJCLJsCWmY\nNcOqhoTAmWKoXOGzIVRJKpgkTn1a8HkDrOsuFMiDa9Pex6eS+IKD8gJ9ZNuBj2lhgzg4cFmGp0jo\n2b71c4qy/PWx8CUc+aJKq4JqbLU4SbjVcodzbG1p+isqXVYFVWIArHNb/ZkJFTH/0MNx3iIXHGGb\nQmhnDe34Wca8LJuEL18hVMUZTMF3qhOAlOoSMMnujnvSEo+y2px1OH01GEKj4Rbs4qL+vnDBWY59\nYqTIdIyLMW1aA9c8jJwgYi6Nzt9hzI6PnQuPd2BrvGMcHuoOfP78eNtcvCQrBDdkwOR+5k3igaSA\nTEOcJM05Bu32eHjxKbLSh4vUscyCjynESgazVhmqwRCOj3Xy1td1Are2dPei2gipvnOR8UVVNPAn\n5vpJSorlZR6+9/DtrkxcINRGQ8e21dLiqKGjga1asbzsKo1agBEwM0h7H/4Odc9thCLsG1mZf9zH\ndls3kcXFJIMD87OnQUeMe0yUXxGmkEb8k0oAZUsQ1WAIH3ygCxYn+2xu6ufHx24xWCt5VoASV+gt\n6vay3zH4FNHYZxRVN3yMDp8zIUF3RhTfYOAIxp6FgPttpKBItmfHMoWYRCi4k0PvZZkBSyDWM4L1\nsLysa0TEqT98EC1LM4RJKgrFYlaiftnPqIZRsdPRydvZcYdo2jrYEI9F3AKzxAEUPe7Np4/yc9IC\nfLKInZlT7LW4nuFzK4IRgFAgph8fu1rjtg/435eoMxgoY0axkxBjRaVsW4zEzo3vfX0GSl/7/b7I\n7q66pfEce5gCFdVJAMyhYPZsVtAQwBGOVTcaZqEaDAGLcjhUpgCuj8NADw/1f6sThizkvqL6IaQx\nFgvbbkz73C4zAx/jSQP6mVaf0DLFoyPHSLGL9/sqke3sOD++bRMSxc5OkplYSQDfwR3oYwSYQx4H\n/u0D+owfWwCFgVB3e2YDMzE+n+308zJ31lA046QGyXmgGioDV9pZXNRJ63ZdEgn7ngGfBdy3aMtC\nyIBmk3J8+jkbzmIkBOtHF3FGQas6+fR4SAV8RlmzqTst+sn2GX5H1LXEsWpgDugD2uP33t9X+w+e\nxxIV8lTQP1sLEbs+vxsbk/G8Xk8Z1IULTj3gce92XfUiVheYAU3R+JslFUzTjsCI3dtCqAZD4Mnf\n23OuRmYIi4vJEFafMY0Ra8H2tRNDtICtn2fbjjF8+sDfQeS1zIevxW4KdavTGbcLIDS809Ex3d4W\nuXhRCQk7LKz2Iu7642Nt17ryMD9ra+OxAzxPPDbWtQnbgi/N2dqPVlb80owtWYa2rKowZU9QFaSA\nSV9x4eTkbOk4NWrUmB6qYUOoUaNGJVAzhBo1aoxQM4QaNWqMUDOEGjVqjFAzhBo1aoxQM4QaNWqM\nUDOEGjVqjFAzhBo1aoxQM4QaNWqMUDOEGjVqjFAzhBo1aoxQM4QaNWqMUDOEGjVqjFAzhBo1aoxQ\nM4QaNWqMUDOEGjVqjFAzhBo1aoxQM4QaNWqMUDOEGjVqjFAzhBo1aoxQM4QaNWqMUDOEGjVqjFAz\nhBo1aozw/wHMmKsU4wD6swAAAABJRU5ErkJggg==\n",
"text/plain": [
"<Figure size 300x300 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"for i,l in enumerate([31,21,11,1]):\n",
" utils.heatmap(numpy.array(R[l][0]).sum(axis=0),0.5*i+1.5,0.5*i+1.5)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<p>We observe that the explanation becomes increasingly resolved spatially. Note that, like for the MNIST example, we have stopped the propagation procedure one layer before the pixels because the rule we have used is not applicable to pixel layers. Like for the MNIST case, we need ot apply the pixel-specific zB-rule for this last layer. This rule can again be implemented in terms of forward passes and gradient computations.</p>"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {},
"outputs": [],
"source": [
"A[0] = (A[0].data).requires_grad_(True)\n",
"\n",
"lb = (A[0].data*0+(0-mean)/std).requires_grad_(True)\n",
"hb = (A[0].data*0+(1-mean)/std).requires_grad_(True)\n",
"\n",
"z = layers[0].forward(A[0]) + 1e-9 # step 1 (a)\n",
"z -= utils.newlayer(layers[0],lambda p: p.clamp(min=0)).forward(lb) # step 1 (b)\n",
"z -= utils.newlayer(layers[0],lambda p: p.clamp(max=0)).forward(hb) # step 1 (c)\n",
"s = (R[1]/z).data # step 2\n",
"(z*s).sum().backward(); c,cp,cm = A[0].grad,lb.grad,hb.grad # step 3\n",
"R[0] = (A[0]*c+lb*cp+hb*cm).data # step 4"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The relevance scores obtained in the pixel layer can now be summed over the RGB channels to indicate actual pixel-wise contributions."
]
},
{