{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "## Backpropagation Exercise\n", "In this exercise we will use backpropagation to train a multi-layer perceptron (with a single hidden layer). We will experiment with different patterns and see how quickly or slowly the weights converge. We will see the impact and interplay of different parameters such as learning rate, number of iterations, and number of data points." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "#Preliminaries\n", "from __future__ import division, print_function\n", "import numpy as np\n", "import matplotlib.pyplot as plt\n", "%matplotlib inline" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Fill out the code below so that it creates a multi-layer perceptron with a single hidden layer (with 4 nodes) and trains it via back-propagation. Specifically your code should:\n", "\n", "1. Initialize the weights to random values between -1 and 1\n", "1. Perform the feed-forward computation\n", "1. Compute the loss function\n", "1. Calculate the gradients for all the weights via back-propagation\n", "1. Update the weight matrices (using a learning_rate parameter)\n", "1. Execute steps 2-5 for a fixed number of iterations\n", "1. Plot the accuracies and log loss and observe how they change over time\n", "\n", "\n", "Once your code is running, try it for the different patterns below.\n", "\n", "- Which patterns was the neural network able to learn quickly and which took longer?\n", "- What learning rates and numbers of iterations worked well?\n", "- If you have time, try varying the size of the hidden layer and experiment with different activation functions (e.g. ReLu)" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "shape of x_mat_full is (500, 3)\n", "shape of y is (500,)\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAU4AAAEyCAYAAACVsznTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzsfXt8FOW99/eXAElKEiKwxEsgUKupIJdqjAXvCBEsCKXa01aPeCMq+uKp1FZffE89Ki09Sj36KpelivSVHq09akFBkhi1rWghWgmQSvEWCEqyYEMSSAIkz/vH7iyzs3N5ZuaZy+7O9/NZyM7OzHOZZ37P7/4jxhgCBAgQIAA/srzuQIAAAQKkGgLCGSBAgAAmERDOAAECBDCJgHAGCBAggEkEhDNAgAABTCIgnAECBAhgEgHhDBAgQACTCAhngAABAphEQDgDBAgQwCT6ed0BKxg6dCgbOXKk190IECBAmuH9998/wBgLGZ2XkoRz5MiRqK+v97obAQIESDMQURPPeYGoHiBAgAAmERDOAAECBDCJgHAGCBAggEmkpI5TDceOHUNzczO6u7u97opvkJubi5KSEvTv39/rrgQIkFYQQjiJ6BkAMwC0MsbOVvmdADwO4EoARwDcwBj7IPbbXAD3x059mDG2xkofmpubUVBQgJEjRyLaXGaDMYaDBw+iubkZo0aN8ro7AQKkFUSJ6s8CmKbz+3QAZ8Q+VQCWAwARDQbwcwDnA6gA8HMiOslKB7q7uzFkyJCAaMZARBgyZIgqBx6pXoHOXZsTjnXu2oxI9QrL7Tlxz1RGMB/pDSGEkzH2JwBf6ZwyC8BvWRTvASgiolMAXAGghjH2FWPsnwBqoE+AdREQzURozUde6TjsefqO+IvduWsz9jx9B/JKx1luy4l7pjKC+dCGclOJVK9ApHZVwqbi903GLePQaQD2yr43x45pHU8CEVURUT0R1UciEcsdOXaoFb3dnQnHers7cexQq+V7phryyyZhxM1PYc/Td6Bl/VLsefoOjLj5KeSXTfLNPVOdY3NijtMFyk0FWdnY/9JiICsbQGpsMm4RTjXWh+kcTz7IWJgxVs4YKw+FDB37NZGV8zUcPbAnTjx7uztx9MAeZOV8zfI99fDAAw/g0UcfdeTeixYtwvDhw5Gfnx8/prYxqBGc/LJJGHLRdWjd+ASGXHSdkBda5D1FcGx1NY2Ye00YV17yKOZeE0ZdTaPl/liB0Xyk+uZgFcpNJVK9DCfPWYRI9bKU2WTcIpzNAIbLvpcA+ELnuGPIzs3HgKEjUPvKu7j+e8sxs3IFbv23P+HtP+8BkFoc6cyZM7Fly5aEY8qNoe9YjyrB6dy1GQf//ByGTV+Ag39+LukFtgKR97TLsdXVNOKJ/6xGa0s7GANaW9rxxH9Wu0o8jebDK3HeaYLNc3/lphKaMk/4Ru4k3CKc6wBcT1F8G8AhxtiXADYBqCSik2JGocrYMUfx9p/3YPnTOxCJHAYDEGntjL9UdjjS3/72txg3bhzGjx+Pf/3Xf036fdWqVTjvvPMwfvx4fO9738ORI0cAAC+++CLOPvtsjB8/HhdffDEAYOfOnaioqMCECRMwbtw47N69O+l+3/72t3HKKackHJM2hqMH9uBY2370dn6VRHCkF3TEzU+heObCOIGyQ+icuKcdDnZN+C/o6TmecKyn5zjWhP9iuT9mwDMfXonzThNs5f2b196HpnBVwv0jtasQeWNVfFOJ1K4SvpE7CSGEk4j+G8C7AMqIqJmIbiai24jottgpGwB8CuBjAKsAzAcAxthXAB4CsDX2eTB2zFE8u/JP6OnpTTgmvVRKwnP0wB4MGDoC2bn5GneLYufOnVi8eDHq6uqwbds2PP7440nnzJkzB1u3bsW2bdtw1lln4emnnwYAPPjgg9i0aRO2bduGdevWAQBWrFiBu+66Cx9++CHq6+tRUlLCPb7s3Hz0yx+MY4dakZUzMOlF7GpqSHhBpRe4q6mBuw0lnLinHQ420tpu6rho8M6H1ubgJFfoNMFW3v/QB+sTFHCR2lXY/9JiFM9YiOKZCxGqnI/9Ly1GqHK+sE3XaQjx42SM/dDgdwbgDo3fngHwjIh+8KC3uxMHWjtVf5NeKjnh6T9omCHRBIC6ujpcffXVGDp0KABg8ODBSefs2LED999/P9ra2tDZ2YkrrrgCAHDBBRfghhtuwPe//33MmTMHADBx4kQsXrwYzc3NmDNnDs444wzuMR492IzeI4fQf9Aw9O1rjC/ArqYGhCpvQ6jytqRr8ssm2XpxRN9TzrHll03CwDMnmnrBQ8MK0dqSTCRDwwot9ccseOdDuTkMPHMi8ssmxbk2abzy+RABOcEeNn2BcC5XeX/p+Q256DpE3liFk+csQmjKvOjJfb04ec4ioK83fu2Im59CpDYc/y6hc9fm+Dr2EmkdcqmqrzzchqEhdbFbeql6uztxvPMr9B80DMc7v0q6hxoYY4buUDfccAOefPJJbN++HT//+c/jPpYrVqzAww8/jL1792LChAnY/9lH+Jc5V2HdunXIy8vDFVdcgZrXX+PSs/Z2d6L3yCEAQFZuPrLzB6MpXIWmlVW+tlIqYZeDnVt1IXJyEvmCnJx+mFt1ofC+WoWeOO80V+iEjlvv/gDihDR0+bwTRBPRTSY0ZV4CMcwvm4TQlCrfunSlNeFU01f2drXj+psmar5Ukk5zwNAR6F90clxsNyKel19+OX7/+9/j4MGDAICvvkrWOHR0dOCUU07BsWPHsHbt2vjxTz75BOeffz4efPBBDB06FPtav8Ku999B6anDsGDBAsy8cjo+fO/PXHrWvp4jGBAqxYBQKY4e2AN2rAdgwKBzZ/he4S5HqPK2pP7ml03i5jQmTx2NBT+txLDiQhABw4oLseCnlZg8dbQT3bUEo83BCc8HwBl9tNH9m8JVOPDmalOE2s8uXWkTq64Gub6yX/5gHO/8CgOGjsCU7+Qja0AO1oT/gkhrO0LDCjG36kJMnjoaxw61Jug0pXv09RzRFdnHjBmDRYsW4ZJLLkF2dja+9a1v4dlnn00456GHHsL555+P0tJSjB07Fh0dHQCAe+65B7t37wZjDJdffjnOqZiIXzz8H1h70x3oPyAXw4YU4f7f/S6p/Z/+9Kf43e9+hyNHjqCkpAS33HILHnjggfjv/fIHo/fzfTj5shtRPHOhmEl1GZHqFcgrHWdJXJs8dbSvCKUSRuJ889r7cOiD9QliPADbompXUwMKx5+IM5EIVFv9OiFisHJDABDfvItnLtRUu6g9awDIPe0sLpWCnbViGoyxlPuce+65TInGxsakYxKO/vNLdvjzbezoP7/UPMdPONrWwo53dST0+3hXBzva1sJ9j+NdHezI3p1se/17bOc9E1jHR+842GPn0PHROwn9V35PV3R89A7bcfcYtuPHY1jHR+8kfRdxf6vz2rppOdv73L0J53Z89A7b+9zPWOum5arnK+/b8dE7Seeq9Uka8/51jxr2T8RaAVDPOGiQ50TQyscM4ZQIyNF/fsmO7N3Jjnd1cE+iVzje1cGO7NnBjuzZEe137G/evktjPt7VwRobGx0jNrwvhF1I/ed5ebyA0TxYmSfpGvnYd/x4DNv73M+E9dvqvDpJ1BPGq7gnzzq2u1Z4CWda6zit6itTHX09RxLUDSJcg9TglgO3KF2fU5FERvNgZZ4kHa987EMvuxEl1y4R0mfA+rzml01CaVUYIODzZTfi82U3xkTxmUnnmnWhkvcpb8Q4lN4aNmUgdEovrERaE04lAZHrK/0MycDTr2AIjh1qRb+CIRgQKuXut5oLlRnDCi/cUt6LsAA7GUlkNA925slJ67ede+eXTcLQS28EO9YNdqwbQy+7EUXlM21vpPI+de/7u2q7euvYaW+BOHjYUr99zOo4UxF2VQySnnT7+39NEHUkEVCkOL1/3aOs4fYRbP+6R4XdU4IVvdUb1TvZ9VevZNMvfoRdf/XK+PdpFz2S9Ln+6pXC+mo0D2bnyUn9rt17S+L59gVnsu0LzkwQ262KyiL65JaOM605zlSFCBWD5IoFAvY8fQcitauw5+k7gKxsoeK0kzt8XU0jnl28Csu2X4I77t+BuppGQ3FNi7NUc4YHxEUSGc2DmXmSoobk1mn5dxEqFzt+sp27NuPzp24A6z2OkfNXY+T81VGxfflNaKtfb1lUtuu760T0mhYCwhmDn5J7iFAxxK/p6kTB6Euw/6XFKBh9KSLVy4SJ0076A0oE8LXPx+CTwyUJorWeuKYVo56VpR6cICKSyGgezM6TpBOVXGvkIq8olYsdP9mupgYMLLsAlNUvft2w6XcBfb04dmi/5Y00VHkbupoakuL580rHcelJ7fr+mkFAOGNwKt2clbRyajrK7Nx89B80LOHY+++/j7Fjx+Ib3/gGFixYEHWTUFyTlTMQbVtextdOPw9tW14SqjDn2eGtxlzzJOlQM/ZocZB9fcyxSCKjeTDLCfnZ8RuIEqhRd6xG6a3hhNRwxVf9FF2ff2hrI7ViSPMiPV9GEk61ie5qakB7Q43p5B5e4vbbb0c4HMbu3buxe/duvP766wm/93Z3oq/nMIoqvosjn2xFUcUcoeI0zw5v1fJulKRDSyTPL8hVvU6KHHIikshoHqxwQm5Zh+1A2Uf09doWla1sGl6k58tIwqk10QNPPy+e3KNf/mDTRNPNtHJffvkl2tvbMXHiRBARrr/+erzyyivx3+Mcc14+OhrfxslzFqGj8S2EKue7mnnGKvekJUIXFOZi7jVhPPLQBlWOlAianOXkqaOx5sUqbHj7J1jzYpWvo4pcsw7bgLKPalE/VkRls5uGFxx6RhJOrYnOKx1nOrmHBLfTyu3bty/hWElJCfbt2xf/LulJwYARNz+F0JR50cw6Ma7AijhtVSSywj2pJeno1z8Lhzt7NA09ANDR3u37GHUjOB1LLgJO9tHKpuE2h56RhBNInui80nG2LNm8aeUuuugijB07FmvXrsXOnTsBnEgrt2rVKvT2RlNrTZw4Eb/4xS/wq1/9Ck1NTcjLy0u4l1KfCSQWZ+s/aBj6eo6A+g1IEJ3ySsclxe7yijpWRSIrL4Jako68vAHo7VWtrBJHaFihbzhLqxuNm9ZhOcz016k+WiXIbnPoGUs4lRPd0fgnW5ZsxsSllTt48CB+9KMfJaSVq6urS7hXSUkJmpub49+bm5tx6qmnJpyTlfM19HZ+ZUjoeEUdKyKRHc5ESQA7O5JLHcthx9jjRFSR1Y3GTeuwHGb6a7aPvETZCkH2gkPPSMKpNtFfPL8o6eGoWbK1IDKt3N69e/Hpp5/i61//OhYsWICrrroKDQ2JfTvllFNQUFCA9957D4wx/Pa3v8WsWbOS+p+dP5iL0PGKOmZFIpGciZ7rkB2R3KmoIr9bx5Vwsr+8RNnKpuEFh56RhNOJiZanlRs/fjzuvvvupHOktHJTp07FN7/5zfjxe+65B2PHjsXZZ5+Niy++GOPHj8cLL7yAs88+GxMmTMBHH32E66+/Pul+y5cvxy233IJvfOMbOP300zF9+vSkc7L653AROl5Rx6xIJJJ70kpOfM//udKWSO5kfSKndG9OueA41V8nibInHDpPeJHfPpkQcikKO7b9zTAEjjdUzQ8p3tTCKe1i+sXJoZjTLnqETb/4Edv3diqzk1PPwulMVGbCTt3KviUHOEMu0zqRcaajt7sTvZ1f4esGdXu0OHBlzZeoUWk+upoa4pl7JE7dLfHTKDlxXU2jaoJqPThVn8hu3SQ9yDm4IRddh4N/fs72fZ3sr3R/tfpKWnC67pIdZKSonino6zmC7PzBhioJTVFHUfMlGvq2LEEv5YbRghdWdZWi6xO5EWsOiBerndQVWjHg+FlHnFaEk6m46GQy+hWGkNU/J+GYGULn54WrBqu6StH1idyINQfEu+A4qSu0SpT9GkElRFQnomkAHgeQDeA3jLElit8fA3BZ7OvXAAxjjBXFfusFsD322x7G2FVW+pCbm4uDBw9iyJAhhm5BmQDGGA4ePIjcXPUQRF7IF64TZWRFwk4tdZH1iZwQo5VwWqwWDavlo82K927BNuEkomwATwGYCqAZwFYiWscYi8tHjLEfy87/XwC+JbtFF2Nsgt1+SH6NkUjE7q3SBrm5uUkRR2bh14Urh6TX1BI43KqlLofTG44eB+e352MVft4cRHCcFQA+Zox9CgBE9DyAWQC0FEs/BPBzAe0moH///hg1apTo22Y0/LxwJUh6TaWILsGrWupObzhWObhUgp83BxGE8zQAe2XfmwGcr3YiEZUCGAVAHgaTS0T1AI4DWMIYe0XtWhFwtXxoGsCthWvFEi5BTa8pYVixuXuJQipsOKkAP28OIoxDagpFLSvNDwD8gTHWKzs2gjFWDuBHAP6LiE5XbYSoiojqiajeqjjuRfqpVIbdxLI8sBu1o6e/jLS2Y034L3jy1zWOFGnTglex5qkIL3JpioAIwtkMYLjsewmALzTO/QGA/5YfYIx9Efv/UwBvIVH/KT8vzBgrZ4yVh0Ihro4pH0rUQjgfny+/KSWsxH6A05uNVUu4FFuu50ghEeLXXtnmSJE2LXgVa+5HGBFGrfXVE/nc1wRVBOHcCuAMIhpFRAMQJY7rlCcRURmAkwC8Kzt2EhHlxP4eCuACaOtGTUPtoUSql2HQhOmq7g2psvs51U+1+wJA4fhpjrkkWbGEy7lUK7AbTulUmWEJqbIOeWC08Wq5vBWVX+Vr6dA24WSMHQdwJ4BNAP4O4PeMsZ1E9CARyV2LfgjgeZbobHkWgHoi2gbgTUR1nMJWodpDCVXOR0fjW6q+b6kiyjvVT637FpXPdMyXTsvirWcJ19Nr8sJqkTYnywxLSJV1yAMeX2A1X02/+xALcYBnjG1gjJ3JGDudMbY4duzfGWPrZOc8wBi7V3HdZsbYWMbY+Nj/T4vojxzyh1Iw+pJ4sTK16AW/PywJZvupxcF89tQNXKoMAI7lOrQStaNF9CTndR6EhhVa4hydTAgiQdTzlThUrzlYIyd2LUd+s87vbo4zrSKH1CA9lIFlF6Dtg9cQqpyf8AAKx1+RoLT3a6SCEl1NDSgYfUlCP7UWiRYHk192gYYqY9qJOjKAo7kOrUTt6HGpaoRYiZycfjhv4ihLnKMdJ3szMFqHciIhPd9I7ar4cTmH6jUH27z2Phx4a3UCYZTWqlYoZvPaexGpXZVAUKXxacHNcaZ1kg9lUoCmcBVaNzyOvOFjAEB1J08Fh28AQFY22ra8gqKK7+Lgn59DVl5hnJtWQi+SJW/4mITjocr5iFQvi4//WHvEcZcks1E7c6suTPLdlNcVApDg3nTexFHY+u5nCe5OepyjEdF2IiGIEkbrUJkAI1Q5H/tfWoyiitmI1K5UfWZORjLpjePQB+sBBgw8cyIGnjkRTeEqgAGlt4Z1Esz8Bv9853mcPGcRQlPmISuvEPtfWoyT5yzSbMvNcaY14VQ+lNKqcJR4vv4kuvf9XZVopoL/ncQZnjxnESLVy1Aw+tL4otLqp1Yki/x4UcWchLrr0viLyq9KupeX86FGHOVEk4cQP/rwBtXjRpyjHtEWBZ51qEYkiipmo23Ly6qRSl6FznY1NaC0Koy2+vVoCldh6KU3AgwYdO6MhHOkPkr/t9WvQ/6YyxCpXoa+rnYc/PNzUaLZ16vajgS3xpnWhFPp/pFfNglDL71Rc1L9HKkgh7yffV3tcaKnt6i0OBj58cgbq1A8Y6Hvxw/Yjy23yjkaEW0RUGZRUvqBSgEbyk1PbvRUcqheSVLyd7Bty0vxd0++GQCJ0l/nrs1o37YJI25+Cof/8W78mtCUeYbtuTZOnqSdfvuoJTLmgdNJWt2ClOBVPp4dd49he5+7VzX5a2tNmG2/qywp6W1rTdjVxMROJCG205dZUx5LSFw8a8pjnvZJCaNkxdL3Pav/jTXcXspaa8KmrnNz/Xd89A7bcfcYtn3BmWz7gjPZjh+PSWhf7d00+76KGCc4ExlnP/DAA+KpscMIh8MPVFVVmbpGEn8Kx09D4djLMeic70QVxyPG4ejBvTj0wWsYeHq5Qz0WC3b8KJpWVuHQB+tROm8FBgwZjkP169Gz/x8YeOZEfPmH/0DeiHEYMHQ4Ondtxr7f3YviGQtxUsVsAMCAocORN2Ic2ra+glOv/vf4jiwd72pqED4XkhtP+6EuAMDhwz14/6+fYdgphRh1Ol9Ag0iMOj2EYacU4uNdLThypAeFg3KRnZ2Ft2r/juoNOzDopDxP+iWH9Dz2PH0H+roPY/8rS9ST+vb1Ij/mMZI3IhpSLH+Ohz54DUMvu8mV56wGqa+l81ag/6BiHN71DkBZKDpvNgYMHR7vU1/3YbRufAKhy+dhwJDh8fENnvQv8XmQ1rUaRIzzP/7jP7584IEHwkbnEUvBHJbl5eWsvr7e1DVSnDqABBFh/ytL0NPyKUpvDccnPBXi15vX3otD77+KoZfdGFeCA1ExTjIcuG0I0MPca8KqovGw4kKseVF9E7QTw24GaolCcnL6+aYee8v6pXFxtXjmQgCplXdB+e4Nueg6HHhzNQadOwMl10YzUErEVVqzheOnoah8pur4ADg2diJ6n0VDwHWR1jpOOeQTKleq97R8mhBt76f0/HoouXYJ+heGVI09AHyXQ9OsG4+SmEnuQgCEEzOrFnY3oKWz83MCDCVClbdpGrw6Y4ZHtd+Kymcm3Ecan/JeXryzae/HqQa5Un3oZTeitCpsy+ndzRBIuf+bllO66MzgImA2QsgNR3MJbvlmmoWVchN+hZ7h1WxSFD8EqmQk4VQSFgC2nN7dDoFEVrbmC+XXl81shJCbxMxK2KcbiNSGEwI2JH9NqYheKkEv8YmVpCheB6pkHOFUIyxNK6uSIhvMwGgHtMqRat0Xfb3Cdm+3YDZCyE1iJrpYmyiEplQhUr0sKbIrNMWcYZQXXodmmoHnUhWP6d1vH6vuSIwl12qW3CT2Pndv/LtVVw2tmtF23STM1KJOF5hxFxLh5uSFqxRPm2660PnBbYkHTvYTnO5IGWNV14JZ66TW+W3169C+bZOmJVtpNeTVyVi9Lh3AY1X3u0VcC2b6rWZVdwpOrjdRngBOehTwWtUznnCahZpFTx57q2b1k8D7Ami5TknEOZOIpxGsuDk5DR6Cz9tvM246otyQnCLUetZwv6znwB3JIajFCA86Z2bCYlYLVTQTCiYZhQrHT0sKSSsqv8q1EEi3/CjtwG8WcV43Kp5+q7nwNIWrcOj99fEEGcjKTkjuYtfX0cmQxfyySSgcPy0esy73P45Ur7BE+L3yZ80445AIKC16Jdf+UtcqaNbSLRHe9m2v4/A/3k14eYysjWagZwxwI2GvCPjNIs7rRsXTbzVDX2lVGIPOnYE9T9+B7i8+wv6XFsct73JvDiueHm54ZBSVzwQ7fiwpbaFVDxSvUuYFHKcFmN2VrSQPcSPLizI1mfzFWXM/v1O4F5yp1KaauGvWIi6y/7wcME+WJT0n95ZY8ENRxXcTMgjJ15nZFGtuJbmhfv0BRN2tDry5OiFqzyzUJEA3RP+AcOpATQyI1K5Cy6tLMfL2ZzRTfilhJcrDjSwveosu0qrOZSgJgJsRPlptyqFXEliNQAIQ2n/erEt2siwp10bB6EtVN1izm6/T0UjxmPWqcDzrkQi4lUpOjkBU14GaGNDy6lLN1Gui4KYTu5YjMa8I7GaEj16bwAnDihbRVFM9rHi8Tmj/zfiETp46GmterMKGt3+i2W8llGsjVDkfbVteRlHFHN9HkEkcLXCiFAtl90dbfVJtR1PwYpwBx6kDNY5M4jSV54nc5XjzMYqAFmfLm7DXC+OMlTa1CLxW0Ter/Xc6X6dcnJYntEZfL06aeI1mjks/JOU2ilm30i+vko8HhNMAXogBcqKoXAQikxnoLbrJU6PtGREAt0pJ2G3TLCG003+7SZb1IF8bSp0kgIQN1o9JuUXrUb1KPh74cRpAhEOwHZcJvzske+GAbqVNLb/JwkG56Ok+7okDfSq4e2UaAj9OARAlBuhZr43gJMcrwhjgRikJEW1qqR5uXTBZ915OETc7RrWA4HoPIRwnEU0D8DiAbAC/YYwtUfx+A4BHAOyLHXqSMfab2G9zAdwfO/4wY2yNUXtucZwinWtFhlxKyYrddvp1AnaJgJnrzbblJDdtNeIpVUNM9eCnpMyucZxElA3gKQBTATQD2EpE6xhjSk/pFxhjdyquHQzg5wDKATAA78eu/afdfomASPcMK5yjFscbqpzveSJXEbDrymT2erO6RycTHFs1qvk56bJV2JHIvIIId6QKAB8zxj5ljB0F8DyAWZzXXgGghjH2VYxY1gCYJqBProInHZeay4TRdVqKbymtnJeJXEXAriuT065QTnoMWI148luIqQhopU/083oWQThPA7BX9r05dkyJ7xFRAxH9gYikaku814KIqoionojqI5GIgG6Lg1HYV+euzfh8+U0IVc5P8MvsObgXny+/SfM6vQSvWv6XqQS7REDrvNaWdsy9JowrL3kUc68JWw4TdTKc02oOUL+FmIpCqq1nEYSTVI4pFafrAYxkjI0DUAtA0mPyXBs9yFiYMVbOGCsPhbytPqiE0Y7Z1dSA4hkL40lpo8RvPtr++j8onrHQ0k5rxem3rqZRCEERBbtEQO88ETH2TiY4NpvY2Y0+eQm/OesbQYRVvRmAvF5nCYAv5Ccwxg7Kvq4C8CvZtZcqrn1LQJ+EwIzSWk+HKZ2bN3yMqjN9X1e7Kd1n89r7oqWBq8IJWXMGnTMTJdf+UvUaL0IjjcDrZC+H3MCTX5CLfv2zcPxYn247VnWASut9fkEuiIBHH96ANeG/2LZmW/H39MKLwWl45cRuByII51YAZxDRKESt5j8A8CP5CUR0CmPsy9jXqwD8Pfb3JgC/IKKTYt8rAdwnoE9CYEZpzRNbrkZcrcWks2S+nMX/UYUfjQpmiYCS+He0d4M4ZSY7kUCTp4721cZjRHBTzV3JKyd2O7BNOBljx4noTkSJYDaAZxhjO4noQUTT0K8DsICIrgJwHMBXAG6IXfsVET2EKPEFgAcZY1/Z7ZMo8GZe4d0xlURCHHfSAAAgAElEQVTyyN4dOPLx1nh2mIFnTkTTyqqEetNqKLl2CYrKr0rol1GGGb8aFcxwXWrEn+kzm3Hwiv9aRMePG48a/ETgeZFKpY4lCHGAZ4xtALBBcezfZX/fBw1OkjH2DIBnRPRDBJTieX7ZJM3sMxKMdsxI9YqEhLP5ZZOQlVeI/a/8AtQvN/FmFP9HF6Yz37gcGukE12OVyPPqAPWIjl83HiW0CPyKx+t8SzhTEUF2JAWUFvJI7SrN7DMSjMqb5pWOQ8urSxMSzkaql+Hk2f8bRefNTjAOlVaFNfWUcphVprtpVHAqCbJVIp+Ty8cf6HGVqWLN1iLkHe3dnhsD0wkB4VRALp7vffbfsP+lxTh5ziIMv+Exy+nd8ssmYeTtzyBSvSzBeh6aMg8l1/7StBuGlbRzVq24VqBFgJYu3mjr5VUj/v36ZyE7O5FDz84m9Ot/Ymm3H+rmItx6XGWqWLP1CLmTqf4yDRkZq25kLZeLwUUV30VoyjwA9pTWWqK1FeOQVWW6k1l75NAiQH19zJa+TcuYpDzW3X0U7Ye6E67l0UfqqTOctmaLUm2cN3EUXntlm+pvflMrpDIyknAaWcvVxGA5kbKaN1BJIAFrORP9qkyXXn699AdKAmaWYGgRf/mxKy95VPVaI8Jh5B7l1MYj0qCz9d3PNH+TuFGrRNpPMeVeIyNFdT2HdSeyr2vds61+nSbnmGqQ6zWNIBGwJ39dg0ce2uCaLtRIH+mmOkMOkaGjepvD3KoLbemfvSqM5kdkJMcJaIvOTviUmbmnHzhHK9AqZ6GG0LBC1NU0qoqUIlx8rDjWS3BLnSGHSIu9lrqhoDAXk6eOxtxrwpbdqnjd8zIBac1x6iXR0LJKG1nIrcCJe/oNvC+5RMD0uCm7ujivOEerEGmxP2/iKNXjF08uA2CfSIuMKedJjuNXpDXh1BItkJXtWjG0TIHWS144KFeVgOm9qCJcfKwUQhMFszkBRFrstXSc0nG7RFpkTHkqi/5pLaorRYvIG6tQPGNhPC2bpNOUF0fLRLFDBPQyrKsRLS2RUrpXqsKKoUekxd6Io7SjxhAdU57Kon9aE04gUbQoqvhuQvSOciGkwgPzK8y+/GovMAB8Z/Z434rUPLAamilKt2oUIWaHSDuh/3eyNIyTSPtibcrSE6HK+YhUL0u5HS4dkWrJKHhw5SWPqrpjEQEb3v6J4+3bKa3hxfNwshihFQTF2qAtWmjFngd+au7CCwu20/CiXLIcVjlKL5KDSAm+i2csRGjKvITSMOjrNfXOuf3upjXhVBMtQpXz0fLqUtVInVSsfRLAGpzirszoEJ3qA++GJG+fiNDXl8gqO539SUrw3fLqUgBAaMq8+PsZTfy9gpvouf3upr2oLoeSA1V+l5/jpuig9QKloyjrFvTmzulKkTzPzetqlWrtq8ENFUOkdhX2v7QYRRWz0dH4NvJGTkhItwjwcY8i3t1AVFcBj3LbbWX1k7+uSXAEl0Skxu37ULtxZ0rlVfQLjMROp3Nr8nB8eolQpHs4Cd6ABTdUDKEp89DdvBNtW17G106vwJGPtyZkVuTlHt18d9Paj1OJUOVt6GpqSPA9yy+bhLzScXGnWzdrn+hFz2xc1+BoBcd0hlEIox9yaxolQnE6BRzPWN3K/tS5azM6Gt+OEs1PtqBw3FSUVoVN1+Jy893NKMIJ6DvdOhGnrgc9IqjUN0kw+3L7rUCbGzAijH7IranXlhsbpFb7WVkkNNrKKDpIeudClfPR0/IxiirmoG3Ly+jau9NUhJLb727GEU69BB96orwT0COCWVnqWeDNvNxOJRT2O4wIox9ya6r1QQ6nuV+tOVi4aLrQaCuj6KCo3nJ+3L96+A2P4eQ5i7B/3X/iwFurublHt9/djDIOyV0WWtYvjTnFz0HuqWWeuBvNvSasGT3zndnjE3ScAJ/xwMhSCkS5iTUvVtkfgA+gZogBYGh48YPhra6mEUsXb/TsGbk1B0ZGG6UrUeeuzQlVW9WMuE6B1ziUUYRTLhZEqpehYPSlaNvyMk6esyierNhNaFk2vzN7PO68e6rphe0nS6kb0LNMA+6V0DV6Tl5a+P0CiVEZNn0Bimcu1D3XS3/qgHBqQOn6IBcTvIhYELnr63GwctjlZvzArQHa43WTozYifDyE0S/z6RT8Fh2kh8AdSQt9vSiqmI22LS9j2PQFCE2Zh7zhY7jjbUXvhiKjZ9ywlPqp/KxV67iWeG+FeBm5NvG4PqVjBJUE0YlB/IKMI5x5peMQqV2ZFDnE+xD9HF2kFe6XlUVgjAnhZrysL64kePkFueho7046T8+Apkb4f/2LjaAswvFjffFjvJuBEfH2g+uTl3AiMYgfIIRwEtE0AI8DyAbwG8bYEsXvdwO4BcBxABEANzHGmmK/9QLYHjt1D2PsKhF9UoOI3c/PqbC0wv1E6stEEgIzIqoawVNWtwSMOWo1wt/by4Bea+GGRrHpXseuew2/1seyC9vuSESUDeApANMBjAbwQyJSrra/AShnjI0D8AcA/yn7rYsxNiH2cYxoAuJcFkRmwRYJNzKfi/KBNOsqpUnwFJgyfYzqeCV/Vh4dsASezcDItckPrk8BxEMEx1kB4GPG2KcAQETPA5gFIP4GMMbelJ3/HoDrBLRrGqJ2Pyslfd2C0/oyO4lw5TAr8vNytGoZ0Hm9DZTg2QyMshE5XVY4gDcQQThPA7BX9r0ZwPk6598MYKPsey4R1SMqxi9hjL2idhERVQGoAoARI0bY6rAdpKuymxeiCIEZkb+uphFEUT2tlfuueLxOl2hmZ1OCjhMwtxkYbVbpbPzJVIggnGohLqornIiuA1AO4BLZ4RGMsS+I6OsA6ohoO2Psk6QbMhYGEAai7kj2u80HpRVdinSQlNvpouw2AxGEgFf3J3GLWiGoPNerGZAkDCu2ZlVPdxeiAPoQQTibAQyXfS8B8IXyJCKaAmARgEsYYz3SccbYF7H/PyWitwB8C0AS4fQKSiu6/LuEdFB2uw1ekV8riw9R1FtArufUul4LSn9PXsLnhUtWQKj9BRGEcyuAM4hoFIB9AH4A4EfyE4joWwBWApjGGGuVHT8JwBHGWA8RDQVwARINR57Dz1b0VIZc5G9taUdWFiUkt5B+19Nt3v2/pxsSE73rrRpoRLtk8UQe+cV31irSjfDbJpyMseNEdCeATYi6Iz3DGNtJRA8CqGeMrQPwCIB8AC8SEXDC7egsACuJqA9RC/8SxpjvMlCkakEpv0N6cfSIgp5Iz6My0Lq+cFCu5RdXtEuWEVH00ndWBNKB8CshJDsSY2wDY+xMxtjpjLHFsWP/HiOaYIxNYYwVK92OGGObGWNjGWPjY/8/LaI/ouFmnr9Mg1HuTLvuPFrX37pgsuU+5xfkqh634ptpNH7AOyd6USkJecaYasi4yCGzyHQrutMwIgp2rfiTp45G4/Z92LiuAX19DFlZpOnrqYRWaGZX19Gkc7OzyZLoz0MUtbhmIkJdTaMjXJtILlFrjKN730TnrrNTsjhiQDgN4HTImBXdTzrpi3is63as+HU1jajduDNule/rY6jduBOjx55mKtOURDhycvsluC1JGJifY6mPPOPXqkEvZYsH7Iu8yjXV1XWUSz3Asxa1xtiRO8q34ctGyLhExmYRqrwtiUBGq2Xa3xGtJBpOt+TETkfWWBUTta5rP6Tu2qTn8qQHnvFPnjoaU6aPUb1ehMirtqa0xiPnHnnXotYYJ994nWZScb8jIJwewspLnW76IqfDRI1EYS09nln9odXYc57xS1yzFlpb2m3pIXkLtwGJ4+Rdi3pjtBO+bFSWw0kEorqHsKL0T8dsO2ZEcbNqCj1RWE+Pp3VdQWEujvYctx1yKofR+HkIm5zjk+7JC961oxynmbWoNUY74cteZioLOE4PYSVhhh8KjXkFK2oKPVFYj2PSuu62uyY7nkhFCTObohXpQ2vtFA7K1R2n3bVot8CaXv0wpxFwnB7CSsIMUUk2UhFW/Bn1rPKPPrxB9ZpIazt38g43oMX9asGs9KG1pm5dMFl3nHbXogjDq1c+1gHh9BBWXG14rkknq7scVtUUWmKikUXbL8k5tAhUTm4/VWOVWenDqsuXXVcxEdnKvMpUlnE1h9Id6Vz8S3SNIbNz5eWGpGz7vImj8Ke6XUnW73R51jxQ+liLqIbJW3Moo3ScXlrh3EK6Wd3lEO26ZMai77Ub2OSpo7HmxSpsePsnmFt1IWo37kwimoWDcjOGaALu11KXI6NEddFWOD+KxH63uktzJiX26Otj8dRuTouGWvcUUZTNDsyuIy0re27uAM/Xn5vwsixHRhFOkZmO/Jq4wM81bpRzJkXzmJk7r/SOTm1IVtaR3zfHTEBGieqAuHpBfhWJ/VzjRs8f0Q9zpwen3MC01tHKJ+ri35VO+gWF4pKMBLCGjCOcojId+XXXd6Ngm1UYzY3Xc6cHpzYkrTG3H+pGXU2jqm71cGcP+vVPfHX9sjlmCjJKVBeZ6cjPIrGaOOsHfayRP6If5k6C2nwt+Gml8DnUmxOJA1er7lkwMAd5gwf4Sr+uBT+sPdHIKHckZf0gwHoaq1Ry+/FLX/WqTfpp7p78dQ1ee2VbwjGn+ldX04hHHlJ3xKdYNS+1V5QI2PD2T4T2xQn4Ze3xInBHUoEy01FdTSPuuH8H5i7uNJ0gwc8isRJ+0cfK5wyI1gwC/DV3dTWNSUQTcG6+Jk8drauzTPUQW7+sPdHIKFFdDhFWcb9ElhhBhD5WlLjl9znTe6Gd0sHedtdkzdDFxu37VLlfJ/SZTojUemsvlUX4jCWcqV7HxQzs6mP96nrlBPSIo1NcnpZ/KgDVdHI8GezNEiWnnrHW2ssvyE3pNZVRorocfrWKOwG7FuFUFLes1svRI47nTRwlqntJkEcGrXmxKvpdw31r67uf6d7LSpST1jNeunijrVyfWmuPKNnopXTD8jMyluP0s1VcNOxG3PBsMl6JXVp1gaxyM1plKgBwldwQCaubuxYRXPF4neYz0rqnlSAFObTWnlZmKskNy+9cZ8YSTrspsZwkFDx1tq1ksrHaP6NNxitRXq8ukFU1jPT70sUb40TD7D1EwermrkUEO9q74/HtvGWY5bA6frW1J4XdqiEV1GUZK6rbsYo7mfDB6N5eJJvQElGl416J8mbrAvGqYSZPHQ0tNz03VTlzqy5McnTv1z/LcHPnlZqMyjCrQWv8ZlUjemNIBXWZEMJJRNOIaBcRfUxE96r8nkNEL8R+/ysRjZT9dl/s+C4iukJEf3ihplfigZOEwujeXhApLZ2adFxrobe2tNuqx20Eq3WBeF5yLeKTX5ArpNY4L5iC61V+VwMvEQQSyzDLGQnJVUwJtXmxspkbuWH5HbYJJxFlA3gKwHQAowH8kIiUFOhmAP9kjH0DwGMAfhW7djSAHwAYA2AagGWx+/kaThqWjO7thVHLqE29he4kR6zVbkFhrqYxzE5lxuxsQlfXUde4/TXhv6C3N5FQ9vYyw01STZoqHGRMpOSMxMJF07kNilY389vumuzbvApGEMFxVgD4mDH2KWPsKIDnAcxSnDMLwJrY338AcDkRUez484yxHsbYZwA+jt3Pt6iraQQR/26sdr0ex2Lk8CzCIdqsWGXUphGHI5ojlvqvpiMzqgtkpzLjwPycpJrqTnL7djZJpTR16wJzREotWEEaq3K92MnMnypBJEqIMA6dBmCv7HszgPO1zmGMHSeiQwCGxI6/p7j2NLVGiKgKQBUAjBgxQkC3zUPiVpRGA4Bvp+QxohgZrXiNWloGJCuGHKM25ZZTLYW/KI5YL2xTmddT+l+ai0cf3qAavqjVP6VR48pLHuW+VgREen7o+YrOvSasW1fJaL3Y6acZo6WfHOZFcJxq7JdyeWqdw3Nt9CBjYcZYOWOsPBQKmeyiGGj51WVlEddOycPtGO3CPLu0njhqRaziaVPicCQORQlReiutZyCVz1BLbiKfCy3w9M/t8EcnMt7LuVAAhmoLnvXiRipDrzPwKyGC42wGMFz2vQTAFxrnNBNRPwCDAHzFea3j4N3JtDgLxhjXzscr0hjtwka/6y12PUOOnv8cL2fgdBVOs2IhT01y3v657cLmRMZ7OXii53jm2+l+8vbVTYggnFsBnEFEowDsQ9TY8yPFOesAzAXwLoCrAdQxxhgRrQPwOyL6NYBTAZwBYIuAPnHDjOhqV3QSJXoZvYB6i13PX0+E76XTL5HZOdQTo4lgqn92xmZmnT356xpsXNeAvj6GrCzC9KvG4c67pxq2wdMHed951Cq88+10DgK/RfrZJpwxneWdADYByAbwDGNsJxE9CKCeMbYOwNMA/h8RfYwop/mD2LU7iej3ABoBHAdwB2Os126fjCBPLyftZKcPbEZJXgvePnCu5k6mxXGcN3FUXE9UUJgLxoDOju6kF0sEN6b2Aj625HWseLwu3mZ+QW5SIS/gBJHQ0hGK2sGdfInMzqHWi2+1MqbVsfFyTMqUdn19LP7dDvFUWzdakBNFpyUIXvgt0k9I5BBjbAOADYpj/y77uxvANRrXLgawWEQ/eCEv2hZpbcfpA5tx7fCNWLt3evwcLWMBkMhxnDdxFGo37owvLLnztZKrEMGNqb2Ax4/1oePYiYiQfv2zkJ1NCa4s0mKX2tLKAel352OzRg6/vPi8HNPGdeoVGjeua+AmnGoSCY/KAkieGzfEcB745TlKyMiQS3nRttkjR2NsTj3W7p2OTw6XxM/R2smUHMfca8K6C1K0HsYoLA6IEtLCQbnIzVXPEC655fhpBzcD5TPgEYO9fvF5OSY1jw2940pozYXeGh1WXKg7N05KELx6X788RwkZSTiBE0XbJm58Am8ePD+BaJrZyXg4NOkcuzHdZiyIHe3deGH9nZq/+20HtwMjMdhp/RsPeOdbKpmshFYkjxJac6F1X6sqCxEw+z744TlKSGvCqVcqI690XLxo2yVvPIvW7G/gg32DTe9kPMkRJK7CrmXQjKO1EefIu4P7yXdOC34zHKiBd76nXzVONQP99KvGad5b/oy0XK76+hhycvrZ3ihFrge/WcrNIK0Jp1yXmV82KV6sLVQ5P6lo24+evgP3/tx80TY9YwuQuDi1CKzyuNbi5CUEvC+E0Q6eKgmM/WY40ILWfCuf94Rzh6Phb81cVnW9gAA5pOAAHqInMnhCD6mw4WkhrQmnXJc55KLrcPDPz2HEzU+hq6khobKldF5XU4NpwqnkJPSs6jximN7i1CIQevpMO0gVjiCV1Q5qz/tQ2xEsXDSdWwrhMfqcN3EUl6irt/5Er4dU2fDUkNaEEzihy2zd+ASGTV+A/LJJqsRR6zgPeHUvPIp/vcWpRSBuXTA5fm2ktT0u0tslbrwcMi+cEvuNxGA/qxvsEiNe7swoazxPf0RziKm84aU94ezctTmuyzz45+cw8MyJlgmkXQwr1vYplKC3OPVccZwQqe0aKuRQ42QeeWgDGrfvE+LcrScGi5obt4uZ8YBHx27mflaCJ6xyiH6zlJtBWhNOSacp12XKv1uBnZeHZ4c1WpxqBELNJUqESG3XNUYOLZFSz7lbBKESJV66XcyMlxgZ6djN3k+vP05wiH6ylJtBWmeA19NlWoHdRAM8yTKsJEzQiz/nTR2nlmpOK2GH1nE96HE8r72yLal/opI6iBIvnUogbTdBhnJNFQ7KRXZ2okRg5n56/eFZv5kC0ioR4GeUl5ez+vp619vVygEp2hfOLKel1S81KFOvSe2pcRJTpo9JiIqSjlt5WYz6qJxDUXMt6j5XXvKoqqsPEbDh7Z9w30cNolUA8vvlF+SCKOrXy3tvP+uEnQYRvc8YKzc6L61FddFwy33CrPjCK64B2lE1atzU1nc/w4KfVgp5ieZWXagZ5in1Sw5Rcy1KvHTSAixCXBVZ7TNVxWc3kdaiumhovSSMwZX6M1pQilBGUIqYRgYpKYej5Adopd7O5Kmj8Z3Z43XPkd9PryyGmez1osRLN3JOWoWWWmPF43WeFNHLBAQcpwnocXZOOofziE5yLoFHdDebOkyEcUQyAKlFxgCJZWHV5rpf/ywc7uyJJ1Jxk4PyswVYS2LQkkCsSkjSOmxtaY97XKipfjIBAcdpAnLuRQ1O7OZWjCQ8VQ6VqcOMuClRxhE91yNlclwlp5iXNyCpeJlTHJSasUzOfZupiuo0rFb7NAP5OgROeFZ4nYndKwQcp0lI3IuWsUAqietl9I6cO9IqaGY2dZhI/a6WP6tRcly3av6kSqipBL2Isp7u40Lch/QilHp6jmPlE3W+5MadQkA4LULP8Vjki2angqCZyBkjcVakccSqwcatEL1UCTWVwBtRZoegGa239kPdplUogH4inlDlbab76RYCwmkRRpZsUS+aCGIhQscn0vnZqr7QrRC9VEs+YTSfIog9b4SSBN71r5WIZ8TNT9ntsqMICKdFGInDgJgXzS/xvKKNI1aIuVsGmlRMPuG0C5GRO5kaeNa/ViIer8KieZHRhNOumCAtVi0rtigfP8Af1lw/+Pe50Qe/bFaiYcexffLU0Wjcvk/TI0INBYW5XOepJeLxOzKacIoSE5x+0fxAsDIJftqsREHP4AXwjfXOu6eaIpy8QYl+SsTDi4wmnKLEhHR80eyirqYRK5+oixsMCgpzcdtdk1NmTtJts9IyeK18oi7B8i7SsNnZkVxpNekcBxLxuIGMJpyAeTFBS7wfyxqw5sUT4r3kB5iJhLSuphGPLXkdx4/1xY91tHfj17/YCMCfLj3pDi19o7wqqwQ9w45WqkE15BfkGr4DIpOKA+5Z6W05wBPRYCKqIaLdsf9PUjlnAhG9S0Q7iaiBiP5F9tuzRPQZEX0Y+0yw0x8rUIoJnbs2654viffSedKOmVd6oiaMmtP6Iw9twPSLzYcqpiLWhP+SQDQl9PYybmd1NQd0vyEV+ijBrL5di9Dq1T6SIzub0NV11DBwI1R5WxKBzC+bZJnI8byfImA3cuheAG8wxs4A8EbsuxJHAFzPGBsDYBqA/yKiItnv9zDGJsQ+H9rsjynIxYTimQvjYrse8ZSL9y3rl6qKFXrOwpkQaaFnTeWxtIpKKeckRPbRDQKsFR2mZcDRIrR33j0VE84dbtjewPycpM3TjTh5nvdTBOwSzlkA1sT+XgNgtvIExtg/GGO7Y39/AaAVQMhmu0JgNV+nXLwfctF1SQ/FiDike6IFPe6Gh/NxKvelSIjqo1ubhFayk9vummw6eckXzYd02xpWXIiOdnX9plUXvUj1iiSGpnPXZkSqVySda/R+ioBdwlnMGPsSAGL/D9M7mYgqAAwA8Ins8OKYCP8YEeXY7I8pWBUTjMR7HuLgV2dqEZhbdWFSMl0gKr7xeBqkggO63xMkq0Et1t5K9ii9MUpEV+sdsOqipyaCf778JiArO+G8zl2b0bz2PlPqNyswNA4RUS2Ak1V+WmSmISI6BcD/AzCXMSbx8PcB2I8oMQ0D+BmABzWurwJQBQAjRoww07RQ8FgBefJj+tmZ2ghW/QGnzRzHdZ7XDujy8RUU5kYzDXVHn2XhoFzcumCysD76YZMw60GgNfasLEoguiJd9NQ8YIpnLESkehnyho+JuxM2rawCCCitCjtqpTfkOBljUxhjZ6t8/gigJUYQJcLYqnYPIioE8BqA+xlj78nu/SWLogfAagAVOv0IM8bKGWPloZB3kj6PeG+URSmVnal5RMs14b8kZTEC+CstqunjsrMJ3d1HHTfEKMfXfqg7TjSB6PfHlryO8yaOEpKf0yxn5geDlJa+VF7S2IkyG0oRPDRlXpI+c9C5M+JEU7rGTrkcLdh1R1oHYC6AJbH//6g8gYgGAHgZwG8ZYy8qfjuFMfYlERGi+tEdNvsjFGquDXml45JcJdRKC5tNspEq4EmAYZeLUvrF5hfkoqvrqKUkEmbBU6f8+LE+YdnxzQRP+CVrE6/fsmhfWC1Hebk7YfHMhUnX2Sn9rQW7hHMJgN8T0c0A9gC4BgCIqBzAbYyxWwB8H8DFAIYQ0Q2x626IWdDXElEIAAH4EICv0qEYRRbx+oylgzO1PImtGswmRjaCMjGz0tjgVLYiM2V03U6Q7KesTW6vaS0VWahyvidRR7YIJ2PsIIDLVY7XA7gl9vdzAJ7TuH6ynfadhlFkUapkdrHL8aoVc1NCmRhZj4sy2x839YC8WYCs6lu1xs7zPPygD/UKaiqyUOV8tLy6FCNvf8b1qKMgA7wB9Fwb1HzGCsdfkXQPLbcJNyDC3cVIfFVLjKyl37LSHysWWqu6QJ7s+f36Z1nSUdt9FqIt1akENQ8Y9PXGiSbgnD5TDRlFOM34gsl/13NtUBLWovKrXIlc4IUIdxc9jkZL6a9VZsJKf8wWSrNDoJREv3BQLnJyT7RdOCgXP753miUx1e6z8HPBOC8gOurIDDIqVt2saK3UqxxrP4CmcFWC1S5SuwqRN1Yl6Fj8lF9QhHinJb5aqSdvpT9mk6jY1QU6pb8TbTRLdUNjKiOjCKfZbEhKvUpR+Uwcen892urXIb9sEiK1q7D/pcU4ec4ihKbMS9Cx+CW/oAhDjci0eVb7oyRmeklU/KoL9Es2/wD2kVGiOmAuHEspCuSXTULprWG0b9uElvVL0fLq0jjRlH4fcfNTaKtfzxW5YEV1YBYixDuRPnki+mMkivtVFyhK1PaDL2emI6M4TsB+0lSl35hENOVo3/Y6V35BN6zyosQ7UZyOiP4YieJOJJYW4YtrZezKds+bOAq1G3cm+HI+8tAGrHyiDrcu8G++01QtyqYFYrxpmn2E8vJyVl9fb/o6pc5S+d3MPbREfbMLxOh+opEOzvhapZmJgA1v/wSA2HGquWPl5PSzHQljpV09uNEnqxDx7rkBInqfMVZueF4mEU67u55TD79l/VLdyAdRcIoAuE2MtWo8WTFW+bE9o3b14HSfeBzQ5BYAACAASURBVKD1nrXVr0P7tk2+MJpqgZdwZpSOM1R5G7qaGhL0ivllk5BXOo5Lr2g2DR2PDtNsImU7cCITD6/rj0i9nNtuOV4Zm6zc32sDGKCdTLio/CrH0725hYwinJHqFUBWdsJDjdSuwufLb+LyszTrN2aUjdpKImU7cIIA8BBj0TknnUggoQevjE1W7u+1AQzQTiYMwDUmwWlkFOGMcpbLEKqcjz1P34G9z/4Y+19ajOIZC03vflrc5GdP3RA/Li2gpnAVPn38R0livdVEynrQ4+ycIAA8xNgJTlfLwd4JeOV4rtXud2aPV83c7idneKX3CgAhTIIbnig8yCjCKRGmSPUy5BR/A21bXkJRxXdVLeNG0OIm88suSFoQ7PgxHN71TpJ4IjrywYizc4IA8BBjv/pV8sJtDteo3Tvvnorfv3on7vk/V7reJ14oVVBt9euFMAlu1RQyQkYZhyTsffbf0LblZXzt9Ar0tHxsWUmtZRGXHz/w5mqAgKGX3phwjhPuGTxGDNGGHB6Dk1fGlQDewGkLupOeKIFxSAOR2lVo2/IKiiq+i56Wj+NiuxV9i5Yzvfw46z2G0qpwknjixM7Jw9mJFnF5uLEgxjqz4IQKSg43agoZIWMc4CXDkDzaJ1K7Ci2vLkXxjIWW6jhrOdNLxweWXZCwWOQLKFR5m274pxWO1KuSE0bO8ZkSY50OPrJyWJWK1H4TmUzYbhCLCGQM4cwrHYfPl9+E4hkLEZoyL6ZQXobiGQuBvl7T4rFeYtVI9TJNMUW+gJRRSMpM82ajipyImJFglyh4GWPtBkEzk509VQisH/PN8tT8cgMZpeMUqRvR2o0jtWGEplRx7dJG/bHSXydeSq8iZ0TArb7z6nFTbS7djmwzgtOhm0HkkAbcitIxAq8C3Q/9TWXjjlt95wkDdbM/IuGHNegWAuOQCprX3osDb65OcMD1Kjs7jwK9c9dmRN5YhaKKOQkOw273OZXdidzqO6+PrBP9cTJbkpuRbamEjCGcnbs249D7rwKEeLLhppVVaApXeZKd3ciHU+JAi2csREfjW3Hrf6R2lab13akXyK9p2njgVt95PQdE90d0VJbcwVxag6HK+cjKGWgrss0vjuuikDGEs6upAaW3hlFaFcaep+/A4X+8CxAw6JyZvoyZlThSqXZ0pHoZCkZfgpZXl6rqmUS/QHL40Z2Id5Nwq+9GdZakvnZ1HUW//omvnZ3+iI7KkrvJRfWGUWOnpFe06lbkF8d1Ucg4HSeQmjoboz47rTvzkyXYrIHFy76r9TU7mzAwPwcd7d22+8OrW1VCz8giETmRBiHJHTBSvSx+31DlfEseLU6CV8eZMe5IEvzgA2YWPH12UpfnJ6IJmK8p5KUrlFpfe3sZcnMH4IX1d9q+v1XfXT1XIz03OauQ2isYfSlaNz6Booo5cbe9VIQtUZ2IBhNRDRHtjv1/ksZ5vUT0YeyzTnZ8FBH9NXb9C0Q0wE5/jKDMRlQ4fhqawlUJuhe53sUPehneDEpGujOr+k8nVQBWkUrGKqf7alUVoZXBSB7AIdIgJNVBl0Kd27a8jFDlfN8zLVqwq+O8F8AbjLEzALwR+66GLsbYhNjnKtnxXwF4LHb9PwHcbLM/ulArvgYGtNVHablS7+IHvQxv+JreC2SH+DmR2cguUslY5XRf7SQgUQtddCLVYaR6RbQabPUyFFXMxpFPtmDgNy9Ey6tLU9ZKb0vHSUS7AFzKGPuSiE4B8BZjrEzlvE7GWL7iGAGIADiZMXaciCYCeIAxdoVRu3Z1nHI44YTuFZQi9W2XfomxU6bijvt3xMW50wc2oySvBW8fOJdL/2lVh2YHRqoBp/WGIuFnh3e1tS3pOEU6mHfu2ozPl9+EovO/h/a/bUDB6EvRtuVlnHTRtcgZMjwjdZzFjLEvASBGPIdpnJdLRPUAjgNYwhh7BcAQAG2MMWlFNQM4zWZ/TMNIn+OEvscpKHV50otR0HkJWlGC0wc249rhG7F273QAfOKi2/HvPKGLytj3/IJcdHUdRfuhbs1r1NpxQ2/r1zh9M6GLduPM88smoXjGQux/aTGKKmajo/EtnDxnUUrrOA0JJxHVAjhZ5adFJtoZwRj7goi+DqCOiLYDUHtrNdlfIqoCUAUAI0aMMNG0PowML6loTJIgifX/+tjN2HxgDL49eDvW7p2OTw6XAOAjfjzx7yKJEK/hR75JzL0mjI72bsNr5P3ljSsXAT/WQleqgCTXI3myG6FVKPt6UVQxG21bXo5Xh80bPsZSch0/wFDHyRibwhg7W+XzRwAtMREdsf9bNe7xRez/TwG8BeBbAA4AKCIiiXiXAPhCpx9hxlg5Y6w8FAqZGKI2jPQ5bpe2cAL5ZZPAzpqFKcO24r2vxsaJJq/voJEOTbgDtgVjitlr/Ki3dRvKAIyeSBNaNz6eUNalKVyFnkiTkPbySseho/HtBIOTnaTdXsOucWgdgLmxv+cC+KPyBCI6iYhyYn8PBXABgEYWVa6+CeBqveudhJHhxem8gm6gc9dmFO7bhMNlP8SkoTtx+sBm09nC9XJ4iiZCVowpZq9JJau8W5AMpU3hKrSsX4qmcBXAYsdtwg4D4gfPFjXYJZxLAEwlot0Apsa+g4jKieg3sXPOAlBPRNsQJZRLGGMSO/IzAHcT0ceI6jyfttkfUzAKexRd2sJtyBfst+9agjE/fhrzx76Npx4+W5joKJoIWXGvMXtNKlnl3UJ+2SSU3hoGO34smoD7+DGU3hoWIkbbYUD84NmiBlvGIcbYQQCXqxyvB3BL7O/NAMZqXP8pgAo7fUgniE6ZpbdgeV4IHt1lQWFu3CijPG4FVowpZq85b+IovPbKtoRjXoeQpjPsJDaW+5sOueg6RN5YlVRcUagulhMZFznkNXhC3UQljlUuJKlt+XGtRcdrQNHyZrMTyWvFmMJ7TV1NI2o37kw6PmX6GFfDMNeE/4LWlnZkZRH6+hiGFXtrbZd0mpTdH6EpVTjw5mo0hatQWiWG67QDuWeLFHGUN3yMp8mVMybJh1+gJ3roRXM43bYSvPXSldZsCZ0d6sdFwko0lNq4AGDru5850cUkyI1pANDXF91hvI7IaqtfBzCg9NZofazSW8MJwSFeQu7ZIs8U5sQ7wouA43QZStFD6VTvpN+oUdtyGOkuJQKgBRH6Qj1VgVWXIq8NQ1qEG9B3oXIaOaGRCTpNSedp1hAqWt2k5W9aMPoST32rA47TA+hV6bMaJyy3Pkp/y62P0t+8FQKNDCh6BECEvtDIzcmqNd9rw5ARgfbKsi/KECramKOmpw9VzsehD1/3NLlyQDg9gBZxtOO2IV+weaXj0BSuQtPKaJJm+eLlJcxGlmq9F1xEOKERYbTKOXqZW7SuphHRSGNtpLplX7S6SUnQpSKLI29/xlPf6kBUFwgeMUUv1M2OFVwphoMBIODwP96Ni+QAuMPstCzVQDRSR8v4M6y4UIioaUQYrYaCuh0CKTcEGSFdLPtOqpvseoqIQkYmMnYKPAXYnK7SJ094DCAh+bHdttUSVsjRr38WrvjOWGx99zPbRMkoMbOfk2dIMJovOby2qotEKiXGUSKocukRvFw0UhaaQROmob2hFiBg6KU34sBbqzHonJkoufaXtu6vRcwkEAHZ/bJw/Fhf/JhVYsZDGP2WYFkJo/kCnM0ypQanN27e6q0iIXJMQZVLj8BrfBENeXG39oZasL5egAFZeYUAAw59sN62HshIf8gYEogmYD38kifPpF4oqB/Am33KTTgdieNFmLIX0UWBjlMwvMqmJF+wPS2f4ND7r6Jw3FS0vLoUI29/Jn6Onb5o6RWNYNVS7MesQmZgNF9e6DTNuKRZgZ0oIatwekxqCDhOgfAym5Lc+lhy7RIMvexGtG15CaHL58UXrl1RTM0izdW3FLcUWy07ojdfZhOtyGE38YVXUpEWRCTycHtMAccpEH6x+DnF9cot0mqcVHY2gbIoSceZypZiPUd7QN8675QF325ort9yzIoINXZ7TIFxKM3gpnJezTgD+C/buR1oGXgKCnNxtOe4pvHKacOVVSOk6PUhyjBjx6gqckyBVT1D4bTV1G14bTnXqrmkBcmtyA1XKbnrWfHMhVzXOB0SaYdoWRkPcGJM8npJ8sQ5ZsYWWNV14NfkqCKQ6jlE5fBDaWKz+tlIa7srGeZ5I8CUa11aB/K1bmd9iIoUslOSWFrzcuu6/LsT1vWMJJx+TY4aIBF+KHGhFaJZOEg932hoWKHjiUTMGCHdWOt2DTPy8WTlDIxnP5L3WcnUqDE/AFA4fpormZMyknA6nb4tgBh4nckI0PYnvXXBZM2Yd55EInakHjO+km6sdTvcIpA4nrzScYhUL4sXjtMi9FobQlH5TFes6xlrVXcynjaAGLhdmlgLev6kWvpXo8qgdizJZnwlJf2ffK1Lx52IFNLLf6AFeT+UPpl6JYvVfDcBuGJdz1jjkJuhkZHqFeiJNKGofGa8jc5dm9FWvw45oZEpqX90A6kQj64FHqOW02swUr0CyMpG68bHAQYMvexGRGpXAawXI+9YI6Qtp4yRvIYi+XlKom3FUMVrHMpIjlPELimBZ+HklY5D66Yncej99dHM2gA+f+oGICsrHtWjdl2mw+1MRiLBE/XktNQjpRdkx4+D+vXD0YN7wY51g/pbqwelBicihXh9MpXnHWuPuOZHnZE6TpHxtDzK9/yySSitCgMEfL7sRny+7EYgKwuUdWLf8tpA5VdPA7/Ho9uBXd2gEbqaGvC1r1eA+vVD7qlnoW3Lyxj4zQtQVDHbtyWueQ1faue1b9uUdD+nPErSmnBqEQMAwlx2eJXv+WWTMPTSG8GOdYMd60bo8nkovTXsGwNV4GngLtwIz436MP4NeaXjceSTLcg5tQyHP3oHOcWnm17rbm2svEyNF8lE5EhrwukWMeBxx+jctRkH3loN6p8L6p+LA2+uBgDfxAzzbABecKVW48T9DrMvvpW5zy+bhMJvXYnDH/0FOad+Ez1f/AMnXXQtItXLTBNoM++SnXXC64fstb+yLcJJRIOJqIaIdsf+P0nlnMuI6EPZp5uIZsd+e5aIPpP9NsFOf5Tg5QYj1SvQvPbehIfduWszmtfex/WwjUSuzl3R0qtgwMj5qzFy/uqo2L78Jhx4c3XCdV6KzEYbgNtcqR8c4J2C2Rffytx37tqMtr/+DwZ+8yL0fPERiipmo/1vG+KuPmZgxq0pE6QXuxznvQDeYIydAeCN2PcEMMbeZIxNYIxNADAZwBEA8vKI90i/M8Y+tNmfJPBwg3ml43Do/VfRFK5C565okbOmlVU49MF6w4fNI3J1NTVg0Dkz41UE88smYdj0u4C+Xgw6d0bCdcjK9mzRGW0Abvu/ajnAr3i8zpH2rMINrlg5958vvwmhyvlJRkn5BtvV1IDiGQvR3bwzVlr3bYQq5wN9vZbVUjwSUib4SdslnLMArIn9vQbAbIPzrwawkTF2xGa73OBRwOeXTYrXkY4bbwgorQobPmwekStUeRtKrv1l4r36oi4hJdcuSbgOfb2WFp0IrplH5+Zm+i4tR/eO9m7fcJ1ucsXyuR80YVqCyK22wUrO5PJnGqleZnkTNmPMcjvNm9uwSziLGWNfAkDs/2EG5/8AwH8rji0mogYieoyIcrQuJKIqIqonovpIJMLVOTMK+PyySRh62QnjzdBLb+R62FZ1LXrXWVl0drhmgF/n5rQlWA49R3cnwi6tcI5uhoXK517iHvU2WN5nyqMeMmvMcnOdeAFDwklEtUS0Q+Uzy0xDRHQKgLEA5D4D9wH4JoDzAAwG8DOt6xljYcZYOWOsPBQKcbVpRgHfuWszDrwpM968tdqzh21l0WlxzYPOmal6fyUXyrMBuGEJlkMvj6fosEurnCNPWKgIvbXa3Eeql6Fg9KWaGyzvps6jkzT7LnmV0NstGBJOxtgUxtjZKp8/AmiJEUSJMLbq3Or7AF5mjB2T3ftLFkUPgNUAKuwNJxG8C0fizkAy4w1DnHtzE3YWnRrXXFQ+U5jO1G0XkMlTR+sm0xAJq5wjT1y6CGOJ2tyHKufj0IcbbXN1PDpJM5KV165CbsBWyCURPQLgIGNsCRHdC2AwY+ynGue+B+A+xtibsmOnMMa+JCIC8BiAbsZYkoFJCbMhl0b5+iK1YfQvOhkAxcMioyGR6wEwV8Mi7YSwSRsA643uTdSvf9TxHkjZcq1uhV1q5d00qkLJ2z/R4ZXK6Dcr4YVKWM2H6RS8yC3rVj7OJQCmEtFuAFNj30FE5UT0G1lnRgIYDuBtxfVriWg7gO0AhgJ42GZ/VCHt+JLFOlK7KuF7aEoVSq5dksCd5ZdNQlH5TLRv2+SqG4VVnake1wz4x1/ULHiqXYoAD+dop3+ijSWiuTqvdJJ6agw/uzVlTJIPadILRl+Cti2voKjiu+hofCtph3Yz+YdIRBOJfI6i8qvi/ZVzze3bNqXcmHggKkO805ytn9eVE9yrqLbdnjdejhOMsZT7nHvuucwK9q97lDXcPoJ9/OjVrOH2EWz/ukd1z9P6PZXQ8dE7bOc9E1jHR++ofm/dtDz+t/ya1k3LXe+rWbxRvZPNmvIYm3bRI/HPrCmPsTeqd1q+3/VXr2TTL36EXX/1Ssv3UcLoGXgNt9eAsr2Oj95hO+4ewz75rx+qzoub7yOAesZBg9I65FIOSRQpqvgujnyyFUUVczSjfNLJjcJIpPOzOGQE0a5ATiUU8buxxO3wReWaAwB2/BgO73onSY3h1/cxI0R1iRiEKufHs0vL/zcSE/wkVjkBP4mRZkRvqwadAN5DvuYOvBkNQR566Y0J68+L9zEo1iaDtONLUTmhKfMSvks7v985A6fglygPM76UdTWNiDpjJMPtDPEBzEO+5ljvMZRWhZPc7/z8PmYExykC6VZ2Vw6/cJxaNcyHFRdizYtV8e9qhhwJqZIhXjRSbX1Kay73tLPQ1dQQz+Mg/eZVvwOOUzBSWReoBz9FefAWZ1PTbQJAVhZlJNEEjNennxJVy9fc1+/6XTwvrdQ/N9PDWUVAODmRrhlf/CQO8fpSahFYxlhGEk3AeH36ZeOPVK9AW/26pHencPw07jXnh00gIJwm4AddoN1Fo7xe2tnl13u142vVMFfGrFt1Vk936K1Pv2z8eaXjEkpcSAS8qHwm95rzwyYQEE4T8INrhN1F44dFpwXeKBxeAptp4Mmn6vXGL4KA+2ETCIxDnPDKVUlN6R+pXYWWV5cidPk8S8YcvxiDzEDppnTexFHY+u5nKVf9UgT01sTI259JWp9STgbgRM6CA2+txqBzZqLk2l96MgYRcfFOxNYHxiHB8EoXqMYhRqqXYdCEaZY5Bz9wHmag5qZUu3En5lZdmJbVL42gtiZaXl2K4hkLVddnXuk4NK2sQlO4CiNufgoDz5wIMODQB+s9kZpESG5eS39pzXGmmouGFpQcouS4b4VjjFSvALKyE663U07BDfC6KaUirK5Rs1JD89p7cej9VzH0shNO5gBcfxekfkjuR1JymkHnzohXQzCCk9JfwHEiNVw0ePog5xALRl+aVA7BlPtQVjb2v7QYocr5KJ65EKHK+dj/0mIgK9tS39wAr5sSkHpVMa3qnM1KDSXXLsHQy25MON8bIyAByrgFiv/DBT94gqQ14cwvm4TC8dPQFK5KUCID8E3aKp4+yMWSQx9uTCjSZXrR9PXi5DmLEKlehpb1SxGpXoaT5ywC+not9c0N8FrRU7EqplVDh1lR1WvRVkLJtb9EaVU4YbylVWFTulavSwMDSP/sSB0fvcO2Lzgznl1FK1PN/nWPepaxRq8PTmXW4c044+b8aGXp2fzkA1xZkK6/emXCOdLn+qtXOtZnUTCTAcjsmvBjdia/ZiBDkB3pBKhff1D/XERqw2haWZXE5osylChF20j1CkRqVyUVvVKKunp9cEIsMcN9uGlI0uJwx06ZyuWmZEak9xPMcoNm14QfRFs5/ML92gIPdfXbh5fjlO+s0g63fcGZqhydCI4qKddlTZg13F7KWmvCqr870QezfeTlVtziyO20l4ocp3z+WzctZ6014aTnkwq5UXnh9/ywCDhOWVYkIL7DUXZ/tNWvAyA+Tlupr5L0h5I+UU1/5XasuN+rFdrhcFPRMV7+PKQ66KHK+ehqavBVcIIWzBoQ0yY/LA919dvHrI5Ta4dzandT6m/09Dle77B6+PTJuXFuWUJrTZh9+uRcx9q0y+E6lcXdLfhB564Hrezte5+7N/5dlOTmxRyAk+P0nAha+ZghnG4TJuVDl0Qvv74IenDbqOBHI4YX8KvhhDH1Z7Tjx2PYjrvHCF3jvHMg+v0OCKcHsKrj9DP8YFX/9Mm5vuXKRcPqfDvFIKjdt7UmzLbfVZbQR5HE3swciN5sA8LpAZSLTFL2yxdvKr7wXnNAmcKJ2hmnU3Okdd89q+9KcvFzwsDKMw6R7QeEUwd+1iv6DXYXpai59rvuTwTszpVTc6SnfpLEdFEE2+ociNrcXSGcAK4BsBNAH4BynfOmAdgF4GMA98qOjwLwVwC7AbwAYABPu3YJZ6ZwMHYhYp5EzrXXnG8qwKk5ku67Z/W/JTy/vc/9jO348ZiE5+k2E5JyHCeAswCUAXhLi3ACyAbwCYCvAxgAYBuA0bHffg/gB7G/VwC4naddEaJ6JnAwduEnbjF4XsZwg+PcfldZkqeFl9JaSus4DQjnRACbZN/vi30IwAEA/dTO0/uI0nEGHIx7sDPXgYRgDLd1nH6Ze6+s6m44wJ8GYK/se3Ps2BAAbYyx44rjqiCiKiKqJ6L6SCRiu1NpEfaVIrA7134LGfQjnJojv8+9Zwk/jCgrgFoAO1Q+s2TnvAVtjvMaAL+Rff9XAP8XQAjAx7LjwwFs56H2gY4zdRDMdYBUAjg5zsT4NHXCOsUmbW6OEUUJJQC+QFRMLyKifizKdUrHHYfeLur3bOiphmCuA6QjhGSAJ6K3APyEMZaUlp2I+gH4B4DLAewDsBXAjxhjO4noRQD/wxh7nohWAGhgjC0zas+LmkMBAgRIf7iSAZ6IvktEzYgadl4jok2x46cS0QYAiHGTdwLYBODvAH7PGNsZu8XPANxNRB8jqvN82k5/AgQIEMANpHXNoQABAgQwg6DmUIAAAQI4hIBwBggQIIBJBIQzQIAAAUwiJXWcRBQB0GTikqGIuj95haB979rP5LFnevtW2i5ljIWMTkpJwmkWRFTPo/AN2k+/9jN57JnevpNtB6J6gAABAphEQDgDBAgQwCQyhXCGg/Yztv1MHnumt+9Y2xmh4wwQIEAAkcgUjjNAgAABhCEgnAECBAhgEmlDOInoGiLaSUR9RKTpgkBE04hoFxF9TET3yo6PIqK/EtFuInqBiAaYbH8wEdXErq8hopNUzrmMiD6UfbqJaHbst2eJ6DPZbxNEtx87r1fWxjoR4+cc+wQiejf2jBqI6F9kv1kau9azlP2eExvLx7GxjZT9dl/s+C4iuoJ3rCbbv5uIGmPjfYOISmW/qT4HgW3fQEQRWRu3yH6bG3tWu4lortm2Odt/TNb2P4ioTfabrbHH7vEMEbUS0Q6N34mInoj1r4GIzpH9Znv8Qkpn+OEDj+ofye79n4gVogNwL4BfGZw/GMBXAL4W+/4sgKttjJ+rfQCdGsctj5+nbQBnAjgj9vepAL4EUGR17HrPUnbOfAArYn//AMALsb9Hx87PQbRg4CcAsh1o/zLZ871dal/vOQhs+wYAT2qsu09j/58U+/sk0e0rzv9fAJ4RMXbZPS4GcA6AHRq/XwlgI6Iler4N4K+ixs+YO6UzXAFj7O+MsV0Gp1UgmnX+U8bYUQDPA5hFRARgMoA/xM5bA2C2yS7Mil3He/3VADYyxo6YbEdU+3EIGL9h24yxfzDGdsf+/gJAK6JVAKxC9Vnq9OsPAC6PjXUWgOcZYz2Msc8Qrb5aIbp9xtibsuf7HqLJukWAZ+xauAJADWPsK8bYPwHUIFqF1sn2fwjgv022oQvG2J8QZTy0MAvAb1kU7yGaNP0UiBl/+hBOTgipf6SBYsbYlwAQ+3+Ywfk/QPJiWhwTKx4johyH2s+laO2m9yQ1AeyP39TYiagCUU7lE9lhs2PXepaq58TGdgjRsfJcK6J9OW5GlAOSoPYcRLf9vdic/oGIpCoMro49pp4YBaBOdtjO2O32UcT4jUtn+AlEVPv/2zt70CiCKI7/nohaiGK08CwkBkQhCAqpbKJio4JgIykCommCaSxsJBZi41djY6OFhULAKKKiIGi0DKmUYCH50CIYI9jYHUGfxbwle+dmbzc7QYzvB8fOzc7sf/6ZvXe3+3I3wNaMXYOq+qTIITLqNKe+sH4B7fRxasAewo87J1wAvhICym3CjzxfXgb97ar6RUQ6gBERGQd+ZLRr8B/Z+z3glKr+suqW3rMO1WrMOW0KzXcE/dBQpBfoArpT1X/Mg6pOZfVfovYzYEhV6yLST/jkfajMuCvqJ/QAD1X1Z6quiveqY4zh/98KnPqX1z/K0xeRORGpqeqsBYdvOeM4CTxW1fnUsWetWBeRu8D55dC3y2RUdVrCkif7gEe08B9DW0Q2AM+Bi3b5VNh7BovNZVabGQlLuGwkXN4V6RtDHxE5THhz6VbVelK/yDwUDR4ttVX1e+rpHeBaqu+Bpr5vC+oW1k/RAww0ja2K96pjjOF/5SSHkgf5yaHVhJvBO1i4qd1p+4ZpTI6cLal7g8YEyfWctqPAwaa6mm0FuAlcja1PuBm+1spbgAkWkmNL9l9Qew3wGjiXsa+097y5TLUZoDE59MDKnTQmh6Ypnxwqop8EhJ1F5yGidi1VPgGMWrkN+GRj2GTlttjerd0u4DP2RZsY3puO387iyaFjNCaHxmL5V9WVEzjt5JgB6sAc8NLqtwEvUu2OEhaPmyJc4if1HcAYIVEwnExuFqTp2QAAAM5JREFUCf3NFhgmbNtm9V00Lo/cTli0blVT/xFgnLD08n1gfWx9YL9pvLdtXwz/BbV7gXngXeqxt4r3rLkkXOIft/I68zJp3jpSfQet30fgyBLPuVb6r+xcTPw+bTUPEbWvAB9M4w2wO9X3jP1NJoHTy+Hdnl+i6U0whnc7zhDhPzPmCa/7PqAf6Lf9Atyy8Y2T+jAVw79/5dJxHKck/1tW3XEcpzIeOB3HcUrigdNxHKckHjgdx3FK4oHTcRynJB44HcdxSuKB03EcpyS/AX4AOD/USWAbAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "## This code below generates two x values and a y value according to different patterns\n", "## It also creates a \"bias\" term (a vector of 1s)\n", "## The goal is then to learn the mapping from x to y using a neural network via back-propagation\n", "\n", "num_obs = 500\n", "x_mat_1 = np.random.uniform(-1,1,size = (num_obs,2))\n", "x_mat_bias = np.ones((num_obs,1))\n", "x_mat_full = np.concatenate( (x_mat_1,x_mat_bias), axis=1)\n", "\n", "# PICK ONE PATTERN BELOW and comment out the rest.\n", "\n", "# # Circle pattern\n", "# y = (np.sqrt(x_mat_full[:,0]**2 + x_mat_full[:,1]**2)<.75).astype(int)\n", "\n", "# # Diamond Pattern\n", "y = ((np.abs(x_mat_full[:,0]) + np.abs(x_mat_full[:,1]))<1).astype(int)\n", "\n", "# # Centered square\n", "# y = ((np.maximum(np.abs(x_mat_full[:,0]), np.abs(x_mat_full[:,1])))<.5).astype(int)\n", "\n", "# # Thick Right Angle pattern\n", "# y = (((np.maximum((x_mat_full[:,0]), (x_mat_full[:,1])))<.5) & ((np.maximum((x_mat_full[:,0]), (x_mat_full[:,1])))>-.5)).astype(int)\n", "\n", "# # Thin right angle pattern\n", "# y = (((np.maximum((x_mat_full[:,0]), (x_mat_full[:,1])))<.5) & ((np.maximum((x_mat_full[:,0]), (x_mat_full[:,1])))>0)).astype(int)\n", "\n", "\n", "print('shape of x_mat_full is {}'.format(x_mat_full.shape))\n", "print('shape of y is {}'.format(y.shape))\n", "\n", "fig, ax = plt.subplots(figsize=(5, 5))\n", "ax.plot(x_mat_full[y==1, 0],x_mat_full[y==1, 1], 'ro', label='class 1', color='darkslateblue')\n", "ax.plot(x_mat_full[y==0, 0],x_mat_full[y==0, 1], 'bx', label='class 0', color='chocolate')\n", "# ax.grid(True)\n", "ax.legend(loc='best')\n", "ax.axis('equal');" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Here are some helper functions" ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [], "source": [ "def sigmoid(x):\n", " \"\"\"\n", " Sigmoid function\n", " \"\"\"\n", " return 1.0 / (1.0 + np.exp(-x))\n", "\n", "\n", "def loss_fn(y_true, y_pred, eps=1e-16):\n", " \"\"\"\n", " Loss function we would like to optimize (minimize)\n", " We are using Logarithmic Loss\n", " http://scikit-learn.org/stable/modules/model_evaluation.html#log-loss\n", " \"\"\"\n", " y_pred = np.maximum(y_pred,eps)\n", " y_pred = np.minimum(y_pred,(1-eps))\n", " return -(np.sum(y_true * np.log(y_pred)) + np.sum((1-y_true)*np.log(1-y_pred)))/len(y_true)\n", "\n", "\n", "def forward_pass(W1, W2):\n", " \"\"\"\n", " Does a forward computation of the neural network\n", " Takes the input `x_mat` (global variable) and produces the output `y_pred`\n", " Also produces the gradient of the log loss function\n", " \"\"\"\n", " global x_mat\n", " global y\n", " global num_\n", " # First, compute the new predictions `y_pred`\n", " z_2 = np.dot(x_mat, W_1)\n", " a_2 = sigmoid(z_2)\n", " z_3 = np.dot(a_2, W_2)\n", " y_pred = sigmoid(z_3).reshape((len(x_mat),))\n", " \n", " # Now compute the gradient\n", " J_z_3_grad = -y + y_pred\n", " a_3_z_3_grad = sigmoid(z_3)*(1-sigmoid(z_3)) # missing gradient da/dz\n", " # J_W_2_grad = np.dot(J_z_3_grad, a_2)\n", " J_W_2_grad = np.dot((J_z_3_grad*a_3_z_3_grad), a_2) # gradient to adjust W2\n", " \n", " a_2_z_2_grad = sigmoid(z_2)*(1-sigmoid(z_2))\n", " J_W_1_grad = (np.dot((J_z_3_grad).reshape(-1,1), W_2.reshape(-1,1).T)*a_2_z_2_grad).T.dot(x_mat).T\n", " gradient = (J_W_1_grad, J_W_2_grad)\n", " \n", " # return\n", " return y_pred, gradient\n", "\n", "\n", "def plot_loss_accuracy(loss_vals, accuracies):\n", " fig = plt.figure(figsize=(16, 8))\n", " fig.suptitle('Log Loss and Accuracy over iterations')\n", " \n", " ax = fig.add_subplot(1, 2, 1)\n", " ax.plot(loss_vals)\n", " ax.grid(True)\n", " ax.set(xlabel='iterations', title='Log Loss')\n", " \n", " ax = fig.add_subplot(1, 2, 2)\n", " ax.plot(accuracies)\n", " ax.grid(True)\n", " ax.set(xlabel='iterations', title='Accuracy');" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "Complete the pseudocode below" ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "iteration 0, log loss is 0.8310, accuracy is 0.482\n", "iteration 200, log loss is 0.5030, accuracy is 0.738\n", "iteration 400, log loss is 0.2648, accuracy is 0.962\n", "iteration 600, log loss is 0.2143, accuracy is 0.962\n", "iteration 800, log loss is 0.1929, accuracy is 0.958\n", "iteration 1000, log loss is 0.1795, accuracy is 0.958\n", "iteration 1200, log loss is 0.1697, accuracy is 0.962\n", "iteration 1400, log loss is 0.1618, accuracy is 0.964\n", "iteration 1600, log loss is 0.1552, accuracy is 0.966\n", "iteration 1800, log loss is 0.1495, accuracy is 0.97\n", "iteration 2000, log loss is 0.1446, accuracy is 0.97\n", "iteration 2200, log loss is 0.1402, accuracy is 0.972\n", "iteration 2400, log loss is 0.1364, accuracy is 0.972\n", "iteration 2600, log loss is 0.1329, accuracy is 0.974\n", "iteration 2800, log loss is 0.1297, accuracy is 0.974\n", "iteration 3000, log loss is 0.1268, accuracy is 0.976\n", "iteration 3200, log loss is 0.1242, accuracy is 0.976\n", "iteration 3400, log loss is 0.1217, accuracy is 0.978\n", "iteration 3600, log loss is 0.1195, accuracy is 0.978\n", "iteration 3800, log loss is 0.1174, accuracy is 0.978\n", "iteration 4000, log loss is 0.1154, accuracy is 0.978\n", "iteration 4200, log loss is 0.1136, accuracy is 0.978\n", "iteration 4400, log loss is 0.1119, accuracy is 0.978\n", "iteration 4600, log loss is 0.1103, accuracy is 0.978\n", "iteration 4800, log loss is 0.1088, accuracy is 0.978\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA6UAAAIZCAYAAABeXpn8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzs3XucXHdd//HXZ++by+6mSbttk7QpbaAtt6KxpYIYoGhBaPmJFyq3KlL1JygKYuHHD/khgqKAN1D7Q5RrS63Kr0ikXBcRC/RCi/RKWtom6S1prptkL7Pz/f1xzibTzW4ym53dkznzej4e++jOnLPnfOab6cy853s5kVJCkiRJkqQitBVdgCRJkiSpdRlKJUmSJEmFMZRKkiRJkgpjKJUkSZIkFcZQKkmSJEkqjKFUkiRJklQYQ6kk6ZgTEf8YEe8uuo5WFhG3RcT6As9/SkQMR0R7UTVIkhaGoVSSFkhE3BcRF8zzOd4ZEZ+cz3McKyJifUSkiHhL0bWUUUrpySmlIViY59XU/z9SSg+klJaklCbm87ySpOIZSiVJzeo1wPb8vwsqIjoW+pzzJTLz+nmgTO0lSWo8Q6kkHQMi4nURsTEitkfEtRFxcs22n4qIuyJiV0R8OCK+HhG/ehTnOCsihiJiZz4086KabS+KiNsjYk9EbImIN+f3r4iIf8v/ZntEfGOmABMRfxERmyJid0TcFBE/UbPtnRFxdUR8PD/HbRGxrmb7MyLi5nzbZ4CeIzyWRcDPAb8JrK09Vr792RHxX3ndmyLi0vz+3oh4f0Tcn7fnf+b3rY+IzVOOcaDnLq//moj4ZETsBi6NiHMj4vr8HA9FxF9HRFfN3z85Ir6Ut9sjEfG2iDgxIvZFxPKa/X40IrZGROc0j7M7Iv48Ih7Mf/48IrrzbXdExItr9u2IiG0R8SP57WfWtMGttUNx8+fBH0XEN4F9wBOmOfd9EXFBRFwIvA34xXw47a359v6I+Pv8sW+JiHdHPtQ2Ii6NiG9GxAcjYjvwzog4PSK+GhGP5XV+KiIG8v0/AZwCfC4/x1siYk1kPeEd+T4n5/9vbI/s/5XX1dR6pOfX7+c17ons/6XnH/KkkiQVxlAqSQWLiOcB7wV+ATgJuB+4Kt+2ArgGeCuwHLgL+PGjOEcn8Dngi8AJwBuAT0XEk/Jd/h74tZTSUuApwFfz+98EbAaOBwbJwkma4TQ3AOcAxwGfBv4pImrD5UX54xoArgX+Oq+tC/gs8In8b/8JeNkRHtLLgOF83+uAV9c81lOAfwf+Kq/7HOCWfPOfAT9K1obHAW8Bqkc416SLyf4tBoBPARPA7wArgPOB5wP/M69hKfBl4AvAycAZwFdSSg8DQ2T/1pNeCVyVUhqf5pz/C3hm/hieDpwLvD3fdiVwSc2+Pw1sSyndHBErgc8D784f55uBf46I42v2fxVwGbCU7Dk3rZTSF4D3AJ/Jh9M+Pd/0MaCSP7ZnAD8F1H5Zch5wL9nz7Y+AIHuenwycBawG3pmf41XAA8BL8nO8b5pSriR7Lp5M9oXEe6aEy5meX08CXg/8WP78/mngvpkeryRp4RlKJal4rwA+mlK6OaU0ShZAz4+INcCLgNtSSv+SUqoAfwk8fBTneCawBPjjlNJYSumrwL9xMNSMA2dHRF9KaUdK6eaa+08CTk0pjaeUvpFSmjaUppQ+mVJ6LKVUSSm9H+gGnlSzy3+mlDbkcwQ/QRayJmvrBP48P8c1ZAH3cF5DFpImyALwJTU9ja8AvpxSujI/3mMppVsi6+H9FeC3U0pbUkoTKaX/ytu8HtenlD6bUqqmlPanlG5KKX0rf7z3AX8H/GS+74uBh1NK708pjaSU9qSUvp1v+xhZECXvWbwkb4/pvAJ4V0rp0ZTSVuD/kIVJ8sd9Ud5rDPBL+X3kx9+Qt3c1pfQl4Eay59Okf0wp3ZbXP10gnlFEDAIvBN6YUtqbUnoU+CDw8prdHkwp/VV+/P0ppY0ppS+llEbzx/KBmvY60vlWA88Gfj9vz1uAj9S0Bcz8/Jogey6eHRGdKaX7Ukr3zObxSpLml6FUkop3MjU9VSmlYeAxYGW+bVPNtkTWW3Q059iUUqrtFbw/PwdkPY8vAu6PbHjw+fn9fwpsBL4YEfdGxOUznSAi3pQPKd0VETuBfrJexEm1YXof0JMPzTwZ2DIl7M7Yc5cHlOeS9VYC/D+y4b4/k99eDUwXOlbk+x1tINlUeyMinhjZ0OaH8yG97+Hg452phsl6z46IJwAvAHallL4zw76Pe27kv58MkFLaCNwBvCQPphdxMJSeCvx8PnR3Z/7v8WyyLximfTyzdCrZFwkP1Rz/78h6Rac9fkScEBFX5cNodwOf5PHPj8M5GdieUtpTc1/t8xdmeH7l7fRGsl7ZR/MaTkaSdMwwlEpS8R4k+5APQEQsJhuquwV4CFhVsy1qb8/yHKvj8fNBT8nPQUrphpTSxWSh4rPA1fn9e1JKb0opPQF4CfC7083Hi2z+6O+TDUtdllIaAHaRDdk8koeAlfljq61tJq8ie//6XEQ8TDZEtIeDQ3g3AadP83fbgJEZtu0FJnscJ3swj5+yz9Qe4r8B7gTWppT6yIY2Tz6GmWogpTRC1r6vyB/LTL2kMOW5QdYuD9bcnhzCezFwex7AJs//iZTSQM3P4pTSHx/m8RzO1H03AaPAiprj96WUnnyYv3lvft/T8vZ6JY9/fhyungeB4/Jh0ZMOPH+PWHxKn04pPZusLRPwJ/X8nSRpYRhKJWlhdUZET81PB1nv1i9HxDn5IjbvAb6dDwn9PPDUiHhpvu9vAice4RxtU87RDXybLHi9JSI680VvXgJcFRFdEfGKiOjPh3HuJhvySES8OCLOyAPj5P3TXaJjKdn8wq1AR0S8A+irs02uz//2tyJbrOdnyeZOzuTVZMNYz6n5eRnwM5EtIPQp4IKI+IX8eMsj4py8l/ijwAfyRXPaI+L8vH3uJutZ+5l8GPDbyYZ8Hs5SsjYZjogzgd+o2fZvwIkR8cbIFitaGhHn1Wz/OHApWe/m4S61ciXw9og4Pp9f/I4p+19FNpfzNzjYS0q+z0si4qfzx9kT2WJOR/OFBsAjwJrJLzVSSg+RzU9+f0T0RURbZAsZHW447lKyecA78zmvvzfNOQ5ZcCk/3ybgv4D35o/lacBrOdhbPqOIeFJEPC//dx4B9jP9c1iSVBBDqSQtrA1kH4onf96ZUvoK8L+BfybrNTydfG5eSmkb8PPA+8iG9J5NNjfwcPMgL5lyjntSSmNkAeiFZD2GHwZenVK6M/+bVwH35cMqf518ziOwlmzBnmGy8PjhyWtXTnEd2eJCd5MNqxyhzuGheW0/SxbSdgC/CPzLdPtGxDOBNcCHUkoP1/xcSzbM+JKU0gNkQ5HfRHbJmFs4OL/wzcB/k81Z3U7WY9aWUtpFtkjRR8h63/Zy5GHSbyabx7kH+L/AZ2oe0x6yobkvIRtW+gOyIceT279JtsDSzfmXDzN5N9m/9/fyum/O75s8zkNk/y4/PuX8m8h6T99G9kXBJrIQeLTv+/+U//exiJicb/xqoAu4nezf7RoePzx4qv8D/AhZD/rnOfTf+L1kAXxn5Ks/T3EJ2b/9g8C/An+Qz5U9km7gj8me9w+TjQZ4Wx1/J0laIDHDehWSpGNQ3lO1GXhFSulrRdejoxcRXwU+nVL6SNG1SJJUJHtKJekYlw/BHMiHH07OW/xWwWVpDiLix8h6DT9zpH0lSSo7Q6kkHfvOJ1vJdRvZcNCXppT2F1uSjlZEfIxsSPQbp6wmK0lSS3L4riRJkiSpMPaUSpIkSZIKYyiVJEmSJBXGUCpJkiRJKoyhVJIkSZJUGEOpJEmSJKkwhlJJkiRJUmEMpZIkSZKkwhhKJUmSJEmFMZRKkiRJkgpjKJUkSZIkFcZQKkmSJEkqjKFUkiRJklQYQ6kkSZIkqTCGUkmSJElSYQylkiRJkqTCGEolSZIkSYUxlEqSJEmSCmMolSRJkiQVxlAqSZIkSSqMoVSSJEmSVBhDqSRJkiSpMIZSSZIkSVJhDKWSJEmSpMIYSiVJkiRJhTGUSpIkSZIKYyiVJEmSJBXGUCpJkiRJKoyhVJIkSZJUGEOpJEmSJKkwhlJpDiLivoi4YJ7P8c6I+OR8nkOSpDKJiKGI2BER3UXXIunIDKWSJEkqjYhYA/wEkICLFvC8HQt1LqlsDKXSPImI10XExojYHhHXRsTJNdt+KiLuiohdEfHhiPh6RPzqUZzjrPzb4J0RcVtEXFSz7UURcXtE7ImILRHx5vz+FRHxb/nfbI+Ib0SErwWSpLJ4NfAt4B+B10zeGRG9EfH+iLg/f//9z4jozbc9OyL+K39v3BQRl+b3D9W+P0fEpRHxnzW3U0T8ZkT8APhBft9f5MfYHRE3RcRP1OzfHhFvi4h78vfnmyJidUR8KCLeX/sgIuJzEfHG+Wgg6VjjB1FpHkTE84D3Ar8AnATcD1yVb1sBXAO8FVgO3AX8+FGcoxP4HPBF4ATgDcCnIuJJ+S5/D/xaSmkp8BTgq/n9bwI2A8cDg8DbyL5NliSpDF4NfCr/+emIGMzv/zPgR8nec48D3gJUI+IU4N+BvyJ7bzwHuGUW53spcB5wdn77hvwYxwGfBv4pInrybb8LXAK8COgDfgXYB3wMuGTyS+L8s8LzgStn88ClZmUolebHK4CPppRuTimNkgXQ8/MhRS8Cbksp/UtKqQL8JfDwUZzjmcAS4I9TSmMppa8C/0b2ZgcwDpwdEX0ppR0ppZtr7j8JODWlNJ5S+kZKyVAqSWp6EfFs4FTg6pTSTcA9wC/lYe9XgN9OKW1JKU2klP4rf49+BfDllNKV+fviYyml2YTS96aUtqeU9gOklD6ZH6OSUno/0A1MfmH8q8DbU0p3pcyt+b7fAXaRBVGAlwNDKaVH5tgkUlMwlErz42Sy3lEAUkrDwGPAynzbppptiazn8mjOsSmlVK257/78HAAvIwvA9+fDg8/P7/9TYCPwxYi4NyIuP4pzS5J0LHoN8MWU0rb89qfz+1YAPWQhdarVM9xfr021NyLiTRFxRz5EeCfQn5//SOf6GPDK/PdXAp+YQ01SU3FCtjQ/HiT7phaAiFhMNlR3C/AQsKpmW9TenuU5VkdEW00wPQW4GyCldANwcT7M9/XA1cDqlNIesiG8b4qIJwNfi4gbUkpfOYoaJEk6JuTzQ38BaI+IyRFI3cAA2QihEeB04NYpf7oJOHeGw+4FFtXcPnGafQ6MNsrnj/4+WY/nbSmlakTsAKLmXKcD35/mOJ8Evh8RTwfOAj47Q01S6dhTKs1dZ0T01Px0kH0z+8sRcU6+HP17gG+nlO4DPg88NSJemu/7m0z/Jlerbco5uoFvk71ZviUiOiNiPfAS4KqI6IqIV0REf0ppHNgNTABExIsj4ow8DE/eP9HgNpEkaaG9lOz97GyyOZ3nkIW7b5DNM/0o8IGIODlfcOj8/P30U8AFEfELEdEREcsj4pz8mLcAPxsRiyLiDOC1R6hhKVABtgIdEfEOsrmjkz4C/GFErI3M0yJiOUBKaTPZfNRPAP88ORxYagWGUmnuNgD7a37emfc6/m/gn8l6Rk8nmx9CPqTo54H3kQ3pPRu4ERg9zDkumXKOe1JKY2RL3b8Q2AZ8GHh1SunO/G9eBdwXEbuBX+fgkKC1wJeBYeB64MMppaE5tYAkScV7DfAPKaUHUkoPT/4Af002b/Ry4L/Jgt924E+AtpTSA2TTXd6U338L8PT8mB8ExoBHyIbXfuoINVxHtmjS3WRTakZ4/PDeD5CNXPoi2RfDfw/01mz/GPBUHLqrFhOubyIVK198YTPwipTS14quR5IkFSMinkM2jHfNlDUjpFKzp1QqQET8dEQM5MOG3kY21+RbBZclSZIKkq8B8dvARwykajWGUqkY55OtvreNbB7oS507IklSa4qIs4CdZAsy/XnB5UgLzuG7kiRJkqTC2FMqSZIkSSpMYdcpXbFiRVqzZk1DjrV3714WL17ckGOVlW1UH9upPrZTfWyn+jSqnW666aZtKaXjG1DSMS8iPgq8GHg0pfSUabYH8BdkK4ruAy5NKd18pOP63rywbKP62E71sZ3qYzvVZ6HfmwsLpWvWrOHGG29syLGGhoZYv359Q45VVrZRfWyn+thO9bGd6tOodoqI++deTdP4R7LLXHx8hu0vJLv801rgPOBv8v8elu/NC8s2qo/tVB/bqT62U30W+r3Z4buSJDWZlNJ/kF1PcSYXAx9PmW8BAxFx0sJUJ0nS7BTWUypJkubNSmBTze3N+X0PTd0xIi4DLgMYHBxkaGioIQUMDw837FhlZRvVx3aqj+1UH9upPgvdToZSSZLKJ6a5b9rl9lNKVwBXAKxbty41alibQ+SOzDaqj+1UH9upPrZTfRa6nRy+K0lS+WwGVtfcXgU8WFAtkiQdlqFUkqTyuRZ4dWSeCexKKR0ydFeSpGOBw3clSWoyEXElsB5YERGbgT8AOgFSSn8LbCC7HMxGskvC/HIxlUqSdGSGUkmSmkxK6ZIjbE/Aby5QOZIkzYnDdyVJkiRJhTGUSpIkSZIKYyiVJEmSJBXGUCpJkiRJKoyhVJIkSZJUGEOpJEmSJKkwhlJJkiRJUmEMpZIkSZKkwhhKJUmSJEmF6Si6gLmoTFTZOzpBpZqKLkWSJEkqnT0j45Tpo/be8cSu/eNFl3HMq6aF/Udv6lB6y6ad/NzfXs+b1/VwQdHFSJIkSSVy9Y2beMs13yu6jMb7yheLruCY94fP6l3Q8zV1KD2oRF/fSJIkSTW+v2UX19y0+bD7bN48ytDu2xp63hvu286irnbe9FNPauhxi7Rx40bOOOOMoss45vXvu29Bz9fUoTQi++8C9y5LkiSphYxWJtg7OlHY+f/26/ew4b8fYmlP54z7VCoVOrZuafi5n3/WIK999mkNP25Rhir3s75Ej2e+DA3dv6Dna+pQClkqNZNKkiRpPlSriee872s8snu00DrWnbqMa37jx2fcPjQ0xPr16xeuIKmBmjqUTvaUSpIkSQAT1cRffPlutu8ba8jxxiuJR3aP8uKnncSPrTmuIcc8GueeVty5pfnW1KFUkiRJx64de8cYr1bndIydo1Ue3TNS9/73PLqXv/zqRpZ2d9DV0ZirH57U38Nlz3kCT1s10JDjSXq8pg6lkx2lDt+VJEk6tlx328P82iduaszBvvaVWf/Jx197Ls84ZVljzi9pXjV3KM3H77rQkSRJ0vzaPTLOB790NyPj9S34c/tDewD4w4uffOAz29G4++67eeITnzirv1na08HT7dWUmkZzh9KiC5AkSceMnfvGGJ9IrFjSRUQwVqkyPlFlcXcHO/eNsW/sYJiKgBP7euYUlprZWKXKtuHZLdzz9bu38g/fvI/li7tob6uv3Z535gm86vw1R1HhQUMjP2T9M0+d0zEkHduaOpRKkiQBPLJ7hPPekw3x/J/rT+ctF57Jaz76Hb67aQcbfusnuOADX6c6ZWTVbz9/Lb/zgtn1wJXFr33iRr5219aj+tsv/e5PctzirgZXJKmVNXUoPXCd0mLLkCRJBfvcrQ8e+P3DQ/fw2PAY19/7GADv/vwdVBO84XlnsGpZLwAf/NIP+NfvbmHr8ChvvGAtJyztKaTu+bJ97xgf+NJdjIxPv8jQDfftYN2py/j5datmddzBvh4DqaSGa+5Q6gBeSZIE/PeWXY+7/ZkbNx34/at3PgrAZc95Akt7OgG4/7F9/MvNW/j0tx9g9bJFvPhpJxEBKwd6j7khvWOVKo/srn/1WYAv3f4In/zWAwz2ddPRdugKtP29nbzq/FO5+JyVjSpTko5ac4fSyZ5Su0olSWppN92/g7UnLOHE/h6+8YNtrD6ul6+/+bmc9Y4vMFqp8twnHX8gkAK85cIzeeMFT+TJf/AF/uQLd/InX7gTgLe96Ewue87pRT2Maf3Wld/lC7c9POu/awv4ypvWs6S7qT/uSWoBvkpJkqSm193RRl9vJ3948VO48f4dnHniUtragk/96nnc99g+zl1z3CF/09XRxidfex6bduwH4I8+fztXfmcTd+Srxk4nAn7lWafxlJX9DX8Mn/72A9xw3/ZD7v+ve7ZxzuoBXjnLxX5O7u8xkEpqCqV4pbKjVJKk1jYyXuXU5YtYs2Ixa1YsPnD/ujXHsW6aQDrpvCcs57z899se3MVX7niUm+7fMeP+m3fsA+ANz1t7yLaT+nvo6Wyvq979YxM8vLfKD7ftPXDfn153J5WJxLIpczaXLe7ilc88lZ/70dnN/5SkZlFXKI2IC4G/ANqBj6SU/njK9lOAjwED+T6Xp5Q2NLjWaeqa7zNIkqRmMFqZoLfOQDiTP3jJk/mDlzz5sPtc+Of/wb/cvIV/uXnLIduedcZyPvWrz6zrXK/5h+/wnR/uh28MPe7+3/vpJ/Gbzz2j7polqQyOGEojoh34EPACYDNwQ0Rcm1K6vWa3twNXp5T+JiLOBjYAa+ah3sfXli905JxSSZJa2/6xibp7Kefiry55Brc9uPuQ+z9zwya++8BO3nDld+s6zvc27+TJy9t43QVPO3BfW1vw3Ccd37BaJalZ1NNTei6wMaV0L0BEXAVcDNSG0gT05b/3Aw+yAOwplSRJD+3az0ilOuee0nqsHVzK2sGlh9zf09nG+64b4bYpqwDPZOVAL89fVeGlz3D1W0mqJ5SuBDbV3N4MB6ZfTHon8MWIeAOwGLhgugNFxGXAZQCDg4MMDQ3NstzH27Qnu/bW/pGROR+r7IaHh22jOthO9bGd6mM71cd20lz8cNtenvtnQwD09Ra3VMaFTzmJC59y0qz+xue9JGXqefWerj9y6oDZS4B/TCm9PyLOBz4REU9JKT3uis0ppSuAKwDWrVuX1q9ffxQlH3Tnw7vhm9+gp6eHuR6r7IaGhmyjOthO9bGd6mM71cd20lxs2p4tPPSWC5/EL503u9VpJUnHhkOvpnyozcDqmturOHR47muBqwFSStcDPcCKRhR4OAfmlM73iSRJ0jFp+94xAH7q7BO9/IkkNal6QukNwNqIOC0iuoCXA9dO2ecB4PkAEXEWWSjd2shCp3NgTqmpVJKklnTLpp0AHL+ku+BKJElH64ihNKVUAV4PXAfcQbbK7m0R8a6IuCjf7U3A6yLiVuBK4NKU5n9NXNc5kiSptU1+3Ohf1FlwJZKko1XXOJf8mqMbptz3jprfbwee1djS6mdHqSRJremerXs5dfmiosuQJM1BPcN3j1mTw3cNpZIktZ7xiSr/uXEbXe1N/XFGklpek7+Km0olSWpVO/eNA/C8s04ouBJJ0lw0dSgNJ5VKktSyduzLVt598sn9BVciSZqLpg6lk+wolSSp9Xz73scAGOh1kSNJamZNHUq9IowkSa1rtFIF4OmrBwquRJI0F80dSh2/K0lSy9q9f5y2gKXddV1MQJJ0jGruUJr/dwEuiSpJko4xu0cqLOnuoK3NL6klqZk1dSiVJEmta/f+cfqcTypJTa+pQ6mjdyVJal2bd+6nr8dQKknNrrlDaT6A18G7kiS1nh9u20u7Q3clqek1dyjN34ecUipJUmt5z4Y72LpnlNOPX1x0KZKkOWrqUCpJklpPtZq44j/uBeAXf+yUgquRJM2VoVSSJDWV+7fvA+DNP/VEzj99ecHVSJLmqqlD6YHhu8WWIUmSFtCOfWMAnHVSX8GVSJIaoclDqQsdSZLUanbvHwdgYFFXwZVIkhqhuUNp0QVIkqQFt3ukAkB/b0fBlUiSGqGpQ+kBdpVKktQyJntKvUapJJVDU4dS55RKktR6tuzcD0Bfr6FUksqguUOpA3glSWo5m/LVd7s7mvpjjCQp19Sv5gd6Su0qlSSpZYxPVFk50HtgwUNJUnNr7lBadAGSJGlB7dw3xnW3PcLKZb1FlyJJapCmDqWT7CiVJKk1POd9XwOgzW+mJak0mjuU+oYkSVJLmbwczMXnrCy4EklSozR1KJ1c6MieUkmSWsvzzzyh6BIkSQ3S1Feddn0DSZJay3GLu1h93CJO6OspuhRJUoM0dU/pJFfflSSp/CoTVbbvHePpq/qLLkWS1EBNHUrtKJUkqXU8smcUgIHezoIrkSQ1UnOH0nBOqSRJrWLXvnEAzjqpr+BKJEmN1NyhdPIXU6kkSaX3pdsfAWBJT1MviSFJmqKpQ6kkSWodH/zy3QBUJvw2WpLKpKlD6eTqu741SZLUOtYOLim6BElSAzX1+BevUypJUutY2tPBmuWLWbVsUdGlSJIaqKl7Sl1+V5LUqiLiwoi4KyI2RsTl02w/NSK+EhHfi4ihiFhVRJ2NlBIsX9JVdBmSpAZr6lAahlJJUguKiHbgQ8ALgbOBSyLi7Cm7/Rnw8ZTS04B3Ae9d2CobqzJRZXi0wjmrB4ouRZLUYE0dSiclx+9KklrLucDGlNK9KaUx4Crg4in7nA18Jf/9a9NsbyrfvOcxwGuUSlIZNfmc0oyZVJLUYlYCm2pubwbOm7LPrcDLgL8A/gewNCKWp5Qeq90pIi4DLgMYHBxkaGioIQUODw837FgA192XXaO0ffsPGRq6v2HHLVKj26isbKf62E71sZ3qs9Dt1Nyh1PG7kqTWNN0b4NTvaN8M/HVEXAr8B7AFqBzyRyldAVwBsG7durR+/fqGFDg0NESjjgXw3S/dDXf+gF/6mefS3laO9/9Gt1FZ2U71sZ3qYzvVZ6HbqblDadEFSJJUjM3A6prbq4AHa3dIKT0I/CxARCwBXpZS2rVgFTbY8GiFxV3tpQmkkqSDyjGn1AG8kqTWcgOwNiJOi4gu4OXAtbU7RMSKiJh8n38r8NEFrrGh9oyMs6Snqb9LlyTNoKlDaTipVJLUglJKFeD1wHXAHcDVKaXbIuJdEXFRvtt64K6IuBsYBP6okGIbZHi0wtIeFzmSpDJq6q8cIx/AayaVJLWalNIGYMOU+95R8/s1wDULXdd82TNSYUl3U39skSTNoBQ9pYZSSZLKbc9IhaUO35WkUmrqUCpJklrDnpFxQ6kklVQ5QqldpZIkldrwqMN3JamsmjqUOnxXkqTW8MjuURc6kqSSqiuURsSFEXFXRGyhncb7AAAgAElEQVSMiMun2f7BiLgl/7k7InY2vtRp6nKhI0mSSm/X/nEARisTBVciSZoPRxwHExHtwIeAF5BdrPuGiLg2pXT75D4ppd+p2f8NwDPmodZpaluIs0iSpCLtHa0A8JST+wuuRJI0H+rpKT0X2JhSujelNAZcBVx8mP0vAa5sRHGSJEn7x7Me0t6u9oIrkSTNh3pWDFgJbKq5vRk4b7odI+JU4DTgqzNsvwy4DGBwcJChoaHZ1HqIiWo2cHd0dGzOxyq74eFh26gOtlN9bKf62E71sZ10JCN5KO3uMJRKUhnVE0qnGyQ70zTOlwPXpJSmnfSRUroCuAJg3bp1af369fXUOKNqNcEXN9DZ1cVcj1V2Q0NDtlEdbKf62E71sZ3qYzvpSEbsKZWkUqtn+O5mYHXN7VXAgzPs+3IWcOhuW1vQ3dHGqOseSJJUWvvHqgD0dhpKJamM6gmlNwBrI+K0iOgiC57XTt0pIp4ELAOub2yJh7e0p5N9466/K0lSWR0cvtvUV7KTJM3giK/uKaUK8HrgOuAO4OqU0m0R8a6IuKhm10uAq1JKC5oQ+3o72F8xlEqSVFajlayntMeeUkkqpXrmlJJS2gBsmHLfO6bcfmfjyqrfCUu7ue+RfVSribY2rxEjSVLZTF6ftKfTnlJJKqOmf3W/5NxTeHhv4ou3P1x0KZIkaR5M9pS6+q4klVPTh9IXP+1kBhcFf/cf9xZdiiRJmgd3PrQbcPVdSSqrpg+l7W3BT67u4LsP7OSBx/YVXY4kSWqwJT3ZbKP+3s6CK5EkzYemD6UA556YvVk5hFeSpPKpVJMr70pSiZXiFX5FbxunLl/Ed364vehSJElSg01MJDpczFCSSqsUoRTgx9Ycx3fu284CX5FGkiTNs0o10W4olaTSKk0offrqAXbuG+ehXSNFlyJJkhpooproaC/NRxZJ0hSleYV/4glLALjrkT0FVyJJkhrJnlJJKrfyhNLBpQD8wFAqSVKpTFSrzimVpBIrTShdtriLFUu62fjocNGlSJKkBrKnVJLKrTShFGDVsl627NxfdBmSJKmBJqquvitJZVaqULpyWS9bdhhKJUkqE3tKJancShVKVw308uDOEapVLwsjSVJZTEwYSiWpzMoVSpf1MjZRZdvwaNGlSJKkBsl6Skv1kUWSVKNUr/An9fcC8KDXKpUkqTRcfVeSyq1UoXTF0m4Atu2xp1SSpLJwTqkklVupQunxk6HU4buSJJVGNbn6riSVWalC6fLFXYChVJKkMqm40JEklVqpQmlPZztLezrYNjxWdCmSJKlBJqqJjnZDqSSVValCKcDxS7rZ6pxSSZJKw9V3JancSvcKv2JJN1sdvitJUmlMVJ1TKkllVr5QurTLOaWSJJXIrv3j9Ha2F12GJGmelC6UDizqYte+8aLLkCRJDVCZqPLA9n0s7ekouhRJ0jwpXShdtqiTnfvHSSkVXYokSZqjkUoVyL50liSVU+lC6UBvFxPVxPBopehSJEnSHI3nofTEvu6CK5EkzZfShdL+RZ0A7HQIryRJTW98IgulnR2l+8giScqV7hV+WT68x1AqSVLzG817SjvbS/eRRZKUK90r/MBkT+n+sYIrkSRJczXZU9plKJWk0irdK/xAr8N3JUkqi7EJe0olqexK9wp/cE6pPaWSJDW7vaMTACzq9jqlklRWpQulA73OKZUkqSxGxrNQ2ttpKJWksipdKO3qaGNxVzs79xtKJUlqdmP5Qkfdrr4rSaVVylf4gUVd9pRKklQCo5Wsp7S7w55SSSqrkobSTueUSpJUApOXhOmyp1SSSquUr/ADizrZYSiVJKnpOXxXksqvlK/wA71d7HJOqSRJTW/UUCpJpVfKV/j+RZ2GUkmSSmDM4buSVHqlfIXv781CaUqp6FIkSdIcjE0YSiWp7Er5Cj/Q28n4RGLf2ETRpUiSpDkYHc9DaXspP7JIkihrKF3UCeC1SiVJanJjExO0twUdhlJJKq1SvsL39+ah1BV4JUlqamOVqr2kklRypXyV7+/tAnCxI0mSmtxopUp3Zyk/rkiScqV8lZ8cvrtrn6FUkqRmZk+pJJVfKV/lDwzftadUkqSmNlapuvKuJJVcKV/lD/SUGkolSWpqo5Uq3YZSSSq1Ur7K93a209Xexk6H70qS1NRGK1W6OtqLLkOSNI/qCqURcWFE3BURGyPi8hn2+YWIuD0ibouITze2zNmJCPp6O9m139V3JUlqZmMTDt+VpLLrONIOEdEOfAh4AbAZuCEirk0p3V6zz1rgrcCzUko7IuKE+Sq4XgOLOh2+K0lSkxsdn3D4riSVXD2v8ucCG1NK96aUxoCrgIun7PM64EMppR0AKaVHG1vm7A30djp8V5KkJjc24ZxSSSq7I/aUAiuBTTW3NwPnTdnniQAR8U2gHXhnSukLUw8UEZcBlwEMDg4yNDR0FCUfanh4+JBjVfaN8OhIatg5mt10baRD2U71sZ3qYzvVx3bS4YxVqnQtMpRKUpnVE0pjmvvSNMdZC6wHVgHfiIinpJR2Pu6PUroCuAJg3bp1af369bOtd1pDQ0NMPda1j97C1nu3H3J/q5qujXQo26k+tlN9bKf62E46nNFKle5OQ6kklVk9r/KbgdU1t1cBD06zz/9LKY2nlH4I3EUWUgsz0NvlnFJJkprcWKVKV7uhVJLKrJ5X+RuAtRFxWkR0AS8Hrp2yz2eB5wJExAqy4bz3NrLQ2RpY1MnwaIXxiWqRZUiSNC+OtDJ+RJwSEV+LiO9GxPci4kVF1DlXYxVX35Wksjviq3xKqQK8HrgOuAO4OqV0W0S8KyIuyne7DngsIm4Hvgb8Xkrpsfkquh79vZ0A9pZKkkqnZmX8FwJnA5dExNlTdns72Xv2M8i+UP7wwlbZGKOVCbq9TqkklVo9c0pJKW0ANky57x01vyfgd/OfY8LAooOhdMWS7oKrkSSpoQ6sjA8QEZMr499es08C+vLf+zl06k1TsKdUksqvrlDajCZ7Sr0sjCSphOpZGf+dwBcj4g3AYuCC6Q60kCvjH42R8Qke3rKZoaHCrzbXcK48XR/bqT62U31sp/osdDuVPpTu2j9WcCWSJDVcPSvjXwL8Y0rp/RFxPvCJfGX8xy22sJAr489WtZqY+MIG1p6+hvXrn9iQuo4lrjxdH9upPrZTfWyn+ix0O5V2PMzAoi7AOaWSpFKqZ2X81wJXA6SUrgd6gBULUl2DjOWLFTp8V5LKrbSv8gMO35UklVc9K+M/ADwfICLOIgulWxe0yjnaumcUgJGxiYIrkSTNp9KG0j5DqSSppOpcGf9NwOsi4lbgSuDSfGHCprF3rALAE09cWnAlkqT5VNo5pe1twdKeDofvSpJKqY6V8W8HnrXQdTXS/ryHdHFXaT+uSJIocU8pZJeFMZRKktScRsazOaXdnaX+uCJJLa/Ur/IDvV3s3Ofqu5IkNaOR8ayntLezveBKJEnzqdShtL+3k532lEqS1JQmQ2mPoVSSSq3codThu5IkNa2RiqFUklpBqUPpQG+nq+9KktSkRifnlHqdUkkqtVK/yh+3OJtTOlFtqhXwJUkSMFoxlEpSKyj1q/zxS7upJti+18WOJElqNmN5KO0ylEpSqZX6Vf74Jd0AbN0zWnAlkiRptkbzOaXdHc4plaQyK3coXZqH0mFDqSRJzWayp7SzPQquRJI0n0odSlfkPaXb7CmVJKnpjFaqdHe0EWEolaQyK3UotadUkqTmNVqpOp9UklpAqV/pF3d3sKir3TmlkiQ1of1jEyzqcj6pJJVdqUMpZL2lhlJJkprP8FiFxV0dRZchSZpn5Q+lSwylkiQ1o72jFRZ3G0olqexKH0pXLOlmm3NKJUlqOvtGJ1jc7fBdSSq70ofSwb5uHt49UnQZkiRploZHHb4rSa2g9KF05bJe9oxU2LV/vOhSJEnSLOwdc/iuJLWC8ofSgUUAbNmxv+BKJEnSbOwdnTCUSlILKH0oXbWsF4AtOw2lkiQ1k72jFRZ7SRhJKr3Sh9KVeSjdvGNfwZVIkqR6TVQT+8ftKZWkVlD6ULp8cRc9nW0O35UkqYnsG6sAsMRQKkmlV/pQGhGsWraIzYZSSZKaxt7RCQAWeUkYSSq90odSyOaVPrDd4buSJDWL4VF7SiWpVbREKD3j+CXcs3WYiWoquhRJklSHyeG7XqdUksqvJULp2sEljFaqziuVJKlJTPaUutCRJJVfS4TSM05YCsDdj+wpuBJJklrT57/3EGsu/zwfHtpY1/6Tc0oXO6dUkkqvJULp2sElAPzg0eGCK5EkqTX93jW3AvC+L9xV1/4Hhu/aUypJpdcSobSvp5MT+3rsKZUkqSBPXzUAwGBfd137u9CRJLWOlgilAE9Z2cetm3cWXYYkSS3paav7AXjqyv669v/+ll2APaWS1ApaJpQ+45Rl3Lt1Lzv2jhVdiiRJLWdfPkd0ZLxa1/7tbQHYUypJraBlQumPnLIMgO9u2lFwJZIktZ59Y5OhdKKu/feMVDjluEXzWZIk6RjRMqH06av7aW8LbrzPUCpJ0kLbP57NER2p1B9Kl/bYSypJraBlQumirg5+5JQBvn731qJLkSSp5RzsKa1v+O6ekXFDqSS1iJYJpQDPO3OQ2x7czUO79hddiiRJLeVohu/29XTOZ0mSpGNES4XSC846AYAv3/5IwZVIktRa9h9FKF1qKJWkltBSofSME5Zw5olLueamzUWXIklSS9k3ls0p3TY8xqbt+w6771fueIQtO/c7fFeSWkRLhdKI4OfXrebWzbu48+HdRZcjSVLLmBy+C/CCD36dlBLjE1VSSofs+/bPfh+o/5qmkqTm1lKhFOB/PGMlXe1tXPWdTUWXIklSy6gNpSPjVd7+2e+z9n/9O6e9dcMh++7YN8Zlz3kCL/vRVQtZoiSpIC0XSo9b3MWLnnoi/3zTZvaOVoouR5KkljB1Lumnvv3AtNtu3bSTkfEqA4ucTypJraKuUBoRF0bEXRGxMSIun2b7pRGxNSJuyX9+tfGlNs6rzl/DntEKn71lS9GlSJJUeiklRitVnn/mCdNu/8YPtrF/bIL9YxN85D9/CDh0V5JayRFXEIiIduBDwAuAzcANEXFtSun2Kbt+JqX0+nmoseF+5JQBzj6pj09cfz+/dO4pRETRJUmSVFqjlezapD+6ZhlfufPRA/d3tAWVauJ1H7/xcfuvO3UZP7H2+AWtUZJUnHp6Ss8FNqaU7k0pjQFXARfPb1nzKyJ49fmncufDe7jx/h1FlyNJUqlNrrbb09H+uPu//pbnHvj9xL4eLn/hmVz+wjN5x0vOXtD6JEnFqieUrgRqVwXanN831csi4nsRcU1ErG5IdfPoonNOZmlPBx+//v6iS5EkqdRe8ZFvA9llYZ5xygAAb/+Zs1g50Mszn3AcAB96xY/w6z95Or/+k6fztFUDhdUqSVp49VwAbLqxrVPXb/8ccGVKaTQifh34GPC8Qw4UcRlwGcDg4CBDQ0Ozq3YGw8PDR3Wsc0+Af//vB/niCTvpai/3EN6jbaNWYzvVx3aqj+1UH9up/B7dMwpAR3sb//o/n/W4bVdddn4RJUmSjiH1hNLNQG3P5yrgwdodUkqP1dz8v8CfTHeglNIVwBUA69atS+vXr59NrTMaGhriaI41MfgIX/nYjSxZ81R+/PQVDanlWHW0bdRqbKf62E71sZ3qYzu1jq72llv0X5JUh3reHW4A1kbEaRHRBbwcuLZ2h4g4qebmRcAdjStx/px72nG0twXX3/PYkXeWJElz0t5W7lFJkqSjc8Se0pRSJSJeD1wHtAMfTSndFhHvAm5MKV0L/FZEXARUgO3ApfNYc8Ms7elk7QlL+P6WXUWXIklS6bUZSiVJ06hn+C4ppQ3Ahin3vaPm97cCb21saQvjrJP6+Na99pRKkjTf2r0EmyRpGi0/uePME5fy0K4Rdu4bK7oUSZJK7flnnVB0CZKkY1DLh9K1g0sAuGfrcMGVSJJUTmeeuJQXnD3IYF9P0aVIko5BLR9KVy9bBMDmHfsLrkSSpHIaq1Tp7mj5jxySpBm0/DvEymW9gKFUkqT5Mlqp0t3RXnQZkqRjVMuH0kVdHSxf3MXmHfuKLkWSpFIarVTpsqdUkjQD3yGAVct67SmVJGmejFYmHL4rSZqR7xDAYF8Pj+4eLboMSZJKad+YoVSSNDPfIYAVS7vZNmwolSSp0R7ZPcJENfHPN28uuhRJ0jHKUAqsWNLN9n1jVCaqRZciSVKp/OCR7JJr24a9HrgkaXqGUuD4JV2kBNv3+oYpSVIjTS5w9CvPOq3gSiRJxypDKXD80m4AtjqEV5Kkhto/PgHAzzztpIIrkSQdqwylZMN3waFFkiQ12vBIBYClPR0FVyJJOlYZSqkJpXvsKZUkqVEeGx5l98g4AEu6DaWSpOn5DgEsX9IFOKdUkqRG+d7WCpe++8sHbi+xp1SSNAN7Ssm+vW1vC3btHy+6FEmSSuHO7Y9f0X5xl6FUkjQ9QykQEfT3drJzvz2lkiQ1Qlsc/H1xVzvttXdIklTDUJob6O1k1/5K0WVIklQKKR38fWlPZ3GFSJKOeYbSXF9vJzv32VMqSVIj1GRS55NKkg7LUJobWNTpnFJJUtOIiAsj4q6I2BgRl0+z/YMRcUv+c3dE7FzI+iZqukpdeVeSdDi+S+T6ezv54ba9RZchSdIRRUQ78CHgBcBm4IaIuDaldPvkPiml36nZ/w3AMxayxiWdB+eQGkolSYdjT2luoLeTnfvsKZUkNYVzgY0ppXtTSmPAVcDFh9n/EuDKBakst7z34EeM7g4/bkiSZuZXl7n+3k52j4xTrSbaXCFQknRsWwlsqrm9GThvuh0j4lTgNOCrM2y/DLgMYHBwkKGhoYYUuG//CJC9n+7c8VjDjlsmw8PDtksdbKf62E71sZ3qs9DtZCjN9S/qIiXYM1Khf5GrBEqSjmnTfXuaprkP4OXANSmliek2ppSuAK4AWLduXVq/fn1DCvz65i8B2QKCK08cZP36BR093BSGhoZoVHuXme1UH9upPrZTfRa6nRxPkxvozYKo1yqVJDWBzcDqmturgAdn2PflLPDQXYBq9eDvHe2OQJIkzcxQmuvPQ6kr8EqSmsANwNqIOC0iusiC57VTd4qIJwHLgOsXuD4mavptd+z1C19J0swMpbmBfMjuDhc7kiQd41JKFeD1wHXAHcDVKaXbIuJdEXFRza6XAFellGYa2juPNR78/Wt3bV3o00uSmohzSnOTPaW77SmVJDWBlNIGYMOU+94x5fY7F7KmWhMLHoMlSc3KntJcn8N3JUlqmOrCd85KkpqUoTR3oKd0xFAqSdJcVc2kkqQ6GUpz3R1tdLW32VMqSVID1A7fXX1cb3GFSJKOec4pzUUEfb2d7N5fKboUSZKa3mRP6SufeQqvOX9NobVIko5thtIa/b0dLnQkSVIDTCRobwve/dKnFl2KJOkY5/DdGn29nc4plSSpAap5KJUk6UgMpTX6ezudUypJUgNUU6LDUCpJqoOhtEZfT6fDdyVJaoAJe0olSXUylNawp1SSpMZw+K4kqV6G0hp9vR3sHqmQvOC3JElzUk04fFeSVBdDaY3+3k4mqom9YxNFlyJJUlNz+K4kqV6G0hp9PZ0ADuGVJGmONu+pEhhKJUlHZiit0d+bhVIXO5IkaW6WdIaXWZMk1cVQWqOv155SSZIaYSIlzjqpr+gyJElNwFBaw55SSZIao1KFznaH70qSjsxQWsM5pZIkNcZEgs52P2ZIko7Md4saB3pKRyoFVyJJUnOrVL0kjCSpPobSGkt6OgB7SiVJmit7SiVJ9fLdokZ7W7C0p8M5pZIkzVGlmgylkqS61PVuEREXRsRdEbExIi4/zH4/FxEpItY1rsSF1d/baSiVJGmOsp5Sh+9Kko7siKE0ItqBDwEvBM4GLomIs6fZbynwW8C3G13kQurr6fS6apIkzdFEFTrsKZUk1aGed4tzgY0ppXtTSmPAVcDF0+z3h8D7gJEG1rfg+ns7nVMqSdIcVewplSTVqaOOfVYCm2pubwbOq90hIp4BrE4p/VtEvHmmA0XEZcBlAIODgwwNDc264OkMDw837Fhje0d4ZG+1Ycc7VjSyjcrMdqqP7VQf26k+tlM5TVQTHW32lEqSjqyeUDrd15zpwMaINuCDwKVHOlBK6QrgCoB169al9evX11XkkQwNDdGoY23Yditb7t7WsOMdKxrZRmVmO9XHdqqP7VQf26mcqilbQFCSpCOp5yvMzcDqmturgAdrbi8FngIMRcR9wDOBa5t1sSPnlEqSNHcJaAtDqSTpyOoJpTcAayPitIjoAl4OXDu5MaW0K6W0IqW0JqW0BvgWcFFK6cZ5qXie9fd2sm9sgvGJatGlSJLUtKoJ7CiVJNXjiKE0pVQBXg9cB9wBXJ1Sui0i3hURF813gQutr7cTwMWOJEmaA4fvSpLqVc+cUlJKG4ANU+57xwz7rp97WcXpz0Pp7v3jrFjSXXA1kiQ1p2qCNkOpJKkOLos3RV9vltPtKZUk6ehlc0qLrkKS1AwMpVMc6CkdqRRciSRJzauaoN2FjiRJdTCUTtHX45xSSZLmolrNrhzn8F1JUj0MpVPUzimVJEmzV01ZKLWnVJJUD0PpFK6+K0nS3Ewke0olSfUzlE7R09lOV0cbu0cMpZIkHY1qfqnvNntKJUl1MJROo6+n0+G7kiQdpQPDd/2UIUmqg28X0+jv7WD3flfflSTpaBwYvmtPqSSpDobSafT3djqnVJKko3Rg9V1DqSSpDobSafT1djqnVJKko5RnUtpd6EiSVAdD6TTsKZUk6ehNHOgpLbgQSVJTMJROo6/HUCpJ0tGqekkYSdIsGEqn0d+brb6b8jdVSZJUvwOr7zqnVJJUB0PpNPp6O6gmGB51BV5JkmZrwoWOJEmzYCidRn9vJwC7RwylkiTNVrWa/dfhu5KkehhKp9HXk4XSXfucVypJ0mwdGL7rpwxJUh18u5jGZE+pix1JkjR7E8nhu5Kk+hlKp7FscRcAO/aNFVyJJEnNp+qcUknSLBhKp7F8SRZKHxseLbgSSZKaT55JaXdOqSSpDobSaRy3qIsI2DZsT6kkSbN1cPXdgguRJDUFQ+k0OtrbWLaoi8f22lMqSdJsVZ1TKkmaBUPpDJYv7mLbHntKJUmarcmeUofvSpLqYSidwfIl9pRKknQ07CmVJM2GoXQGy5d085hzSiVJmrUDodSeUklSHQylM1ixuIttrr4rSdKsTVSz/7bbUypJqoOhdAYrlnSze6TCWKVadCmSJDWVg8N3Cy5EktQUDKUzWL6kG4Dtex3CK0nSbFSrDt+VJNXPUDqD5Uu6ABzCK0nSLE0kV9+VJNXPUDqDFYZSSZKOSt5R6uq7kqS6GEpnsHxxNnx3myvwSpI0KweG75pJJUl1MJTOYLCvB4BHdo8UXIkkSc1lourwXUlS/QylM+jtaqe/t5OHdxlKJUmajYOr7xpKJUlHZig9jJP6e3jYnlJJkmbFUCpJmg1D6WEM9vXYUypJ0ixN5Jf4dviuJKkehtLDsKdUkqTZqx64JEzBhUiSmoJvF4dxYv//b+/Oo+Oq77uPf76jGS2jffciecU2scEYEAbCJkJCIUmBNE3i0KSkSeukLWmWtnlI08NJ0ydPSdInp3lOaRuapFmBkN0QJ8AJCBIwxgZsYxvbsY0XWbYlS9Zm7Zrf88dcibEsWXfske5Ier/O0Zm5d+5c/eYrjX7z0f3d383Wic5e9Q/9yxcAAIxrKJQaw3cBAD4QSs9iVkG2nJMaO7hWKQAAfg3PvksoBQD4QCg9i1mF8cvCHGvrDrglAABMHV4m5ZxSAIAvhNKzeCOUcqQUAJBezOwWM9ttZnvN7J4xtnmvme00sx1m9uBktS0WGxq+O1nfEQAwlYWDbkA6m12QI0k6ypFSAEAaMbMMSfdLepukekmbzGydc25nwjZLJH1W0jXOuZNmVjFZ7RscnuiIVAoAGB9HSs+iICesvKyw6k8SSgEAaWW1pL3Ouf3OuT5JD0u6fcQ2fyHpfufcSUlyzjVOVuOGZ9/lUCkAwAeOlJ6Fmam6JKpDLV1BNwUAgERzJR1OWK6XdOWIbZZKkpk9JylD0uedc78euSMzWytprSRVVlaqrq7uvBu361C/JGnDhg0qzCKYjqWzszMl9Z7uqJM/1Mkf6uTPZNeJUDqO+SVR7W3qDLoZAAAkGi3puRHLYUlLJNVKqpL0WzO7yDnXetqTnHtA0gOSVFNT42pra8+7cQeee13auVPXXXuNSnIzz3t/01VdXZ1SUe/pjjr5Q538oU7+THadGL47jnml8SOlQ5M2AACQBuolVScsV0lqGGWbXzjn+p1zr0varXhInXCDQ7PvMnwXAOADoXQc1SVR9Q3EuFYpACCdbJK0xMwWmlmmpDWS1o3Y5ueSbpQkMytTfDjv/sloXHffgCQpJzNjMr4dAGCKI5SOY35JVJJ0sPlUwC0BACDOOTcg6W5Jj0t6TdIjzrkdZvYFM7vN2+xxSc1mtlPS05L+3jnXPBnt6x2IySRlhvmYAQAYn6/eYrxroZnZx8zsVTPbYma/M7PlqW9qMOZ5oZTJjgAA6cQ5t945t9Q5t9g590Vv3b3OuXXefeec+7Rzbrlz7mLn3MOT1baYc1yjFADg27ihNOFaaLdKWi7p/aOEzge9Dm+VpC9L+mrKWxqQucU5ChmhFAAAv5wbfSYmAABG4+dI6bjXQnPOtScs5urMGQCnrEhGSNUlUe1vYvguAAB+TJsPAQCASeHnkjB+roUmM/trSZ+WlCnpLaPtaCKuhSZN/HV0ikO92vL6sSl9TSOuyeQPdfKHOvlDnfyhTtMPR0oBAMnwE0r9XAtNzrn7Jd1vZndK+kdJd42yTcqvhSZN/HV0Nvbs0jd+u1/XXHe9IhlTc3trA48AACAASURBVNIGrsnkD3Xyhzr5Q538oU7TjxPnlAIA/POTsPxcCy3Rw5LuOJ9GpZslFXnqH3Q6cIIhvAAAjIcjpQCAZPgJpeNeC83MEi/G/Q5Jv09dE4O3tDJfkvT7xs6AWwIAQPpzpFIAQBLGDaU+r4V2t5ntMLMtip9XesbQ3alscXmezKQ9xzuCbgoAAGmPTAoASIafc0rlnFsvaf2Idfcm3P9EituVVnIyM1RdHCWUAgDggxOhFADg39SctScAK+YUaPuR9vE3BABghos5JjoCAPhHKPVpZVWRDrV0qbWrL+imAACQ1hwXKgUAJIFQ6tPKqkJJ0qtH2gJuCQAA6Y8DpQAAvwilPl00Jx5Kt9UTSgEAOBvH8F0AQBIIpT4VRiNaUBrVtvrWoJsCAEBaY6IjAEAyCKVJWFlVpC2HW+PXXwMAAKOKOUcoBQD4RihNwhULinW8vVeHWrqCbgoAAGnLcagUAJAEQmkSrlxUKknauL8l4JYAAJC+yKQAgGQQSpOwpCJPJbmZeuH15qCbAgBA2nJOMmY6AgD4RChNgplp9YISjpQCAHAWjnNKAQBJIJQm6apFJTrS2q3DnFcKAMComA8QAJAMQmmSrl9aLkl6endjwC0BACA9OXGkFADgH6E0SYvK87SgNKqndhFKAQAYTfyc0qBbAQCYKgil5+DGCyu0YV+zuvsGg24KAABph9l3AQDJIJSeg5surFTvQEzP7zsRdFMAAEg7MU4qBQAkgVB6DlYvLFFeVlhP7DgedFMAAEg/DN8FACSBUHoOMsMh3by8Ur/aflS9AwzhBQAgEcN3AQDJIJSeoz9cNUftPQN6dg9DeAEASOSc40gpAMA3Quk5uvaCMhVHI3p0a0PQTQEAIK3EHEdKAQD+EUrPUSQjpLdfPFtP7jyujp7+oJsDAEDaYJojAEAyCKXn4b011eruH9TPt3C0FACAIc45jpQCAHwjlJ6HlVWFumhugX7wwkE5pr8HAECSd6SUVAoA8IlQeh7MTHeunq9dxzr08qHWoJsDAEB6cHzAAAD4R59xnm5fNUd5WWF9d8OBoJsCAEBaiDlmOgIA+EcoPU+5WWGtuaJaj207qsMtXUE3BwCAwJFJAQDJIJSmwJ9ft0ghk77+7L6gmwIAQOCcmOgIAOAfoTQFZhVm692XVemRzfVq7OgJujkAAATKufi8CwAA+EEoTZGP3rBYgzGn+5/aG3RTAAAIVIwJ6QEASSCUpsjCsly974pq/WDjIb1+4lTQzQEAIEAM3wUA+EcoTaFPvnWJMsMh/evju4NuCgAAgYkP3w26FQCAqYJQmkIV+dn6i+sW6ZevHtXG/c1BNwcAgEAwehcAkAxCaYp99IZFmluUo3/42avqHRgMujkAAEw65xwfMAAAvtFnpFg0M6z//a6LtK/plL7+zP6gmwMAwKSLOXGhUgCAb4TSCXDjsgq9c+Vs/ftTe7XrWHvQzQEAYFKRSQEAySCUTpDP37ZCBTkRfeKhLerpZxgvAGDmcI6zSgEA/hFKJ0hZXpb+9T0rtft4h+771a6gmwMAwKQKcagUAOAToXQC1S6r0IevWahvP39Av9x2NOjmAAAwKWIcKQUAJIFQOsH+163LVDO/WH/7oy3afqQt6OYAADDhnOOcUgCAf4TSCZYVztB/fuByFUcztfa7m9XU0Rt0kwAAmFDOSUYqBQD4RCidBOX5WXrggzVq6erTh/7nRbX39AfdJAAAJowTw3cBAP4RSifJxVWF+s8PXK7dxzr059/ezIy8AIBpi+G7AIBkEEon0Y3LKvTV963SpoMt+tj3XyKYAgCmJYbvAgCSQSidZLddMkf/8q6L9cyeJv3Z/2zSqd6BoJsEAEBKOTmOlAIAfCOUBmDN6nn66nsv0YsHWvSBb25UWxfnmAIApg+uCAMASAahNCDvurRK9995mbYfadO7/uM5HThxKugmAQCQEk4M3wUA+OcrlJrZLWa228z2mtk9ozz+aTPbaWbbzOw3ZjY/9U2dfm65aJa+/5ErdbKrT3f8x3N6YX9z0E0CAOC8xRzDdwEA/o0bSs0sQ9L9km6VtFzS+81s+YjNXpFU45xbKenHkr6c6oZOV1cuKtXP//oaleZm6oPf3Khv/e51OcY9AQCmsPjsu8RSAIA/fo6Urpa01zm33znXJ+lhSbcnbuCce9o51+UtviCpKrXNnN7ml+bqp391jWqXVegLj+3UR7/3EueZAgCmLCdxTRgAgG9hH9vMlXQ4Yble0pVn2f4jkn412gNmtlbSWkmqrKxUXV2dv1aOo7OzM2X7CtKd1U5lsUw98tpxveXLT2rtyiwtK8lIyb6nS40mGnXyhzr5Q538oU7TEMN3AQBJ8BNKR+tXRh1famYfkFQj6YbRHnfOPSDpAUmqqalxtbW1/lo5jrq6OqVqX0G7UdJ7D53UJx7eon95sUsfevMCfeaWZYpm+vlRjW061WgiUSd/qJM/1Mkf6jT9MNERACAZfobv1kuqTliuktQwciMze6ukz0m6zTnXm5rmzUyXzivWrz95ne66er6+/fwB3fq132rDPiZBAgBMDUx0BABIhp9QuknSEjNbaGaZktZIWpe4gZldKunrigfSxtQ3c+aJZob1T7dfpIfXXiXnpPf/9wv6+EOv6Ghbd9BNAwDgrJivDwCQjHFDqXNuQNLdkh6X9JqkR5xzO8zsC2Z2m7fZVyTlSfqRmW0xs3Vj7A5JumpRqZ741PX6xE1L9PiOY7rp/z6j/6zbp96BwaCbBgDAqOKz7wIA4I+vExWdc+slrR+x7t6E+29NcbuQIDuSoU+9banefVmVvvDYTn3p17v0/RcO6tNvW6o7Lp2rjBBdPwAgfXBOKQAgGX6G7yJNzCuN6ht31eh7H1mt4tyI/vZHW/X2r/1WT+48zrVNAQBpw3FOKQAgCYTSKei6JeVa99fX6t/vvFR9gzH9xXc36/b7n9Ovtx9TLEY4BQAEyzmOlAIA/COUTlGhkOmdK+foiU9dr/v+6GK1dffrY99/STf/27P6yUv16h+MBd1EAMAM5Ua/chwAAKMilE5xkYyQ1qyep998+gZ9bc0qhUOmv/3RVtV+pU7//ex+tXX1B91EAMAEMLNbzGy3me01s3tGefxDZtbkTUC4xcz+fLLaxkRHAIBk+JroCOkvnBHS7avm6rZL5uipXY36+jP79cX1r+mrT+7RHZfO1YoIR04BYLowswxJ90t6m+LXE99kZuuccztHbPpD59zdk90+JjoCACSDUDrNmJluelOlbnpTpXY2tOs7zx/QT1+u10MDMf3y6Au688p5unlFpbLCGUE3FQBw7lZL2uuc2y9JZvawpNsljQylgYgx0REAIAmE0mls+ZwCfemPV+qeWy/U//lhnZ5v7NLHH3pFRdGI7lg1V++pqdKKOYVBNxMAkLy5kg4nLNdLunKU7d5tZtdL2iPpU865wyM3MLO1ktZKUmVlperq6s67cV2nujQQjaVkX9NZZ2cnNfKBOvlDnfyhTv5Mdp0IpTNAcW6m3rEoU/d96AY9v++EHtlcrwc3HtK3nz+gFXMK9J7Lq/TOS+aoLC8r6KYCAPwZ7UDkyNmFHpX0kHOu18w+Juk7kt5yxpOce0DSA5JUU1Pjamtrz7txOZvrFAn3KBX7ms7q6uqokQ/UyR/q5A918mey60QonUEyQqbrlpTruiXlau3q0y+2NOiRzYf1+Ud36p9/+ZrevLhUf3jJHP3BilkqzIkE3VwAwNjqJVUnLFdJakjcwDnXnLD435K+NAntGvrenFMKAPCNUDpDFUUzddebF+iuNy/QrmPtenRrg9ZtbdBnfrxN//iz7bphWbluu2SO3nJhhXKz+DUBgDSzSdISM1so6YikNZLuTNzAzGY75456i7dJem2yGhdj9l0AQBJIG9CFswp04awC/d3Ny7S1vk3rtjTosW0NenLncWWFQ7r2gjLdvCI+eRJDfAEgeM65ATO7W9LjkjIkfcs5t8PMviBps3NunaS/MbPbJA1IapH0oUlrn0ilAAD/CKUYZmZaVV2kVdVF+tw73qRNB1r0+I5jemLHcf1mV6PMXlXN/GLdvHyW3ra8UgvKcoNuMgDMWM659ZLWj1h3b8L9z0r67GS3K/69JSOVAgB8IpRiVBkh01WLSnXVolLd+87l2nm0XU/uPK4ndhzXF9e/pi+uf02Ly3NVu6xCNywt1+qFJcqOcJkZAMBQKAUAwB9CKcZlZloxp1Ar5hTqk29dqsMtXXpy53E9vbtR33vhoL75u9eVHQnpqkWlql1arhuWVWhBaVTGLBcAMGPRBQAA/CKUImnVJVF9+NqF+vC1C9XdN6gXXm/WM7ub9MyeJn3+0Z3Sozs1rySqa5eU6WrvaGt5PueiAsBMEXOOI6UAAN8IpTgvOZkZunFZhW5cViFJOth8Ss/uaVLd7iat29KgBzcekiQtrczT1YtKdfXiUl25sFTFuZlBNhsAMIHcyCumAgBwFoRSpNT80lx98OpcffDqBRoYjGl7Q7s27GvWhv3NemRzvb6z4aDM4jP+Xr2oVKsXFuvy+SUcSQWAacSJ65QCAPwjlGLChDNCw7P5/mXtYvUNxLStvnU4pP5g40F967nXJUnzS6O6fH6xauaXqGZBsS4oz1MoxCcaAJiKOFIKAEgGoRSTJjMcUs2CEtUsKNHHb1qi3oFBbT/SrpcOtmjzgZN6ZneTfvryEUlSYU5El88v1uXzi3XZvGJdXFWovCx+XQEg3fUPxtTe069QYSjopgAApgg+5SMwWeGM4eC59nrJOacDzV3afKBFLx08qc0HT+qpXY2S4rM4XlCep5VVRVpVXaiVVUW6cHa+ssJchgYA0smGfc3q6Y9p2wkOlwIA/CGUIm2YmRaW5WphWa7eU1MtSTp5qk9b6lu19XCrttW36Zk9jfrJy/WSpMyMkN40O18rq4q0sqpQl1QXaXF5njIY9gsAgSmKRiRJJ3sIpQAAfwilSGvFuZmnze7rnNOR1m5tq2/TVi+s/uyVI/reCwclSdmRkC6cVaDlcwq0fHb89k2zCpSTyRFVAJgMswqyg24CAGCKIZRiSjEzVRVHVVUc1dsvni1JisWc9p/o1NbDbdp5tF07Gtr02NY3LkcTMmlhWa6WzynU8tkFWjEnHlbL8pjxFwBSjsEqAIAkEUox5YVCpgsq8nVBRb7e7a0bOqK6s6FdOxratfNou14+eFKPbm0Yfl5ZXpaWzcrTkop8LZuVr6WVeerqZ7gZAJwP81Ipl4QBAPhFKMW0lHhE9eYVs4bXt3X1Dx9N3X2sQ3saO/XI5sPq6hsc3mb25t9oaWU8pMZv87WkMk/RTN4uADAewigAIFl8ysaMUhiN6OrFpbp6cenwulgsflR1z/EO/WrDVg3mlmrP8Q5t2N+svoHY8HbVJTlaXJ6nRWV5WlSeq0XluVpcnqeK/CwZn8IA4DT8VQQA+EUoxYwXCpmqS6KqLokq43imamtXSZIGY06HWrq0+1iHfn+8Q7uPd2h/0ylt3N+i7v43jqzmZYW1sCxXi8tztajcC6xleVpYlssESwBmHMIoACBZhFJgDBmhNy5Rc8tFbwwBjsWcjrX3aH/TKe0/0an9Tae0r6lTmw6c1M+3NJy2j7lFOVpYlqt5pVEtKI1qXkmu5pdGNb80ynBgANMSI0cAAMniUzGQpFDINKcoR3OKcnTtkrLTHuvuG9TrJ04Pqweau7T+1aNq7eo/bdvy/CzNL4l6gTUeVueVRDW/NFfF0Qgf7ABMafwFAwD4RSgFUignMyN+jdQ5BWc81tbdr0PNXTrYckoHm7t0sDl+u2Ffs3768pHTts3PCmt+WVTVxVFVFedoblGOqoqjmluco6riHOVnRybrJQFAUgijAIBkEUqBSVKYE9HFVYW6uKrwjMd6+gd1uKUrHlZbunSo+ZQONHdpz/EOPb27UT39sTP2FQ+qOV5QjQ4vVxdHVZAT5kgrgEDwpwcAkCxCKZAGsiMZWlKZryWV+Wc85pxT86k+1Z/s1pGT3ao/2aUjrd2qP9mtA82n9NzeEzqVcEkbKT75UlVxfIjxrMJszSnM1qzCHM0uzNbswmzNKszmnFYAE2LoOqUcMgUA+MWnUiDNmZnK8rJUlpelVdVFZzzunFNrV78XVLtUf7J7+OtoW7e2HG5Vy6m+M55XmBNJCKlvBNbZhfEgO7swW7lZ/IkAcG7IpAAAv/jECUxxZqbi3EwV52bqorlnDg2W4sODj7f3qKG1R8fau3W0rUdHW3t0tC2+vK2+Tc2jBNeC7LBmF+aooiBLFfnZqizIUkV+lpqODSj/YIsq8rNVnp+l7AiXvgHgIY0CAJJEKAVmgOxIhuaX5mp+ae6Y2/T0D6qxvVcNbd061uYF1rZuNbT1qLG9R3sbO9XU0auBmJMk3b9lw/BzC3MiqsjPUkVBlirzs1V+WojNHn6MIcPA9Mc5pQCAZPEJEYCkeHCdVxq/RM1YYjGnlq4+rX/qd5q37GI1dvSqsb3Hu+3V8Y4ebXy9RU0dveobjJ3x/PyssMrz40ORy/IzVZobv1+al6myvCyVD63Lz1JuZgaTNQFTGO9eAIBfhFIAvoVC8fNb5xVkqHZZxZjbDZ3n2tjRq8aOnuHA2tjeq6aOXp3o7NWe45060dl8xvVbh2SFQ965tJnD59QOhdfSvEyV58XDa2lupoqjmQqF+AgMpAPeiQCAZBFKAaRc4nmuy2adOaNwor6BmE529Q2H1ebOvvjtqT6d6OhVU2evjrb16NUj8fNeB73hw4lCJhVHM1Xifc+SqHebGxleP/Q1tBzlSCwwIYbfV7y9AAA+EUoBBCozHFJlQbYqC7LH3TYWc2rr7teJzl6dGAqv3v2Wrj6dPNWnllN92tfUqZMH+3Syq3/UEDv0fUsSAms8zEa8MOt9eeG2OJqpomiECZ0AH8iiAIBkEUoBTBmh0BtHYJdUjr99LObU0TOglq54WB0KrYkB9qT32JHWbjV39qq9Z2DM/WWFQyqKRlSUk6nCaERFOZH4cjRThUP3c+IBtjAnoqaumDp6+pWXFeaoLGYcfuMBAH4RSgFMW6GQqTAaUWE0ooVlY888nKh/MKbWrv7hsHrSC7GtXf1q7+5Xa1e/Wrvjy4daurStPr7c03/mxE6S9PfPPqFwyFSYE0kIspkqGl7O9IJtRAU5ERVkR1SYE1ZBdnw5Kxwi0GJKGX1sAgAAYyOUAkCCSEZI5flZKs/PSup5Pf2DahsKrV19auvu18ZXXtWseYuHQ2xrd7/auvrV2NGjPcc71NbVr47esY/MSlJmRkgFXkjNz4moIDs8HF6H1o+6zlvOjhBqMbmci8dSfusAAH4RSgEgBbIjGcqOZJx2bmxm0y7VXr/orM8bGIypvWdArV3xc2A7evrV3jOg9u5+tff0q7174Ix1Da3dw8u9A6MfoR0SybCEkHpmeM3PDisvK6y87IjyssIqyA4rb3hdWPlZBFsAADCxCKUAEKBwRmh4YqVz0dM/qI6eAS/AjhVo4/eHtjna1jO8zVjDjhNlhCweUrPCys8OnxFkh5eHg2xY+dmR4XA79DgzHgMAgNEQSgFgChs6QpvscOMhfQMxdfYOqLNnQB29/ersGYgv9w6oY+h+Tzzcdnj3O3sH1HyqTwebu4bXdfcPjvu9QiblZsVDa152PLjmZoX19srxgzGmDs4pBQAki1AKADNYZjikkvC5H6kdMjAYOz3IDgfdoSDbP7zc0fNGuG3r7k/RK0G6yM8Ka80V1bogoynopgAApghfodTMbpH0NUkZkr7hnLtvxOPXS/o3SSslrXHO/TjVDQUApK9wRig+q3A0+XBbV1eX+gYhMGam+969kp8rAMC30HgbmFmGpPsl3SppuaT3m9nyEZsdkvQhSQ+muoEAAAAAgOnLz5HS1ZL2Ouf2S5KZPSzpdkk7hzZwzh3wHuPEIAAAAACAb35C6VxJhxOW6yVdeS7fzMzWSlorSZWVlSkb2tPZ2ckwoXFQI3+okz/UyR/q5A91AgBgZvMTSkebv/+cJtdzzj0g6QFJqqmpcbW1teeymzPU1dUpVfuarqiRP9TJH+rkD3XyhzoBADCzjXtOqeJHRqsTlqskNUxMcwAAAAAAM4mfULpJ0hIzW2hmmZLWSFo3sc0CAAAAAMwE44ZS59yApLslPS7pNUmPOOd2mNkXzOw2STKzK8ysXtJ7JH3dzHZMZKMBAAAAANODr+uUOufWS1o/Yt29Cfc3KT6sFwAAAAAA3/wM3wUAAAAAYEIQSgEAAAAAgSGUAgAAAAACQygFAAAAAASGUAoAAAAACAyhFAAAAAAQGEIpAAAAACAwhFIAAAAAQGAIpQAAAACAwBBKAQAAAACBMedcMN/YrEnSwRTtrkzSiRTta7qiRv5QJ3+okz/UyZ9U1Wm+c648BfuZseibJx018oc6+UOd/KFO/kxq3xxYKE0lM9vsnKsJuh3pjBr5Q538oU7+UCd/qNP0xM91fNTIH+rkD3Xyhzr5M9l1YvguAAAAACAwhFIAAAAAQGCmSyh9IOgGTAHUyB/q5A918oc6+UOdpid+ruOjRv5QJ3+okz/UyZ9JrdO0OKcUAAAAADA1TZcjpQAAAACAKYhQCgAAAAAIzJQOpWZ2i5ntNrO9ZnZP0O2ZbGb2LTNrNLPtCetKzOxJM/u9d1vsrTcz+39erbaZ2WUJz7nL2/73ZnZXEK9lophZtZk9bWavmdkOM/uEt546JTCzbDN70cy2enX6J2/9QjPb6L3mH5pZprc+y1ve6z2+IGFfn/XW7zazPwjmFU0sM8sws1fM7DFvmTqNYGYHzOxVM9tiZpu9dbzvZgD6Zvrm8dA3+0PfnBz65vGldd/snJuSX5IyJO2TtEhSpqStkpYH3a5JrsH1ki6TtD1h3Zcl3ePdv0fSl7z7b5f0K0km6SpJG731JZL2e7fF3v3ioF9bCms0W9Jl3v18SXskLadOZ9TJJOV59yOSNnqv/xFJa7z1/yXpL737fyXpv7z7ayT90Lu/3HsvZkla6L1HM4J+fRNQr09LelDSY94ydTqzRgcklY1Yx/tumn/RN9M3+6wRfbO/OtE3J1cv+ubxa5S2ffNUPlK6WtJe59x+51yfpIcl3R5wmyaVc+5ZSS0jVt8u6Tve/e9IuiNh/Xdd3AuSisxstqQ/kPSkc67FOXdS0pOSbpn41k8O59xR59zL3v0OSa9JmivqdBrv9XZ6ixHvy0l6i6Qfe+tH1mmofj+WdJOZmbf+Yedcr3PudUl7FX+vThtmViXpHZK+4S2bqJNfvO+mP/pm+uZx0Tf7Q9/sH33zeUmL991UDqVzJR1OWK731s10lc65o1L8j76kCm/9WPWaMXX0hmdcqvh/GqnTCN6wly2SGhX/A7NPUqtzbsDbJPE1D9fDe7xNUqlmQJ0k/Zukz0iKeculok6jcZKeMLOXzGytt4733fTHz2x0/O6Pgb757OibfaNv9idt++bw+e4gQDbKOq5vM7ax6jUj6mhmeZJ+IumTzrn2+D/ERt90lHUzok7OuUFJq8ysSNLPJL1ptM282xlZJzN7p6RG59xLZlY7tHqUTWd0nTzXOOcazKxC0pNmtuss287kOk03/MySM6N/9+mbx0ffPD765qSkbd88lY+U1kuqTliuktQQUFvSyXHv0Lq820Zv/Vj1mvZ1NLOI4p3eD5xzP/VWU6cxOOdaJdUpfv5AkZkN/fMq8TUP18N7vFDx4WrTvU7XSLrNzA4oPizxLYr/d5Y6jeCca/BuGxX/ILVavO9mAn5mo+N3fwT65uTQN58VfbNP6dw3T+VQuknSEm9mrUzFT1ReF3Cb0sE6SUOzYN0l6RcJ6//Um0nrKklt3iH6xyXdbGbF3mxbN3vrpgXvHIFvSnrNOffVhIeoUwIzK/f+Cyszy5H0VsXP8Xla0h97m42s01D9/ljSU845561f481st1DSEkkvTs6rmHjOuc8656qccwsU/5vzlHPuT0SdTmNmuWaWP3Rf8ffLdvG+mwnom0fH734C+mZ/6Jv9oW/2J+37ZpcGM0Gd65fis0LtUXx8/eeCbk8Ar/8hSUcl9Sv+X4uPKD4m/jeSfu/dlnjbmqT7vVq9KqkmYT8fVvxk7r2S/izo15XiGl2r+JCCbZK2eF9vp05n1GmlpFe8Om2XdK+3fpHif5D3SvqRpCxvfba3vNd7fFHCvj7n1W+3pFuDfm0TWLNavTHDH3U6vTaLFJ/BcKukHUN/n3nfzYwv+mb6Zh81om/2Vyf65uRrRt88dm3Sum82b8cAAAAAAEy6qTx8FwAAAAAwxRFKAQAAAACBIZQCAAAAAAJDKAUAAAAABIZQCgAAAAAIDKEUSAEze967XWBmd6Z43/8w2vcCAABjo28Gpg4uCQOkkJnVSvo759w7k3hOhnNu8CyPdzrn8lLRPgAAZhr6ZiD9caQUSAEz6/Tu3ifpOjPbYmafMrMMM/uKmW0ys21m9lFv+1oze9rMHlT8gsQys5+b2UtmtsPM1nrr7pOU4+3vB4nfy+K+YmbbzexVM3tfwr7rzOzHZrbLzH5gZja0PzPb6bXlXyezRgAATCb6ZmDqCAfdAGCauUcJ/431OrA259wVZpYl6Tkze8LbdrWki5xzr3vLH3bOtZhZjqRNZvYT59w9Zna3c27VKN/rjyStknSJpDLvOc96j10qaYWkBknPSbrGzHZKepekC51zzsyKUv7qAQBIP/TNQJrjSCkwsW6W9KdmtkXSRkmlkpZ4j72Y0OlJ0t+Y2VZJL0iqTthuLNdKesg5N+icOy7pGUlXJOy73jkXk7RF0gJJ7ZJ6JH3DzP5IUtd5vzoAAKYe+mYgzRBKgYllkj7unFvlfS10zg39N/bU8Ebx813eKulq59wlkl6RlO1j32PpTbg/KCnsnBtQ/D/AP5F0h6RfJ/VKAACYHuibgTRDKAVSq0NSfsLySmT2BgAAAOxJREFU45L+0swikmRmS80sd5TnFUo66ZzrMrMLJV2V8Fj/0PNHeFbS+7xzY8olXS/pxbEaZmZ5kgqdc+slfVLx4UUAAEx39M1AmuOcUiC1tkka8Ib6fFvS1xQfnvOyN6FBk+L/CR3p15I+ZmbbJO1WfJjQkAckbTOzl51zf5Kw/meSrpa0VZKT9Bnn3DGv4xxNvqRfmFm24v/J/dS5vUQAAKYU+mYgzXFJGAAAAABAYBi+CwAAAAAIDKEUAAAAABAYQikAAAAAIDCEUgAAAABAYAilAAAAAIDAEEoBAAAAAIEhlAIAAAAAAvP/AcU8f4GiW6Y0AAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "#### Initialize the network parameters\n", "\n", "np.random.seed(1241)\n", "\n", "W_1 = np.random.uniform(-1,1, size=(3,4))\n", "W_2 = np.random.uniform(-1,1, size=(4))\n", "num_iter = 5000\n", "learning_rate = 0.01\n", "x_mat = x_mat_full\n", "\n", "\n", "loss_vals, accuracies = [], []\n", "for i in range(num_iter):\n", " \n", " ### Do a forward computation, and get the gradient\n", " y_pred, (J_W_1_grad, J_W_2_grad) = forward_pass(W_1, W_2)\n", " \n", " ## Update the weight matrices\n", " W_1 = W_1 - learning_rate*J_W_1_grad\n", " W_2 = W_2 - learning_rate*J_W_2_grad\n", " \n", " ### Compute the loss and accuracy\n", " curr_loss = loss_fn(y, y_pred)\n", " loss_vals.append(curr_loss)\n", " acc = np.sum((y_pred>=.5) == y) / num_obs\n", " accuracies.append(acc)\n", "\n", " ## Print the loss and accuracy for every 200th iteration\n", " if(i%200==0):\n", " print('iteration {}, log loss is {:.4f}, accuracy is {}'.\n", " format(i, curr_loss, acc))\n", " \n", "plot_loss_accuracy(loss_vals, accuracies)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "#### Plot the predicted answers with mistakes in yellow" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.3" }, "varInspector": { "cols": { "lenName": 16, "lenType": 16, "lenVar": 40 }, "kernels_config": { "python": { "delete_cmd_postfix": "", "delete_cmd_prefix": "del ", "library": "var_list.py", "varRefreshCmd": "print(var_dic_list())" }, "r": { "delete_cmd_postfix": ") ", "delete_cmd_prefix": "rm(", "library": "var_list.r", "varRefreshCmd": "cat(var_dic_list()) " } }, "types_to_exclude": [ "module", "function", "builtin_function_or_method", "instance", "_Feature" ], "window_display": false } }, "nbformat": 4, "nbformat_minor": 1 }