lesson1-intro.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "toc": true
   },
   "source": [
    "<h1>Table of Contents<span class=\"tocSkip\"></span></h1>\n",
    "<div class=\"toc\"><ul class=\"toc-item\"><li><span><a href=\"#Intro-to-Pytorch\" data-toc-modified-id=\"Intro-to-Pytorch-1\"><span class=\"toc-item-num\">1&nbsp;&nbsp;</span>Intro to Pytorch</a></span><ul class=\"toc-item\"><li><span><a href=\"#Pytorch-tensors\" data-toc-modified-id=\"Pytorch-tensors-1.1\"><span class=\"toc-item-num\">1.1&nbsp;&nbsp;</span>Pytorch tensors</a></span></li><li><span><a href=\"#Pytorch-Autograd\" data-toc-modified-id=\"Pytorch-Autograd-1.2\"><span class=\"toc-item-num\">1.2&nbsp;&nbsp;</span>Pytorch Autograd</a></span></li><li><span><a href=\"#torch.nn-module\" data-toc-modified-id=\"torch.nn-module-1.3\"><span class=\"toc-item-num\">1.3&nbsp;&nbsp;</span>torch.nn module</a></span></li></ul></li><li><span><a href=\"#Linear-Regression-with-Pytorch\" data-toc-modified-id=\"Linear-Regression-with-Pytorch-2\"><span class=\"toc-item-num\">2&nbsp;&nbsp;</span>Linear Regression with Pytorch</a></span><ul class=\"toc-item\"><li><span><a href=\"#Gradient-Descent-with-Pytorch\" data-toc-modified-id=\"Gradient-Descent-with-Pytorch-2.1\"><span class=\"toc-item-num\">2.1&nbsp;&nbsp;</span>Gradient Descent with Pytorch</a></span></li><li><span><a href=\"#Simplified-GD-Loop\" data-toc-modified-id=\"Simplified-GD-Loop-2.2\"><span class=\"toc-item-num\">2.2&nbsp;&nbsp;</span>Simplified GD Loop</a></span><ul class=\"toc-item\"><li><span><a href=\"#Models-in-Pytorch\" data-toc-modified-id=\"Models-in-Pytorch-2.2.1\"><span class=\"toc-item-num\">2.2.1&nbsp;&nbsp;</span>Models in Pytorch</a></span></li></ul></li></ul></li><li><span><a href=\"#Logistic-Regression\" data-toc-modified-id=\"Logistic-Regression-3\"><span class=\"toc-item-num\">3&nbsp;&nbsp;</span>Logistic Regression</a></span></li><li><span><a href=\"#Dataset-and-Data-loaders\" data-toc-modified-id=\"Dataset-and-Data-loaders-4\"><span class=\"toc-item-num\">4&nbsp;&nbsp;</span>Dataset and Data loaders</a></span></li><li><span><a href=\"#Two-layer-neural-network\" data-toc-modified-id=\"Two-layer-neural-network-5\"><span class=\"toc-item-num\">5&nbsp;&nbsp;</span>Two layer neural network</a></span><ul class=\"toc-item\"><li><span><a href=\"#Lab\" data-toc-modified-id=\"Lab-5.1\"><span class=\"toc-item-num\">5.1&nbsp;&nbsp;</span>Lab</a></span></li></ul></li><li><span><a href=\"#References\" data-toc-modified-id=\"References-6\"><span class=\"toc-item-num\">6&nbsp;&nbsp;</span>References</a></span></li></ul></div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import pytorch libraries\n",
    "%matplotlib inline\n",
    "import torch \n",
    "import torch.autograd as autograd \n",
    "import torch.nn as nn \n",
    "import torch.nn.functional as F\n",
    "import torch.optim as optim\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Intro to Pytorch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "PyTorch consists of 4 main packages:\n",
    "* torch: a general purpose array library similar to Numpy that can do computations on GPU\n",
    "* torch.autograd: a package for automatically obtaining gradients\n",
    "* torch.nn: a neural net library with common layers and cost functions\n",
    "* torch.optim: an optimization package with common optimization algorithms like SGD, Adam, etc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Pytorch tensors\n",
    "Like Numpy tensors but can utilize GPUs to accelerate its numerical computations. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create random tensor\n",
    "N = 5\n",
    "x = torch.randn(N, 10).type(torch.FloatTensor)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[-2.0993, -1.3890,  0.1010,  1.2254, -0.8571,  1.0801,  0.7751, -0.7115,\n",
       "         -1.7981, -0.5221],\n",
       "        [-0.7605, -1.3645, -1.2078, -0.9818, -1.2611,  0.4692, -0.6510, -0.5530,\n",
       "          0.5525,  1.1476],\n",
       "        [-1.3559,  0.6765,  0.6093, -1.1212, -1.6670,  0.4180,  0.7307, -0.0307,\n",
       "          0.5006, -0.6180],\n",
       "        [ 1.1176,  0.3221,  3.2936, -2.1257,  0.1021,  0.3484,  1.6506, -0.3950,\n",
       "          0.1117,  1.9128],\n",
       "        [-0.8439, -1.4928, -0.7386,  0.5823, -0.0436, -0.2074,  0.6052,  0.4644,\n",
       "         -1.5036,  0.4270]])"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[-2.0993, -1.3890,  0.1010,  1.2254, -0.8571,  1.0801,  0.7751, -0.7115,\n",
       "         -1.7981, -0.5221, -0.7605, -1.3645, -1.2078, -0.9818, -1.2611,  0.4692,\n",
       "         -0.6510, -0.5530,  0.5525,  1.1476, -1.3559,  0.6765,  0.6093, -1.1212,\n",
       "         -1.6670,  0.4180,  0.7307, -0.0307,  0.5006, -0.6180,  1.1176,  0.3221,\n",
       "          3.2936, -2.1257,  0.1021,  0.3484,  1.6506, -0.3950,  0.1117,  1.9128,\n",
       "         -0.8439, -1.4928, -0.7386,  0.5823, -0.0436, -0.2074,  0.6052,  0.4644,\n",
       "         -1.5036,  0.4270]])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# reshaping of tensors using .view()\n",
    "x.view(1,-1) #-1 makes torch infer the second dim"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Pytorch Autograd\n",
    "The autograd package in PyTorch provides classes and functions implementing automatic differentiation of arbitrary scalar valued function. For example, the gradient of the error with respect to all parameters.\n",
    "\n",
    "In order for this to happen we need to declare our paramerers as Tensors with the requires_grad=True keyword. Here is an example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = torch.tensor([1., 2., 3., 4., 5., 6.], requires_grad=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "x.grad"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(188., grad_fn=<SumBackward0>)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "L = (2*x**2+1).sum()\n",
    "L"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "L.backward() # computes the grad of L with respect to x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([ 4.,  8., 12., 16., 20., 24.])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x.grad"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## torch.nn module\n",
    "A neural net library with common layers and cost functions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`nn.Linear(5, 3)` creates a linear transformation ($A\\cdot X+b$) of a $N \\times 5$ matrix into a $N \\times 3$ matrix, where N can be anything (number of observations)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "D = 5 # number of input featutes\n",
    "M = 3 # neurons in the first hidden layer\n",
    "linear_map = nn.Linear(D, M)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Parameter containing:\n",
       " tensor([[ 0.1861,  0.3188,  0.2535, -0.1935, -0.2338],\n",
       "         [ 0.2288, -0.0527, -0.0345,  0.3245, -0.1615],\n",
       "         [-0.2683,  0.1347,  0.2408, -0.0245,  0.4171]], requires_grad=True),\n",
       " Parameter containing:\n",
       " tensor([-0.1231,  0.4153, -0.3564], requires_grad=True)]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# parameters are initialized randomly\n",
    "[p for p in linear_map.parameters()]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[torch.Size([3, 5]), torch.Size([3])]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[p.shape for p in linear_map.parameters()]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  Linear Regression with Pytorch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The goal of linear regression is to fit a line to a set of points."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Here we generate some fake data\n",
    "def lin(a,b,x): return a*x+b\n",
    "\n",
    "def gen_fake_data(n, a, b):\n",
    "    x = np.random.uniform(0,1,n) \n",
    "    y = lin(a,b,x) + 0.1 * np.random.normal(0,3,n)\n",
    "    return x, y\n",
    "\n",
    "x, y = gen_fake_data(50, 3., 8.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEGCAYAAAB/+QKOAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAV+0lEQVR4nO3dfZBldZ3f8fdHwCyIFUEG5WkccImRNbHKtIjGtXAJESiqiJabYLLBpdyl2BV1949EKpXaTW2SjVr7R7JilkwphfwR1FWjRMcHah9ES3Gnh5UniWF2ZGF2iDRisc6CgYFv/rgX0zSnu2/39Dnnnnvfr6qu7nvPufd+59Sd8z2/7+/hpKqQJGml5/UdgCRpOpkgJEmNTBCSpEYmCElSIxOEJKnRkX0HsJVOOOGE2rFjR99hSNJg7Nmz5+Gq2ta0baYSxI4dO1hcXOw7DEkajCR/udo2S0ySpEYmCElSIxOEJKmRCUKS1MgEIUlqZIKQJDUyQUjSBPYtHeSTu+9n39LBvkPpzEzNg5CkNuxbOsjFH/4GVZDAF97zRs7YdmzfYbXOFoQkrWP3fY9QBY8/+RRVo8fzwAQhSet47Y7jSeDoo44gGT2eB5aYJGkdZ2w7li+8543svu8RXrvj+LkoL4EJQpImcsa2Y+cmMTzDEpMkqZEJQpLUyAQhSWrUWoJIcl2Sh5Lctey5X0xyd5Knkyys8doLknwvyd4kV7cVoyQNXZsT+NrspL4euAa4YdlzdwFvA/7bai9KcgTwEeB8YD+wO8lNVfXd9kKVpOFpewJfay2IqroFeGTFc/dU1ffWeenZwN6q2ldVTwCfAC5pKUxJGqy2J/BNYx/EKcADyx7vHz/XKMkVSRaTLC4tLbUenCT1oamU1PYEvmmcB5GG52q1natqJ7ATYGFhYdX9JGmoVisltT2BbxoTxH7gtGWPTwUO9BSLJPVueSnp6KOOYPd9j/w0GbQ5gW8aS0y7gTOTnJ7k+cClwE09xyRJvelrLajWWhBJbgTOBU5Ish/4bUad1h8GtgFfTPKdqnpLkpOBj1bVRVV1KMlVwFeAI4DrqurutuKUpL7tWzq4Zpmor7WgUjU7ZfuFhYVaXFzsOwxJmljf95pIsqeqGuelTWOJSZLmxnpDVfu8k900dlJL0txYq3+h79aFCUKSerRW/8Jao5e6YIKQpJ6tNlS17zvZmSAkaUr1fSc7E4QkTbE+72TnKCZJUiMThCSpkQlCktTIBCFJPelzEtwk7KSWpMOw3jpKa72uz0lwkzBBSNImHc5Jvu9JcJOwxCRJm3Q4t/zc6CS4PspRtiAkaZMOZ6bzRibB9VWOMkFI0iYd7kznSSfB9VWOMkFI0mHoYqbzzN1RTpK0Nfpak8kEIUkD0MeaTI5ikjRoTaN7pn0C2lDYgpA0WE2je4Cpn4A2FCYISYPVNLoHmPoJaEPRWoJIch1wMfBQVb1q/NzxwCeBHcB9wD+tqh81vPY+4MfAU8ChqlpoK05Jw7Xa6J4+78I2S1JV7bxx8ibgIHDDsgTxIeCRqvpAkquB46rq/Q2vvQ9YqKqHN/KZCwsLtbi4ePjBSxqMprWQNrs+0jxKsme1i/DWWhBVdUuSHSuevgQ4d/z3x4E/BZ6TICRpUk2je/q8C9ss6XoU00uq6kGA8e8TV9mvgK8m2ZPkis6ikyT91LR2Uv/DqjqQ5ETg5iT/q6puadpxnECuANi+fXuXMUrSTOu6BfGDJCcBjH8/1LRTVR0Y/34I+B/A2au9YVXtrKqFqlrYtm1bCyFLmjfOoxjpugVxE/BO4APj359fuUOSFwDPq6ofj//+x8DvdBqlpLk1hBv5dKW1FkSSG4FvAa9Isj/JuxglhvOT3AucP35MkpOT7Bq/9CXAN5LcDvwZ8MWq+nJbcUrScodzj4dZ0+Yopnessum8hn0PABeN/94HvLqtuCRpLX2tnDqNprWTWpJ60dfKqdPIBCFJKziPYsTVXCVJjUwQ0pxzSKdWY4lJmmOHO6TTNY9mmwlCmmNNy2VPeqJ3vsDss8QkzbHDGdLpfIHZZwtCmmOrDemcpHTkfIHNG0ppzgQhzbmVQzonLR05X2BzhlSas8Qk6Vk2Ujo6Y9ux/LPXbueMbcc6GmpCQyrN2YKQZsRWlS02Uzoa0lVx34ZUmjNBSDNgK0/QmykdHc5oqK027fX9IZXmTBDSDNjqE/RGl5qYlqviobRkhrKUhwlCYvqvOtfT9wl6Wq6Kp6kl02Ro3zMThObeUK461zINJ+hpuCruO1GuZYjfMxOE5t60X3VOahpO0H2bhkS5miF+z0wQmnvTfNWpjZvWRDnE71mqqu8YtszCwkItLi72HYYGaGi1YQ3TNH7PkuypqoWmbbYgJKb3qrMr03jimkVD+56ZIKQ5N8TOU3XDpTakOTekpR/ULROENOeWd54WxdKP/6/rKQloMUEkuS7JQ0nuWvbc8UluTnLv+Pdxq7z2giTfS7I3ydVtxSjp/w8NffebXw7AR/7kL7j4w98wSWyBoS9g2GYL4nrgghXPXQ38UVWdCfzR+PGzJDkC+AhwIXAW8I4kZ7UYpzT3zth2LNte+LcIsdS0RZ7p2/l3N313sAm3tQRRVbcAK79hlwAfH//9ceCfNLz0bGBvVe2rqieAT4xfJ6lFQxynP81moW+n61FML6mqBwGq6sEkJzbscwrwwLLH+4HXrfaGSa4ArgDYvn37FoYqzZdpnoU8RLOQcKdxmGsanlt1Nl9V7QR2wmiiXFtBSfNgaOP0p9ksJNyuE8QPkpw0bj2cBDzUsM9+4LRlj08FDnQSnSRtoaEn3K6Hud4EvHP89zuBzzfssxs4M8npSZ4PXDp+nSSpQ20Oc70R+BbwiiT7k7wL+ABwfpJ7gfPHj0lycpJdAFV1CLgK+ApwD/Cpqrq7rTglSc1aKzFV1TtW2XRew74HgIuWPd4F7GopNEnSBJxJLUlqZIKQJDUyQUiSGpkgJEmNTBCSpEYmCElSIxOEJKmRCUKS1MgEIc2Bod+4Rv2YxtVcJW2hZ25cUwUJfOE9bxz0AnLqji0IacbNwo1r1A8ThDTjZuHGNeqHJSZpxq1345p9SwcHfVMbtccEIc2B1W5cY/+E1mKJSZpj9k9oLSYIacp0OSTV/gmtxRKTNEW6Lvms1z+h+WaCkKbI8pLP0Ucdwe77Hmn9pL1a/4RkiUmaIpZ8NE1sQUhTxJKPpokJQpoylnw0LdYtMSW5KslxW/mhSd6X5K4kdyf5jYbt5yZ5NMl3xj+/tZWfL0la3yQtiJcCu5PcBlwHfKWqarMfmORVwK8CZwNPAF9O8sWqunfFrl+vqos3+zmSpMOzbguiqv4tcCbwMeCXgXuT/G6Sl2/yM18J3FpVj1XVIeBrwFs3+V6SpJZMNIpp3GL4P+OfQ8BxwKeTfGgTn3kX8KYkL05yDHARcFrDfq9PcnuSLyX5udXeLMkVSRaTLC4tLW0iHElSk3VLTEneC7wTeBj4KPCvqurJJM8D7gX+9UY+sKruSfJB4GbgIHA7o6Sz3G3Ay6rqYJKLgM8xasU0vd9OYCfAwsLCpktfkqRnm6QFcQLwtqp6S1X9YVU9CVBVTwOb6iOoqo9V1Wuq6k3AI4wSzfLtf11VB8d/7wKOSnLCZj5LkrQ567YgqmrVEURVdc9mPjTJiVX1UJLtwNuA16/Y/lLgB1VVSc5mlMh+uJnPkiRtTl/zID6T5MXAk8C7q+pHSa4EqKprgbcDv5bkEPA4cOnhjJzS4fF+AdJ86iVBVNXPNzx37bK/rwGu6TQoNfJ+AdL8ci0mrcn7BUjzywShNbl4nDS/XItJa3LxuG7Z36NpYoLQulw8rhv292jaWGKSpoT9PZo2JgipQ2vdb9r+Hk0bS0xSR9YrIdnfo2ljgtBUm6VO20nuN21/j6aJCUJTa9Y6bS0haWhMEJpak1xxLzftrQ1LSBoaE4Sm1kauuIfS2rCEpCExQWhqbeSKe6OtDUnrM0Foqk16xW19X9p6JgjNBOv70tYzQWhmWN+XtpYzqSVJjUwQkqRGJghJUiMThCSpkQlC6sBaq7hK08pRTFLLhjLLW1qplxZEkvcluSvJ3Ul+o2F7kvx+kr1J7kjymh7CnEte6W49bwSkoeq8BZHkVcCvAmcDTwBfTvLFqrp32W4XAmeOf14H/MH4t1rklW47nOWtoeqjxPRK4NaqegwgydeAtwIfWrbPJcANVVXArUlelOSkqnqw+3Dnh+sZtcNZ3hqqPhLEXcB/TPJi4HHgImBxxT6nAA8se7x//NxzEkSSK4ArALZv395GvBsy7UtOr8Ur3fY4y1tD1HmCqKp7knwQuBk4CNwOHFqxW5peusr77QR2AiwsLDTu05Whl2iGeqU75KQsTbNeRjFV1ceAjwEk+V1GLYTl9gOnLXt8KnCgm+g2bxZKNEO70h16UpamWV+jmE4c/94OvA24ccUuNwGXjUcznQM8OoT+B0s03XOEkNSevuZBfGbcB/Ek8O6q+lGSKwGq6lpgF6O+ib3AY8DlPcW5IUMt0QyZSVlqT0YDhWbDwsJCLS6u7O/WrLMPQtq8JHuqaqFpmzOpNXhD6zeRhsK1mLRpzrqWZpstCG2Ko4ek2WcLQpvi6CFp9pkgtClbNXrIMpU0vSwxtWTWR9ZsxZBey1TSdDNBbLF9SwfZdeeDXPMnewmZ6RPf4Y4emoWZ59IsM0FsoWeuiJ986mmefGo0v8QT3+qc5CZNNxPEFnrmiviZ5HDUERnMia+Pkpgzz6XpZoLYQsuviIviqjf/LBf9vZOm/sTXZ1+Ak9yk6WWC2EJDvSK2L0BSExPEFhviFbF9AZKamCA02JaPpHaZIAQMs+UjqV3OpJYkNTJBNHD5B0myxPQcLv8gSSO2IFZwldJmtqqk+WMLYoV5GfK5kZnTtqqk+WSCWGEehnxu9ITvRDppPpkgGsz6kM+NnvDnpVUl6dl6SRBJfhP4FaCAO4HLq+ony7afC3we+P74qc9W1e90HObM2ugJfx5aVZKeq/MEkeQU4L3AWVX1eJJPAZcC16/Y9etVdXHX8XWh75sJbeaEP+utKknP1VeJ6Ujg6CRPAscAB3qKo3PT0uHrCV/Sejof5lpVfwX8HnA/8CDwaFV9tWHX1ye5PcmXkvxcp0G2yGG0koai8wSR5DjgEuB04GTgBUl+acVutwEvq6pXAx8GPrfG+12RZDHJ4tLSUktRbx07fCUNRaqq2w9MfhG4oKreNX58GXBOVf36Gq+5D1ioqofXeu+FhYVaXFzcynBb0XcfhCQ9I8meqlpo2tZHH8T9wDlJjgEeB84DnnVWT/JS4AdVVUnOZtTS+WHnkbbE+r+kIeg8QVTVt5N8mlEZ6RDw58DOJFeOt18LvB34tSSHGCWRS6vrpo4kzbnOS0xtGkqJqS2WriRt1LSVmNSCaRk+K2l2uJrrjHD4rKStZoKYEQ6flbTVLDHNCNdLkrTVTBAzxOGzkraSJSZJUiMThCSpkQlCktTIBCFJamSCkCQ1MkFIkhqZINSqfUsH+eTu+9m3dLDvUCRtkPMg1BrXh5KGzRaEWuP6UNKwmSDUGteHkobNEpNa4/pQ0rCZINQq14eShssSkySpkQmiI5MO91xrP4eMSuqSJaYOTDrcc639HDIqqWu2IDow6XDPtfZzyKikrpkgOjDpcM+19nPIqKSupaq6/9DkN4FfAQq4E7i8qn6ybHuA/wJcBDwG/HJV3bbe+y4sLNTi4mI7QR+mfUsHJxruudZ+k76HJE0qyZ6qWmja1nkfRJJTgPcCZ1XV40k+BVwKXL9stwuBM8c/rwP+YPx7sCYd7rnWfg4ZldSlvkpMRwJHJzkSOAY4sGL7JcANNXIr8KIkJ3UdpCTNs84TRFX9FfB7wP3Ag8CjVfXVFbudAjyw7PH+8XPPkeSKJItJFpeWltoIWZLmUucJIslxjFoIpwMnAy9I8ksrd2t4aWNnSVXtrKqFqlrYtm3b1gYrSXOsjxLTPwK+X1VLVfUk8FngDSv22Q+ctuzxqTy3DCVJalEfCeJ+4Jwkx4xHK50H3LNin5uAyzJyDqMy1INdBypJ86zzUUxV9e0knwZuAw4Bfw7sTHLlePu1wC5GQ1z3MhrmenmbMTl8VJKeq5d5EG3ZzDwIl7CQNM/Wmgcx9zOpXcJCkprNfYJwCQtJajb3q7l61zNJajb3CQJcwkKSmsx9iUmS1MwEIUlqZIKQJDUyQUiSGpkgJEmNTBCSpEYztdRGkiXgL/uOowcnAA/3HUTPPAYeA/AYPGMjx+FlVdV4r4SZShDzKsniamupzAuPgccAPAbP2KrjYIlJktTIBCFJamSCmA07+w5gCngMPAbgMXjGlhwH+yAkSY1sQUiSGpkgJEmNTBADkeSCJN9LsjfJ1Q3b/0WSO8Y/30zy6j7ibNt6x2HZfq9N8lSSt3cZXxcmOQZJzk3ynSR3J/la1zG2bYL/D387yf9Mcvv4GLR6X/s+JLkuyUNJ7lple5L8/vgY3ZHkNRv+kKryZ8p/gCOAvwDOAJ4P3A6ctWKfNwDHjf++EPh233H3cRyW7ffHwC7g7X3H3cN34UXAd4Ht48cn9h13D8fg3wAfHP+9DXgEeH7fsW/xcXgT8BrgrlW2XwR8CQhwzmbOCbYghuFsYG9V7auqJ4BPAJcs36GqvllVPxo/vBU4teMYu7DucRh7D/AZ4KEug+vIJMfgnwOfrar7Aapq1o7DJMeggBcmCXAsowRxqNsw21VVtzD6d63mEuCGGrkVeFGSkzbyGSaIYTgFeGDZ4/3j51bzLkZXDrNm3eOQ5BTgrcC1HcbVpUm+C38HOC7JnybZk+SyzqLrxiTH4BrglcAB4E7gfVX1dDfhTY2Nnjeew1uODkManmscn5zkzYwSxBtbjagfkxyH/wy8v6qeGl08zpxJjsGRwD8AzgOOBr6V5Naq+t9tB9eRSY7BW4DvAL8AvBy4OcnXq+qvW45tmkx83liNCWIY9gOnLXt8KqMro2dJ8veBjwIXVtUPO4qtS5MchwXgE+PkcAJwUZJDVfW5TiJs3yTHYD/wcFX9DfA3SW4BXg3MSoKY5BhcDnygRsX4vUm+D/xd4M+6CXEqTHTeWIslpmHYDZyZ5PQkzwcuBW5avkOS7cBngX85Q1eKK617HKrq9KraUVU7gE8Dvz5DyQEmOAbA54GfT3JkkmOA1wH3dBxnmyY5BvczakGR5CXAK4B9nUbZv5uAy8ajmc4BHq2qBzfyBrYgBqCqDiW5CvgKoxEc11XV3UmuHG+/Fvgt4MXAfx1fPR+qGVvVcsLjMNMmOQZVdU+SLwN3AE8DH62qxqGQQzTh9+DfA9cnuZNRqeX9VTVTy4AnuRE4FzghyX7gt4Gj4KfHYBejkUx7gccYtao29hnj4VCSJD2LJSZJUiMThCSpkQlCktTIBCFJamSCkCQ1MkFIkhqZICRJjUwQUkvG96S4I8nPJHnB+L4Er+o7LmlSTpSTWpTkPwA/w2jRvP1V9Z96DkmamAlCatF4raDdwE+AN1TVUz2HJE3MEpPUruMZ3bDmhYxaEtJg2IKQWpTkJkZ3PDsdOKmqruo5JGliruYqtWR8J7dDVfXfkxwBfDPJL1TVH/cdmzQJWxCSpEb2QUiSGpkgJEmNTBCSpEYmCElSIxOEJKmRCUKS1MgEIUlq9P8AiyIjmE5AvAEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "plt.scatter(x,y, s=8); plt.xlabel(\"x\"); plt.ylabel(\"y\"); "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You want to find **parameters** (weights) $a$ and $b$ such that you minimize the *error* between the points and the line $a\\cdot x + b$. Note that here $a$ and $b$ are unknown. For a regression problem the most common *error function* or *loss function* is the **mean squared error** ($\\sum_i (\\hat{y}_i - y_i)^2$). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "def mse(y_hat, y): return ((y_hat - y) ** 2).mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Suppose we believe $a = 10$ and $b = 5$ then we can compute `y_hat` which is our *prediction* and then compute our error."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5.274337634869753"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_hat = lin(10,5,x)\n",
    "mse(y_hat, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "def mse_loss(a, b, x, y): return mse(lin(a,b,x), y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5.274337634869753"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mse_loss(10, 5, x, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So far we have specified the *model* (linear regression) and the *evaluation criteria* (or *loss function*). Now we need to handle *optimization*; that is, how do we find the best values for $a$ and $b$? How do we find the best *fitting* linear regression."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Gradient Descent with Pytorch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For a fixed dataset $x$ and $y$ `mse_loss(a,b)` is a function of $a$ and $b$. We would like to find the values of $a$ and $b$ that minimize that function.\n",
    "\n",
    "**Gradient descent** is an algorithm that minimizes functions. Given a function defined by a set of parameters, gradient descent starts with an initial set of parameter values and iteratively moves toward a set of parameter values that minimize the function. This iterative minimization is achieved by taking steps in the negative direction of the function gradient.\n",
    "\n",
    "Here is gradient descent implemented in [PyTorch](http://pytorch.org/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((10000,), (10000,))"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# generate some more data\n",
    "x, y = gen_fake_data(10000, 3., 8.)\n",
    "x.shape, y.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Wrap x and y as tensor \n",
    "x = torch.tensor(x)\n",
    "y = torch.tensor(y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(tensor([-1.7388], dtype=torch.float64, requires_grad=True),\n",
       " tensor([-0.4666], dtype=torch.float64, requires_grad=True))"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create random Tensors for weights, and wrap them in tensors.\n",
    "# Setting requires_grad=True indicates that we want to compute gradients with\n",
    "# respect to these tensors during the backward pass.\n",
    "a, b = np.random.randn(1), np.random.randn(1)\n",
    "a = torch.tensor(a, requires_grad=True)\n",
    "b = torch.tensor(b, requires_grad=True)\n",
    "a,b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "119.15936085954331\n",
      "0.8409551256331668\n",
      "0.09556075190939882\n",
      "0.09040722863895637\n",
      "0.09001990870986476\n",
      "0.08974451465023489\n",
      "0.08953283665450877\n",
      "0.08937002512870058\n",
      "0.0892447984281157\n",
      "0.08914848014131739\n"
     ]
    }
   ],
   "source": [
    "learning_rate = 1e-3\n",
    "for t in range(10000):\n",
    "    # Forward pass: compute predicted y using operations on Variables\n",
    "    loss = mse_loss(a,b,x,y)\n",
    "    if t % 1000 == 0: print(loss.item())\n",
    "    \n",
    "    # Computes the gradient of loss with respect to all Variables with requires_grad=True.\n",
    "    # After this call a.grad and b.grad will be Variables holding the gradient\n",
    "    # of the loss with respect to a and b respectively\n",
    "    loss.backward()\n",
    "    \n",
    "    # Update a and b using gradient descent; a.data and b.data are Tensors,\n",
    "    # a.grad and b.grad are Variables and a.grad.data and b.grad.data are Tensors\n",
    "    a.data -= learning_rate * a.grad.data\n",
    "    b.data -= learning_rate * b.grad.data\n",
    "    \n",
    "    # Zero the gradients\n",
    "    a.grad.data.zero_()\n",
    "    b.grad.data.zero_()    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([2.9704], dtype=torch.float64, requires_grad=True) tensor([8.0155], dtype=torch.float64, requires_grad=True)\n"
     ]
    }
   ],
   "source": [
    "print(a,b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Simplified GD Loop"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Linear(in_features=1, out_features=1, bias=True)"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# linear tranformation with input dimension=1 and output dimension=1\n",
    "nn.Linear(1, 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Models in Pytorch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Sequential(\n",
       "  (0): Linear(in_features=1, out_features=1, bias=True)\n",
       ")"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# simple way of specifying a linear regression model\n",
    "model = torch.nn.Sequential(\n",
    "    nn.Linear(1, 1),\n",
    ")\n",
    "model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "# equivalent way of specifiying the same model\n",
    "class LinearRegression(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(LinearRegression, self).__init__()\n",
    "        self.lin = nn.Linear(1, 1)\n",
    "        \n",
    "    def forward(self, x):\n",
    "        x = self.lin(x)\n",
    "        return x \n",
    "model =  LinearRegression()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Parameter containing:\n",
      "tensor([[-0.7872]], requires_grad=True), Parameter containing:\n",
      "tensor([-0.8784], requires_grad=True)]\n"
     ]
    }
   ],
   "source": [
    "# note here we have just two parameters, why?\n",
    "print([p for p in model.parameters()])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([10000])"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x, y = gen_fake_data(10000, 3., 8.)\n",
    "x = torch.tensor(x).float()\n",
    "y = torch.tensor(y).float()\n",
    "x.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([10000, 1])"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# you have to be careful with the dimensions that your model is expecting\n",
    "x1 = torch.unsqueeze(x, 1)\n",
    "x1.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[-1.0105],\n",
      "        [-1.2466],\n",
      "        [-1.6118],\n",
      "        ...,\n",
      "        [-1.0586],\n",
      "        [-1.1920],\n",
      "        [-1.5389]], grad_fn=<AddmmBackward>)\n"
     ]
    }
   ],
   "source": [
    "y_hat = model(x1)\n",
    "print(y_hat)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Use the optim package to define an Optimizer that will update the weights of\n",
    "# the model for us. Here we will use Adam\n",
    "learning_rate = 0.1\n",
    "optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "117.57083892822266\n",
      "0.09010167419910431\n",
      "0.09003133326768875\n",
      "0.09003132581710815\n",
      "0.09003134071826935\n",
      "0.09003133326768875\n",
      "0.09003134071826935\n",
      "0.09003133326768875\n",
      "0.09003134816884995\n",
      "0.09003133326768875\n"
     ]
    }
   ],
   "source": [
    "for t in range(10000):\n",
    "    # Forward pass: compute predicted y using operations on Variables\n",
    "    y_hat = model(x1)\n",
    "    loss = F.mse_loss(y_hat, y.unsqueeze(1))\n",
    "    if t % 1000 == 0: print(loss.item())\n",
    "       \n",
    "    # Before the backward pass, use the optimizer object to zero all of the\n",
    "    # gradients for the variables\n",
    "    optimizer.zero_grad()\n",
    "    loss.backward()\n",
    "    \n",
    "    # Calling the step function on an Optimizer makes an update to its\n",
    "    # parameters\n",
    "    optimizer.step()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Parameter containing:\n",
      "tensor([[3.0131]], requires_grad=True), Parameter containing:\n",
      "tensor([7.9950], requires_grad=True)]\n"
     ]
    }
   ],
   "source": [
    "print([p for p in model.parameters()])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Logistic Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "# generating fake data\n",
    "# Here we generate some fake data\n",
    "def lin(a,b,x): return a*x+b\n",
    "\n",
    "def gen_logistic_fake_data(n, a, b):\n",
    "    x = np.random.uniform(-20,20, (n, 2))\n",
    "    x2_hat = lin(a,b, x[:,0])\n",
    "    y = x[:,1] > x2_hat\n",
    "    return x, y.astype(int)\n",
    "\n",
    "x, y = gen_logistic_fake_data(100, 1., 0.5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 1, 1, 0, 0, 0, 1, 0, 1, 1, 1, 0,\n",
       "       0, 0, 1, 1, 0, 1, 1, 0, 0, 0, 0, 1, 0, 1, 1, 1, 1, 0, 1, 1, 1, 0,\n",
       "       1, 1, 1, 0, 0, 0, 1, 0, 1, 1, 1, 1, 0, 1, 1, 1, 0, 0, 0, 1, 1, 1,\n",
       "       0, 1, 0, 0, 0, 1, 0, 1, 1, 0, 1, 1, 1, 0, 0, 1, 0, 1, 1, 1, 1, 1,\n",
       "       1, 0, 1, 1, 1, 0, 0, 1, 1, 1, 0, 0])"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7ff3200323a0>]"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEGCAYAAACO8lkDAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAA/5klEQVR4nO3dd3RU5dbH8e+eSU9ogVAEFEQsgIKK2L12EAtiu6LXq4hi7x17vfaOBcVyffViV+wdFSkKKkpREQVFSkJPn3L2+8czaIQASZiZM5nsz1qsTKac83NMsuc8VVQVY4wxpqaA3wGMMcakHisOxhhj1mLFwRhjzFqsOBhjjFmLFQdjjDFryfA7QDy0adNGu3Tp4ncMY4xpVKZOnbpEVYtqeywtikOXLl2YMmWK3zGMMaZREZF563rMmpWMMcasxYqDMcaYtVhxMMYYsxYrDsYYY9ZixcEYY8xarDgYY4xZixUHY4wxa/GtOIhIZxH5RERmicgMETkvdn+hiHwgIrNjX1v5ldEYY1KWKsycmbDD+3nlEAEuUtVtgF2As0SkB3A58JGqdgc+in1v0ohGF6FV76LRP/yOYkzj9PPPsN9+0K8f/JGY3yPfZkir6kJgYex2qYjMAjoCg4C9Y097GhgHXOZDRJMAGl2ILjkE8ACF1q8hGV18TmVMIxGJwL33wjXXQGYm3H03dOiQkFOlRJ+DiHQBtgcmA+1ihWN1AWm7jtcMF5EpIjKlpKQkaVnNRgp9CRoFLQf1IDTB70SmDjS6FPWW+R3DLFoE118PBx7ompSGD4dAYv6M+14cRKQAeBk4X1VX1fV1qjpKVfuqat+iolrXjTKpKHM7QIHM2Pd9fAxj6sIrfwot+QdavBdexQt+x6mVRhfhLTsRb8kgNDTV7zjxVVUFo0e7PoZOneC77+DVV6Fjx4Se1tfiICKZuMLwrKq+Ert7sYh0iD3eASj2K5+JP8noirR+AWl2MdL6OSSzh9+RzIaUPQCE3L+y+/xOUytdeQWEJkNkFrr8VFTV70jxMX489OkDp5wCn3/u7uvaFUQSfmo/RysJMBqYpap313hoLHBi7PaJwOvJzmYSSzK3QvKHIpm9/I5i6iLYEQgCGRDs7Hea2nkrcf1YgFbhrk4bsVWr4KyzYM893ZXDu+/CXnslNYKfVw67AycA+4rIt7F/A4FbgQNEZDZwQOx7Y4xPpNUoyDkMcgchLUf6HadW0vwakOZAJjS7GpH4/mnzqj7GW7wD3uK+aPXEuB57LaowYAA8/DCcey5Mnw79+yf2nLWQdLj86tu3rzaW/Ry0ejLoSsjeG5Esv+MYkzZW/y2TBDS5eIt3Bl3uvgl2JFD0SdzPwZIl0Lw5ZGXBxx9Dbi7sumv8z1ODiExV1b61PeZ7h3RT4pU9gq4Yjq68FF0+3O84xqQVEUlIYXAHz63xTV58j60Kzz0H22wDt8YaSvbdN+GFYUOsOCRT5VjQStAKCE1ENep3ImNMHUirkZCxDWT0QlrFsVP+99/h0EPh+ONh883hiCPid+yNlBbbhDYa2XtDxR+AQkZ3RIJ+JzJxoqoQnQOBQiRQ6HccE2eS2RNpE+exMS+/DEOHQjQK99wD55wDwdT5m2DFIYmk2cWQ2Qt0FeQc6nccEyeqiq44A6onuCGGrR5HsnbyO1ZaU68MtBoJtvY7SsN17Qq77w4PPeRupxgrDkkkEoDcgX7HMPHmLYbq8UAIFLTcikMiafXn6PIzAQ/NG0qg+cV+R6qbcBjuuMOthTRyJOywA7zzjt+p1sn6HIzZWIGWINmxb3IgY2s/06Q9Lb0bqAbCUPE4qpHknVsVr/xZvGWn4FXUo5lp6lTYaSe48kooKXFrJKU4u3IwZiOoRqD6c2h2AYS+h4yuSP4wv2Olt4zNIPITEAFpiZuglyTV46DsdjewJPQVmtEFyeq97udXVMB118Fdd0G7dm7Zi8MPT1LYjWPFwZiNoCsuhNBn7pvcIwkUnO5voCZAmt+ISgvwliLNzk/c8NXaRP9wC0aC61/yFgDrKQ5Ll8Ijj8DJJ7smpZYtk5EyLqxZyZiNUT3ODU3WCqh6z+80TYIEmhFocT2BVg8iGVsk9+S5AyHQGsh2y4pk/WPt56xY4UYfqULnzjB7Njz2WKMqDGBXDsZsnKydIfylu52d3LVv4sWrGAtVr7lZ+3knJPeTeCMjgUIo+gC8JRBou/Zw9NdegzPPhMWLYY89XD9Du3a+ZN1YVhyM2QjSaiRUvQVkQs5BfsepNw1/B6uuBiohNBWCnSBnX79jpTSRTAiuscHO4sVunsKLL8J228HYsdC31lUp/sbzPFYUr6RFUXOCKTTHAaw4GLNRRLIgd7DfMRouutC1nSuA59rUTf2ous13fvgBbr4ZLrnE7dK2AZXlVZy325XM/2kBbTq25sEv/0PzwmZJCFw31udgTFOWvVdsGe5s15aee3C9D6GqqFbGP1uqmzsXqqtdcX3wQZg2DUaMqFNhAPjyra9Z9Gsx4eoISxcu59MXErzaaz1ZcTCmkdPwTLR6UoPW6hLJRVq/hhR9gBR9WO+lP9RbhS4ZiC7eHm/p8aiG6p2h0YlG3T7OPXu6EUjg9l3Yun7zW4o6t8bz3EqygYDQtnNqzfa2ZqWNpBqGyI8Q3MTW1DFJ55U/C6W3gQQga1ek1cP1PoZIEILtGxag8nWIzgc8iMxwcz5y9mvYsRqDGTNg2DCYPBkGDoR//7vBh+qx61ac/cDJfPzceHY5dEf6DdwhjkE3nq/FQUSeAA4BilW1V+y+64BTgZLY00ao6tv+JFw/1TC69J8Q/cXdUfgMkrmtv6FM01I5BqhyfQbVH6Madh2myRIo5M8GCNXY92lq9Gg44wy358Kzz8KQIRu9XeeAofsyYGhqDgDwu1npKWBALfffo6p9Yv9SsjAAEJntCkNsnLtWPO93ItPUZO0M5AKZENwcInPxlg3DW3EB6i1L/PlzBkL+yZDRC5pdgmRtn/hzJtvqDdF22AH++U+YNQuOOy4p+zj7ydcrB1X9TES6+Jlho/ztUjwXMnr6FsU0TdLsCsjY0u2hnHc0umQgeEuBIOqVIoWPJ/b8Ikiz86DZeWj4e7zivYEQ0uJ2JHuPhJ474crK4Kqr3NfHH4ftt4dnnvE7VdL4feWwLmeLyHci8oSItKrtCSIyXESmiMiUkpKS2p6ScBIoRAqfg9wh0OwKJO9YX3KYpkskiOQdgxScCtICvBW4NqYIeAuTmkVXXOqWk/CWoCsuSOq54+7996FXL7jvPsjOBs/zO1HSpWJxeBjoBvQBFgJ31fYkVR2lqn1VtW9RUVES4/2dZPZwU/nzj7WZpcZXIgJ5x/11R079h6VuXICaf05S8U9LHSxbBiedBP37u6Lw2Wduee1AI/3v2Qgp91+sqotVNaqqHvAY0M/vTMY0GqHJf90uH53UU0uLOyHYFQKbIC3vT+q546aiAt54w81XmDbNDVFtolJuKKuIdFDV1dfDg4HpfuZJF175GKh+H3IGEsg7yu84JmFqft5L7pWsZG6DFDXCxQcXLHAL411zDXTqBL/+6kYkNXF+D2X9H7A30EZE5gPXAnuLSB9cw+lc4DS/8iWb24f499g+xAU17g+BtwoCrRvUdKXVX0Dpf1i9fo5mbNYodyrT6FKoehsyOiPZe/sdJyVJi9vRFeeBViItbvU7TmrzPNfRfMklEArBkUe6fgYrDID/o5WG1HJ3cq+FU4Sqhy4fDqEvQYKxORO90PBsdNkQ0CrI3htaPlD/AhFdo2MyuiBuuZNFNYQuPdx1uEoALbiCQL4NAFiTZG6FFL3rd4zU9/PPcOqpMG4c7L23u3LYIsnLf6e4lOtzaLKi81xhoAq0HC1/EgCteAK0FAhB9Wd/Tbirj5wDIdgWyHLDb7Mb4QzW6GI3XJPq2C5c4/xOZBorz4ODD4avv4ZRo+Djj60w1CLl+hyarEBrN9pDwe1DHPthDXYCsoGq2PNqHdm7XhJoDm3eja1B32btNegbg2AHt7mKt9BNSso5zO9EprGZPh26d3ejkP77X9e/0LGj36lSll05pAgJNEdaPQ3ZB0HBaUj+qe7+/FMh71+QtQfS6tEGr98kEkSC7RpnYQBEMpDWLyEtbkNaP0sgd6DfkUxjUVUFV17pJrHdfbe7b+edrTBsgF05pBDJ6o1k3ff3+yQLaX6pT4lSiwTyIae/3zFMY/L5565v4ccf3fyF05rM+JaNZlcOxiSBeuV4y0/HK94br7zpLMHgq7vvhr32cnsuvPcePPkkFKbxwoBxZsXBmCTQ8seherxbXqL0djQyz+9I6SsScV/33RfOPx++/97t1GbqxZqVjEkGrQJWr88jQNjHMGmqpMQVg6wsd5XQp4/7ZxrErhyMSQIpOBUyugPZkHc8kmFDJ+NG1e2vsM028OKL0LXrX8tsmwazKwdjkkAChUib1/2OkX7++AOGD4e334ZddnEznnva0vnxYFcOxsSBqodX/hzeqlutPyGZVOGbb9yezuPHW2GII7tyaMLc/tc/xfa/rv/kOvMXLX8Eyh4BqtHKV6Dt54hk+x0rPf3wg9uy8/bb3US2X36BnBy/U6Udu3JootxaRUehy45HS/ZFwz/4HalxC32Lm8WubnmPZGzR2dSEw3DzzdC7tysOc+a4+60wJIQVh6YqPBOiv8X2vy5HK1/yO1GjJvknADkgeZDZGwLtN/iapkirPsFbeizeyhvcasN1NWUK9O3rtu0cNMjt42zrISWUNSs1VcGOxBZywu1/vY2faRo9yd4Tit6FaDFk9rJdAWuh0WK3nDhVEJ6BBlsjBWdt+IWRCBxzjJvM9tprrjiYhLPi0ERJsAhaPe2uGDK3Q3KP8DtSoyfBTSC4id8xUpeuqvFNCKKL1v/88eNhp53cQnmvvAJdukDLlgkMaGqyZqUmTLJ6E2hxI4G8o+2TbiOnWoVWj0PDs/2Osm7BbpCzPxAEKUTyT6n9eStWuPWQ9twTHnjA3denT1oXBs/zmD97IRWllX5H+ZPfO8E9ARwCFKtqr9h9hcDzQBfcTnDHqOpyvzIak+pUo+jSY9wughqFlvcgOam3Z4eIIC3vRr0bQHJrXyH41VfhrLOguBguvdTdTnPRSJRL9ruen6bOISMzg3vH30SXnp39juX7lcNTwIA17rsc+EhVuwMfxb43xqxL9A+IzAUtB6pSfnCBBApqLwxXXw1HHAFt28LkyXDbbZCbm/yASfbLd/OY/fUvVFeEKF9ZwdiHU2Mfbr+3Cf1MRLqscfcg3L7SAE8D44DLkpcqfam3DC19EBCk2VkN3hvCpJhgOwjkgxcBMiFrN78T1Z2q62jOyXF7OOfmuj2dMzP9TpY0rTdp9edqH9l52WzWo5O/gWJEfV6DJFYc3qzRrLRCVVvWeHy5qq41Q0tEhgPDATbddNMd582zWakb4i09BsLT3TeZ2xFoPcbfQCZuNLoIKt+AjE0h+8DG0Yf0669uf4VNNoGnnvI7ja+mj5/Faw++y5Z9N+eoCw8lEEhOo46ITFXVvrU91mhHK6nqKGAUQN++fW2VrbqI/ApEatw26UKC7aHgVL9j1E00Cvff7+YsBINuprMqNIaCliC99tiGXnuk1nByv/scarNYRDoAxL4W+5wnfRScjtuPOjt225gkmz0bdtsNLrwQ9tkHZsyA009v0oUhVaVicRgLnBi7fSLgy1KWXvl/8Rb1xivZL20WUgvkD0OK3kOK3iOQP9TvOKaRUlW8ldfiLd4Bb9nJqNZj+GVeHixdCs89B2+8AZ39H5VjaudrcRCR/wETga1EZL6IDANuBQ4QkdnAAbHvk0q9Uii9DaiE6B9oadIjJIwEN3GTtYxpqPBXUPU6aBmEvoKKF9f//EmT3NWBKnTs6PZzHjLErhZSnN+jlYas4yGfB2kHcbt14b6KLexlzF9qfqYUqG1YKkBZmetXuP9+t3rq/PnuSiG4jueblJKKzUq+k0AetLgLgp0gc0ek2Qi/IxmTOjJ3hNwhIIWQvSfkHrX2c95/H3r1gvvugzPPdH0L1oTUqPg+lDUe+vbtq1OmTPE7hjEG3LyFrbZycxYefxx2393vRGYd0nIoqzEmhajCm2/CAQe4CW3vvusWyrO9Fhota1YyjZpqJd7yM/GK98Are8jvOE3TH3/A4MFw2GHuSgFg662tMDRyVhxMo6blz0L1Z+AVQ9kjaHim35GaDs+DUaOgRw/Xx3DnnW5UkkkL1qxkGrkwf25aJMKfM8BN4p1/vltSe5994LHHoFs3vxOZOLIrB1MvWj0Jr/gfeCX7p8SndMn7F2RuC5LrRs1kbOt3pPQWibghqgDDhrlmpI8+ssKQhmy0Uj2oVwaRnyCjGxJokfDzpSJv8c6wenuNYHcCRW/5G8gkz7ffuoLQqxc8/bTfaUwcrG+0kl051JF6y9Al/dHlp6Al+6PRhX5H8omu47ZJW1VVMGIE9O3rOp8PPbROL5s1eTaX9b+Ru099JKV2ODN1Y30OdVU9AbxyoALIhqqPIP9ffqdKOml5L7ryUpAspOXtfscxiTZtGhxzDPz0Ewwd6jqdCze8D0g4FOayA26gsqyKzOwMQLnwsTPiHs/zPJ686n9889F0Dj3jQPqftE/cz9FUWXGoq4wtAC/2jUDm1n6m8Y1k74a0He+a2KrGopHfIOegxrF/QCOnGgYtRwItk3fSNm3cYnnvv+/mMNRRqDJEuDoMQLg6wqJ5JQmJ9/7Tn/Lq/e9QXVHN3Bm/seWOm9N1280Scq6mxpqV6kgyt0ZajYK8fyOt7keyam2mazJ02RB01X/QlVegZff4HSftaXg2WrwbWrwb3oqLSGhf4Ztvwgkn/LVQ3tdf16swAOS3yGfQ2QeRkRkktyCHk244NiFRVyxeQTTsRqgFAgFWlKxKyHmaIisO9SDZuxBofhWSvbffUXylGnId81QDlVD9qd+R0p6WjwJdBUSg6gOI/hb/k5SUwHHHuT6Fb7+F4thWKg28Kjz9rhN5cfFoXip5gh67bBm/nDX0P3lf2nRqTSAYoMeuW7LdXj0Scp6myJqVTL2JZKGZ/SDyPaCQe7jfkdJfsCOQhSvIQKB5/I6t6vZXOO88WLUKbrgBLrsMsrI2+tAFLfPjEHDdWrVtwX9/fpCq8ipyC3ITeq6mxoqDaRApHO1mJgdaNvkmtmSQgjNRLYfwT0jBaUhgrW3VG66yEq68Erbc0s1b6NG4Pn2LiBWGBLDiYBpEJAty9vc7RpMhkoU0vzJ+B/Q8+O9/4dhjXYfzuHG214L5m5QtDiIyFygFokBkXRM1jNkYGl0M0fmQua0reClII/PRimch2BHJOxaRjfy1nTULTjkFJkxwReLkk90KqsbUkLLFIWYfVV3idwiTnjT8HbrsBCDg2vRbv5JyBUI1gi47GrzlQBYaXYw0v6hhBwuH4bbb4MYbIT/fzXI+4YS45jXpI9WLg+9UPdBSkOY2lj/NaMXLoLGZu9E/IPKjW6cplWgpeKtwc2yqIPxNw4916qmuIBxzjNu6s127eKU0aSiVh7Iq8L6ITBWR4Ws+KCLDRWSKiEwpKUnMBBuNLkFL9kWLd0WX/dtNQjLpI7MPUKMjM9jRryTrJi0ha1eQPCAHyR9av9dXVMCKFe72RRfB66/D889bYTAblLIL74nIJqq6QETaAh8A56jqZ7U9N1EL72nZI2jZ/UAEJA9p+RCSvVvcz2P8oapQ9SYanonkHo5kbuV3pFqpRiE8HQKtkYxOdX/hxx+7q4Xdd3edz8asoVEuvKeqC2Jfi4FXgX5JDxFoC2TGAnkQaJP0CCZxRATJPZRA88tStjAAiASRrN51LwwrVrgO5/32g0DAdTgbU08p2ecgIvlAQFVLY7cPBG5IepDcwyH6O4QmQe6xSGZiZnkaEzcTJsBRR7nZzZdeCtddB7k2B8DUX0oWB6Ad8GqsAzgDeE5V3012CJEA0uw84Lxkn9qYDfLKRkL5YxDshLR6Agm2dUNSt9rKrY+0ww5+RzSNWEoWB1X9Bejtdw5jUpVGF0DZI6BV8NxU+GgvGDsTNtkEPvnEt1zRaJTXHniH+T8uYNDZB9GlZ2ffspiNk5LFwRizIUGYF0Iu/QP5rBLdLRtWroRWcVxWowGev+01nrvlFUKVIT4Z8wX/m/8oufk5vmYyDZOyHdKm8dHq8Wj5aDQy3+8o6S0aRe57Ftl3HnwdwrtzF/j0a98LA8Dsr3+huiKEKoRDEVYsXul3JNNAVhxMXGjVe+jys9DSu9Glg1HP1tVPmMpKuP9+ZN8DkFlzCVw0EcnY8O5syTD43IPJzssmJz+bHrtuSfuubX3JEQlHWDS3mGgk6sv504E1K5m40OovgNX7BEcgOhcC2/mYKM1UV8PIkXDmmVBQAJMnQ9u2Dd5rIVG226sHz8x5kGWLVtClV2dfVhVYtbSUM/peysriVbTp1JqRX91KfvO8pOdo7OzKwcSF5AwEckDyQVpARne/I6WPCRNg++3dDOexY9197dqlXGFYrVW7lnTr3YWgTyu8fv7yJFaWrKK6MsTShcuZ/OZUX3I0dlYcTFxI9i5Im1eQFrchbd5AxMbWb7SyMjj3XNhjD3f77bfdukgpLBqNcv1RdzIg61jO3+MqKsurkp6hfde2f16xqKe+NW01dlYcTNxIxhZIzoFIoJnfUdLDSSfBgw/CWWfBjBlw0EF+J9qgqe9/x5T3pxGNRJn9za988HTyt5Dd8YDenHnvUHY5ZEfOf3Q4PXZN3dnvqcz6HIxJJUuXuuaiwkK4/nq48ELYrfGs55WTn+22HWX1Dm3+DGM9aNh+HDRsP1/OnS7sysGYVKDqVkvdZhu44AJ3X8+ejaowAGy75zYcffFhtO/alv3/tSf7Hr+H35FMA6Xsqqz1kahVWY1Jivnz3SikN96Avn1h9GjYruEjvYp/X8K7T3xM+y5t2f+EvQgE7DOgqd36VmW1ZiVj1kHD30F0CWTvkbgd4t5/H44+2u3SduedcN55kNHwX8twKMxZ/S5n1ZJSMrMzWbpgGUOuOCKOgU1TYR8pjKmFV/ECuvQEdOVF6LJ/Efcr7NXH23Zbt7T299+7oaobURgAVpasonxFBV7Uo7qimmmfzoxDWNMUWXEwpjaVrwCVoOVuox2N04zvcBhuvdWNPPI86NABXnkFunWLy+Fbb1JI9x03J7cgh+zcLA47o39cjmuaHmtWMqY22XtBeBYQhWB7kDgMz/3mGxg2zH0dPBjKy6FZfIf9igh3fXIdM774kdabtKLTlpvE9fim6VhvcRCR5kCRqs5Z4/7tVPW7hCYzxkeSfwZkdIVoCeQeishGXGRXVblhqXfcAW3awEsvwZFHxi/sGjIyM+i9d8+EHd80Dev8iReRY4AfgJdFZIaI7FTj4acSHUxEBojIjyLys4hcnujzGVOTiCA5ByH5/0YCG7naaSQCY8bAiSfCrFkJLQzGxMv6Pg6NAHZU1T7AUOAZEVk97CGhi7qISBAYCRwE9ACGiEiPRJ7TmLhatQquucZdNRQUwLffuiGqKbCstjF1sb5mpaCqLgRQ1S9FZB/gTRHpBCR6ckQ/4OfYjnCIyBhgEGBDL0zqe+MNOOMMWLgQdtkFBg6EFi38TmVMvazvyqFURP4cQhErFHvj/kgnukGzI/B7je/nx+77k4gMF5EpIjKlpKQkLidVbxle6d14ZY+imvwFw0wjV1wMxx4Lhx3mrhAmTnSFwaSVSDjid4SkWF9xOAMI1GzOUdVSYABwSoJz1dZs9berFVUdpap9VbVvUVFRXE6qy/4N5Y9D2YPoyivickzThAwd6oal3nADTJ0K/fr5ncjE0fLilZy01bkMzDmOG46+E8/z/I6UUOtsVlLVaQAiMl1EngFuB3JiX/sCzyQw13yg5s7knYAFCTyfE/kFiLh/4ekJP12qUQ2hZQ9DdC6SfyqS2XS6edRbiVY8i0g+5A2p+4zoefMgP9+NQrrrLjd3oUfTed+akjcfeZ9Fc4tRVb56bxozJ/5Er9239jtWwtRlfN7OuD/UE4CvcH+kd09kqNh5uotIV3G/pccCYxN8Tsg9EsgFciDvpISfDkC9UlQrN/zEJNDS+6F8NFS9FZsVnBq5kkGXnQRlI9HSO9GVV274BdEoPPCAWxzv0kvdfVtvbYUhjTVv3YxghtvASD2PZq3yfU6UWHWZBBfG7f8Y+6vJr6qa0OspVY2IyNnAe0AQeEJVZyTynADS/AbIGwKSi2R0TfTp8Moeh7K7gSC0vB/J2Sfh51yv6Gwg1teiIfBWQrCJbNoT+RF31QiEv17/c2fOhFNOcX0K/fvDtdcmPJ7x38HD92ferPnMGP8Dg84ewGY9Om/4RY1YXYrDV8DrwE5Aa+BRETlKVY9KZDBVfRt4O5HnWJOIQIKaUjQ0Fa18G8nuh+TEljQou4fVzVhaepvvxUHyT0NDk0A9yN4bAu18zZNUuYOg6m235lHe8et+3iuvwJAhbnjqf/8L//pXym7XaeIrIzODcx9MdHdr6qhLcRimqqvXw14EDBKRExKYKe1oZB667GSgEq18CVrlINn/gEAheMVAEIIdN3SYhJOsHaBoPHjLIVj75vBexVgIT0RyBiHZu/iQMjGk+S2QewxIDpK5zdpPiETconi77uoKwn/+A21t+0mTvjbY51CjMNS8L5Gd0ekn8gv8ufxCCMI/ACCFT0HW3pAzEGl5R4MPr9ESvJUj8FZeiUaXbFRUCTRDMjattTBo1cew6mqofBldfhoa+XWjzvW3Y0eXot6yuB2vvkQEydp+7cJQXu5WSz3ggL8Wyhs92gqDSXu28F4yZO0EgZbgAQQgZwAAktENKXx0ow+vy8+AiOuS0fCPSJuXNvqYtYr8BITcbQlAdJ5bf2gjeeVPQ6krjtr8GgJ5x2z0MePio4/g1FPh11/htNMgFIIcf7a9NCbZbMnuJJBAAdLmbaTVE0jRh0jGZvE9QfR3IOr+RX/f0LMbLudgkAKQPAgUQeZOG35NXZTdjys6ISi7Lz7H3BirVrnVU/ff3zUljRsHjzxihcE0KVYckkQk1zVbbOwibrUpOBvIcv8Kzo7/8WMkozNS9AlS+BzS5k0kEKehfMFOuEFpGRDcND7H3BjBIHz2GVx2GUybBv/4h9+JjEk6a1ZKA4H8E9CcgwCQYJt6v15Va+1jqI0ECiAQ3xFd0moUGhvSKwUXxfXYdbZwoetkvu02N6nt++/tSsE0aXblkCYk2KbehUE1irf8HHTx1nhLDkO9lQlKt34SbEegxW0EWtyCBFsn9+SqroO5Rw8YNQq+/NLdb4XBNHFWHJqy0FcQ+hxQiMyByhf9TpRcc+a4foVTToHttoPvvrMmJGNirDg0ZYFmbsIbAEHX2dyUDB8OX33lOps/+QS23NLvRMakDOtzaMIksyfa7EKoeN4Nt81N6KT31PDdd26uQlERPPqoaz7q1MnvVMakHLtyaOIC+ScRKHqHQIsbEEnjzwrV1XD11bDjju4rwBZbWGEwZh3S+K+BMTETJrh+hVmz4IQT4OabG3yocCjM2499RFV5NQcP35+Clum9Mqdpuqw4mLjwyh6Bsgch2N5N9stIgfkKAE89BSefDJ07wzvvwIABG3W4e4Y/yqcvTkSjHp+9NJGRX94an5zGpBhrVjIbTaNLoewBIATR+bE5Cz6rjO1FMWCAWxtp+vR1FobPXprI9UfdyXtPfbLBw37/+SxClSHCoQhzps2NY+CmKxqJcucpD3F81zN46toxqCZ6i3pTF1YczMaTTP76UQq45TX8smSJazo68EC3UF779nDHHdCsWa1P//Grn7n9pJGMf2UyD549mqkfTFvv4Q85/UCy87LIyc9mn2MTvedV0/Dh/33GuDETKJ63hJfvfpPp43/wO5LBmpVMHEigOdriNrdxUXBTpNklyQ+hCmPGwHnnwfLlcMUVbre2wPo//yyYs5hAwM0O99Rjwc+L2PGA3ut8/j8vGcQO+21LVXk1vfZI3y0ik6m6IvTX1YIIVRXV/gYyQAoWBxG5DjgVKIndNSK28Y9JYYHcgZA70J+Tl5S4foU334SddnKrqW67bZ1e2m/g9rRq1xLPW05Byzz2OHLDe1R032HzjU1sajjgxH8w7vkvmDV5NrscvAM7HrCd35EMIKnWvhcrDmWqemddX9O3b1+dMmWtbSdMU1FeDrvsAkOHuiuHYLBeL4+EIyyaW0LbTduQlZ2ZoJAmXagqL9/7JpPemMp+x+/JQcP28ztSg4nIVFXtW9tjKXflYEyd/PSTG5L6yCNuobxvv613UVgtIzODTt07xDefSVuT3pzK09c8T1V5NT9+9TOb9ehEj1238jtW3KVqh/TZIvKdiDwhIrWucS0iw0VkiohMKSkpqe0pJh2Fw3DrrW4tpLFj3SgkaHBhMKa+lsxfiuf91UdSMt+/HQwTyZfiICIfisj0Wv4NAh4GugF9gIXAXbUdQ1VHqWpfVe1bVFSUvPDGP19/Df36uc7mQw6BmTNdH4MxSbT3sbvTpmMhGVkZdNyiPTsfvIPfkRLCl2YlVd2/Ls8TkceANxMcJ+nUW4mW3g1aihScnzoTxlKZqutPWLQIXn4ZjjjC70SmiWrWqoAnf7iPVUtLad66GYENjIhrrFKuz0FEOqjqwti3g4HpfuZJBF1xKYTGA1E09DXSdpzfkVLXZ5/B1ltD27bwzDPQogW0SsBuesbUQyAQoGVRC79jJFQqlrzbReR7EfkO2Ae4wO9AcRedB4QBD7xiv9OkppUr4fTT3f4Kt9zi7uvSpc6Foa6j8MpXVTB/9kI8z9vwk41pQlKuOKjqCaq6rapup6qH1biKSB8FFwDZQBbkn/bn3aqKV3ofXsm+eCuvQTXqW0RfjR0LPXvCY4+5pS9WF4c6UFXuO/MxBmQdy0lbncvShcvX+dyfv/2VIZ1P4/TtL+byA2+0AmFMDSlXHJqCQG5/pO14pOgTAs3O++uB0JdQ8QRE50Pl61CVdt0tG3b//TBoEBQWwqRJcOedkFf35TjmzvidD/47Di/qsfCXxbx459h1PvfV+96msrSK6ooQsybP5rdZf9TpHJ7n8cF/P+X/bnqJkvlL65zNpJ8FcxYx5f1paTmrO+X6HJoKCdTSXqkVoLL6G9DKpGbyjSqsWuX6E/75T6iogAsvhKyseh8qtyCH1S1KwcwgBS3XXVi69OxMdl4W1RUhQChs37JO53j+ttd49uZXiITCvPHQezw772EyMu1XqamZNm4GVx5yC4FgkDYdC3nkmzvSahKl/USnkuy9IHtPqP4IMntB7iC/EyXe3Lmub6GszHU+t2sHl1/e4MO179KWcx4cxgu3v84WO2zOURcdts7nHnHBwUQiUX6ZNo/B5w2keevaF+db07RxM6iOfVIsXV7OyiWltO5gneRNzQfPfBr7YOHmPvz+wx90693F31BxZMUhhYgEkVYP+B0jOaJRGDkSRowAEfjPf+J26AFD92XA0H03+LxgMMiQywfX+/iHnH4g07/4gUAgwOa9N6vzFYdJL9vutQ2fvjCBUFWYYEaQ9l3Sa76VFQeTfPPnw9FHuz6Fgw5yS2Bs2njmeuwxeGcenno7S/5YxrZ7boOIbPhFJu0c+O+9yWuWx28z57P3sbuR3yK9dgW04mCSr7DQ7bXwzDNw/PHuyqGR6bxVRzpv1dGXc0cjUYp/X0KbjoVkZqVPG3djIyLsecTOcMTOfkdJCCsOaUKjfwBBJNje7yi1+/JL13T03HNu9NGkSY2yKPitsqySs/pdQfFvJTRrVcBDU2+nVdv0noxl/GFDWdOAV3o/WjIALdkfr+IFv+P8XXm5G3m0667w1VcwZ4673wpDg3z17rcsmb+U6ooQq5aW8ukLE/yOZNKUFYd0UP44UA2EoGyk32n+8uGHbtOde+6B005zC+X16uV3qkatXZe2f64IGggG6LB5O58TmXRlzUrpINgZor8AAcjYwu80AJQuKyX78hFkZWbCp5/CXnv5HSktbNW3G5c8cSYfPfs5/Q7egZ0Hpu6KoNWV1bxy31tUllZxxPkHp/1aROkm5XaCa4imvhOcRovRspEg2UjB2UiguX9hXnmFqVXNueaUJynUcnY/+SBOH3nahl+XYkrmL+WqQ/5D8W9LOPGGYzj8bJ+2QG3EbhpyDxNe/wov6rHp1h0ZNa3W1feNj9a3E5w1K6UBCbYl0OJ6As1H+FcYFixwy2gfeSRLLr6GUFWIRdWZvProR4Sqw/5k2giPX/4sc2f8TtmKckZd8gwrl6zyO1Kj89NXcwhXhYmGo/z2Q92WJjGpw4qD2Tiq8Pjj0KMHvPMO3HYb0/Y7hszsDESEZq2bkZnV+FovAxmBv/WZS8A60OvryAsOJjs3i+y87Ea9z3I4FOae0x7hlF4X8NZjH/gdJ2ka32+tSS033wxXX+2W1n7sMejenXPKKmlW1JJli1ZwwjVHN8pJYqfeejwLZi9k0a/FDL15CM0L67a0hvnLoLMOYof9t6O6IkS3Pl38jtNgbz76AR/+3+eEKkM8fP5T9Np9azbr0dnvWAlnxaEeNDwbXXkBaDXS4j9IVq1NdekvEoFly9wGPKee6tZDGjYMYjti5RbkcsbdJ/mbcSMVtm/FfV/c7HeMRi9eEwWrK6sJV0coaJn8Wcily8uIRtzy+RIQyldWJD2DH6xZqR505UUQ+Qmi89AVZ/uXw6vAW3E53pKj0epPk3vy775zcxYGD3aznNu1cwUiTbdKNP775uPvObJoGEe3G8aTV/8v6ec/7Iz+bNKtPRIQdj1sJ7beuXvSM/jBl99oETlaRGaIiCcifdd47AoR+VlEfhSR/n7kWyet0bGqEf9ilN3n9nqITEOXn4N6KxJ/0qoquOoq2HFH+O03t59zijQXVZZXcdOQexjW6wI+/t/nfscxcTZ6xHNUV1QTCUcZc+trhEPJHeDQsqgFT8y8l3dDYxjx7Hlpu2f0mvz6r5wOHAF8VvNOEekBHAv0BAYAD4lIMPnxaictboVAW5CWSMs7/QvileC2GQW370OCL3N//hm23971Lxx3nJvMdswxKVMcnrv5ZSa89iW/zZzPXcMeZsmCZXE79vefz+Ksfpdx1WG3srx4ZdyOa+qu/WZFf+6Xkd88z7e9M5pKUVjNl3dZVWcBtXVUDgLGqGo18KuI/Az0AyYmN2HtJKs30na83zGQgrPQ6gmgKyD3SCS4SWJPuMkm0LEj3Hsv9E+tizmA0mXlRCOxLT5FqCqPz65c0WiUKw++hcqyKoIZAe47/VGue+XSuBzb1N35j55Gdn42yxev4OSbj2uUAxwao1TrkO4ITKrx/fzYfWsRkeHAcIBNG9Fyz/EgGd2g7QQgjEh2Yk7yzjtui8433nAL5X34YWLOEwfHjRjM1A+mUfzbEg4+dX86de8Ql+NGIx6hqvCft1eWlMbluKZ+Clrmc8kTZ/kdo8lJWHEQkQ+B2pYIvVJVX1/Xy2q5r9Yp3Ko6ChgFboZ0g0I2YiIBIAGFYckSuOAC+L//c3MXFi6Ebt3if544artpEc/MGYmqxvVTZVZ2JsP+cxyjr3iW3IIcTr/7xLgd25hUl7DioKr7N+Bl84GaA4g7AQvik8islyqMGQPnngsrV8K118IVV0B2gq5MEiARzQ1HX3QYR5x3MIFgwJozTJOSaj0sY4FjRSRbRLoC3YEvfc7UNKjCgw/C5pvD11/Dddc1qsKQSMGMoBUG0+T4NZR1sIjMB3YF3hKR9wBUdQbwAjATeBc4S1WjfmRsEjwPRo2CxYvdPIXXXoMJE2xZbdNkRSNRolH7kwM+FQdVfVVVO6lqtqq2U9X+NR67WVW7qepWqvqOH/mahB9/hL33dvssjB7t7isqgmDKjBw2Jqnee/ITDin4F4Na/Juv3vvW7zi+S7VmJZNo4bDbrrN3b/j+e3jySde3YMx6jH3oXU7Z9gLuP+uxP5eSSDcjz3uCSChCdUWIB88Z7Xcc36XaUFaTaNdcA7feCkcdBQ88AO1TdM9pkzLmTJvLqEufoboixKJfitl8uy4cctoBfseKu/yWeVSWVSEBoUUbH/dESRFWHJqCykq3UF7Hjm6Y6s47w+GH+53KNBJlK8r/nB0cjUYpXV7mc6LEuOWtEdx/9uNk52ZzwaONb4OqeLOd4NLduHFuYby2bWH8+JRZ8sI0Hp7ncdM/72b8K1+yaY+O3D3uBpq3tiXM08H6doKzK4d0tXIlXHqpG43UrRvcdJMVBtMggUCAa168mGg0StAGLDQZ1iGdjr7/3s1ufvxxuPhit8z2Pvv4ncqkgMryKn6cMofyVfVfrNEKQ9NiVw7pRNVdHXTr5voVRoyAvk10QyKzllVLSxne+yIqS6vIzMnkkW/uoM0mhX7HMinKrhzSgSo8/bQrCBUVbqG8V16xwmD+Zsp731K+qpKK0koqy6qY+PpXfkcyKcyKQ2M3dy4MGAAnnQRZWW5UkjG12HSbTqjnljYXgS69mtZqxqZ+rFmpsfI8txbSiBHuN/3BB+GMM2y7TrNOW2zflWtfvoQvXp1Mv4N2YNs9t/E7klmDqvLMDS8y5b1vOfCkfThkuH/zSaw4NGYvvAD/+Ac8/DA0sT0tTMPs1L8PO/Xv43cMsw6fvjCBF+8cS1V5Nb9+9xtb9OnC1v382bPaPmY2JqEQ3HILLFrkrhDeegvefDPtC8O8WfN576lPWDyvxO8oxiTUskUr/tzVUALC8sX+bU1rVw6NxeTJMGwYzJgBzZvD2WdDixZ+p0q4OdPmcv7uV4G48faPz7iHok6t/Y5lTELsf8JejH3oPRb9upjNe2/Gjgf29i2LFYdUV14OV10F993nlr948004+GC/UyXNtx9PJxqJEg5FyG2Ww8yJP/GPo3f1O5YxCdG8sBlP/nAfFaWV5DXL9XUfEWtWSnUjRsC997rO5hkzmlRhAOi9d08CGUGyc7MA2GbnLXxOZExiiQj5zfN832DKl7WVRORo4DpgG6Cfqk6J3d8FmAX8GHvqJFU9fUPHS7u1lZYtg1WroEsXKC52ey/suaffqXzz6/fzmDnxJ7bfb1s26WaryBoTL6m4ttJ04Ajg0Voem6OqfZIbJ0WowksvwTnnwJZbwqefugXz2rb1O1nceZ5HZVkV+c3zNvjcrttuRtdtN0tCKmPMan7tBDdLVX/c8DObkAUL4Igj4JhjXN/C/fen7UJ5SxYs419dz+TINkMZcfAtti1jAv06/Teeu+UVvv1kut9RTCOTin0OXUXkGxH5VESaRlvK5Mluobx334Xbb3ff9+njd6qEeWvUByxbuJxoxOP7z2cxa+JPfkeqk2gkys1D7uWINkO585SH8GKzjVNV8W8lnLvrCJ6+9nmuOvQ/ViBMvSSsOIjIhyIyvZZ/g9bzsoXApqq6PXAh8JyI1Lolk4gMF5EpIjKlpCR5499VQ3jLT8NbvAPeyhGobsQfiEjEfd1uOxg82K2mesklkJHeg8hadygkI8v9N2rUo0VR49h1a9zzE5j05hRKl5Xx6QsTmPzW135HWq9fvvuNQDCAF/UIV0eY2UiKsEkNCfsrpKr7N+A11UB17PZUEZkDbAms1dusqqOAUeA6pDcubT1UvgrVE4EqqHobcg6C7Hpe4EQibgTSU0+5q4T8fLeXcxNx0Cn7svCXRUwf/wODzh5A5606+h2pzv4cv6FuqYNU1mO3LcnKcaO8vKjHrofu6HOihilbUc5jl/8fZcvKGHrTEDptuclaz5n6wTTuHPYQ2bnZXPvSRdZHFQcp9RFVRIqAZaoaFZHNge7ALz7HWsOaa9rX8+Jr2jQ3mW3qVDjsMLeFZ35+3NI1BsFgkFNvO8HvGPX2j2N2ZeIbU5j6wTR2G9SPXQ5J7T+2zQubMXrmPcyc8BObb7cpbTct8jtSg9wxdCRfvv0N0UiUmRN/4n+/rz2O5cZj7qZ8pduj4vaTHuThqXcAMPntr7n3tEfJb5HHtS9f3Kg+iPjNl+IgIoOBB4Ai4C0R+VZV+wN7ATeISASIAqeramotM5p7GFR/DqFJkHswZO1Wt9eFw3D99XDbbVBYCM8/D0cfnbadzukoIzODq8Zc4HeMemle2Czli9iG/PHzIiJh1wS7bNEKVHW9cwBWX9CpKjcecxfVFSGWLljGXac8zL2f35SMyGnBr9FKr6pqJ1XNVtV2scKAqr6sqj1Vtbeq7qCqb/iRb31Esgi0uo9Au8kEml9T94kqwaAbmnr88TBzphuVZIXBmA06+eYhZOVkkpmdwZArBtf6O3flmAsobN+SDt3accmTZ/15/1+FArxoajcDphpfJsHFW8pOglu1yl0tXHIJtG8PVVWQk+N3KmManfKV5YSqwrRq1/Jv93/8v8956prn6bhFB6549lyaFzb72+PjX53MvaePIr9FHte/eildenZOYurUt75JcFYcEuXtt+H002H+fNfx/O9/+53ImLSyamkpx3Y6jXB1mIzMIP1P3pfzHx7ud6xGZX3FIRXnOTRuJSWu6ejgg6FZM/jiCysMxiRAOBQB3IdbL+pRVV7tb6A0Y8Uh3q66Cl58Ea69Fr7+Gna1FUSNSYTWHVpx3JVHkpmdSYfN2zH0xmP9jpRWrFkpHn7/3fUndO/urhyKi6FnT//yGGNMHVizUqJ4Hjz0kFv64vTY4rFFRVYYjDGNnhWHhvrhB7d/81lnuaajxx/3O5Gpo5+/+ZV/djyVQwr+xXtPfuJ3HGNSkhWHDVBvFd6Ki/GW/gsNfePu/OQT6N3bbb7z1FPw3nvQtauvOU3dPXDO4yxbuILqimruPWOUrQprTC2sOGyArroRqt6B8JfowpNQDcMuu7id2WbNghNPtMlsjUxOfg4ScP/PMjKDvu+4ZdJLqDrMpy9O5OuPvk/59bfWJ6XWVkpJ3mKoqEbuWgbvlsO3y6FZW7dwXhM3c+KPPHn1GNp3acsZ95xEXrNcvyPVyUWPnc4tx93HqqWlnDPyFAIB+4xk4ueKATfx09Q5oHDciCMYcsURfkdqECsOGyBT9oLTnkfmhtCTdkZoHH8AEy1UHeby/jdRWVZFZlYGEhAuHLXBHV1TQttNi7h3vK2xY+IvGony/Wcz/1y245MxXzTa4mAfmdalshKGD0f6nwrBzuiHbyNPTnIT2wyhyhDh6jDgJiMVz0venhrGpKpgRpDuO3YjOy+bnPxsdj201lGijYJdOaxLdjbMng2XXIJcdx3kbXiv46akoGU+g84+iNcefIfsnCxOsglIxgBw5yfXMW7MFzQrLGD3w/v5HafBbBJcTYsWwRVXwC23QIcOblOeNN+VbWOVrywnKzeLzKzMdT5HVVk8r4SWbVuQk5edxHSpIRwKE414TfK/3aQ2mwS3IapuSGqPHvDcczBpkrvfCsMG5bfIX29hiEajXLr/DQzreQFDOp3GvFnzk5guPqLRKIvmFhMOhev92m8/mc7gwqEc3vJEnr/9tfiHMyZBrDj8+iv07w9Dh7riMG2a28/ZxMUv0+bxw1c/E6oMUbaynDceetfvSPVSWV7FaX0u4ZSeF3DC5mexbNHyer3+0YufprqimmgkypNX/c/mVJhGw5fiICJ3iMgPIvKdiLwqIi1rPHaFiPwsIj+KSP+Eh7nxRpg4EUaOhM8+g623/vOhUHWYyw68gQFZ/+TyATc16JNjU1fYoRXquabL7NwsOnbv4HOi+vn6g+8onldCdWWIVUtLGTdmQr1e37pja4IZbmvZvOZ5NmzWNBp+/aR+APRS1e2An4ArAESkB3As0BMYADwkImtu2hxft9/uZjqfeSas8Yv7+UuTmDnxJ6IRjxlf/MDnL09OaJR01LpDK25643J2G7QTQ644gsPOGuB3pHppt1kRnucBbiRK+83b1uv1F48+g90P34k++/Ti9g/qsXOgMT7zpVFdVd+v8e0k4KjY7UHAGFWtBn4VkZ+BfsDEhIVp02adD+UW5Kz3e1M3ffbpRZ99evkdo0G22L4rlz51Nh/891N2GtCn3kMTWxa14OoXLkpQOmMSJxV6XE8Gno/d7ogrFqvNj923FhEZDgwH2HTTTRMSbNfD+nLIaQcw4fWv2H3wzo1+o3bTMHsdtSt7HWX7cpimJWFDWUXkQ6B9LQ9dqaqvx55zJdAXOEJVVURGAhNV9f9ij48G3lbVl9d3Lt/3czDGmEZofUNZE3bloKr7r+9xETkROATYT/+qUPOBmjuAdwIWJCahMcaYdfFrtNIA4DLgMFWtqPHQWOBYEckWka5Ad+BLPzIaY0xT5lefw4NANvBBbPTGJFU9XVVniMgLwEwgApylqjYw3Bhjksyv0UpbrOexm4GbkxjHGGPMGmxGjjHGmLVYcTDGGLMWKw7GGGPWkhZLdotICTBvIw7RBlgSpzjxZLnqx3LVX6pms1z109Bcm6lqUW0PpEVx2FgiMmVdE0H8ZLnqx3LVX6pms1z1k4hc1qxkjDFmLVYcjDHGrMWKgzPK7wDrYLnqx3LVX6pms1z1E/dc1udgjDFmLXblYIwxZi1WHIwxxqylyRaHlNrH+u+5jhaRGSLiiUjfGvd3EZFKEfk29u+RZOZaX7bYY769Z2vkuE5E/qjxPg30K0ssz4DYe/KziFzuZ5aaRGSuiHwfe4983QxFRJ4QkWIRmV7jvkIR+UBEZse+tkqRXL7/fIlIZxH5RERmxX4fz4vdH9/3TFWb5D/gQCAjdvs24LbY7R7ANNyqsV2BOUAwibm2AbYCxgF9a9zfBZju83u2rmy+vmdrZLwOuNjvn69YlmDsvdgcyIq9Rz38zhXLNhdo43eOWJa9gB1q/nwDtwOXx25fvvr3MwVy+f7zBXQAdojdbgb8FPsdjOt71mSvHFT1fVWNxL6dhNtYCGrsY62qvwKr97FOVq5Zqvpjss5XH+vJ5ut7lsL6AT+r6i+qGgLG4N4rU4OqfgYsW+PuQcDTsdtPA4cnMxOsM5fvVHWhqn4du10KzMJtpxzX96zJFoc1nAy8E7vdEfi9xmPr3MfaB11F5BsR+VRE9vQ7TA2p9p6dHWsufMKP5ogaUu19qUmB90Vkamw/9lTTTlUXgvtjCLT1OU9NqfLzhYh0AbYHJhPn98yvzX6Soh77WEeAZ1e/rJbnx3W8b11y1WIhsKmqLhWRHYHXRKSnqq5KgWwJf8/+drL1ZAQeBm6Mnf9G4C5c8fdDUt+XetpdVReISFvcpls/xD4pm/VLmZ8vESkAXgbOV9VVsY3T4iati4Om6D7WG8q1jtdUA9Wx21NFZA6wJRDXzsSGZCPJe3/XNaOIPAa8magcdZCye6Kr6oLY12IReRXXBJZKxWGxiHRQ1YUi0gEo9jsQgKouXn3bz58vEcnEFYZnVfWV2N1xfc+abLNSY9vHWkSKRCQYu705Ltcv/qb6U8q8Z7FfitUGA9PX9dwk+AroLiJdRSQLOBb3XvlKRPJFpNnq27jBGX6+T7UZC5wYu30isK6r1qRKhZ8vcZcIo4FZqnp3jYfi+5752evuc4//z7j24G9j/x6p8diVuFEmPwIHJTnXYNwnzmpgMfBe7P4jgRm4ES9fA4f68J7Vms3v92yNjM8A3wPfxX5ZOvj8czYQN5pkDq5pzrcsNTJtHvs5mhb7mfI1F/A/XLNpOPbzNQxoDXwEzI59LUyRXL7/fAF74Jq1vqvx92tgvN8zWz7DGGPMWppss5Ixxph1s+JgjDFmLVYcjDHGrMWKgzHGmLVYcTDGGLMWKw7GJJiIvCsiK0TEzwl5xtSLFQdjEu8O4AS/QxhTH1YcjIkTEdkptiBbTmwW8gwR6aWqHwGlfuczpj7Sem0lY5JJVb8SkbHATUAu8H+qmmrLUhhTJ1YcjImvG3BrKlUB5/qcxZgGs2YlY+KrECjA7dCV43MWYxrMioMx8TUKuBq3P8htPmcxpsGsWcmYOBGRfwMRVX0utrz6BBHZF7ge2BooEJH5wDBVfc/PrMZsiK3KaowxZi3WrGSMMWYtVhyMMcasxYqDMcaYtVhxMMYYsxYrDsYYY9ZixcEYY8xarDgYY4xZy/8Dg9BYYOazYCkAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "t = np.arange(-20, 20, 0.2)\n",
    "import matplotlib.pyplot as plt\n",
    "plt.scatter(x[:,0],x[:,1],c=y, s=8);\n",
    "plt.xlabel(\"x1\"); plt.ylabel(\"x2\");\n",
    "plt.plot(t, t + 0.5, 'r--')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = torch.tensor(x).float()\n",
    "y = torch.tensor(y).float()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Sequential(\n",
       "  (0): Linear(in_features=2, out_features=1, bias=True)\n",
       ")"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model = torch.nn.Sequential(\n",
    "    torch.nn.Linear(2, 1),\n",
    ")\n",
    "model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([100, 1])"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model(x).shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "x, y = gen_logistic_fake_data(10000, 1., 0.5)\n",
    "x = torch.tensor(x).float()\n",
    "y = torch.tensor(y).float()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "learning_rate = 0.1\n",
    "optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3.3247745037078857\n",
      "0.01566370017826557\n",
      "0.01058468222618103\n",
      "0.008095240220427513\n",
      "0.006499259267002344\n",
      "0.005346684250980616\n",
      "0.004459547344595194\n",
      "0.0037507289089262486\n",
      "0.0031708863098174334\n",
      "0.0026887478306889534\n"
     ]
    }
   ],
   "source": [
    "for t in range(10000):\n",
    "    # Forward pass: compute predicted y using operations on Variables\n",
    "    y_hat = model(x)\n",
    "    loss = F.binary_cross_entropy(torch.sigmoid(y_hat), y.unsqueeze(1))\n",
    "    if t % 1000 == 0: print(loss.item())\n",
    "       \n",
    "    # Before the backward pass, use the optimizer object to zero all of the\n",
    "    # gradients for the variables\n",
    "    optimizer.zero_grad()\n",
    "    loss.backward()\n",
    "    \n",
    "    # Calling the step function on an Optimizer makes an update to its\n",
    "    # parameters\n",
    "    optimizer.step()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Parameter containing:\n",
      "tensor([[-17.5781,  17.5879]], requires_grad=True), Parameter containing:\n",
      "tensor([-8.7648], requires_grad=True)]\n"
     ]
    }
   ],
   "source": [
    "print([p for p in model.parameters()])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Dataset and Data loaders "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Nearly all of deep learning is powered by one very important algorithm: **stochastic gradient descent (SGD)**. SGD can be seeing as an approximation of **gradient descent** (GD). In GD you have to run through *all* the samples in your training set to do a single itaration. In SGD you use *only one* or *a subset*  of training samples to do the update for a parameter in a particular iteration. The subset use in every iteration is called a **batch** or **minibatch**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "model2 = torch.nn.Sequential(\n",
    "    torch.nn.Linear(1, 1),\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [],
   "source": [
    "from torch.utils.data import Dataset, DataLoader"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [],
   "source": [
    "def lin(a,b,x): return a*x+b\n",
    "\n",
    "def gen_fake_data(n, a, b):\n",
    "    x = np.random.uniform(0,1,n) \n",
    "    y = lin(a,b,x) + 0.1 * np.random.normal(0,3,n)\n",
    "    return x.astype(np.float32), y.astype(np.float32)\n",
    "\n",
    "# create a dataset\n",
    "class RegressionDataset(Dataset):\n",
    "    def __init__(self, a=3, b=8, n=10000):\n",
    "        x, y = gen_fake_data(n, a, b)\n",
    "        x = torch.from_numpy(x).unsqueeze(1)\n",
    "        y = torch.from_numpy(y)\n",
    "        self.x, self.y = x, y\n",
    "    \n",
    "    def __len__(self):\n",
    "        return len(self.y)\n",
    "    \n",
    "    def __getitem__(self, idx):\n",
    "        return self.x[idx], self.y[idx]\n",
    "    \n",
    "fake_dataset = RegressionDataset()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we are going to create a data loader. The data loader provides the following features:\n",
    "* Batching the data\n",
    "* Shuffling the data\n",
    "* Load the data in parallel using multiprocessing workers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataloader = DataLoader(fake_dataset, batch_size=1000, shuffle=True)\n",
    "x, y = next(iter(dataloader))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(torch.Size([1000, 1]), torch.Size([1000]))"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x.shape, y.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [],
   "source": [
    "learning_rate = 0.1\n",
    "optimizer = torch.optim.Adam(model2.parameters(), lr=learning_rate)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "78.26276397705078\n",
      "0.1252632588148117\n",
      "0.09439030289649963\n",
      "0.08993063122034073\n"
     ]
    }
   ],
   "source": [
    "for t in range(200):\n",
    "    for x, y in dataloader: \n",
    "        \n",
    "        y_hat = model2(x)\n",
    "        loss = F.mse_loss(y_hat, y.unsqueeze(1))\n",
    "    \n",
    "        optimizer.zero_grad()\n",
    "        loss.backward()\n",
    "    \n",
    "        optimizer.step()\n",
    "    if t % 50 == 0: print(loss.item())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Parameter containing:\n",
      "tensor([[3.0088]], requires_grad=True), Parameter containing:\n",
      "tensor([7.9979], requires_grad=True)]\n"
     ]
    }
   ],
   "source": [
    "print([p for p in model2.parameters()])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Two layer neural network"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "# generating fake data\n",
    "# Here we generate some fake data\n",
    "def sigmoid(x):\n",
    "    return 1/(1 + np.exp(-x))\n",
    "\n",
    "def gen_nn_fake_data(n):\n",
    "    x = np.random.uniform(0,10, (n, 2))\n",
    "    x1 = x[:,0]\n",
    "    x2 = x[:,1]\n",
    "    score1 = sigmoid(-x1 - 8* x2 + 50)\n",
    "    score2 = sigmoid(-7*x1 - 2* x2 + 50)\n",
    "    score3 = 2* score1 + 3*score2 - 0.1\n",
    "    y = score3 < 0\n",
    "    return x, y.astype(int)\n",
    "\n",
    "x, y = gen_nn_fake_data(500)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEGCAYAAABiq/5QAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAACjlElEQVR4nOydd3hUVROH37M1vUBC7733IkWKKE2lKFUQBBUVEURFwAIIgqAgUgSlKKCIoDSVXqRJ7713CCQkIXX7Pd8fNwTykZ7dTYC8z8NDsnvvObOb3bnnzpn5jZBSkksuueSSy5ODJrsNyCWXXHLJxb3kOv5ccskllyeMXMefSy655PKEkev4c8kll1yeMHIdfy655JLLE4Yuuw1ID0FBQbJEiRLZbUYuueSSyyPFgQMH7kgpg///8UfC8ZcoUYL9+/dntxm55JJLLo8UQogryT2eG+rJJZdccnnCyHX8ueSSSy5PGLmOP5dccsnlCSPX8eeSSy65PGHkOv5ccskllycMlzl+IcRPQohQIcTxBx7LI4TYIIQ4l/B/oKvmzyWXXHLJJXlcueKfB7T+v8eGAZuklGWBTQm/P3HEx5gwxZqy24xccskw0nYUJXIASsxkpLRmtzm5ZBKXOX4p5TYg4v8ebg/MT/h5PtDBVfPnVFbN3sDLwX15Obgvmxdtz25zngh2/3OAkS99zYrpa8isDLmUCtK0HCXqU5SwVijhPZCOECdbmrORSiwyohdY1kPcz8jY77PbpAwhpUSJHocS2hQlahRSKtltUrbh7hh/fillCEDC//lSOlAI0U8IsV8IsT8sLMxtBrqaOUMXYrfasVnszProl0yN8d+KvSz4YglXT99wsnWPH1dP3+DLbt+yc8U+5g5fyLY/d2dqHBk3Exk1Ckx/gOMS2PYjoz5xrrEuQirRKHc6otyqgHJ3cOYdnowC6Uj4xQz2i06z0S1YtoBpCSghYF4BlnXZbVG2kWM3d6WUs6SUdaSUdYKDH6o4dtWcbF+2hz8m/U14SKRL5shbKBChEWi0GoKK5M3w+VsW/8dXPafy6+g/ea/+cKLuRLvAyseH21fC0GjVj7nNYiPkwq3MDWTZDTwYnpMgY7Nsn1swLQH7WUABy79gy2QVvKYQGJsBRhBeCJ83nWikO7A88LMEac42S7Ibd0s23BZCFJRShgghCgKhbp4/RXb/c4Cp784mPCQSjUbDn5P+4pdLMzAY9U6dZ8zfw/jxw/lodFre+fa1DJ9/fMdpLPHqB1giuXH+Fv5Bfk6xzW6zM77nVA5tPkaTTg0YOONNhBBOGTu7qN60EoVKF+D6mZt4+nrQomeTzA3k1R2ijgAKYAP8EX4jnGhpUqT9CjJqOGBD+I1G6CtmfjDhDWgBG0gJwitzwwgBAVPVFbPwR2i8M29TdmBsAYYGYNkO+trg8Xx2W5RtCFe2XhRClAD+kVJWSfj9GyBcSjleCDEMyCOl/DitcerUqSNdqdUTHR5D92JvYzXd36wyeBqYe2IyBUqkGI3KFk7uPsvQ50YjNBry5PfnxyMTMXoanTL2+vlbmPruHCzxFjx8PPh88QfUa1PTKWNnJw6Hg9Ard8hbKBCDhyHT40j7eXCEgqEuQjh3QfD/KHc6gP0UIEFTCE2+LZkeS0obMnokWPeBVzc03q87y8wcjzRvRtpOIDzbIHRlstsctyOEOCClrPP/j7tsxS+EWAQ0A4KEENeBkcB4YIkQ4nXgKtDZVfNnBFOsWV0JJaDVaSlcpgDBRTMeinE1lZ4qx5zjk7lxLoSKDco5zemDGuoS//f744BWq6VgqfzpOjYmMhadXounj+dDzwldGXCi81DfXwUhtMk8GQPIB37OPELoEf7jsjTGo4g0r0Pe/RgwI+N/gqCNCG3O+05nB67M6ukupSwopdRLKYtIKedKKcOllC2klGUT/v//rJ9sIX/xYNq80QKNRuCbx4cB0/oyddc4tNpkvpA5gPzFg6n1bDU8vT2cOu4zrzSmdqvqeAd40bRLA+q2ruHU8Z2N1WJj7c//smHBVuw2e5bGMsWa+HXMH3Qp+Cad8r3Ozr/2OcnK5JG248jQesjbVVBi5zz0vPAbnRCSMYDfGJfa8rgirYdQ92USLqCOy9loTc7CpaEeZ+HqUM89rBYbeoPukY9rPymM6DCBgxuPIoSgYfu6DP910EPHKIqCRpP6+uafH9fz/aCfsVvvXzyKli/ET6emON3mRLvCe4Jtb8JvOkT+gwiR9EJ+77v5qH4e1RDTKLDuBc9uaHzcG2KStqPIiFcBDWgCEUGrEOLhO7kUz1fiwLoNtIUR+mqpH2s9goxfCPpKCK9eCJG1NbW0HQclFgz1sjSW20M9jyLO3sjNxbUc2XICS7y6L3Nw49Ekz928cIsPm48kIuQuLw9+gX5fv5riOLM//jWJ09fqtOR39d6OJhD162cHoSe5r+Kj6vATMS0F09+AGWKnIo31EPqqbpte6KtB0GqwXwJ9zYw5felAhndSN7KlRPp/icbzRRTTKoibB4YaCN+PEUKPVCKQkb1BxoN5HRItwjvlz1taKHE/QcwUEAIMTRCBUzM9Vkrk2HTOXHJJi0Yd6uHh44GHt5EmnRskee6X0X8QfiMSxaGwYvoaQq/dSXGcwAIBiSm2PoHeNH+lMcN+eS/N+ZX4v1Bu10MJa420X86Q7cJvFBieBl1lROBshMi+NZi0HUOJmYa07HLywHGoWVCoTkyJc+746UBoCyOMjTOegaTcAsd11ZljAtPfSPtViBoO9iMQv1hd4QM4bidGk8CUkDqbBeJ/V8eR8WBZ75K9ttwVfxaIDI0iPjqeQqULJLs6O7XnHIu+WkbRcoXoPaaby+8oIkOjsJqs5C/unrqH7OTY9lMc/vcEnt5Gug3rSMeBbZM87xPog06vxWa1IwCjZ8rZPONWf8LMD+ZhMOp557s+BBXKk+b8UlohWk23xBGFjP4CkefndNsvtHkReX5MOqYSA0ooaEtmOVSQXqT9CjK8J2BGxhkhz1yEoa5zBvfsoq747afB0BQM9ZwzrguQ5g2qw/Z4HqErAZp8oMkDSgQIDRibgxKp/iwBbKAkFJbqyoK+QkIWlgbh1T1rxhjqgekW4ABdGZfc+eU6/kyy8699jO3+HQAtezVl0Mx+SZ43xZoY+txoTLFmDngYQMCbEzJ/+5cWW//Yxde9pwHQfkBr+n3dy2Vz5QTGdv+O8JtqbsDK6Wt5aVDSnOw+Y7oRGRLJlVPX6f1F11RrHQqVLsCYlZmRjXrwC5m1RABpO4WMeAWkAvrKkOeX5LN9nI39dIIzk4ADbMfASY5faHwRQSvUbLEMOC9pO46MfA8wI/wnIIyZrL1IJ0r8XxDzOUgLxP0Ewf8iNH4QtBxMq0FXFGFsqlY8GxqBZTNoghBe6vdZCB3kWahWMmvzq+dmAeE3EvRV1AJBzy7OeIkPkev4M8nCL/9MzPtfPXsj73zXJ8mKPvZuPHa7Wt5uNVvZsXwv/sF+dBzYFr3B+Sv/X75YgtVsA2Dp5FW8OeHVRz9GnCr3b3+TuxX28vXks8UfuGx2IQxI/0kQ8yVo8qhf1iwg4xcmhEYA20l19ZmVoq30YqiLWokrAKmubJ1MRj+HMmoYKKocibw7GJH/gNNtSoJtL8h7VdkKOK6BpjJCkwe8eyYeJoQGEfg9UokH4ZnkdQmhBX1Zp5gjhB68ujllrJTIjfFnkpJVi6P30COEICCfP3pD0mtoUOE8NO5QD71RjxAQcvE280cuYcb781xiT7GKRdAbdGg0guAieVP8slnNVlbN2sDq2RuxWtQLhZSSTQu3M2/E79w4/2gIj326aDDBRfOSv0Qwwxc+nM3jDjSerdDk244maCVCVzRrg+nKAAlZPUKCNn11B1lFaPIggtciAqYggtYhdCXdMm/qPHin43oXJTzbAx5qhbMmOM1aDaHxeuQXVbnpnJnEHG9h4ZdLibwVSfdPXqJwmYIPHSOlJORSKL3LDkhcoJatXYoZ+yY43Z74GBMLRi0mJiKOniM6UbBk8o7j8/YTOJSQAVOndQ1GLR3CXzPWMnvor1hMVnz8vVh4ZWayBUxPItfO3GDp5H/IVzyYLh+1Q6d3zU2ylAoy7mewn0R49UQYck7FtLQdV2UODHURhocyA10w3xnk3UEg74V66rt+TvtVcFwBfW2EJnOSFjmR3HROJxMbGcv2pbsIuRiK0cvIgGmvP7QKEEJQqFR+mnVtxO6/96Mokk4fvugSe7x8PXl70muAesGxWW3JhpSObT+JJSFEdWzbyYTHTmOOU/V/bFY7YdcjKFahsEvsfJSwWW0MavQZMZGxGDwMxN2Nc9k+jRAahJvz3NODtF9EhvdAFTgzqHsPhurpO1dKMK9C2s8jPNun+25C6Msjgtdm3uhMIHTFQFfMrXNmJ7mhnkyy+OuVhFwMRXEorJv3L1dPXU/x2E8WDuLbraOZc/xbnunW2KV2RYZG8Vq593jeqwejXv4GRUkqwdusS8OEFEgPmnVrBEDbN1tg9DLg6etBkXKFKFymgEttTI0b50P4stu3fPf2j8REZq/6ZVxUfIKcB1hNVs4fupT43Obfd/Ba+YGM6DCBuCj3pym6DdtJ1E1sRf1nO5LuU6XpT2TUpxA3ExneEcWeSWXUxwgprUjLHvUOIxt5rFf8d25GMKHXNCJv3eWdya9R+7n0rVTSg3eAN1qdFsWhIBWJp0/K8glCCMrWKuW0uVNj1Y/ruX31DlKRHNhwlJM7z1Cl8f1NwkEz+9GkUwMQgprPVAGg5jNVmX3sW25fDqNSw/Joda7JJjHFmtj6x2788vrQ4MU6ycZJh7T4gjs3ItBqtUTcusvoFUNdYkt68A/yo16bmhzceBSpSDp/1B5QL66T+s7AarZx+0oY80Ys5t0pfbPNTpdiqA/CgLpGlGBsmv5zbftJlLKW8RDeDhm05onVy1GLwrqD44KavRU4DZGR99OJPNaOf2r/2RzdehLFoTCiwwSWhv2Mh5dzRM26De3AzfO3OH/oEt2HdyRfMdfnzm/6bTtzh/9GwVL5+GzxBwTm83/oGP8gP3Q6LQ6bA6ko+ObxSfK8EIJazz5cfl6wZP4U9wWcxUfPjOLKyRsIAV2GtOfVEUk1+qSURIREIhWJXbFz41z2bjQLIRi59COunLyOf5AveQqoLaJtCZviAIrdQXz0w200pfUwWHeA4Sm3xMVdhdAGQ/BasB5R5Qi0+ZFKJDLyLbCfB6/X0PgOTP5cz05I0z+oMtao+veWreD1kvteQDYipQUZMwnslxA+/UGTF+znALUPgIz/Pdsc/2Md6omPNqE41FCH1WTj5aA+7P7HOalhHl5GPlk4iJ9Ofsdzr7r+jxcTGcuk12cSdu0OJ/47zeyhyXfvavvms7R5owWla5Tg3al9KV4pi9kmTsJmtXHuwCUs8RbMcRb+W773oWOEEHT/5GX0Rh0GDz19vuzOzQu3mDZgDr+PX86JnadZ+f1aQi7edpvdGo2GklWKJTp9gHxFg3h58AsJ0g7B9BqVNNda2s4gI3ohY6chI/oibSfcZm9GkZY9yLgFqbaRFJo8CI/miIRMIxk3G2zH1TzzuNnIFDpxCUNd8J8MPLDXpH9ypJFlzGSIXwTWrcjIPkjhA8IT1e16Oq1eIjM81lk9F49eYVjrL4m8dTfxsYKl8rPg/HQnWuceou5E073IW9isdjQaQZMuDfn0t/ez26wMMajxp1w8cgWAHp+9TLehHZM9LvL2XfRGPZ6+HnQr/BZRYdFo9VqkoqDVadEZdPx0agp5CwYme35GWL9gC9MHzMXb34uxqz6hVLXiWRpPWg8hTX+pOjWYUb/kAgwNEIEzEcJ5MtpZRZo3Ie8OBhQ1Lz14U7qKj5TosRA///4DgbPRpLJyleZ/kZZ/ER4tsm2Fmx0okf3BsjHhNwMieBNIG9K0DKErDh7tXJ4WmlJWz2O94i9VrThLbs4mf/FghACNVkOeAgHZbVam8A/yo+9XPfD08aBwuUK88VWPbLVHURR++HA+vcu9x8+fLUqXnsg3G0cyaGY/Ri79KEWnDxCYPwCfAG/MsWZiImKRUmK32nHYlcQitQc3WjOLw+Fgcr8fMcWauXMjgmnvPiyPnBGU+MXIiNfAvAywAkbUTVEHWA+A6Z8s2+xMpGUH6sXJCtjT30NXX537rkMgbKlr0wiP5mj8R2fY6UslDiVmGkrMt0jlbobOzQkIn/4gfAC9KgWhzY/QFUHjO1DNcsrGWoDHOsZ/j7GrP2HG+z9j9DTw3vQ3Mnz+9YRYc5GyD+fqu5NOg1+g0+AXstWGe/y3fC+rZm3AHGdh2ZRVVHm6InVb1Uj1nEXjl/PbuGV4+Xjw9caRaW54e/t706hjPfauPogjoQpap9eh1WkpX7d0ll+DEAKtTovdakdoBAavzHfnAsD0F6q4FoAOvF+DuAXqYwLIQat9AOHRCmlapko2CB9VcyY95+mrI4UxoWetEdKQLM4s8u4HYP0PUJCWnYigP10yT2aRSixYNoAmP8LY8KHnhb4K5NsFSmyO29B+Ihx/8YpFmLDu80ydO33gT6z6cT1Co+HVEZ3oPvzJ2JhKi/gY0/2mZUJginl4g/NB4qLjWfTVchS7QuzdeOYMW8iE9Wn/TT77fTCXj1/FN68v8dEmzh+6RPVmlQkIfnhjO6NoNBq+WD6Eae/OwS/Ijw9mvZ21AY3PqLFvFNAWAO/3QdrAvBaMTcCjTbKnqfnufyHtFxGeL6lhADcgjE9B3j/UTVpjw3QrWApdUcizSNWs0ddyXYGV/STq3QjgOOeUIaVlp6q8aWyJ0PikfUJK40gFGdEV7Goat/T9CE0yUsxCGEGbsy748JjH+DOLw+5g75pDRIfHMLHvjMTHvQO8WRExz2125GQsJguftBnH8f9OU+vZqoxeOTRFDaKw6+Gc2HmGiX2+x2KyojPoaNalIUMXpC19/CghpVSdoXIbPNoiNAHpOk+J+xVivgEsIHwRwVsyLCMsHeEgjFlyZjkNJW6uqksP4PUqGr8hWRzv3vuM2lwl6J9Mq6BKJQIZ+jSJGUv6umjyLsySfa4gt3I3A3zx8kQO/3scm9Wu3qInXBsD82d9lfm4YPQ0MmnLF2keF3LxNm/XHAIChEZQpmYJCpcpSP8pfbJsg9VsxRxnwS+vb7rPiY6IYdqAuUTeussb43tQoZ5zhLUgQYzMo0XGT0yS725VLxya9Nd9KDGTVFVJNBA4/bHZQNV4v440NgdpR+jLZX1A81oS32fHFZB3QaQtwZ0sIgC0xcGhisnh0Trr9rmRx3pzN7PsWX0QU6xZjf0i0Om16PRaPl30fqbGk1Ky7c9dzBvxO1dP30jyXHhIJAc2HCH27uNZ/Xlw41EcD+S6dxz4PJ8t/gDfwKytTE/sPMPLwa/TtXA/Zryffh38qe/MZsfS3RzZcoKhLcfgcDiyZIczEJ5dSBQJ05UEbfqlA6S0Q9xs1JWnRU0hzAaktKNEfYoS9ixK7I9pn5BOhK6Uc5w+JDhnT/WftrjqvDNrl9Ag8i5B+I9CBM5MNsyTk8ld8SdDhXpluHD4MkIjaPxSfWo/V53KjcpnusBp46/bmPrObCwmK8unruaXi9/jl8eXq6dvMKD+MIQQeHgZmX38W/zypH/1+ihQoX5ZEKjpmBLK1ytDfIyJC4cvU6xi4VR18lNj/ojfMcephTB/z1xHr1Fd8AlIOzwSdiMCu0119uY4Cw6bA63WDbr3qSCMDSFoFThugqFmBrtxaUEEgowAdBm6aDgV07IH2izOQBrqIww1sseWFNB490TqSt2P8Wex2Y3Q+IBnytlpOZlcx58M49d/zqZft+Plq+rZpNWsOy1O/Hcac7wqgialJORiKH55fNm5ch9Wkw2H3YGUcHz7aRq2z76iDldQunoJJm0ZzdEtJ6jZoip5CgTweqX3iY8xIYRgxv4JFCqdcW2gAiXzozeewWaxYfA0YExnRk6/r19leOsvsZqtvPLpyxg8spjJ4ySErihkQtpZCAF5f1FX+po8CN+PXWBdOpDxJGmzKHPmHWxy2TdPIrmOPxk8vT144a3nnDbes682ZdOv29FoNeQtFEjJquqqrHzd0ugMWtXxK0ri448bxSoUYub7PzP3k4WUq12auOh4TDFmdHot/y3fS+eP2mV4zHcm90ajE4ReuUPvL7qmu7lN5YblWR4xD5vV7jT5juxG6MogAr9365zSskNNsfRorlboenZS6xTsJxLaLDZIe5BHFCklKDdBkxchUtboysnkZvW4idtXwrhx/haVGpRL4nD2rz/Cif9O06BdHcrVznpuek7krxnr+HHIAqwmK0YvA4pDwWaxY/QyMHrlMGq1qJrdJuaSAaT1ADKiD2rxlwci75/Oi8PncKS0q0V6tiMgPBB5/1B79OZQcrN6spn8xYOTbYJep2V16rR0nmpoasTHmBjT5VsuHrnMyx+8QJcEtUlX4+njkVilKITgrUm9CbkYSo1mlXOdvguRli3IuF/BUAvh/bbzGrjbTnG/9aUmoU2kcxy/tPwHjstgbJ3jip4AsJ0A+3HAAtKKjF+E8Bue3VZlmFzH/wTxx8S/OJKQprpg5BIavFiHouVd33DlmR6NObnrDAc2HOW5Xk1p906rbG9dd/y/00SERFL/+VoYPR+PkM+DSPt1ZORAwAzWvaoypFdX5wzu0RxivwP0ajWyk+LmSvxKiB4BKKCZCcGbnKJtJG2nQUar3bWy2sBem0+VVAbACNoSWTUvW8h1/E8Qdpv9vqaOIFG51NVotVoGzeznlrnSw98/rOPHj35BaATFKhRm+p6vsuVCJC3bEuLkzyAM9ZwzprSCEgNKqCrFIAGsSMc1UnuFUolF3h2oNl7x7oXGp3+KxwptYQjeoEoM6yoiNE7KRLNuJjHPXolRc+R1WetjocQthJgJ6nthaJTlvRChLQiBM5Hxv4C+BsJZF1M3k5vH7wLiY0zcvhKWLuEyd9L5o3aUrV0KL19PXn7/hRwj2exuNi3crspDx5q5cPgycVHxbrdBWvcjIwdA/E/IiDeQtjNZH9N2BhnaABnWBBk3D7SVAQMIf9AURbnTCSXqU6Q0P3xu3Dyw7lHTQmN/QKYlvKYJRBjqOc/pAxifR61n8AJtEGiLZH1M0yLArGYdWTaqdQ9ZRBgboQn8AY2PE8NnbiZ3xe9kTu05x9DnRuOwO3jqxTp89vvgZFeTsXfjmDN8IbGRsbw2prtbBOD88vgydec4l8+T02nwYh0uHL6MVCT5igfj7Z+15tp3bkYwtttk7tyI4K2JvWjcMR3aNbbTJI2TnwN9+SzZIeNmqxr5SLBsgbx/ITQeSKnAnVaABeynkZoghO/g/zv7/+/+lISLkUToK2TJrvSi8WyJ1C0E+xUwNkUIJ6Ta6uuC/SrgAG2xDNZIPL7kvgtO5vfxy9U+rcCuv/dz50YEwUUe3qSa2Pd79qw6hMPh4MTOMyy66rxqx1xSp8uQ9hStUJjwm5E8071RlsM8Mwb9zMldZ1EcCl/1mMKy8J/T3jfweAZip+DUOLm2KGBAbYwOQhug6gXZLyMTAz02UO48dKrw7oO07gb7GfDqgTSvhzhVplp690Hz0IXC+UjHbVAiwdjYaXcSwu9T0JdHKlEIry5pn/CEkC2OXwgxGHgDdclzDOgjk7v/fAQpVrEw+9cfwWqyotNr8QlMvpr08onr2G3qbWdEyF2klNm+4fmkIISgYbvkC+UuHb/KmM6TMMWZ+Whu/3T1abaYrEhFXb0rikzX3onQFnogTl7BKY5O+LyDlPFgP4fwfuu+SJy2OHi+oFbXavIgvN96+FyNLyLvb4m/K7frc69FIPELIQuOX9ovAQ6ELuXuW9J+ARneCVUcywjBqxCaTOroPIAQOvDqlur+xpOI2x2/EKIwMBCoJKU0CSGWAN2Aee62xRX0GtUFjUbDtTM36Dq0I57eDxd4HN9xittXwtRfEvrPOtvpXzhymS+7fkt8tIkWPZ+mWZdGnDt4kdI1SjhVmOxxIOTibZZPXU1w0bxsXrSDa2duAjCmy7esiJyfxtnwzre9uXrqOndDo3hjfE88fTzTNa/QBDi1/Z4QhmRTC4UQCP9xSL+RgCF9nzV9JTUbCEBfMdM2KXE/QcxkQCC9eqHx+yj5Ay3/grQAdhBSbVzj4bwiylySkl2hHh3gKYSwAV7AzWyyI1M47A5VfyYZjRe9Qe0VmxrrF2zFblVX+x6eRpp0esrpNo5/dSrXz6oNZP6Y+Dd/TPobg1GPEILRf+UWTd3DYXcwsOEnRN2JwZDQ7vEe6d2cL1KuEL9ccG/lbGbISGqkCJiGjF8AUkF4v5b5SePmci/0hOlXSMnx66ujugWHmi6py9p+Ry6p4/YtaSnlDWAicBUIAaKklOvdbUdmWfvTZl7w7kl7/14c2HAkU2NcOXEt8WeHQ6FAyXzOMi+Rh8INEqxmGxaTlYOZtPtxJC46ntjIOKQisZisBBcNomCp/ATm9+fTRRkLb0gpE8N3jyJSiUSJ7I9y5yWwnUDj0x+N74CsafzrKqI2W9eBNuVQjzDURQT+iPB5D5F3EULnXPkSaf4X5U57lMhBauese49LBSVmMsqdl9TUzycEtzt+IUQg0B4oCRQCvIUQPZM5rp8QYr8QYn9YWJi7zUyR7wf9hN1mxxJvZfp7czM1xrmD93ub+gR6Z1miODk+njcA/2BV+VJv1CGEwOhlwOhpoF7bWk6f71HFN9CHum1q4uFtxOhp4NURnVlwfjpLQuZQr03NdI9z/vAlXg7qy/NePZg/crELLXYdMnqMmg1kP46M7KfWBGQREfAdePcD7z6IPKn3NBbGBgifAQh9pSzP+yBSiVFrFOyn1JTOmIn3nzSvhrh5ajVuzNdIa9YWRVLakOZ1SMvWHJfO/SDZEep5FrgkpQwDEEIsAxoCvz54kJRyFjALVK0edxuZEj4B3pjjLGg0ItOSwhXql+PM3nMIjaD2c67pV1q+bhn+vD2XcwcvcunYVcrWLsX5g5coWDo/0eExXDtzI0tVuwc3HmX/+iPUf74W1ZtWdqLl7kUIwahlQ7h45Ap+Qb7kKxqUqXHmDv+NmEh1Jfn7+OV0+uAFvP0z1kUr21EigHt3LDa1bWQWUyqFxgfhOyjF56XtFDJmAmgCEX6fO2VD9+FJzNxPnbUnvM4ElAiSqIo++FxyQykRavhLm/znRN599/7eiGcPRBa7hrmK7HD8V4GnhBBeqGV6LYBHRoFt7KpPmDZgDh7eRgb/+HB2RPrGGM66n//F4GGgZW/XdksqW6tUYlPzQqXz83rlwUSHx6I4FMauGp4pp31y91lGdJiAJd7KX9+vZcrOsZSuXsLJlrsPjUZDmZolszRGQH5/dHotdpsDrV6H3pg+tVBXIaUVGfUxWPeD58sIn/fT3NQVvh8jI3qrtQA+AzLc/jHjNkpV8ExGAjqkNCECf3D6PEIbjPTqDfFz1QvMgxciz/YQ/5uqD6SrAsZGKY6jxC+F6JGq7b4foPHu+/BBlh0kXjzNayDX8atIKfcIIf4EDqK+Q4dIWNk/CpSqVpzJ28ZkaQxPbw86DEi+8baziboTzb+//0fBkvkIyOdPTHhsYmP0Lb//lynHf+Hw5cRG6xqthkvHrmbY8R/bfoqvekxBaDV89vtgKtZ/tDON+k9+DWu8hdtX7/DGVz0Sdf6PbjvJ1Hfn4BPgzfBfByYr1OcspLSD7SBo8qhVuObNgFkNZRibgSH10JXQV4J8ewGHmwqdFFVDBwA7OEJcNpPGbwjSdzCgTXIBFBp/CFoDmBEijWys2O9IbP4eOw2Sc/yGumA9rP7s0TzrhruIbMnqkVKOBEZmx9zOwOFw8O2bP7J39UEatq/DoJn9stysJSNYzVaiI2LJWzAQgPmjFrNz+T6e7vwUPT/rlPjBttvs9K8zlLuhUWi0GnqN7IxGq0Gr06IzaKmZycyep16ozc+fLUKn12Lw0FM7E+qiY7tPJvxmJABf9ZjCgvPTM2VLTsE30IfPl3yY5DEpJZ+/OF5tOqMRfNP3eyZuGuWS+aWUyMjXVblgqYCxNUlCGPeagqeB+tlxj1sQQov0GZwo+CZ8h7p4vuRfl/qa05GCqy0GSjggQZt8mFQEzgLzKlV2wtgy88a6mNzK3UywY+ketv2xE3Ochc2/7aBem1o06uAcka20uHLqOu83+gyLyUqtZ6vS9s1nWfrtP5jjLIRcuk2FemWp26oGABG37nI3NAqrWf3SH9p8nBkHJrD9z92UrFqMuq3Tv3n5IMFF8jLv7FQuH79GqWrF09Xy8GEeWHU9poVrUkqsFvW9l4rElNB32DWTRSfElhN6CNtPgr4m2I+Cx/OqdEEOROPTD+nVA4TeORINLkQETkPGTFKbv6dQ0CaEETxfcrNlGSfX8WcCR5JUSeE2lUuAZd/9k9iY/fC/J6jUIGm+c0zE/VS1vIUCKVi6AKFXwlAUSYueTShYMj9dhmRdh98vjy/VmmQ+++LTRe8z/tWpaLQahi8cmGV7ciIajYb3f+jH1Hfn4OnjwcAZb7puMvF/mWHKTTTB/yR9yLQOzH+DsTkar5ddZ0sGcfVegrMQmjwI/7HZbYZTyHX8maBJp6fYvnQ3BzYcpV7bmm7tk+sf7IfeqMNmsQOSZ7o3Yvc/Bzi95yxlapakccf7dx5arZZpu8exb80h8hcPpnzdlPOo3U3Vpyuy8PJMp47psDsY+8p37PnnINWaVuSLFUMxZPMma6vXmtPqNdfHeoXQIjXBaiNxgP+LV0vbMYgaApjBshVF+KPxfNblduWSM8ltvfgIcen4VQY2+AS7zYFGK/h8yYc89XxtQHV6Wl0Wm0w84uxcuY+vek7BHGfBw9vIgGmvp+l0pZQc33Eam9VOjeaV3bpX42ykdR/y7iBAIvy/RRjv972VptXIqOEk6t3jhQhej9A6v3gwl5xDbuvFx4Btf+7CHG8BCV6+nklWs0+60we1UO1B7mXWpMbcT35j5fQ1IARNXn6KIT+/6yrzXI4w1EXk25n8k8YmoPEG5Z7jt4NlA3j1cJt9ueQcHt3ljRPYu+YQHzYbyQ8fzGPdvM08792Dl4P7cHJ36k0osotytUsnyv0qikLRCq5vm5gaDoeDo9tOcuXktbQPdgN1WtXg+X7PEVQ4D827N6ZJ57Q1kDb+shVznNqUZesfKThNF3H7ShirZm1IUsntKoTGBwJ+QJVtBtCC7tFOoc0l8zyxoZ7wkEh6lxmAxWTF4GnAbrUnbtKWrl6cHw5NTPX8KyevceHIFWo9W5WAYH+n2pYa/63Yy6ndZ2napWFiYVZ28cnz4zi+/RSKQ+G9GW/QqnfOzVtOiXE9prBz5V6kIqnUsDzfbEw9y1hKycldZ9EbdZSrXTrxcYfDwcmdZ/EL8qV4xbQ7R0XevkufCoNUbR8J49d9RpXGmVfBTC/Ssg1pXgeGpmg8c266YS7OITfU83/ERMQm5DeDzWxDq9eiOBSEEHj6pp7Te2LnGYa2HI1Go8HoaeDnM1MzmdKYcRp1qOe21NHUMMWZObD+SOLF8q8Z6x5Jxz/k5/5smF8Zm9VOqz5p2z/lnVlsWrgdKaHr0Pa8+nlnAEa0n8CxbepF8P0f+/Fsz9Qrss8fuoSUEku8FSEEhzYfd4vjF8YmCGMTl8+T3UhpQ8YtAOUOwruX2is3q2M6QkHGgbbEI5+C/MSGeopXKkK9tjXR6rX4Bfny8bwBlKxajEoNy1Ohfhl6lurPhN7TsFnvF75IKVk/fwsz3v8ZS7wVU6wZi9nKmrmbOLT5WI4WZXI2Hl5G8hULQqvXYvQyUL2pc4W13IXeoKftm8/S/t3WeHilLVu88ZdtmOMsWOItrJq1EYDwmxHsXXNI/TyYrKyYtjbNccrWLoVGq8HD26gWwblIs+lJRcaMVwvD4n9GhnfJ8ndTMa1FhrVA3mmPjB7hHCOzkcd6xX9o8zH2rztMnVY1qPlM0ipVIQQjlnxI7N04PH080Oq0NO/WiNN7z/HRM19gibdw93YUlRqU58W31VvidfP+Zfp7P2GJV/XF9UY9NoudBSOXgICWvZvx3vQ33P46swMhBFN3jmX1nE0E5vOnVd9Hb7WfGcrUKsnZ/RcQQlC5QTkAlkz6674GWMIxaREQ7M/so5M4uPEYZWqWpFS14q4yOVWkZZcq82Bs7nRVzOxCKtFg3kRiHwAlDLWbWPoa5CRL3Mz745n+QPqNQIiMpwpLJQqUGNAWzta7hsfW8Z89cIHP203AEm9h5fS1fLttdJKY7D3+P0RjibciNOofRFGURCcPcGbf+cTfDZ4G2g9ozepZG4mLigdg86IdT4zjBwjMH0CPT3NOIZA7GLvqE/75cQMGo57n+6l58NZ4KxqNQFEkWp2Wzh++mK6xggrnpWXvZi60NnWkdS8y8i3ACnGzIO/fTtfBzw5kRHdQQhN+04LxubR1eNJCVx7sFwA7aPKRGdcpLbsS3m8JHm0RAROyZlMWeGxDPaqQmLoMs5it/PTJb+m63avapCKNO9RDZ9BRukZJ2rzRIvG51n2ewcPbiKevB0XKFaTPmG5Ub1YZo6eqc1/1adfHaLOTVbM38EHTESz+ZuUTFdZ6EG8/L7oOaU/HgW0T00V7fN6JohUKY/DQ03VoewqXyXo82S1Yj6DqJCqAFuw5M5stI0hpBft5EhUytUXVngBZRPh/Ad6vg2dnRJ7fMrVal7EzUO88LGD+W139ZxOPbVZPeEgkfcoPxBSrNow2ehn5fPFg6icUPGWWiFuR3LocRtlaJdEb9FgtNjbM34KU0PK1ZtleKeoqTuw8w7CWYzDHq8VRw38d5NaK5VycT5IG58KICFqN0ARmt1lZRonoA7ZDICX4foDGu3d2mwSAEjUKTEsBKwg/RL6dmQoXZYQnLqsnb8FA3vz6VX74YB5Wsw0hBOb4rHcUylMgkDwF7n851Ft+1zSFtlps3LoUSoESwekqRnIlESGRD4TAJOE3U29YkR3cuRnBl12+Jex6OG9N7EWTTg3SPimbuH4uhL2rDlK+XhkqN8ye/rJCV1qVJLafAX0NVaL4MUAEzgLrfyACEIYa2W1OIsJvOFL4gHIT4f22y51+ajy2jh+gVZ/m/LdiL0e2nKBG88o06vDorFBj78bxdq0hRIVF4xPgzcyDXydbL7Bx4Tb2rz1M826Nsnw3kxr12takaIVCXDxyheAieWnWLeWGFdnFzPd/5tSecygOhfGvTqNe21rpytRxN+EhkbxbZyg2qw2NRsPYVZ9QvdnDfRGktEL870gZi/B6BaEJcLotQlsAtAWcPm52IoRe7T+QwxDCiEip2bybeawdv8GoZ/zaz7LbjEyxZ9VBosJiMMdZkIpkx7K9vPBW0juLfesO891bs7DEW9ixfA9T/kt/J6yLR6+g1WkoXqlouo43ehqZvmc8MZGx+AR450hNG6vZlrj3IBXpVtXUjHDxyGUQYLPYEUJwdNtJCpcryOXj16hQr0xiwoGMGqFqu6MgzWsRQX9lq925PD481o7/UaZw2QL3N1CFoEi5hzcMr5+5mejcNBoNN86FpMvxzx76Kyu/XwMSeo7oRLehHdNlkxACvzy+6X4N7ubtb3tz7cxNIm/d5fXxPfBKoxDvHuEhkcwfuRi9QUfvL7ril9e5r1FKyapZGzix6yxt+j5D+bpl0Bv0ePpKHHaFc4cu8Wvxt9Eb9Hj5eTHn+LeqDbYDJKYQ2s851SZnIs3rkLZjCI+2bk8JlVKCjFJj5iLnLUZyKo/t5i5A6LU7bP9zN8UqFUlsTvIosX3pbrb+sYuG7evwTPenH3r+zs0I3qk1BHO8lcBgP2Ye+gZvP680x23n92ripneeAgEsvjnb6banl2tnbjCmy7fER5sYPOstaj+X8W5eWeWtmh9x5fg10AiqNKrAxM2jnDr+poXbmfzWj1jiLRi9jPx0cjIGTwNHt57k7IGL/Pnt3zhsagMVD28Phv3yHo061EOJWwAxE0FowNgSTcDXTrXLGUjzeuTdIYAJhBciaB1Cmz/951u2qBW2+hoIn3cRIv1ig4pigjvPqambIgARtAahzZuJV/H48sRt7sbHmHin9seYYkxotFo+mP02z3RvnN1mZYinX36Kp19OWWgsqFAeFlz4nluXQilctmC6M4pKVC3Guf0XEBqRrmIjVzLx9RlcPn4VKeGLThNZeXeB2wtbbl0MVZvrOODamZtOH//yiWtYTOrKXaMV3LocRrUmlWjSqQHHtp/CYXckHqs4HJSsqubSa7x7IQ31QMar3bQyibRfRcb9CJq8CJ93sp7T/uDYtmPcl3oWYL8M6XT80nEDGTkQMIN1P1KTF+GdAbXQ2En38/XlXWTcbITfsAxY/+Ty2N4bhVy8jc1sw2axY4m3sH/94VSPl1I+krnpnt4elKxSLENppONWfUK3YR3o8enLfPZ78i3kMoqUkm/7/UBbz+6899Rw4qLi0nWe3epIbNyeXTH57p90RG/Uozfq6TWqs9PHb9m7Kd5+Xnh4GylYKj8VHmgs33FgWwKC/NDqNBQpV5BJW7+gUOn7m61CXwFhqJXpi6GUEhnxippGGPcTMuqLLL+eBxEebdX+ssJbbfKuz0AfZ8cd9W4GACs4rmdscvl/nxeR8zbycyqPbajHarbyRuXB3L0TjeJQGLl0SIrhnvOHLjG01Rji7sbz5tc9efn9F5xg9ZPFyV1nGNpyDOY4CzqDjl6jutB9WNp7BxePXmFEhwmYYkwMnvU2jTvWd4O1D3NgwxGmvDMbh93BkJ/fpUbzKk4dPy46ntCrdyhavhA6fdIbbYfdQVxUvNP3FkDNDJK3q5HYeF1XGU3QcufO4QgFxxXQV8nQ3YSUDmRkX7AeAOGDyLskQ5XDUolAhrUFGQHa4pB3DRpN+oMYUokDGQOa/I+86FpKpBTqeWwdP6hftkObjlG4bEFKVkn5A/XRM6M4suUEAFq9lr+iFmR73vyjxvnDl3i/8WdY4q0YPPS8/lUPXhr0fHablW761fiQS0evAuCX15elYT9ls0UZQzGtVvcDtIUQAVOSxLqVqM/BtBKQiIBvER4ZrzuR0gxKDEIbnIFzJGBLtYm6lFLV0tEEZKrZujqHBSE8Mnae9RAysg9IO3i0QRPwTYbnfhRIyfE/tqEeUMvrG3esn6rTB/AJ9EarU98KnV6X280qE5SpUZJXR3QmX7EgGravxwtvP1pa71K5vwBSlPshhOM7TvHTZ79x+N/j2WFWupBKLER9DMp1sB1UlSkfQPiNRgT9jQj+N3NO33YCGdoQGdYc5e4H6QqJSsdNZFhT5O1qKHeHpniOEAKhzZcpp594fgadPiTIJ8h4wArm1UjHHfVx+wWk9QBSOlIfIA2U2Jkot6qhhLVGOkLUsa1HUCL6okSNQCrxWRo/qzzWjj+9DJrxJjVbVKNU9eJ8+fewXMefDi4du8I7dT7mndpDuHDkMgBdP+7Awssz+XTR+zlWuuJBp/4gQ35+l/zFgwksEMCni9R9j4tHrzCs9ZcsGrecz178itN7XZNSuXXJTgY1/oxZH/+SZKM3/Ti4Lw+qgDQneVYIgdAVR2iDMmWfjP0BZCyqk9xwv6F7qufMTdh4VcC8Juelo+pKAQl7AsIIGh+U+JXIOx2Rka8j7/bP9NDSEQqx3wNmcFxGxnyHlBZkZG+w7gDTMmRM9mZoPbZZPRkhMH8AX635NNVjpJQoioJWm3tRABjdeRLXz6ormS86TWTBuenZbFHqREfE8EHTEVw9eYPGL9fns98HJylCK1e7NL9empHknAuHL9+P/Uo4f+gyFeo5t13hjfMhfNPneywmKxcOXSJ/iWDa92+d7LHSdg7sJ8HQIEmTdKHxR/p+pOrPa/IhfIdkyAbFdhoi3wTlDni+hPD7MmnMW1sc1UlaQGhBpGMvQpsH0KvnIEGTs+o/hO8HSLTguIrweRshPJDxvwBm9Rpq2YaUpsxlQIkH3apGvbDIeJD3entYwZG97UpzV/zp4Ph/p+kQ2JsXvHrwz4/rs9ucHIE5zpLszzmV1bM3cuNsCFJK9q09zPEdp9M8p3bLahg8DXj5eaI36qnXpobT7YoKi0Zo1a+hzWon/MbDGkhK7AyUW9WR4S8go0Yg7zyPVJIep/Hugyb/ETTBGzK4QRoJ4Z1BuQ041L0A+7EkxwjfgeDVEwzNEYE/qf1700B4vwGe7UBXCfwnPNQBS1p2ooQ+jRLWAmk7mW57nYUQRjR+H6MJnI7QJ2zkG2oBHoAWNPkTfs7E2Jo84DcWNIXBUB/h+4EqfufZBdCB8Eb4DHTSK8kcuSv+dDD9vbnER6u5yt8P+pnn+z332GYBpJcP5/bny67fIqXko7mZvy12Fz4BPmh1Wuw2B1JR8PZPu9AtT4FAfj49hQuHL1OqWnH8g/ycblf5emWo3rQS+9cdJjB/AC++0yrJ89J+FWIfaAKCCdCC7QQYHy7qyzC2syTpIoNDTc98ACEMCL+hGRpWCCPCf2yKz8u774O8q/4cNQQRtCrNMaVlG9L0DxieRuOVvp4HGUH4fgy6EkjHHYRXtyx9xzVe7cGrfdLH/EcifQeB8ERkc+ppruNPB355fdFoNSgOBU/vrP3BzPEWjmw5QYGS+dLVlDunUqdldVZEzs9uM9JNm9ef4eLRyxzdepJ2/VulW9PIL4/vQ93bnIlWq+XLv4cTFx2Pp49HMhpI//+7FhCgc1LvB30F1dFLB6CA5ysIXRnnjJ1u0naw0nYGGTkAMIN5HVIbgHDGhQ9Q4ldC3DTQlUH4f4PGhWEpVwjtZYbHOp3TWdy5Ec7E12cSGxnHu1P7UrF+5uK8dpudt2sOIfTaHRSHwog/PqJem8xXZObyZKDE/ax2yNIUAM8XEcaWCJ3zFg3SEabKGOvKI/TuaSYkLTuRUUNBGBAB09LU+JHmdcioYWqzcwwI348Q3q9l3Q5HGDKsOWAF9OD1Chq/1Pf7HiWeOMkGZxJUOK9TVD5vXrjN7SthiTHx9fO35Dr+XNJE490HvPu4bHyhDQbPDi4bP9k5jQ0R+ban/wRDI7UyWJGAB3gkvwGecazcv+NwJFxYHn9yN3fdSHDRvBg8DWh1Wjy8jdRs4dzq0CeJ0Gt3GNzkc/pWGsTBTcfSPiGXRxqh8VE7hOX5HRG8Se0j4IxxtYUTLqp60BYF77eQ9ktImfMTFrJCtjh+IUSAEOJPIcRpIcQpIUTObZXkRDy9Pej4XlvyFc3LM688Tds3ns02W2xWG588P442xu580mYsNqstxWNNcWZWTF/D6jmbsNvs6Z4jMjSKOy7q1DW53w+c3HmGa6dvMqL9hBTz8x8HosNj+HHIAmZ9vICYyNjsNifbEMKoahdpvJ06rsb3AzQFTiDyLoPIvsg7HZBhzz6UOfU4kV2hninAWillJ6GW7KWdYvEYcGDDERZ/vQJznIXI37bTvFsjp2vCpJcdy/ZybNtJ7DY7x3acYvufu3nmleQ3yz57Xi1eEhrBsW0nGbrgvTTHX/vTZqa+OweAV0d2TpduT0Ywx1pQEqpt7Ta7Wnn7mN6/ft5+Amf2nUcgOLv/YhLZaGm/Bo7LoK/ldIf4xGHZAko4YFJDSuZ14NU9u61yCW7/qggh/IAmwFwAKaVVyoS8rmzAarZyas85ou5Eu3yusOsRPLiXHnY93OVz/j8Oh4MJvacx8fUZWM33V/ke3innLJ/eew6r2YYl3pqoaZQW80ctxmaxYbPYWDRuWZbt/n/endaXPAUDMHoaGDTzzce62vrqqes4bA7sNjuXT9wv/JHWQ8g7LyDvDkSGt0NKUyqj5JIm2mL3FT+FSChcezzJjjVSKSAM+FkIcUgIMUcI8dBSRQjRTwixXwixPywsLMuTOhwOxr86lfYBvRj10jfYbXYsJgtv1RzC0Jaj6VV6QJIvlSt4+uX65CuaF71RR/7iwTTqUM+l8yXHnn8OsmP5XqwmK0KAf5Avbd5oQYN2D238J9KkUwM8fDzw8Dby3GvN0jVPkXKF0Oq1aLQa8pdIv7BXeilToySLb8zmn7iFtOnbwunj5yQ6f/giBk8DBg8DXYa0S3xcmv4BTOqGpBIBtjPZZ2Q6kTLlkGJ2IwzVwf9rMLYG3y8QxobZbZLLcHs6pxCiDrAbaCSl3COEmAJESyk/T+kcZ6Rz7vxrH1/1nIo51oyHt5GBM94kX9EgPm8/HlOMGQR0H96Rvl++kqV50kJRFKLuxOAf5JvpvrUOu4OvX5vOgQ1Hefql+rz3/RtJxpJS8vfMdexfd4TnejVN0sxl19/7GddjSuL7MPjHt1IM8Txo8+HNxzF46KncqEK6Cluiw2OY9/nv2Cw2eo/uSlDhx7czkpSS4ztOY/A0UL5OaZfMceN8CEKIJFr90rwGeXcYqpSCFyJ4o1o1mgORShwy4lWwnwB9XUSenzItzJZL+slJ6ZzXgetSyj0Jv/8JuLxtjkajSVKgqNVqKFSmQKIqo9HTSJkaru9GpdFoCMznn6Ux/v39P3au3Ic5zsLGX7fx1It1qN+2VuLzO1fuY/bQXzHHWTi46SiFyxakVDX1trX+87V4pntj/lu+l9otq9G0S9qrGo1GQ61nq2XIRr+8vgyc8WbGXtgjysS+M9i2dDdSkfT47GWn72cAFC7zcM9l4dEGAgxgPw0ebdJ0+lJKp1acS2kH81pAUecXqQjzmf8B+3lAgu04WP4Fj1YpH59DkPZLoNwFffUs9fSVUkGa/lL7KBubIYzPJPu3kPYLyKjhIG0I/7Eu62HsdscvpbwlhLgmhCgvpTwDtABcLtZRr21NWvRozI7le6nTsjpNuzREq9PyzaaRbPx1G5UalKdJp0cjuUhxKPf3CsTDnatCLt7GntDDVaPVcOtyaKLj12g0DP7xLQb/+JY7TX6s2bxoB3armu20atYGlzj+lBAeLVC/Qiljtdj49PlxHNlygmpPV2Tcmk+d0m9CRg0Hc4J2lXkDInBaygdrArifLy9BZG3x4w6U+L8g+jO1S5ihASJwZqbHkjHjIP5XQAHTEqT3Owjf9x8+7u77YFdlNGRkf0S+LZmeMzWyK6vnPWBhQkbPRcB11SkJaDQa3v/hLd7/IanDq1CvrNMVF11N8+6N2LlyH4c2H6dRh7rUf77W/z3fmD8m/kVcVDz5iwdneLX+IHFRcez+5yB6Dz13b0dRvl6ZZMMZUkpiImPxCfDOdAjrUaVMzZJcOHwJodFQpVGF7DbnIXYs28PpPeeQiuTM/gts+3M3z/ZskvWBLdtI7Ldr/S/1Y40twfuEutIXgci77yC1pRF5ZiM0gWqLSNMfYDuN8OqC0OeA9zF+AffVOrdkXq0TEt6rews0CaZVkIzjV+WvE1Z10nWa/dni+KWUh4GUdxNzSRW9Qc+oZSlL7+YtGMgvl2YQfjOCfMWCMi0lbbPaeKf2UCJCIrGYrOj0WrR6LRPWj6Byw/KJx1ktNj56ZhTn9l8guEhepu35yiWCZjmV8Ws/ZdWsjRi9jLR9M+dtNHv7JXVW6RGoSxcez6oOTADG5qkeKoRQpZA9OyDvtAcsYD9J/K2ZeBYYBqbfIWYCYEKal0Pwv9mva2OolbD6tqWq1iltx8F2EoxNUi4s82wHsdNRnb8GPJJ/v4Tfl8i7A1TtJL+URe6ySq5kQzYQdj2cC4cvU/Gpsi5zkAajnoIl82dpjNtX7hBx6y4WkxUAu82BokiObDmRxPEf3HCUy8euYrc5uHMzkk2/buel9x+dtotZxdvfmy5D2qd9YBqc3nuO03vPU69NzSSbuFmlXttadBzYlu1Ld9OoY32eeqG2U8YVfmPA+AzgAGN6L3jqIkRKGN+/CNv/OYRf0FtMXhtAwfwJdw9SAUdIQngo+xC+HyO1xUAJQ3i9knxM3rIHGfkmIEB8A8HrVQnm/x/L+12kvhqYt4Khlro/k9ycxkaQ76D6swsVgHMdv5u5duYG79YdhtAIdHodc45/S2D+gOw2K1nyFQvCN48PUlGwmm3oPfRoNBpqt6ye5Lg8BQMSi6m0Og15Cz38wc8ldY7vOMWwVl+iKJKfP13ET6enkLegc95HIQR9x75C37HOzVgTQgMeGbvDEbriSN/BXN4/j13r/XE44G5YFMt+LM27IxPuRLSlQOf+8KsSOxviF4GhtrqxKgwI756pniOtO4B7Hc+0YDsNxof3CoUQCGMTMKYdYnOH5Huu43cze1cfwma1Y7fa8fT14Oi2UzTtnDM3lQ1GPT8c/Jr187YQeTuK4KJ5qfVctYfkpMvVLs2AaX1Z9/O/1GlZnSbJvB4pJdMHzmXdz/9SukZJxq3+BG+/x79g22q2sm/tYQILBFDpqXIpHndky0msFhtSkeiNOs4fuuQ0x5/T0Hj3xb9MeyQDASt6o568RcshgjaCchN0FRHCva5J2k5C7DRU2ec7oK8G3q+meZ4wPo2Mm68WfKFVZa4fAXIdv5upUL8sWp0GxaFBKpIyNUtkt0mp4rA7WDR+OQ67A6lI6rSqnuxxrfs8Q+s+z6Q4zpl951k/bwuWeCvn9l/gr+/X0n34S64yO0cgpeSjZ0Zx+fg1pCJ5a1IvXngr+Sb0ddvUYNH45Wg0Ap1eS4V67tbEdy9BhfPy6aL3WTxhBWVrl6Lzhy8itHrIZF/gLCNNavaOBHCoG7npOE0Y6kHe38F2CoyNkw3z5ERyHb+bqdywPF+t+Yxj209Rr03NZPOz/5+zBy5w6dhV6rWp6faw0LFtp7Db7JhizOgNOvatOUzR8oUzPI7OoONesaDQCPQezmvGrigKR7eexNPX02UFVJkhPjqes/svJjZQ37Bga4qOv1zt0sw8MIHzhy5TvVmlJ2JzvGG7ujRsVze7zVDR1wJjKzD/DboKCK9u6T5V6CuBi/LtXUWu488Gqj5dkapPp6/hxcGNRxnRYQJCCDy8jcw7MxVvf/eJcZWpVRKpSLQ6LRqdhooNUg5XpDpOjZJ0H/4Sf89cT8WnyvLi28k7wMzwVY8p7F51EKlIen/Rhc4ftkv7pHRgijUxf9QSosKi6fFZJ4qUTfsi/SBefl4ULJWP0Kt30Gg1afZeKFq+cKYuqrlkHSEEImACMCHNYy0mCwYPwyPdfjW3A1cOZ8bgn1k+ZTUAXn6ejFv9aZKMGndw4chl9q09TNWnK7pk7oObjrF54TZqPVstTfmI5Gil75pYxFa4bEHmnZnqFLvGvzqVbX/uwm51EJjfn99vzMrwlz0mMpZNC7eTt2AgjV+q/0g7iycdu83OJ23HceTf4xQqW5Ap/32JXx7XtWl0BjlJsiFHcXDTMVbP3kiVRuVpP6BNjvti1mtTi9WzNwICnUFHicru79NbunqJxB61kaFRhF4Jo1T14ugNWQ/XXD19gxHtx2OJt7J1yS68/b2o/3zG0g1LVy/B5RNX0Wg1VGvqvFvuKyevY7OoFbl3Q6Nx2B3o9Bn7yvgG+tBhQPKpe7k8WhzceIzTe86hKJLbl0PZMH8rLw9+IbvNyhRPtOMPuXQ70ens+ecA3gHePPdq03Sde+N8CGHXwqnUsDwGo/Pi1f9PnZbVmbh5FBePXqX+87XcGub5f87sv8BHzUchBBQuV5Bpu8Zl2BH+PzfOhaDRqpW+dpudKyevZ9jxf7NpBKvnbMbL14PWfVPeYM4ovUZ14cuu3wLQ9s1nE1+rlJL1C7Zw+N8TPPdqU2q1cF0zdlD3MCJvRxEQ7PdYy0/ndPyD/R5IW9biH/zo7sM80Y7/XuwVwGK2cu3MjXSdt2/dYb546Rs0Og1FyhVi2u5xma6OTQ85RVZizZyNmOPUnOXrZ0O4dOwqZWuVwmF3cHDjUbz9vajUIGOhoOrNKhOYLwDkXXQGXbKpoGnh7e9N5w9fzPB5adHgxTosuvYjplgz+Yvfl5beuXIf096diyXewvY/d/PDoW8yHP9PLxaThfcbf86Vk9cJzO/P9/vGExCcMZ0bRVGY/t5c9qw6SJNOT9Hvm14ZvrO9evoG3745Eynhg1lvUbxS0Qyd/zhQvk5p+n3zKmvnbqb2c9V45pXG2W1SpnmyRFX+j0oNylGkXCGMXkZ8/L1o9VrqZef3WD1rAxaTFVOMmaunrnPrUmiS5+NjTOxff4TQa3dcYXa2UbZ2KTy8jIBapZ+vmJp6N+qlbxjT5VuGthzDkol/ZWhML19PZh+bxLfbRvPLxe8pUCLfQ8fYbXa+6DSR9gG9GNdjCg6Hg61/7qJTvr68Vn4gV0+n74KdGfzy+iZx+gBXT93AZlF15TVaQciFWw+dJ6Vk6x+7+GX0H9xM5vn0cmD9UW6cC8FmsRF5O4rNv+3I8Bjb/9zNhgVbCb16h39+3MC+tYczPMYXL0/k5M4znNx5hpEdv07zeIdDlQ7vlP91Jr4xw+WtMfesOsCE16azdclOl87T7p1WzNg/gde/6vFIa1KluuJP6JYVLKW88H+PV5NSHnWpZW5Ab9Azbfc4Qi7cJm/hPHim0oXqQao2qcT+dUewWmwYPY0EFb4vh2uKNfFm1Q+IvRuH4lCYvG0MZWq6Xu7ZFexbd5hlU1ZRoW4Zeo7oRNs3nkWj0XDh8CVav94C/yA/FEVhz6qDiama6+b9S5ePMpZVY/AwpCqJvXXJLvavO4w5zsKuv/fz34q9THh1KjaLnejwGKa8M4tJ/36R6dfpcDhYMXU1187cpP2ANpSsUizZ425dDkVn0NGsW0P+mLgSm81BUOG8VG3y8L7CmrmbmPH+PGxmK8umrGLh5Zl4+WZc4CuoSJ4kVdH3LrYZwWKyJpEkt8RnvJF4XHR8oiJsXFTq4mEOu4PZQ39ly+L/sFnsbF28k6eer03jjvXTPd+ZfecZ32saGo1g+K+DUv0OnT90iTFdv8USb2X7n7vxD/bLtpamD3LnZgQ3zoZQrm7pdPsWd5Gi4xdCdAG+A0KFKrT9mpRyX8LT84BaKZz6SKHVailSrlCGzuk4sC3+QX7cOHeT53o3w+hpTHzu9N7zxN6NIz5a1R3ZtnTXI+n4w66H88VL32AxWTm27SR+Qb50fK8tbV5PWqKv0WgoXaMEV05dV+UcsqAEmhL3wnEqamrp/VCFyPLKa/GElfw2bhlWk4Uti3ey6NoPePokddLzRy1h8dcrEMCAaa/z6+WZ3L4cSpHyhZLd5D669WSig7Xb7Ny+EpbiBSU1ytUuzeBZb7Fh/hbqtqmZqa5tzbo1YvNv2zm67SQ1W1SlYfuM584P/qEfY7p+i5Qw+Me3Uz12/KtT+W/FvsSNcRAPSYc/yI3zIYzs8DVRYdH0n9qX5l0bMbrzJEKvqnfMX3abnGqm1vWzNxM/A1JRuHLyerY7/vOHLjG4yedoNBr8g/2YdXRS4t1yTiC1Ff8nQG0pZYgQoh7wixDiEynlMkhXUVuOwRxvYcOCrRg89Dzbs0mWN8iEELTokXzaYbGKRZCK2vDC4KmnYv3M5b1nN5G37yI06p/ZarIRcvF2isdO/HcUG+ZvxTvAyyVxzyadn2LvmoPsW3uYRh3q0uDFOny6aDBTB8zBL48P7//YL0vjnz90MdFJ26x27oZGP+T4//hmJbaEHsW/jvmTNq+3oGTVlHuytu77DDuW71FbTxYPpmj5pIuLsOvh6PTadBXktXjlaVpkIs31HgajnvHrUmxwly7qP1+bv2N+BdLWktm75lBiKAwBT71QK9UL1rQBc7l66gZSSr55bTqNO9ZL7G8AJPk5Oeq0qoFvXh8AjF5GGr+k3lmc3HWGCb2no9Nr+eS39xMz09zBlsX/YY67f2d1dv8FqiVzZ5hdpOb4tVLKEAAp5V4hRHPgHyFEEZLcOOZ8Pn9xPCd3n0UIwdFtpxjyU3+XzZW3YCBTdo5lx7I9lKtTOklnrEeJ0jVKULlheY5uO4Wnjwft322d4rHefl50eM91KYtarZah899L8ljD9nUztXJNjo4Dn2ffmsMgoOJT5ShQ8uF9hgKl8nP99A2EVkOximmn1NZoXoVZRyZx61IolRuVT5L9tOCLJfw+fgUIGPzjW+nOJMtu0nL4oVfD+PXLpeQpEJC4wm/SqQFDfn43jXFRl5IPeJWhvwxk3CvfIYTg4/kDUj3fJ8Cbn09N4frZEAqWzp8YVhndeRLhNyMBGPfKd8w98V2q4ziT8vXKYvQyYIm3IqWksIs2/zNLigVcQoidwKsPxveFEL7ACqCxlNJt9y1ZLeBq6/lK4gokuGhefrvyg7NMe6yRUhIeEolfXl+XpqzmBCJDo4i8dZfilYskm6F152YEv47+E6OXgVdHdMYnIONptVJKfhu7lPkjFyfGywuUzMcvF77HalHvqgqWzOeU7ljOwBRnZvTLEzmz/wJt33yWN77qkfjcsin/8MfEvyldsySfLByEl68nvcoO4PblMDQaQekaJen8UTsav1QvzYy3kIu3GdnxfqinqZM64XXK/zpRYdEAFCpTgPlnU+kQ5gJ2LN/DmX0XaNa1oVvvNh4kMwVc7wAaIUQlKeVJAClljBCiNZB+IYscQKOO9djzzwGklIkhGofDQUxELH55M9/0/HFHCEFQoZzZvNvZBObzT7UXclChPLz/Q9ZCSntWHeT3CSsSnb5Gp6Fo+cLERMbydq0hRN+Jxdvfix8OfU1AsD8ndp5h0VfLKFaxCK+N6eb2i+/fM9dxZOtJbBYbK6ev4emXn6J8ndJcPxfC3E8WYTVZiQqL5o+Jf9H7i66EXglDcSgoDrVi+ez+C+xfd4hXPn051d4QBUvlZ9aRSem2K/RqGH9M+hv/IF+6DGmf4oVy+K8D+arnVLQ6LR/Pe/iuwWqx8dOnv3HlxDW6DetI9aaV021DemjcsX6GNrTdSYqOX0p5BEAIcVwI8QvwNWoLmq9Ru2f94hYLncCwX97j4IajGDwMVGtaieiIGN6r/wmh1+5QvFIRvtvxpds2XkyxJka0/5oz+87zXK+mDJj2eo6rFs7FNUTdiU7i9Gs2r8LwhQPZ/c8Bou/EYI6zIKXCzhX7aNq1IcNajcEcZ+HQpmNYLTb6T37NrYuUh4IBCQ/YrfbETT5FSqwJex8vvf8CK6evASAg2I/lU1dhtzk4sP4ov111zl22lJL3n/6c8JuR6PRabl0J46M5yYduaz9XnT9vz01xrIVj/uTvGeuwmm0c236aRdd+wDfQxyl25nTS8ymqDxQFdgL7gJtAI1ca5Wy0Wi11W9ekerPKCCHYungnd25GYLfauXH+FntWHUw81hRnZsbgnxnTZRKXT1xzui2rZ2/ixM4zmGLNrJ+/hRM7zzh9jlxyJk06N6B4xcIIjaB87dJ8seJjfAN9KFy24H3lUgSFyxUkJiI2MU5uNdv46/u1DKg/HKvZ6jZ7273TkmpNKuIT6M2L77SiXILyaYnKRWk/oA1GLwOlqhWnZe+mnD1wgZ6fv0zHQW154Z2WxEXFY7PYkYok7EY4W5b8hynWlGWbHHYHd25EoDjU5kDnD17K9Fghl0ITL1pSSmIj47Js36NCeip3bagdlT1RV/yXpJSurcZwMUFF8qJJyFiRiiS4yP1wxvcDf2Lzb9uxW+0c2nycP27PcWpVrkarSbLCd3cJvsPu4J8f1xN5O4p2/VuRp0DO0A+Pi45Hb9C5Lb5tijWxc+V+gorkcfotfkp4envw/b4JWM3WJK+z0lPl+HjeALYv3U3D9vWo3rQyUkoatKvL9mW7UewKUpFcP3OTvWsOUa9tLbeEfTx9PFPMBnpzQk/enNCTk7vPMqD+cECg99BhijEjpSSocB4MHvpESepv3/iBoCJ5mX10UpY+8zq9jpa9m7Fl8U6klHQb2iHTY3Ub2oG9qw9iMVlp2L5uspv6jytpqnMKIY4AK4ExQF7gR8AmpezkevNUnK3OKaXkz2//Zvc/B3i2Z5MkuemDGn/KyZ1nAdVJr4xa4NQwkNVsZdwrUzi56wxtXm9Bny+7O23s9DDj/Z9ZPXsjdpud4CJBLLgwPdtDTfNHLmbR+OXo9DrGrhruckesKApv1fiIW5dCkRLemvgqL77dyqVzPsidG+GEh9ylTM0SqS4qpJSsnrORmYPnYYm3ojNouZdJPXjWW7Ts1cw9BqfC169NZ8OCrYCanXPPnegMOn67MpNxPaZwePNxAIyeBmYf/zbLvaCllFw+cQ1vfy/yFc1a4xarxUZcVHyq+zuPMilt7qYn1PO6lHKElNImpbwlpWyPeiF4ZBFC0PnDdkz694uHCpJ6j+qK0dOA3qjjhbeec3rs3+BhYNSyISwJmeN2pw9w/L/TWExWHHaF0Kth9/Otswmb1cZv45bhsDmwxFuYM2yhy+eMDo/h+tkQzHEWLPFq0Za7OLjpGK+VH8hHz4xi6HNjSG3hJYSg7RvP0vuLrtR+rhp6ox57QtvOGe//7DabU6Ns7VIYE74jOoMeg4ceD28jrfs0JzB/AM26NsTD24jBw4BvHp8kVe6ZRQhBySrFsuz0Qa1xeFydfmqkGeqRUj601JZSPjIbuxml1rPVWBwyG1Os+bHMaOk4sC1T3p6F0Gqo/Vy1bE8d1Oq0ePl5EhsZh86gI38qkgTR4TGMfOlrrp++SddhHemUSUlcv7y+FCiZj7Br4QihZn25ixXTVmOJV+P0p3afJfTqnYe0gB7k3iKl0wcv0tpwP5nOYXO43Nb00P7d1uj0Wi4dv8YL/Z5Dq9NgMVkpW6sUAM+/+RzBRYIIuXibJp0bZFnK++Tus+xbc4jqzSq7tTr3/OFLXDxyhdotqzu9F7LVYuP8oUsULJnPbR32chuxPIFcO3OD6PBYKj5VNkeksp4/dIk5w34lsGAg/Se/lmJmxfeDfuLvH9bjsDkweOj5+fQU8hVL2WmmRlxUHNv+3E1QkbzUbVUjC9ZnjIVjl7Loq2VYTFZ8A7xZdP3HJJIfD3LnZgT//raDAiXz0aB9XZ73fCVxw7d4pSLMOT7ZbXbnBK6cus67dYdiibdi9DTw9cYRGVaDzQxHtp7g0+fHqdX4HgbmnZ3qtOwfm9XGu3WHJYYdJ20ZRbnazmsfmtuIJZdEclp7vzI1S6ZPUkCSpLozK2sWb3/vh8J87qDbsA74BHhx/WwI7fq3StHpWy023q0zlOiIWHR6La+Pe4U3J/RgzrDf0Bt1vPf9G262PPu5cPjyfU0eKTm7/6JbHP++NYcS79I0Wg0Xj1yhejPn7ENdPHKFW5dCMcWqcufr5/3rVMefEtm/3HuMuXU5lF5lBtDGozsLvliS3eY88vT4/GXK1i6FT6A3vUZ1STVEklPRarW0f7cN707pq/ZyGDCHtp6v0L/uUKIjYhKPiwiJJDYqHrvVjjnOwoENR+n0QTv+il7Aisj5bstEyknUfKYKeg89Xn6e6A166j/vHjmUOq1qYPQyYPQyoNVpKVkt42J7KZG/xP3PsIeXkbJucPqQG+pxKV/1nMq/v+9AKhK9UccvF2c4PT6Yi0rs3Tg0Wk2mpI+zi3MHL/JB0xGY4yzo9Fq6DetI7y+6Amplef86Qwm5eBvFoTB0wUCefilnVoG6g7thUYRdCye4SF4uHLlMqWrF3RYPB1Um+sLhy9RtU5PgInmdOvb5w5dYN28LZWuW5LleTZ2aZZcb6skGvHw90Oq0CeqCAr0h9+3ODFazlbuhUQn1Fw/fpK6YvpofP/oFIQRD5r1L866PRn2hwUOPTNDal5CQrqmi1WqZunMshzYdJ3/xoFSVQB93zuw7z0fPfAFISlYtxuRtY9xe/1K+bhnK1y3jkrHL1ChJme/cK92eG+pxIX3HvUKtZ6tRuGwBPp73Ln55fd06/5VT19PdTjKnEnLxNt2LvU2fCoP4oMkI7LaHJXp/+mQRdqsdm8XGnGG/ZoOVD+NwOLhxPgSLKeWmJ8UrFaVOq+og1ELCXX/vT5LeafQ08tQLtZ9opw/w9w/rMceZMcdZuHTsKldOXnfr/NfP3uT4jlM4HDkjk8oZ5C5BXYhvoA9j/xmeLXMv+GIJS75Wyy16fPYy3Ye/lC12ZJU1czcRGxGLokguHr3Cqd3nqPp0xSTH5C0UyI1zt9BoM9ehytlYzVYGNvyU62dvYvQyMmPfhBT3I+7ciAQJikPh3IFLmOPMD/UCeNIpU7MkWxbvxBJvQWgEQUXcl2a9dclOvunzPRqthkoNy/PVmk+zveDRGeSu+LOByNt3ee+p4XTK9zp/zVyXrnOiw2OY9MZMxnafTMillJui3GP5lNVYTFYsJisLxy7jtfID+e7tHxNL6B8VCpUugN5Dzf1WHEqyX/pxaz6lUcd6NOncgM8Xf5Cl+eJjTCyfupq1P/+b6ffq2PZT3LxwC0u8ldjIODb8sjXFY596oRYe3kaMngaKViiMRw5r0ZcTaNe/FW9NfJUX3nqOb7eMxi+P++6cV0xfo/bXjjVz+N/jabadfFTIthW/EEIL7AduSCkzV4nziPLTJ79x7uBFHHaFHz6Yx9MvP5Vm9eDYV77jyJYTKA6Fs/svMv9c6triJaoU5fSe80gpsZqt3DgXwp0b4ZSvWyZb0hgzS8vXmhF1J5rjO07Trn+rZMv9C5bMz8g/P3LKfENbjuHCETVt8MzecwyamXEp5vzFgxPz7Q1GPYXLpNyEo8dnnShRpRh3Q6Np3q3hY7GaTA6b1ca8z3/n0rGrdBnSnhrNq6AoCnduRBCQzz9V7SGNRuNWSY0HqdK4AucOXsRmsROYzx9P38fjwpydoZ5BwCnALxttyBaklBnuYRZy4XZiteadG+FpHj965VCWTPyL25fD2PXXPsxxFhSHxBRjzozJGeL03nOc3nOeum1qpOr00oNGo6Hrxx3o+rGTjEuDs/svJDrtg5uOZWqMIuUKMfLPj1gzdzPVm1WiWdeGKR4rhMixmu3OZPHXK1kxfS1Wk5Wj204y/9w0Rnf+lvMHL+Lp48H0veMpUCJlkbR9aw/x9WvfozfqGPnnRy7baP1/XhvTjQIl8hF2I1ytTM6EYOO5gxdZM3cTZWuXonWfZ3LExT1bHH9C+8bngbFA1u7NH0H6jH2FS8evcetSKK+O6pwurZA+Y7vzzWvfg5T0HNE5zeN9A314fewrOBwOxnb/jv+W76Vk1WK06tvcGS8hRY7/d5phrcagOCQ/f7aIuScnE1TYuelvrqRRh7rsW3cEpKR1n8y/V3Vb16Ru65oZPu/EzjMc3nycms9WpdJTru3XvPufAywcu5RSVYvRf0qfFIvJnEHIhdvYHpCUPrTpOJeOXsFqtmG32lkzdxN9xqSsXfVVj6nERMYC8HWf75nrpqplrVbL8/2ey/T5kaFRfNBsJOZYM0YvIxqhoVUWPlfOIrtW/N8BHwMpBuuEEP2AfgDFijmvYMLZ2Kw2YiJiCcwfkO4red6CgXy/d3yG5mnetRF1WlbHYXcQEJx+USmtVsuIJR9maK6scHTrSWwWO4pDQW/Ucf7Q5UfK8X/6+2AObTqOl6+HW6pCH+T8oUsMbTkaq9nGovHLmLpzHKWquSajJzI0ijFdv8VqsnLxyGUC8vun6nizSqcPX+S/lXuxmqzUaVWDik+VRVHUOyu9h56CpQqker5G+4CUuTZjW5OKovD7+OWc3HWW9gPauFWiI/RKWGLTGku8hbMHL+QIx+/2zV0hxAtAqJTyQGrHSSlnSSnrSCnrBAfnzArNkEu36V70bXqW7M/QlmNcnu7lG+iTIaefHdRpVR29UY+nj1rDYLXYGNP1W5ZPXZ2qEmVOQavVUqdldbc7fVDDTKCmdgohOHfwosvmiouKT3RINoudiJBIIm5Fcmbf+WRTZrNKySrFWHJzNr9emsHIPz+icJmCfL7kQ+o/X4uen3emZe/UG85//seHFCgRTNEKhRn+68AMzb1q1kZ+G7ecPasO8sXL33D7SlhWXkqGKFW9OIXKFMDTxwNPHw/a9M0Z+2vZseJvBLQTQrRFbeziJ4T4VUrZMxtsyRL//LCe6PAYpCI5teccZ/ZdcPntuSnWhM6gy7LKoasoV7s0Mw9M4NzBSxQqnZ+PnhmFJd7KnlUH8Qnw5rleqX/Bcxo2q41v3/yB4ztO8+I7rejyUTuXzVW7ZXV0eh1aPy0ajYZaz1Zz2VyFyxSgWfdGbJi/Fb+8vtRtU5PeZd9DCEHRCoWZunOs04ukDB4G8hQwcOn4VTYs2EqZGiUY89ewdN0pV29amV8uzsjUvCEXb2M1qWEmodEQfjPCbXIfeoOeabvHcfHoVfIXD8oxCze3O34p5XBgOIAQohnw0aPo9AEKlMiHwUOPJd6KdCjkKRDg0vl+HfMnv375p9qw5J/hThOKcjZFyxemaPnCHN12Ek3CbbnVZOXq6UevmGzVrI1s+3M3VpOVBaOWUKtFVcrUdE2VZf7iwfx06jvOHbxEuTqlXaoTL4Tgozn9GTD1dQweeib3+wFznFpsdvXUda6cvO6SMFPUnWjeb/QZ8TEmPLyM2G0OWvZu5vR5HuSFt55j3c//Yo4zU75OabdtDN9Db9BTvo57NHjSS24efxZo2+9ZOn/UntrPVUu4FXVd6za7zc4vo/9IbFgy62P3tESQUhIXHZ8Yj80IFZ8qS9HyhTF6GfH29+LQ5mO8HNyX38YtdZp9x3ec4oOmI5jQaxpxUc7vmWqOMydm+QgB5viUK3GdQZ4CgdRvW8ttzUE8vIxoNBrK1i6d2FBFo9EQXNQ1+zK3LoclhvzM8RZO7TnnknkepFDpAiy6/iPzzk7jm00j3S73kBPJ1spdKeUWYEt22pAVtFotvUd1cc9cDzQs0eq1LvtiPojD4eCzF8ZzaNNR8hbKw9Rd4zIkMqc36Jm6aywhF0NZ9cN6Vny/BrvVwW9jl9GkUwOKlCuUadti78ZxaNMxxveahtVk5fSec2j1Wj6a2z/TYybHC2+1ZPvSPVw4cpmnX36Kyg3dH/vPLNERMRzadJwSlYtQvFLRVI998e2W6PQ6Lh69Qts3WjhNb/7/KVm1GPmKBxN6NQypSFq91swl8/w/BqPe6eJqjzK5kg3ZzO0rYfy7aAdFyheiUYd6KcY7hRB8s3Eks4f+SkA+P96d2tfltp347wwn/juNw64QfjOCVbM20Gtkxi50Wq2WImULojPoQCY0uM+iXfExJt6s+gFxUfGJsVub1c6d62nXN2QUnwDvhzKwYiJjuXLyOiWrFsPbz8vpczoD9T36EFOsCcWhMG71p1RrUinF49U2j67feDQY9czYN56z+y9QoFT+x7LL3aNAruPPRkxxZvrXGUpcVBx6o553Jr9G2zeeTfH4MjVLMmF9OhqWOImAfP6JYQ6dQZclSekuH7fnxK4zXD5+jZcHv5Cl1f65gxeJjzElNq9AgJevJ33GvpLpMdNL2PVw3qrxEQ67A4OHgdnHJqW6YeewO9BoNW4v2rl49ArmOHNiwd5/K/am6vidwabftrN8ymoqNyxHv296PRRSkVKy+OsV7F93hNavP0OVxhVTGCn7CQ+J5PLxq5SvWwafAO/sNsfp5Dr+bCT0ShhWs9r43GG3cPjfE6k6fndTrEJhBs9+m7++X0uVpyvS+vVnMj2Wb6AP324Z/dDjl09cY0yXbzHHmflwbn9qtaia4hih1+6w7Y9d5Em4AGl1WnQGLR/Pf49G7eu6JXa7+58DWExWrCYrikPhwPqjtOjxdJJjbFYb5w9dZsvvO1gxfS0+Ad5M3DzSrSqbxSoWRqPVoDfq0WiFSzOEQFWwnPzmD1hMVi4fv0qhMgVp/27rJMds+2MXC79cijnOwum95ylRuShlarhXjjg9XDtzg3frqdlGRk8jc05861Z9IHeQ6/izkcJlC5KvWBBh18NRHEqOTHVs8crTtHjl6bQPzCRfvzadq6dUmd3RnSayInJ+sseZYk28U+tj4qPj0ep19BrVGcUhKVurJLWfq+4y+/6fUtWKc2/xLqWkRJWksXO7zc6ghp9y7UwI5jh1tR0dHsOU/rPpNbIL1ZtXzlTZf0bxy+PLDwe/4b8Veyldo4TLO3ZFh8cgEjK47DY7kbfuPnRM2PXwRNkRjUYQfiMiRzr+nSv3YzXbVFul5Ni2UzTqUC+7zXIquY4/G9HpdXy/bwJHtpygYKn8FKuQs3rhuoN7oSQgsSlJcoRcDMVmsWG3ObDbHFw6fo2h8wa4w8QkVG5YnpF/fsT+dYd56sU6lK5eIsnzV05e5/rZ+04fQGgEp/ecY9RL31DruWqMWjrELbbmLx7MS4Oed8tcFeqXpeYzVdiz6iB5C+XhhXdaPnTMs682Yfm01USE3KV45aLUdPFdSGYpX7c0Or0Wh82Bw6E2f3ncyG296CQib98l6k6MeoudTJeo7CYuOp5Tu89RvFKRHJXdcO7gRb54eSLmOAsfzn2HBi8+1CUOUJuPv1nlAyJDo1AcDkYt+5g6Ld230k8vu/7ex4j2XwOg02sJLBhAbGRcYqxdaATrbItzhFCXKzDHWzB6GlJ8fQ6Hg5iIWPyD/HL0e7B//RFO/Heap16sk+Ny8DNCSq0Xcx1/Btj51z6WfPMX5euU4o0JPROrZw9uPMqI9hNACGo9W5Uvln+coz7U8TEm3qgymLioeBSHwnc7vnxopfooEB9j4vDm4xQuWyDN9MTsone597h5/hYARk8Df8f+yrQBc1g/fyuKQ6FYxcL8cPCbbLYy+zDFmlAUmWOzoR43nsieuw67A5vVjodX1lUHLxy5zNhuk7GabZw7eJE8BQPp+nEHAJZM/AtLQlrhvrWHuRsa5dZG0Glxes854qLiiY82AbB96e5H0vF7+XrSsH3d7DYjVTy8jAgBUoJPoDdCCN6d0pdytUsTFxVPaxero+ZkNv++g4l9vkdK6P/da9mmse8urGYrd8OiCSqcJ8dFAXKWNU7k5O6zvJS3Dx0CejFvxO+ZHkdKydjuk3m33jCsZhsANouNsAdyxsvWLInR0wACPH088Al0TfqXw+5gw4Kt/P3Deq6evsGol75mZMev0xSdKlaxcKLwl9HLQEUX6wk9yXyy6H3K1i5FyarF+GK52kRAq9PSuu8zvDz4Bbz9k3427DY7l09cwxRryg5znUJcdDzrF2xhcJPPedG3J6M7T0q2e9nsj3/BZrFjt9qZO/y3bLA080gpMyQymNgruvxAPmiafK/o7OSxXfHPHbaQ+Bj1y/T7+BV0HdoBz0y0tbt54Ra7/tqfmI2AAP9gP15+/37TsN6ju+Id4M3NC7fo9MGLLhNQm/LOLDYv+g+kRGhE4l3GrUuh/Hh4YornBRXOy5T/vmT7sj1UqFeWem0yrhOfFntWH+TMvvM8/VL9J7o5ePGKRfh+74R0HWuOtzCg3jBCLt7GYXfQsH1dPpzb/5EKgzjsDt6tO4zQq3ewWdSF0a6/9/Pxc6Pp/GE7nnqhNrcuh3Lx6BXyFAgkIuQuUsrElNxHgZO7z/LZ8+MwxVkYMK0vz7+Ztj7/6jkbiYmIRSqSi0eucHrPuRxVt/DYOv68hfOgM+iwW+0YPPWptnZLDd88PtzTrzV4GujwXhv6ju2eJCVPp9fRbWgHJ1idOgc2HsWSjFZMREhkmueWrFo8XQ759pUw5gxfiN6g480JPdMVstr9zwG+7DYZq8nKn5P+5qfTU3IrMtPBvM9/58rJ64m//7diH97+8/lwzjvZaFXGuHMjgjvXwxOdPoDdaufo1pOc2XeBD2a/zeR+P6DRaNAZtDz1Qm2EgLcm9c5GqzPG9+/NJSZS1YGaPmAubV5vkWbopmCpAhg8DFjiLSgOhbyFc9b34bF1/O9Nfx2AO9fDeWN8j0wX9/jl8eXLv4ez6KvllKxWjN5fdHVLHnZyPNujCcumrAIhKF6pCBcPXwag38ReTpvjk7ZjuX7mJkIjCLl0m8lbx6R5zqndZxMvSEIjuHb6Rq7jT4PI0ChWfr82yWOKQyH06p1ssihz5C0USEB+f+6GRmG32tHo1DRIxaEgNIL/lu/FarahOBS8/Dxp83oLnnqhdnabnSF8Ar3RaASKIjF6GdOVuNG6b3OiwqI4sfMM7fq3TrZXdHby2Dp+30AfPlk4yClj1WhehRrNqzhlrKzw2phu1Hq2GuZ4C3VaVU/cc/j/EJbD4eDC4cvkKRCQ4e5XYdfDURQJiuTWpdT3Dv5bsZdFXy0nuEgejF4GtDot3v5ebpe9TQ1FUbh9OYzAAgFO2eR3Fg67I6kDEeDl48lrY7olOc6csGL08vV0s4XpQ6fXMWP/BHYs3UOBkvkoWbUYb9UYginOhHQonN5zFqERaIUWxSEpVe3Ry4n/6Kd3+abPdGIiYhkw7Y10OX6NRkP34S+5wbrMkZvOmYMJuXgbKSWFSqfelu5BpJQMa/UlJ3edQSqSL1Z8nKHK1mVTVjFn2K+A4IM5b/NsjybJHhceEkmvMgOwmqwYPPQ8/9Zz1GtTi4pPlU2MUd/7bGVXaqvdZufDZiO5cPgyBg89U3d/RZGyWWv+7kx+G7eURV8tJ3/xYD7/40MKlymATq+uxRx2B8unruanTxchFclb3/aiw7ttstnihwkPiWTfmkOUql6ccrXVfHdTnJmZ7//Mxl+3YbPY0Rl0dBr8As/0eJqSVR49x/8o80Smc+Y0ou5EExMRS+GyBVN1hqFXw/ju7dkc2nwMjUbQ47OXeeWTl9M1R/jNCI5tP5UYc1363aoMOf6XBj1Pix5Po9FqUpXmTdK6z2pXe6k+UFC1bMoqZg35Be8ALyas+9xlzUtS48y+C1w6dlXV1jHbWDNnI29OeNXtdqTEK58k/3d12B0MbvK5qlWfsC6bO2xhjnP8MZGx9Kv+IZZ4K0jJl/8Mp0bzKnh6e6Az6HDY1apsrVZDy9eaUbT8k1eZ/sekv/jp00X4B/kyYf3naHVapr03F61Ww3vfv0HBkvmRUmKz2jO9D5kZHtt0zpzGwU3H6FHiHd6uOYSx3b9L8ThzvIX+dYexb+0h7FY7VrONPyb9ne55/PL64uFtvJ+6Wb8MUkr+mrGWb/p8z4mdZ9Icwz/IL0099qLlC9G0a0M0Wg2B+fwTaxpArbKd9fEvOOwOou/EMGPwz+m235kEFc6jhq1QN+bTUgS12+xsX7qbfesOZ2t/4MsnrnHp2NUk+tU5qS7kHpeOXcVutWOJt2AxWdm75mDic92GdSS4aF6ERtCqT/MsqbE+qsTHmPjpk9+wW9Wexj8O+YXP243n0MZj7F9/hNGdJnHl1HU6F3iDF3168v2gn9xmW+6K3038Pn65ujIC/lu+l6g70fgH+T103J3r4Vji7mfu3OuBml4MHgam7hrHXzPWUrBUftr1b8WauZuZ9fGvWOItbPtzF/POTsuSxPI9u4b89C6DZryJ3qhPcgej1WrQ6XU4bKoksV8e1zT1SIv8xYMZveJj/p65jooNytGqT+rFUyM6fM2xbScB6DiwLX3dIPOcHEGF83BPCU6j1VCsYmFGuknfJyOUrFoMnV6Hh7cRKaFu6/tpwvmKBvHrxRkoipLjipcyi5SS21fC8Pb3SlejGq1Og0arhYTvgbe/F1F3YtRFhYS7oVH8MvoPou9EI6Xa5rPLkPZukVTJdfxuolS1YpzYeQab2YqnjxEvv+Q36wqWyk+BUvm4dTkUq9nGUy/U5sPZGUvvK1K2IP0n90n8/fKJq/ezboQg9OqdLDv+exg8DA89ptVpGfvPcGYOnkeeggEMmtnPKXNlhlrPVkuUJLbb1DuolDZ5D6w/kigat3XJrmxz/P5BfkzcPIp/flxP6eolaNe/VY50nr6BPsw6OumhGP+D5ES7M8ukN2fy7287EEIweuXQNKWujZ5GRi79iB8/WkBw0bz0/64P+9cd5ts3f0AIQf8pfTm67SQ6gx6bxYbQCDx9Ml5rlBlyN3fTIOTibbYv3U2p6iWyJApms9pY9NVybl0OpeuQ9qlqzZjjLRzZcoICJfNRvGKRTM95j0vHr/J+488StGKKMOW/LxM3EXMyl45dIT7GTKUG5bK8QXxs+yk+eX4cNrONniM60fOzTg8d82HzkZzZdwEhoM3rLej/XZ9kRso4YdfD+XXMn3j6GHl1ROeHqndzyfnERcXxcvDriRXJ1ZpWYtK/X2RqLJvVhhACnV7Hoq+W8efkfzB6Ghg0sx/129Zyptm5m7uZIToihndqf4wl3oJWr2PYL+/RuGP9TI2lN+jT3bbQw8uYoQ+Aw+5gfK9p7F93mPptazFk3rtJag1KVinGwsszuXEuBK1emxj3zsmsmL6aOUMXIjSCRh3qMeyXgVkab9bHv2BO6Nj1y+g/6Ppx+4cqrMet/oR/f9+Jp48HT7+cub9zcgx9bjQ3zt9Co9Vw49wtxvw1zGlj51RMsSZ+Gf0nMZGx9Pj0ZQqUyJfdJmUJD28PvHw9iL2rdssrXinzC7IHxR1/G7cMc5wFD28jGo37st9yHX8qXD9zE6nIRA34w/+eyLTjdyXbl+5m99/7McdZ+G/FXkp8UxRPX0/qtalJwVJq4YjFZOWzF8djjrcQEOzHZ7+/z2/jluPl58nbk3onu9/gSmIiY/nurR+5dTmMN8b3oOYzSTtvrZy+NlGSYvOiHQxd8F6WVv1BhfOgTdBY9/TxSLagz+hppPX/7QPE3o3jk+fHcenoFdq83oJ3Jr+WrB2Lv1nJ4gkrKFK+EGNWDk3yft66HIriUFAcCldOXsv0a3iUmPT6THb+tR+Hzc6hjcf49dKM7DYpS2h1Wr7dNobfxi0juGheeo3snOUxIx5sViMh8nZUlsdML49PAM4FlKxWHJ9Abzx9PTF6GWnauUF2m5QsDzoixaEwf9QSZg1ZwNu1hhBxS5Vz2L50N/HR8ZhjzUSFRTO8zTh2/bWfLb//x4Re091u84zB89i5ch9n91/g83YTsJqtSZ6v3KgCBk8DOr2OEpWLIoTg6LaTLPxyKecOXszwfINnvUWTTg2o0bwyE9aPSHfsecX0NZw7cBFznIU1czclO/ety6EsGLmYmIhYzu6/wMKxS5M83+nDdhg89Bg89PRIJsT0OHL5xDVsFhuKIgm7Hp6tWVLOokTlonyycBBvju+J0TPrxYCNOtajSPlCaHQaCpUt4NS7zLTIXfGngqe3B7OOTOTIlpMUq1g4x6akNX6pPrv+3s/eNYcILhrExaOXQaoN0s8fuky9NoEUq1AYkeDsFEUSHx2PlOrdzO3LoW63+e7tKOwJwncOmx2bxZZko3jQzDcpU7MksXfjaPdOK07uOsMnbcdiNdv4ffxyZh76JkPFWH55fDNVya3TaxEJt+BSymT3Rv7/DuD/f+/7ZXfavtECvVHvtE11d3Hl5DX2rDpIxafKUfXp9IuMvTqiM1+/pi4oXnynVap3a79PWM6qWRup+nRFPpj99iOx/+QMPL09mLFvAqZYM54+Hm4tdHwy3uEs4O3vneM14LU6bWIM/My+83zYfBQajUCj1VC+rpppUevZanww+y32rj5Es66NuHTsCgu++AONRvDGhJ5ZtmH7sj1M7vcDHt4ejF75cZq9VN8Y34Mz+84TFxVPt+EdH9rw1Bv0dBhwv2Dp9J7zKHZFlZfWCi4cvuyWKtwO77Xl1O5znNl3gfYDWlOq2sNCd/mLB/PamG78PmEFxSoUpsenDxdlPYox7tBrd3jvqU+wWmzo9FrGrvok3b17m3ZpSLWmlTDHW1LVqTmz/wK/jlmKJd5CxK27VKxflhffebx1+h9ECJEtchy5WT2PIXvXHmLCq1OJvRvPK5++TO9RyW8qx0TGotNr8fRJ+sGLvH2XldPX4pvXh3b9W6UpMy2l5EXfnol1CmVrlWLG/rSliaWUOOyOdK3wrp8L4d06Q0GodzJzjk8mMJ9/muflknl2/3OAr3pOIT7ahEarofcXXXnlE+fqzxzddpJPX/gKc6wZvVFHnzHd6fxRO6fO4S6unr7BpNdn4HAofDj77RwhT56b1fMEsX7+FmIi45CKZPGE5bR7p2WylZ8pFaF80HQEIRdD0eq1XDl5nQ9mvZ3mnPeyiIQAvTF9H6t7KW3poUjZgsw99R0XDl+mQr0ybt+MfhKp1KAceqMeL19V7K7+885NNQSo0rgCjTvUY8vinZSuXoK2b7Zw+hzuYnTnSVw9eQ0pYWTHb1hw3v17Z+kl1/FngIMbj3Lp2FUav1Sf/MWDs9ucFPHx90Kr02K32hFCoM+ABojD4eDGuVuJq/FTu8+meY4QgpHLhvDdWz/g6evJkJ/fzYr5KRJUKM8TLfccExnL9bMhlKxazC1Ko355fZl7YjLHtp+idI0SLpEW1mg0DF3wHkMXvOf0sd1NfFQ89wIo8dHx2WtMGjz2jl9Kyc6V+4i4dZfm3RrhE5C54pkdy/cw/tVpKHYHv475g4oNynP9zE1e+fQlWvd5xslWZ43Xv+pBeMhdbpwLoe/Y7hl6zVqtlmbdGrHrr31IRfLSoOfTdV6tFlVZcP77zJqcSxqEXLzNO7U/RioS37w+/Hh4ols6dfkH+eXIFOaM4HA4GP/qNHYs20OFemUYt+bTTHXjS4vBs95idOdJSCn5IIPV9u7msY/x35O+VRRJ/uLBzD0xOd275ztX7uP3CSsoU6skdpuDNbM3AmqmhwQcNgcGDz3zzk5zi77Gg5jizKyduxmdQUerPs2dquwnpeT03vP4BHg9kYqKOZHfJ6xg3ueLcNgVvHw9GPbrIBq8+FDoNpdkUAXRJmKKNWPw0PPm168mSRxwJvf86fZlu5ny9my8/DwZvXJotslRP7Ex/j2rDmJOED27eT6E+Oj4xAyS6+dCOLv/AtWaVnoohHDnRjhjX/kOq8nKhcOXaN69MUYvAxqNBonEakpoNSdEkrZz7uLzF8erYRghOLL1BJ8tGuy0sYUQVKxf1mnj5ZJ1Slcvjs6gx2G34HAoFC2fM1OLcyIeXob7vSE0GpeGyYQQKIrChFenYTXbiA6PYXK/H5m6c6zL5swMbnf8QoiiwAKgAKAAs6SUU1w1X4seT3PhyBU0GkGxSkXwSrg9vnj0CoMafYoQAq1Oy9yTk8lT4H6OdUxkXOKdgc1qx+Ch5/u947l6+ibFKxZm1MsTuXXpNh0Hts1Qo5S0UBSFY9tO4e3vlaqG/em95xI7cB3ffspp8+eSM6nbuiZD5w/g0L/HadalYZZqSszxFg5vPk6BkvkoUTllzajo8Bg+feErrp68TsdBbXltdLcUj83JVG5UgU4fvsj6+Vuo+UxVnuvV1KXzCSESa2ZU/5Lz6mSzY8VvBz6UUh4UQvgCB4QQG6SUJ10xWbv+rSlVvQSRt+5S//laic78wPoj2Kx2HDYHXr6enNx1Nkkss0TlojTqUJcti3fiH+RH5w/bUbBU/kRxtZ9OfucKc/my22T2rT2MVCR9x3XnpYHJx9ibd2vMlsU7AUmr11KXG87l0ccUa2L/usPcvHA705oudpud+aMWs3L62kQV0s9+/yDFHriLv17BuYMXcdgc/Dnpb5555WmKZUAiPKcghKD3qK70HtU102Mc2XqC7Uv3UKtF1TTreoQQjPzzQ757exY+Ad58OLd/pud1FW53/FLKECAk4ecYIcQpoDDgEscPUKVRhYcfe7oiOr1WDd1ISbnapQBVEXLhl0sJKpKXwbPe5v0f+mH0MmZYXjYmMpb5IxdjMVnpNbJLuvYAFEVhx9I9ibelq2ZtTNHxfzD7bZ7t2QSdQUelBuUyZFsujx6zhy1kwy9bsVnsnN57jj9D52ZYNmDh2KUs/XZVktDk+gVbUnT8Wp02caEkIUeuXO8RHhJJXFQ8RcsXcnoF7JVT1/n0+a+wxFtY+9NmVZK5RdVUz6nbuiYLL890qh3OJFtj/EKIEkBNYE8yz/UD+gEUK+b8jZGK9csyactoTu06S51W1clXLBiH3cEHzUYSdzcOnVGPOc7M+z+8lanxx3abzJEtJ1EUhVO7zjLn+OQ0z9FoNJSoUpRrZ26i1Wqo1qRSiscKIajeLH1VlLmkjJSSpZP/YefKfTTr1oh2ObRqNPRKGDaLHQC7zYEp1pxhx3/9bEgSp2/0NKSqKd91aAfO7r/AxWNX6fJROwqXSbtSesX01ayft5W6bWrw2uhubpEh2L5sD+N7TkEIQfPujflwjnMzaq6euoFGq74Oh93BxSOX03T8OZ1sc/xCCB9gKfC+lDL6/5+XUs4CZoGa1eMKG8rXKU35OvebR5jjLZhizEgJNrONq6dvZHrsa2duYrepX9Rbl9KnhWO12ChUtiC3r4RRtUklBkztm+n5HyfsNjtLJv7FzfO3eHnwC6lmSJjizFjiLQQEp6+qd++aQ8wfuRhznIVzBy9SskqxDGnSuIteo7pwbPspLCYrz/d7Nt2v70G6fNSOPasOYrfaKFSmAL2/6EbjjvVSPN7bz4vx6z5P9/gnd51h7vDfMMdZuHbmBiWrFKNZ10YZtjOjLPpqWeJ+1/p5//Le9284Ncut5jNV8Pb3RgiBRqOhUSrv2aNCtjh+IYQe1ekvlFIuyw4bksPbz4uWrzVj46/b0AhBz8/TL70acuk2E/vMwBxnZuDMfrw6sjNT+88BoOuwDukaY+3czexbcwirycrhzcc5e+BihrNrrBYbUlGcoh6YU1jwxRKWTV6F1Wxlx7I9LAmZnWznr0Obj/F5u/E47AodBrTmrYm90xw7/GYk8oH+BOE3I5xqu7MoV7s0f4bOxWqyZrqRS9lapVh8cxYx4TEEFw1y+mo86k5M4piKQ+Fu2EPrOZdQqnpxrpy4hs1qJyB/AHrDw27tnlpomZolM/y6fQK8+enkZC4du0qR8oXwy+PrLNMB9a5z/7rD3A2LpvFL9V1SY/D/ZEdWjwDmAqeklN+6e/60+GDW2/T8vBPefp4Z+oJ91WMKp/ecR0rJ5+3Gs+TmbOq1qYnd5iBf0aB0jWGz2BKdkBBkOE1051/7GNttMopDYcC013m+33MZOj+ncvHIlURtfqvZRnREbLIVvPNG/J6oF7RsympeG9MtzQtg085P8cekv7h18TaFyxakfgrx7pyA3qBPUzcpLTy9PVzmWOq0qk65OqU5tu0kBUsX4LlXm7hknv/nvWmvE1w4L5G379J1aIeHHPuSiStZMHIJCEGb15/h3SkZv5P29PGkUoPySR6TUrL0u384uOEorfs+Q5NOmZNtX/z1ChZ+qUp5L5+6mu/3jk9MC40Ki8YvyDdJYyVnkB0r/kbAq8AxIcThhMc+kVKuzgZbkiW9jvpBYu/GJ27KmhI6PT2YHpoe2r7Zgh0r9nLuwAWadW2U4ZDDzPfnJd7y/vjRghzh+EOv3eHYtlOUr1cmXWqa18+FsOTrFQQVzkO3YR0xeBjo/GE7Dv97HBDUbFElRWnjQqUKcO7ARWwWO15+numSqvD292buicnERsbhm8fHrdK4meHKqeuc2nWW6s0qJzbZySnoDXombh6FxWTB4GFw23tp9DTS+4uUM3ZWTF2TuHBYM2dTphx/cmz7YxfzR6hhwqNbT1K0QuFMFWpt/3N3Yq3RhcOXsZisSCkZ1PBTrp+5SXDRIKbv/SpdDd7TS3Zk9ewAcva3KxMMmvEmI9pPwGa1M2jmm5kaw9PHk8lbR2fahryFAgm9dgepKPgHZ7+IWdj1cN6s+gFSkUgpmbb7q1TzxhVFYfDTnxEVFoPeqCMyNJpBM96kerPK/HppBlFh0RStUDhFhzJg+usYvY2E34zgtdHd0p2JpdFo8Mvr3Nt3V3Dp+FUGNvgEUDf3Zx2dlCPlnnNamLFc3dLcDYtGKpISVVL+/KWFlJIfPpzPv4t2UOvZahSvVBSbVd3H0+g0hF69kynH/3SnpxL3E4tVLIzR08DWJTu5dSkUm9VO+M0Iti7ZxQtvOW8h99hX7rqL6s0qs+LufODhRhzu4rPFg5k2YC5Ws9VpjcKzwsmdZ5BSvQPSG3Uc3HA0VcdvibcQExGLlBKr2cbFI5cTnwsI9k9zQ9PDy4gpxsSpXWf5a8ZaBs3sl+E03KwQFx3PzfO3Er68Dzs/h8PBvM9+5+i2k7zYvxXP9shYKOTIlhMoDgWr2Yanjwcnd57JkY4/pzHsl4GsmLYaq8lGx0FtMz3Ooc3HWfXjBiwmK5sWbqdBu9r4B/kRGxlLkbIFqflMlUyN2/XjDpSuUZKohBi/EIKgInkTe2MLjSC4qHMlYXIdvxPJ7jBBUOG8fLH843Qde2TLCaa+Oxu/PL4MXziQfMWcrzZavl4ZkBKDhx6hEVRrmnJ6Kqh3PM27NWb70t0AdB+efu33k7vO8GW377hzIxypSDb/toOnXqjjNj2b0Gt3eKfWx9isNnzz+DDr8MSH9ojW/byF5dNWY4m3cuHwZcrWLJlYEJgeqjerjEajweilbmwf3XaSf37cwEvvP5+skJrD4eD4jtMEBPtlaJ7HDQ8vI92GdszyOA6bPXGFD6oczJJbc7HEWwgqnCfTiwwhBHVb1UjyWJVGFXh3ah82/7aDhu3rUq9NzayY/hC5jt/NxEXFsfufgxQomY/KDcunfYITMMdb+PmzRdy5Hk7PzztRokoxPm83HlOsGaERTHx9Jl9vGOH0eQuUyMeM/RM4uPEYlRqWS7MrF8DH8wfwyqcv4RPgnWwPgZT4vN14osNjkzzmTgHCHUv3YIo1JebaH9x4jKdffirJMRG3IrFb1XaTGq2Gu6HRFE/9WpiEklWK8d6MNzi7/yJefh4sm7wKi8nK2f0XmH3824dkkz9vN4Hj20+hOBQGznyTlr2aZek1PunUblmd4KJ5uX05DFAL3Dy8DPi7KEzYpm8L2vR1TX+CXMfvRuw2O/3rDCPiViRSqjKuLV55Ot3nx96N49alUIpVKpKhPOWZg+exYcFW7FYbhzYdY3HI7MSVi1Rk4ma0KyhSrlCGdGWEEJlSBL2XzQPqrXHjl+o7pXFIXFQc5w9dpnjlIqmGmopXLoJGqwXsKA6FIsmIqLV5vQVr5mwi/GYklRqUo0rjhyvKU2Ph2KX8Pn45QggC8vsnZn0JjSAqLJqCJfMjpeTf3//jwpHL7F93ODFLbOX3a3MdfxbRaDRM2/0VE3pNI/TqHd6a2Cvd+xmRoVHcOBdCmZolE0XiQi7eZsnEleQpGEjXIe2TTVF2FbmO342EXQsnPCQi0UltW7KLFq88TWRoFN5+nqn+4a+fC2FAvWEoDoWgInmZsX9CulUGr525kegk4mPM2Cx23v+hH9PenYOXnycDv38jzTEcdgf71x/B08cj1Yri7OL9WW8x+c0fMHoaGPP3cKfcTUXdiebNqh9iMVkQQjDz4NcpNiOp/Vx1hvzUn/0bjtCsS8OHNvkuHr1CfHQ8889PwxJnwcvPK8OhwY2/bE3M/jDHWwgI9iM6IpbqzSpTLqEQcdXsjfz44Xz1MybUTUe9QUf1FMJsVrOVUR2/4ej2UzRsX5ehCwY4PXXwcSIwnz/j136WoXMun7jGwIafAIKAYD9+PDIRDy8j7zf+jMjQKPQGPRE3Ixk0s59rjE6GXMfvRoKK5ME/yI+oO9GA4Kl2tZnQexpbFu/E4KFn0r9fpKjIufm37cTHmJCK5M6NcE7uPJNquf2DvDqiM5+/OB5FkbTs3RQvX09avdY8Q+JuY7p8y4GNR0FKug3tQI/POqX7XHfwbI8mGd4sTYvD/57AHG/GFKNuTu/6a3+qjWmadmlI0y4NH3p8xfTVzBm2ECEET71Yh09/ez9T9tRuWZ07NyJQFEmpqsX47r8viY824RPgnXgROfHf6cSLg9HTSOu+zSlVtTit+ib/t960cDtHt5/EEm9l19/72b/2MPWfz7m1DPcID4nEbrW7tBNexK1Itv2xm8JlC1C3deZj7FsW/6feVUs1nfHU7nNUblguMdPIarZy/vBlp9mdHnIdvxvRG/TMPPA1O5btoUCp/BQtX4hpA+Zit9qxW+38Nm4pI/74KNlzS1QuitHDgDneguKQFCyd/hzums9UZfHNWcTHmDPdMGbX3/sTFR3/1959BjZVtQEc/58kbToYhZZRWgqyQRABRWT5skSQoYIsAUEUQURBQPYQAUFAkPH6qiA4GMpQEBGQPWSUvfco1UJbWkpn5nk/pITV3aYpzfl9oUmTe5+k4eTec5/zPJt+2JHnBn5HKFu9NFaLbapEo9VQsXa5LG1nzfyN9rO8Hb/8zaglH2YpEaD/rF5Ufa4S8TEJtOjZGK1W+0hu90tvNWXXqv1otIKSZYvTb+abafY1tv0uORYp0WawB7IzSCn564cd7FjxN0e2nEAIQaeP22er6mZqjElG+tcZTmx0HFqthgFz3kq1057VarWVYk7lb1qxdjn0nnoMCQasViuBlfzRe+pp2s2WyCAldB2R/YvPmZF3/8r5VCHfgrR+pzkACbGJaLUaTIC7h1uaC3IadahHYnwSJ3edoUXP/9inHG6FRRMVFk25mmXSPEX3Luyd5aX+AFXqVuDi0asIIajTImNnGo+7MlUDmbZpLHvXBlOzSXVqNKrKjavh3Po3mip1K6DVZWxKpEajqoSHRGK1WChdOfV1COnRarU0eyPta0I1X3iSBSe/4MaVcKo9X+mRQT8xPokTO88QULEkARX8adqtIYe3HOfw5hM06Vw/T/9tf5v7JwtHLcWQYLDf98v0tbw5oTOJcYnM7PMVl0+E0G30a9k++wu/fov4mARMSSZMwIH1R1Ic+O0F4jSCsb8M4bnWj15XavBKXT5ePIDTe8/RpEtD+wLRjxe/T5fhr+Dt453rvaTzfevFvO7knrMsnbyK0lUC6DOlW6Yu8BzbforRbT5DCFvq5PTN4x2WUpoYl8hfP+zEs6AHTbs1dOg88JGtJ/j9q01Ufb4iHQa1ydVc/LQEbzjCJx1noNFoqFC7HDO3TcjQ+202mVn/7RbiYuJp826LHK/1klHGJCN9nxpC1M0YrBYLk/8YRc0XHp8Kr590mMHuX+8V8tVoNZSuXIoFJ2excNQSVs1ah8lga5r0/YW5+AVkPffdbDLzTo0h3PrXNrU24seBKabMdiz+FjGRsQAUC/Rlacj/srxPR3DZ1ot5XfUGVZiyfnSWnrt6zh/2o5/Tf5/n5rUIhy3oSYo30Pj1elmqCpkZ4dcjGdtuKoYEI7tW7+OniSsZteTDPDHvvGb+BvuUzbkDF4gIvZWh8h46Nx3t3nN+uedrp0OJunHbnsW1denux2bgl1Jy6b4FfR7eHjTpWp9Gr9WjT/XBRP0bhdlkS5VFCPt1jqzSuemYHzyVQ5uO4V+uRKrX3rwKeSUXpwNvn6yfUee2vHEopWTJ/UebJqMJvadj0sFWzV7HG2X70y2oP398+5dD9nFXZOite0f4EhLuJDKlm8M6c2ZKtecrofdyR2gEHt4e+OSBshiZ4V+uBFqdFo1Wg95Lnyezs1KTEJtIeEiE/XbZJwP56Jv+zBv4HSGnQ4m7nYBGI9BoNbR5t0W2WlPe5VXQk0Yd6qXZAvWT3z6m2vOVeLJBFcavHJKl/VitVpISsvdFlVnqiP8x5nFflUV3T3duR9zJ1KKnjPpp4kr7wqTvx/1MiTLFKVbalzJVA3N8X5WeKc8TNYI4s//CvXLJqcymhF4IY9/vB6lStwLVGzq+hn6XEa/iU6ww/1wM4+W+LXI17zonFPDxZn7wVLYt38MT1YPSbSHobFJKdq3aR0xkLE26NqBs9SD+uRAGYM+eMhnvVbD1KVaYZaFfpzn9dvNaBOu/3UzJJ4rTsneTbE8jPlE9iC/3ZL2R+vVz/zC48Thio+Jo1acpH37VN1cqAKg5/jzAYrEwucss9q07xJP1qzBp3YgMLQw5uu0kY9p+hkarpXhpX746/Hm2y/ampF+dYVw9HgIC3PS28gtWi5VRSwdRv13ODx5Wq5WDG4/y30GLMCQaGbLgPZ55seYDj4m6EU3vyh9iNJjQ6jR8unYEtZpmrSvSb3PXs2HRNuq8WJM+U7rlmWsKrm7RuOWsnrUOaZUEVPJn9u5J7Pn1AEVKFKZOC9vn4dj2U0zq/AUSGL1s0AOfgZ0r93Lp2DWadm2QXFDNRLegftyJjMXNw42uI1/jjdEdnPTqbKb2mMPWpbuRyaVNpv01lp+nrUFKyYAv38p2BVY1x5+HHVh/hIMbj2EymDmz/zxbftplz/y5S0r5yJHA002q882xmfx76SY1GlVxyKAPMOn3kSweswxDook9aw5gircdZf25YItDBn6NRkPdVrWp2yr1lbeXk7+IzEYzZhOc2HUmSwP/mf0XWDDSlinyz4UwytUok27mTFacO3iJmIg71G5eI830SuWe/esO2efqrxwPQavT0rz7g9k6Nf/zJCtuLnzkuVuW7GLWu19jSDTw29z1/HhpPkaDifiYRKxWiSHByMndZ3MkTrPJjJQyS///fEsVQafXYUoygRDMeW8BV09dBwk3rkzNUMvWrFCHNnmA3tPdXldGINDftyJXSsnnvebR0q0zfZ4cxO2ImAeeW6p8SZ55saZDS+H6lSrK0O8GMGzxALwLe6HVafDw1vN0FqsRZtWvc9fTLagf416ZRpmqAbh7uOFZ0BO9pz5TF3//+PYvhjQZz6rZ64iJuINGc69rlG1xXdad3HOWRWNtFTgf2N9/xjOp8xeMeGlStrbvSpp3b4yHtx4Pbz3Fy/gx7/0FXDsTmqHnntp7zpb4IG3/h8Iu38TXvwhVnquIZwEP9F7utHm3BXMGfEvfmkP4c+GWLMX495pg2hXqSbtCPdn80450H3/nViwjWn5Kj/ID2LJ0Fz3Gd6JZt0ZUea4i41cOtS/qklISfTMm3e1llZrqyQOklCwctZSty3ZRtloQ3Ua/RvUGtjou54IvMrTpBJLiDWh1WjoPb0/vT7ty4fBl1i/YTMXa5WjVp1muVQaNCL3FXz/swL9cCf7TuX6u7Tf0QhjvPj0UY6IRN72OzsNfof2Alzi+4zTlapalYBFvfvhkBUhJj/Gvp5p9dHL3GUa2mkxSvAEPbz0jf/qANfM3cGTLSYKqBTJ716cUyGJ2xuXj1/ig/igMCUb0Xu7M3D4R31JFeKf6YOJuJwC2FMTfbn+fK+318jKLxcLkrrPZu+YgVetVZPL6USm+J8d3nmblF+s49NcxjElGCvp483PYt+keXZ/ed57hLSYiNBp8/Yvw9dHpuHu4YzFbOLPvPF6Fvdi5Yi8rZv6OMdGI3tOd+QenpXrd6tqZUJZP/ZXiQX68MbqD/fpOt6B+RITeAqBAEW9+vbU4zbhm9fuajYu2YTFZcNO7sTz06wd6QWxdvpsZveYDMPjbfrTo8UKa20uPmurJIRazhc97zSN4w1HqtanDkIX9s53TLoTg7c/e4Nqp6xzbfooRLSfRc0InOg1th2dBT/tFTq1Og5tex7ngiwxpOgFDvMFer6f1283T2EPOKRboS7dRGS+XnFPMRrP9Gq/VKjEmmvApVtje7m5QwzGcDb4IEi4cvsycv6ekuJ3If6LsX1bSKom6EcO0TeMwGkzZbtB98ciVe1+EEi4cusw3w3YTH5Ngf0yp8iUyXGMpPzu06TgHNxzFbDJz/uAlNv+wg7b9H015fapxNZZMWokxuYNWYryB+JiEdNOKq9WrxPStE5jcZRYRoZGsmPk7b4zugFanxS/QlwHPjiD+TgKW5BRQoRHERcenuC2L2cLgRmOJi47DTe9G/O0E3p/bBwCf4oWI/DcKpKSwX/pZXoZ4AzJ5BTzYponu17RLQxokX3R35Fm8murJpN2r9/P3mmBio+LYtWofe9fmzJmI1Wpl/x+HSYxLwpBgYOOirQAEVQmg36xeBFUN4KkXqrHss18Z3Hgs5uSia0kJBi4euZIjMeRlZZ8sTbsBLdF7ulOuRhCdhrV74Peh58OwmCxYzBZCz4eluI2Ni7cx74PvsFis6Nx0+AX68kIn2xdHVgf9pAQDJ3efIepGNHVerImb3s3e9rFu61pEhdkqsQJo3bR8uWey0/s25AV6r3vTm4gHpzcf1mlYe/Se7ui99Dz3cu0MryX5c8EWIkIiMSQYWTp5FeHXIwHb/H/c7XgsJovtbyGgbuvaVK1XMcXtJMYl2epkSVvP5ysnQ+y/G7dyKM+8WJOnm9bg07XD042p96SulKpQEndPd7qP7ZBie1a9p97hXczUEX8mPTw1llMzZRqNhvJPl+Xa6VA0Ws0DBdja9G1Bm74tGNp0AsbE5PQ1AR7eeiS2RWCJcYl4FvDMmWDyqL6f96Tv5z1T/F230a+xcOQSJKR4RhJ/J4Ev+3+DyWBGaAT12tRhwuph2crgSUow0O/poUTdjAEpmbXrUxafm8PFI1coV7MMPsUK897s3kzoMAOr2UL/Wb0eixaP6UmMS2T1nPUgJa8MbI13Ia9Mb+OpxtXo8FEbNv+4k9rNn0rzgnqdFjX58cp/uRN5h6BMpBC76XUIjYDkdV13S2wEViqFu4cbSfEG3D3dmH9gapqNagr4eNOkSwN2rdwH4sHPV8myxTO1ALNEmWIsOjsnw493FDXHn0kWs4Vpb84jeMMR6r1ch6GL3sux8gXxdxLYuGgbBXy8ada90SPbnTdwIX8u3ILJYMLbx5vRywYxvfd8EuOS0Hvq+fro9Ew3eM9P7s61plSILj4mntdLvI3JaEaTXK9/7C9ZW3Bz1/GdpxnbdioJsYkAdB7+Cm9/9sYjjzObbDX6HZ33L6Vkzbw/ObzlBK36NHNY97GRL03i2I5TIKFa/crM2DrBIfvJrjtRsUzuMpuQs6H0GPe6fTpUSskf3/zFse2neLFXk0e6X6Xk7gXiAj7eaX55R4TeYs38DfgFFKVtvxczXM/JUdQcfw7R6rSMWvKhQ7btXcgrzbK/faf3wKuQJ+EhkXQd+SrnD10m4U4iSfEGrGYre9ce5OW+OdeQ+XGT0oB/JyqW8JBIyj5ZmoHz32bBiCX4BRTl3RkpnzlkRqkKJbFabfO1ei89leqkXL1T56YDx2TaPmDnir0sHLmUpAQDhzcfZ35w6hcrMyL8eiSjWk8hIiSSNyd2tn82Lxy+bF/QdymXywlnRqGiBZm2aewj9wshaPPui7R598UMb0sIQanyJdN8jNVq5YP6o4gKu42bu44bV8PpN+PNTMedG9TA/xhx93DnrcndAFtzh+9GL7tXk0SITJ0Gu4IrJ0MY1GAMVikpVb4Ec/d9Rqs+OdfKzq9UUb7YMZGty3ZTpW5F+4Xm3JIYl8jetQfxDShKzReeJOzyTftKVo1WQ3hIZLYG/gUjlnD9TChWq2TB8J9o9kYjCvsVou17LVkxYy0gaNPPdQ80HmZIMBB94zZWixVDopEze887O6RUuezAHxsdh8VsyVLRMYvF4rDqlFarlZtXI/ApUTjNlL9Zff9HZPLUhkarYfgPA6nRyPFlC1JjTDJy5UQI/uVLOK365MM2Lt5un4YJu3yT8wcv2dNkc0rF2uWyXKc/O6xWKx82GEPY5XCklLw7owfNujdm1ew/SIxNJKCCf6pdtzJK5661zZFbJQhh+xl4c0JnGr76HFLKDPVRdhWeBTyp27o2R7acQFplmmfvzuaSA/+2n/cwvdd8kJKen3Siy/CMNUEwGkyMajWZ4ztOUemZCkzfOj5H87EtZgvDmk/kfPBFdHodX+6ZnPoRmxAIYbu47KbX0fDVujkWR2YlxifRv9Ywom7eRqPRMG//ZzlSJCu7ytcsg4e3nqR4A9IqKfmEYyqXOkNsVBzXz/2LObl38rble2jbryVLrv6XqBu3KVbaN9sHJ+9M7U7Y5ZvcuBxO78ldH/hCL1+zbLa2nV9NWD2MswcuUtivIAEV/J0dTqpcMp3zu1FLMRlMmIxmfpy4MsPP27v2IOcOXkRKuHoqhO3L9+RoXBePXuXCocsYEo3ExySw7n+bUn3skAX9CaoaiG9AUd6a1JUe5QbQqdQ7BG88mqMxZcSpPeeIvhlDYmwSibGJbPs5Z9+XrGrevTHvze5Nq7ebMe2vcbne7MKRChYtgH+5Eui9bCtbG75mqxXv7uFOybLFc+SMtEgJH2bt+JRl179WjdozSKPRUK1epTw96IOLHvH7lytBZOgtLBYrfgH3BoPE+CQObjxGybLFUjx9L1jEG+ypx4KCRQs88pjs8C1VxJ4uqvfUE1g59aPmoCoB9joeb9cYzM1rtpK1U7rNTnf1YE4LqFgSS/KiFDcPd8rVKJOr+0+NEIJWfZrl6Lx+XqHRaJi7dzI7V+7DL9A3Q5kpinKXSw78o5cNYsHwn0hKMPDWFNvFUovZwsDnRtpa5FmtDFv0Pi+8/uDFulrNatB15KtsX/439drWocErmZ9euX7uH8a/+jmx0fF8MO9tGnWoZ/+dX6miTP5jJL/N/ZPKz1agbb+MZR3cn4vujMqS/k+UYOqG0WxZupsaDatk6X1RMs+7sHe+/FJTHE/l8ScLu3KTd6p/hCF5afjzbZ9h4pr0V+Jl1pAm4zmx87Rtbt7Djd9jf0z3tNxoMLF79X48C3hQr02dR1Z+Xjp2lUmdZ2FMMjJs0QCebpK7xdMeZ3eiYhnbbipXT4by6get6DWxi7NDyjVxt+P5tNMXXDkZQqdh7ek4uI2zQ7I7sesMx7af4tlWtaj8THlnh/PYUnn86fALKErBogWwRsai1Wmo27qWQ/ZjG7QF9jmjDBjbbiqn/z4HQNv+Len7eY8Hfl++ZlkWnc0bXapyU0ToLSxmS7baTa6YvpbzwZcwmyysnPk7Tbo2dEiDmbxo+dRfOb7zNGajmUVjllG/3TPp5qrnhjP7LzCy1SSMiSaWT/uNrw5No3TlAGeHla+45MXdlLi5u/HfQ5/zzrTujFk+OFOLOzJj0NfvElQ1gKIlfRi15MMMXYQ7uvUkSfEGkuIN7Lmv2XRqrFYr0TdvY7FYciJkpws9/y9zBnzLLzPWYjHbXtParzbSq9JA+lQbxE+THrxAv/2XPYxrP431Czanu22NTgP3nUHdLdHsCiwW6wMlSKzWvHH2f+7ARawWW2lijUZw8cjVHNu2lBJjcp0rq9VKbHTcI2VYHE1KSeiFMG6FRefqfu+njvjvU6R4YV79oLVD9xFY0T/TzRWeblrdfsTfIJ20zaQEA4MajiHkdCh+AUWZd2DqY10fxmgw8UH90cRGx6H3cCc2Oo4+k7uxdPIqjEm2/8C/TF9D9zEdAVsZ6xlv/RdDgpEjW09QLNCXZ19K/eyt07D2nDtwkUvHr9Hxo7YudWTZZfgrnNx9lpAzoXQY3IbAinkjE6Vu61osGrMMN70OnZuWp5vkTEP40AthfNR4HDERMTTv+QLHd5wmIvQWlZ8tz/QtE7JdnTWj5n2wkA3fbQMp+Xjx+/Y2krnJKQO/EOIl4EtACyyQUk51RhyPi0/XjmD3qn14FPBIt/5K8J9H+PfiDUxGM7fCotm2fA/tB7zksNiklARvOEpUWDSNX38er4I5WyguLjqOpPgkkGBINHLh0GUAgqoGEhNxBynlA6lzN66E2y9wW61Wwi6Hp7l970JeTN346LJ+R0upo5qjGZOMttaZyfst7FeIuXtTLl/tTKXKl+S7s19y6ehVKj9b/oFyx3evd3kXsi2Wysx7uHTyKm6HxyClZMuSXQgBFpOFK8dDOLL5eKaa+WSVxWLh96822UutL5m8yjUGfiGEFpgPtABCgWAhxFop5em0n+m63PVuNO2WsXaAfoG+9lN2jVZDsdKP1q/JSatmreP78T8jpWTlrHV8e3xmjg5oRUr4ULv5UxzbfgpplXT8qC0AY3/5iB8nrsBsNNN9bEf7459tVQvfgKJEXI+kYNGCNH69XmqbdgqzyczYdlM5tOk4VepWYNrmcXh6eyClZM573/Lnwq2UrlyKz7eMp0jxzK8qT4nVamVKt9nsSk79/HLPJPwCHPu5yC5f/yL4+j9acHBMmymc2XcBgFcGtqLPlEeL4qWmSInC6Nx1mAwmhBBodRrMRgtWq6RISZ+cCj1NWq2WYoG+RIbeQuum44kaQbmy34flelaPEOJ5YIKUsmXy7ZEAUsrPUntOXqrO+TjYuHgbm3/cyXNtatNhUBuHHll+2HCMfRpKq9OyMnxhljtYpcZqtXLlRAg+xQunOBg8zGK2EH49Er+Aog7rQ5xVe347wLSec20VVb3c6f9FL17u24KLR68wqOFYDAm2Tmsdh7RNsdJnVpw7eImhTcaTFG9Ao9XQ8aM2vDOtR/pPzGOklLR062w/Wg6s5J+pEseJ8UnMHWDradt9TEeunLjGwU3HaNm7CS/1buqosB9x42o4S6esprBfQbqN7uDQbmx5KasnALh+3+1Q4LmHHySE6Av0BQgKcs634uOqZa8mtOzVJFf21aRLfS4fuwpAQEV/vAtnvjZ7ejQaTaZKBGh1WvyfKJHjceQE78Je9/orC2F/vzy89Pb7NTpNjk6ZFSpawH4WqHPXUaSET45tOzcJIXiqcTXOBV8EoH4m14t4envw8eL37bfrt3+WN8Z0TOMZjlGybHE++qZfru/3fs444n8daCmlfDv5dg+grpRyYGrPUUf8edvRbSeJCovm+fbPunwv2fRIKVk6eRVblu6mbqta9J3ew35N4vf/bWTlF+uoVKc8Q7/rn6NdmLYu382qmb9TpV5F+s18M8+dCWWUMcnIjhV78S7kxfPtnlHdzNKR2hG/mupRFEXJp1Ib+J2Rxx8MVBRCPCGEcAe6AGudEIeiKIpLyvU5fimlWQjxPrARWzrnd1LKU7kdh6IoiqtySh6/lHI9sN4Z+1YURXF1qmSDoiiKi1EDv6IoiotRA7+iKIqLUQO/oiiKi3ksGrEIISKAa1l8uh8QmYPhPA7Ua3YN6jW7huy85jJSymIP3/lYDPzZIYQ4mNIChvxMvWbXoF6za3DEa1ZTPYqiKC5GDfyKoiguxhUG/m+cHYATqNfsGtRrdg05/prz/Ry/oiiK8iBXOOJXFEVR7qMGfkVRFBeTbwd+IcRLQohzQoiLQogRzo4nNwghSgshtgkhzgghTgkhPnR2TLlBCKEVQhwRQqxzdiy5RQjhI4RYKYQ4m/z3ft7ZMTmaEGJw8uf6pBBimRAi33X9EUJ8J4QIF0KcvO++okKIv4QQF5L/Tb//aDry5cB/X0P3VkA1oKsQoppzo8oVZmCIlLIqUA8Y4CKv+0PgjLODyGVfAhuklFWAmuTz1y+ECAA+AJ6RUlbHVtK9i3OjcojFwEsP3TcC2CKlrAhsSb6dLfly4AfqAhellJellEZgOdDeyTE5nJQyTEp5OPnnWGyDQYBzo3IsIUQg8DKwwNmx5BYhRCGgMbAQQEpplFLedmpQuUMHeAohdIAX8K+T48lxUsqdQNRDd7cHvk/++XvglezuJ78O/Ck1dM/XA+DDhBBlgVrAfieH4mizgY8Bq5PjyE3lgAhgUfIU1wIhhLezg3IkKeU/wAwgBAgDYqSUm5wbVa4pIaUMA9vBHVA8uxvMrwN/Sh2YXSZvVQhRAFgFDJJS3nF2PI4ihGgDhEspDzk7llymA2oDX0kpawHx5MDpf16WPK/dHngCKAV4CyG6Ozeqx1d+HfhDgdL33Q4kH54WpkQI4YZt0F8ipVzt7HgcrAHQTghxFdt0XlMhxE/ODSlXhAKhUsq7Z3MrsX0R5GfNgStSyggppQlYDdR3cky55aYQwh8g+d/w7G4wvw78LtnQXQghsM37npFSfuHseBxNSjlSShkopSyL7W+8VUqZ748CpZQ3gOtCiMrJdzUDTjsxpNwQAtQTQnglf86bkc8vaN9nLfBm8s9vAmuyu0Gn9Nx1NBdu6N4A6AGcEEIcTb5vVHKPYyV/GQgsST6wuQz0dnI8DiWl3C+EWAkcxpa9doR8WL5BCLEM+A/gJ4QIBcYDU4FfhBB9sH0Bvp7t/aiSDYqiKK4lv071KIqiKKlQA7+iKIqLUQO/oiiKi1EDv6IoiotRA7+iKIqLUQO/omSTEGKDEOK2K1UHVR5vauBXlOybjm39hKI8FtTArygZJIR4VghxXAjhIYTwTq4NX11KuQWIdXZ8ipJR+XLlrqI4gpQyWAixFpgEeAI/SSlPpvM0Rclz1MCvKJkzEVstqCRsjUEU5bGjpnoUJXOKAgWAgkC+a/2nuAY18CtK5nwDjAWWANOcHIuiZIma6lGUDBJC9ATMUsqlyX2d/xZCNAU+AaoABZIrKvaRUm50ZqyKkhZVnVNRFMXFqKkeRVEUF6MGfkVRFBejBn5FURQXowZ+RVEUF6MGfkVRFBejBn5FURQXowZ+RVEUF/N/ZAiDVp+GYPIAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "plt.scatter(x[:,0],x[:,1],c=y, s=8);\n",
    "plt.xlabel(\"x1\"); plt.ylabel(\"x2\");"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Sequential(\n",
       "  (0): Linear(in_features=2, out_features=2, bias=True)\n",
       "  (1): Sigmoid()\n",
       "  (2): Linear(in_features=2, out_features=1, bias=True)\n",
       ")"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model = torch.nn.Sequential(\n",
    "    torch.nn.Linear(2, 2),\n",
    "    torch.nn.Sigmoid(),\n",
    "    torch.nn.Linear(2, 1)\n",
    ")\n",
    "model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [],
   "source": [
    "x, y = gen_nn_fake_data(10000)\n",
    "x = torch.tensor(x).float()\n",
    "y = torch.tensor(y).float()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [],
   "source": [
    "learning_rate = 0.01\n",
    "optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.6346913576126099\n",
      "0.1373707503080368\n",
      "0.113670714199543\n",
      "0.11172331869602203\n",
      "0.11058656871318817\n",
      "0.10969038307666779\n",
      "0.10906001180410385\n",
      "0.10862652957439423\n",
      "0.10833132266998291\n",
      "0.10811644047498703\n"
     ]
    }
   ],
   "source": [
    "for t in range(10000):\n",
    "    # Forward pass: compute predicted y using operations on Variables\n",
    "    y_hat = model(x)\n",
    "    loss = F.binary_cross_entropy(torch.sigmoid(y_hat), y.unsqueeze(1))\n",
    "    if t % 1000 == 0: print(loss.item())\n",
    "       \n",
    "    # Before the backward pass, use the optimizer object to zero all of the\n",
    "    # gradients for the variables\n",
    "    optimizer.zero_grad()\n",
    "    loss.backward()\n",
    "    \n",
    "    # Calling the step function on an Optimizer makes an update to its\n",
    "    # parameters\n",
    "    optimizer.step()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Parameter containing:\n",
      "tensor([[1.0664, 2.2842],\n",
      "        [0.3983, 0.4265]], requires_grad=True), Parameter containing:\n",
      "tensor([ 2.7789, -5.6357], requires_grad=True), Parameter containing:\n",
      "tensor([[-6.0012, 19.7024]], requires_grad=True), Parameter containing:\n",
      "tensor([-3.9404], requires_grad=True)]\n"
     ]
    }
   ],
   "source": [
    "print([p for p in model.parameters()])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Lab\n",
    "* Get an AWS account and ask for GPU access.\n",
    "* Add a dataset and data loader to the neural network training.\n",
    "* Modify the neural network example to solve a regression problem.\n",
    "* Find a tabular dataset to fit a linear or non-linear model. Note that you need to scale your data. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# References\n",
    "* https://pytorch.org/docs/stable/index.html\n",
    "* http://pytorch.org/tutorials/beginner/pytorch_with_examples.html\n",
    "* https://hsaghir.github.io/data_science/pytorch_starter/"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.11"
  },
  "nav_menu": {},
  "toc": {
   "base_numbering": 1,
   "nav_menu": {
    "height": "116px",
    "width": "251px"
   },
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": true,
   "toc_position": {},
   "toc_section_display": "block",
   "toc_window_display": false
  },
  "widgets": {
   "state": {},
   "version": "1.1.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}