Lab03.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8PsD0Lw0g_mL"
   },
   "source": [
    "# Lab 3: Minimizing Cost"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "UPg1vs52g_mQ"
   },
   "source": [
    "Author: Seungjae Lee (이승재)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "A1VVTA6Xg_mR"
   },
   "source": [
    "<div class=\"alert alert-warning\">\n",
    "    We use elemental PyTorch to implement linear regression here. However, in most actual applications, abstractions such as <code>nn.Module</code> or <code>nn.Linear</code> are used.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "HSf_efZxg_mS"
   },
   "source": [
    "## Theoretical Overview"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9hOXEnwKg_mS"
   },
   "source": [
    "$$ H(x) = Wx $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "juqvz7hSg_mT"
   },
   "source": [
    "$$ cost(W) = \\frac{1}{m} \\sum^m_{i=1} \\left( Wx^{(i)} - y^{(i)} \\right)^2 $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "VFNYs7EBg_mU"
   },
   "source": [
    " - $H(x)$: 주어진 $x$ 값에 대해 예측을 어떻게 할 것인가\n",
    " - $cost(W)$: $H(x)$ 가 $y$ 를 얼마나 잘 예측했는가"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "UPBuQBPJg_mV"
   },
   "source": [
    "Note that it is simplified, without the bias $b$ added to $H(x)$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "l9OqqeHgg_mX"
   },
   "source": [
    "## Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "_EzXq4xBg_mY"
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "import torch.optim as optim"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "KCD_9kMcg_ma",
    "outputId": "0c5c2cf3-04bf-4039-d356-e240a5a1aa97"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<torch._C.Generator at 0x7f6e0195fdd0>"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# For reproducibility\n",
    "torch.manual_seed(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "DX-yid2yg_mb"
   },
   "source": [
    "## Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "HMDL0D3Kg_mc"
   },
   "source": [
    "We will use fake data for this example."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "SWh1HBeqg_mc"
   },
   "source": [
    "기본적으로 PyTorch는 NCHW 형태이다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "GycghV9Og_mc"
   },
   "outputs": [],
   "source": [
    "x_train = torch.FloatTensor([[1], [2], [3]] )\n",
    "y_train = torch.FloatTensor([[1], [2], [3]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "sdyERsQEg_md",
    "outputId": "4a3e17ad-eb2b-42df-b819-19b8b0407168"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7f6df60a1f60>]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAD8CAYAAABw1c+bAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xl4VPXZ//H3DYR9h7ATwr6DYlgU61pZ3BC1rVap2lra2v66I+CKohW1j61ttUjVutT1IQFRUcSqBTcUELKxhUUg7FsIkECW+/fHjH0iJjDAJDOT+byuK1dmvuc7M3eGwycn55y5j7k7IiISP2pEugAREalaCn4RkTij4BcRiTMKfhGROKPgFxGJMwp+EZE4o+AXEYkzCn4RkTij4BcRiTO1Il1AeVq2bOnJycmRLkNEJGYsWbJkl7snhjI3KoM/OTmZxYsXR7oMEZGYYWZfhjpXu3pEROKMgl9EJM4o+EVE4oyCX0Qkzij4RUTijIJfRCTOHDf4zayumX1mZsvNLMvM7ilnTh0ze8XMcsxskZkll1k2OTi+ysxGhrd8ERE5UaGcx38YuMDdD5hZAvChmb3l7p+WmfMjYK+7dzOza4AHge+ZWR/gGqAv0A5418x6uHtJmH8OEZGYNPuLXB6et4ot+wpo17QeE0b25IrT21fqax53i98DDgTvJgS/jr5Q7xjg2eDtmcCFZmbB8Zfd/bC7rwdygCFhqVxEJMbN/iKXyWkZ5O4rwIHcfQVMTstg9he5lfq6Ie3jN7OaZrYM2AHMd/dFR01pD2wCcPdiIA9oUXY8aHNwTEQk7j309koKir6+A6SgqISH562q1NcNKfjdvcTdTwM6AEPMrF+4CzGz8Wa22MwW79y5M9xPLyISVTJz89iSV1jusi37Cir1tU/orB533we8D4w6alEu0BHAzGoBTYDdZceDOgTHynvuGe6e4u4piYkh9RkSEYk5hUUlPPj2SsY89hE1rPw57ZrWq9QaQjmrJ9HMmgZv1wMuAlYeNW0OcEPw9tXAe+7uwfFrgmf9dAa6A5+Fq3gRkViyaN1uRj+6kL9/sJYrT2/P1DH9qJdQ82tz6iXUZMLInpVaRyhn9bQFnjWzmgR+Ubzq7m+Y2b3AYnefAzwFPG9mOcAeAmfy4O5ZZvYqkA0UAz/XGT0iEm/yC4uY9tZKXli0kQ7N6vH8j4bwre6BPRsN6tSq8rN6LLBhHl1SUlJcbZlFpDr494rt3DE7k237C7nprM78fmQP6tcOf0d8M1vi7imhzI3KfvwiIrFu94HD3PN6NnOWb6FH64Y8dt1ZDEpqFumyAAW/iEhYuTuvLdvCPa9nceBwMb/+dnduOa8btWtFT4ccBb+ISJjk7ivg9lkZfLBqJ6d1bMpDVw+gR+tGkS7rGxT8IiKnqLTU+deiL3nwrZWUOtx5aR9uPCuZmhWdrxlhCn4RkVOQs+MAk9PS+XzDXs7u1pIHruxPx+b1I13WMSn4RUROQlFJKTMWrOPRd9dQN6EGD189gKvP6ECgTVl0U/CLiJygjM153Jqazoqt+7m4fxumXN6XVo3qRrqskCn4RURCVHCkhD+/u5p/LFxHy4Z1eGLcGYzs2ybSZZ0wBb+ISAg+WbubyWnpbNh9iGsGd2Tyxb1pUi8h0mWdFAW/iMgx7C8s4oG5K3nps40kNa/PizcP5axuLSNd1ilR8IuIVGB+9nbumJ3BzvzDjD+nC7/5dg/q1a55/AdGOQW/iMhRduYfZsrrWbyZvpVebRoxY1wKAzs2jXRZYaPgFxEJcnfSluYy9c1sDh0u4XcX9eAn53aNqnYL4aDgFxEBNu05xG2zMli4ZhdndGrGg1f1p1ur6Gu3EA4KfhGJayWlznOfbPjvdW7vubwv44Z1okaUtlsIh+MGv5l1BJ4DWgMOzHD3R4+aMwG4rsxz9gYS3X2PmW0A8oESoDjUftEiIpVtzfZ8bk1N54uN+zi3RyL3j+1Hh2bR3W4hHELZ4i8GfufuS82sEbDEzOa7e/ZXE9z9YeBhADO7DPiNu+8p8xznu/uucBYuInKyjhSX8vcP1vLY+znUr1OTR747kLGnt4+JdgvhcNzgd/etwNbg7XwzWwG0J3A5xfJcC7wUtgpFRMJo2aZ9TJyZzqrt+Vw6oC1TLu9Ly4Z1Il1WlTqhffxmlgycDiyqYHl9YBTwizLDDrxjZg484e4zTqpSEZFTcOhIMY+8s5qnP1pPYqM6/OMHKVzUp3Wky4qIkIPfzBoCqcCv3X1/BdMuAz46ajfP2e6ea2atgPlmttLdF5Tz/OOB8QBJSUkh/wAiIsfzUc4uJqWls2lPAd8fmsSk0b1oXDc22y2EQ0jBb2YJBEL/BXdPO8bUazhqN4+75wa/7zCzWcAQ4BvBH/xLYAYELrYeUvUiIseQd6iI++dm8+rizSS3qM/L44cxrEuLSJcVcaGc1WPAU8AKd3/kGPOaAOcC15cZawDUCB4baACMAO495apFRI7j7cyt3PlaFnsOHuEn5wbaLdRNiP12C+EQyhb/cGAckGFmy4JjtwFJAO4+PTg2FnjH3Q+WeWxrYFbwSHkt4EV3fzschYuIlGdHfiF3v5bFW5nb6NO2Mf+8cTD92jeJdFlRJZSzej4EjnuOk7s/Azxz1Ng6YOBJ1iYiEjJ353+XbOa+N7IpLC5lwsiejD+nCwk1q1e7hXDQJ3dFJOZt3B1ot/Bhzi4GJzdj2lUD6JrYMNJlRS0Fv4jErJJS558fred/3llNDYOpV/TjuiFJ1brdQjgo+EUkJq3als/E1HSWbdrHBb1acd8V/WjXtF6ky4oJCn4RiSmHi0t4/P21PP5BDo3qJvDoNadx+cB2cdNuIRwU/CISM5Zu3MvEmems2XGAMae1465L+9AiztothIOCX0Si3sHDxfzxnVU88/EG2jSuy9M3pnBBr/hstxAOCn4RiWoLVu9kcloGufsKGDesE7eO6kmjOG63EA4KfhGJSvsOHWHqGytIXbqZLi0b8OpPzmRI5+aRLqtaUPCLSFRxd+ZmbOPuOZnsPVTELed15ZcXdle7hTBS8ItI1Ni+v5A7Z2fyTvZ2+rVvzLM/HELfdmq3EG4KfhGJOHfnlc83cf/cFRwpLmXS6F7cfHZnaqndQqVQ8ItIRG3YdZDJaRl8sm43Qzs3Z9pVA+jcskGky6rWFPwiEhHFJaU8/dF6Hpm/moQaNbh/bD+uHax2C1VBwS8iVW7F1v1MTE0nfXMe3+7diqlX9KNtE7VbqCoKfhGpMoeLS/jbezn8/YO1NKmXwF+vPZ1LB7RVu4UqpuAXkSqxeMMeJqams3bnQa4c1J47L+lDswa1I11WXDruIXMz62hm75tZtpllmdmvyplznpnlmdmy4NddZZaNMrNVZpZjZpPC/QOISHQ7cLiYu1/L5DtPfEJhUSnP3DSYR757mkI/gkLZ4i8GfufuS82sEbDEzOa7e/ZR8xa6+6VlB8ysJvAYcBGwGfjczOaU81gRqYY+WLWD22dlsiWvgBvOTOb3I3vSsI52NERaKJde3ApsDd7ON7MVQHsglPAeAuQEL8GImb0MjAnxsSISo/YePMLUN7JJ+yKXrokNmPnTMzmjk9otRIsT+tVrZsnA6cCichafaWbLgS3A7909i8AviE1l5mwGhlbw3OOB8QBJSUknUpaIRAl35430rUyZk0VeQRG/vKAbP7+gG3Vqqd1CNAk5+M2sIZAK/Nrd9x+1eCnQyd0PmNnFwGyg+4kU4u4zgBkAKSkpfiKPFZHI25pXwJ2zM3l3xQ4GdGjCv24eSu+2jSNdlpQjpOA3swQCof+Cu6cdvbzsLwJ3n2tmj5tZSyAX6FhmaofgmIhUE6Wlzkufb2Ta3JUUlZZy+8W9uWl4stotRLHjBr8FTrB9Cljh7o9UMKcNsN3d3cyGEDhbaDewD+huZp0JBP41wPfDVbyIRNa6nQeYnJbBovV7OLNLC6Zd1Z9OLdRuIdqFssU/HBgHZJjZsuDYbUASgLtPB64GfmZmxUABcI27O1BsZr8A5gE1gaeD+/5FJIYVl5Tyj4Xr+dO7q6lTqwYPXtWf76Z01AexYoQF8jm6pKSk+OLFiyNdhoiUIzM3j4mp6WRt2c+IPq2ZekU/WjeuG+my4p6ZLXH3lFDm6oRaEQlJYVEJj/57DTMWrKNZ/do8ft0gRvdro638GKTgF5Hj+mz9HialprNu10GuPqMDd1zSm6b19cnbWKXgF5EK5RcW8eDbK/nXpxvp0Kwez/1wCOf0SIx0WXKKFPwiUq73Vm7n9lmZbNtfyA+Hd+Z3I3rQQO0WqgX9K4rI1+w+cJh7Xs9mzvIt9GjdkMeuO4tBSc0iXZaEkYJfRIBAu4XXlm3hntezOHC4mF9/uzu3nNeN2rX0QazqRsEvImzZV8DtszJ4f9VOTuvYlIeuHkCP1o0iXZZUEgW/SBwrLXVeWPQl095aSanDnZf24cazkqmp695Wawp+kTi1ducBJqWm8/mGvZzdrSUPXNmfjs3rR7osqQIKfpE4U1RSyowF63j032uoW6sGD189gKvP6KAPYsURBb9IHMnYnMetqems2Lqfi/u3YcrlfWnVSO0W4o2CXyQOFBaV8Kd3V/PkwvU0b1Cb6defwah+bSJdlkSIgl+kmvtk7W4mp6WzYfchvpfSkdsu7k2T+gmRLksiSMEvUk3tLyzigbkreemzjSQ1r88LNw9leLeWkS5LooCCX6Qamp+9nTtmZ7Az/zA//lZnfntRT+rV1nVvJSCUK3B1BJ4DWgMOzHD3R4+acx0wETAgH/iZuy8PLtsQHCsBikPtFy0iJ25n/mGmvJ7Fm+lb6dWmETPGpTCwY9NIlyVRJpQt/mLgd+6+1MwaAUvMbL67Z5eZsx441933mtloAhdNH1pm+fnuvit8ZYtIWe5O2tJcpr6ZzaHDJfz2oh789Nyuarcg5Tpu8Lv7VmBr8Ha+ma0A2gPZZeZ8XOYhnxK4qLqIVIHNew9x26xMFqzeyaCkpjx41QC6q92CHMMJ7eM3s2TgdGDRMab9CHirzH0H3jEzB55w9xknWKOIlKOk1Hn+kw08NG8VAFMu68O4M9VuQY4v5OA3s4ZAKvBrd99fwZzzCQT/2WWGz3b3XDNrBcw3s5XuvqCcx44HxgMkJSWdwI8gEn/WbM9nYmo6Szfu45weifxhbD86NFO7BQlNSMFvZgkEQv8Fd0+rYM4A4ElgtLvv/mrc3XOD33eY2SxgCPCN4A/+JTADAhdbP8GfQyQuHCkuZfp/1vK393KoX6cmj3x3IGNPb692C3JCQjmrx4CngBXu/kgFc5KANGCcu68uM94AqBE8NtAAGAHcG5bKReLM8k37mJiazspt+Vw6oC1TLu9Ly4Z1Il2WxKBQtviHA+OADDNbFhy7DUgCcPfpwF1AC+Dx4JbHV6dttgZmBcdqAS+6+9th/QlEqrmCIyU8Mn8VT324nsRGdfjHD1K4qE/rSJclMSyUs3o+JHB+/rHm3AzcXM74OmDgSVcnEuc+ztnFpLQMNu45xPeHJjFpdC8a11W7BTk1+uSuSBTKKyjiD2+u4JXFm0huUZ+Xxw9jWJcWkS5LqgkFv0iUeTtzG3e9lsnug0f4ybld+M23e1A3Qe0WJHwU/CJRYkd+IVPmZDE3Yxu92zbmqRsG079Dk0iXJdWQgl8kwtydmUs2c9+bKygoKmHCyJ6MP6cLCTXVbkEqh4JfJII27TnEbbMyWLhmF4OTmzHtqgF0TWwY6bKkmlPwi0RASanzzMcb+OO8VdQwmDqmL9cN7UQNtVuQKqDgF6liq7fnc+vMdJZt2sf5PRO5b2x/2jetF+myJI4o+EWqyJHiUh7/IIfH3s+hYZ1aPHrNaVw+sJ3aLUiVU/CLVIGlG/cyKTWd1dsPMOa0dtx1aR9aqN2CRIiCX6QSHTxczB/fWcUzH2+gTeO6PH1jChf0UrsFiSwFv0glWbhmJ5PTMti8t4DrhyUxcVQvGqndgkQBBb9ImO07dIT73lzBzCWb6dKyAa+MH8ZQtVuQKKLgFwkTd+etzG3c9VoWew8d4ZbzuvLLC7ur3YJEHQW/SBhs31/InbMzeSd7O/3aN+bZHw6mbzu1W5DopOAXOQXuziufb+L+uSs4UlzKpNG9uPnsztRSuwWJYsddO82so5m9b2bZZpZlZr8qZ46Z2V/MLMfM0s1sUJllN5jZmuDXDeH+AUQi5cvdB7nuyUVMSsugT9vGvP3rc/jpuV0V+hL1QtniLwZ+5+5LzawRsMTM5rt7dpk5o4Huwa+hwN+BoWbWHLgbSAE8+Ng57r43rD+FSCWb/UUuD89bxZZ9BbRtUpfByc2Zl72NhBo1uH9sP64dnKR2CxIzQrkC11Zga/B2vpmtANoDZYN/DPCcuzvwqZk1NbO2wHnAfHffA2Bm84FRwEth/SlEKtHsL3KZnJZBQVEJAFvyCnlt+Rb6tmvMkzek0LaJ2i1IbDmhv0nNLBk4HVh01KL2wKYy9zcHxyoaF4kZD89b9d/QL2vvwSMKfYlJIQe/mTUEUoFfu/v+cBdiZuPNbLGZLd65c2e4n17kpOXuKyh3fGteYRVXIhIeIQW/mSUQCP0X3D2tnCm5QMcy9zsExyoa/wZ3n+HuKe6ekpiYGEpZIpXq4OFipszJqnB5O3XUlBgVylk9BjwFrHD3RyqYNgf4QfDsnmFAXvDYwDxghJk1M7NmwIjgmEhU+2DVDkb8aQHPfrKBb3VrSd1aX/+vUi+hJhNG9oxMcSKnKJSzeoYD44AMM1sWHLsNSAJw9+nAXOBiIAc4BNwUXLbHzKYCnwcfd+9XB3pFotHeg0eY+kY2aV/k0jWxATN/eiZndGr+tbN62jWtx4SRPbnidB2ukthkgRNxoktKSoovXrw40mVIHHF33kjfypQ5WeQVFPGz87ry8/O7qd2CxAwzW+LuKaHM1Sd3Je5tyyvkjtmZvLtiOwM6NOFfNw+ld9vGkS5LpNIo+CVulZY6L32+kWlzV1JUWsrtF/fmpuHJ+uStVHsKfolL63cdZFJqOovW7+HMLi2YdlV/OrVoEOmyRKqEgl/iSnFJKU9+uJ4/zV9N7Vo1mHZlf743uKOueytxRcEvcSNrSx4TU9PJzN3PiD6tmXpFP1o3rhvpskSqnIJfqr3CohL+8u81PLFgHc3q1+bx6wYxul8bbeVL3FLwS7X22fo9TEpNZ92ug1x9RgfuuKQ3TevXjnRZIhGl4JdqKb+wiIfeXsXzn35Jh2b1eO6HQzinh1qBiICCX6qh91Zu5/ZZmWzbX8gPh3fmdyN60KCOVnWRr+h/g1Qbuw8c5t43snlt2Ra6t2pI6s/OYlBSs0iXJRJ1FPwS89ydOcu3cM/r2eQXFvGrC7tzy/ldqVNL7RZEyqPgl5i2ZV8Bd8zO5L2VOxjYsSkPXTWAnm0aRboskaim4JeYVFrqvPDZRh58ayUlpc6dl/bhxrOSqanr3oocl4JfYs7anQeYnJrBZxv2cHa3ljxwZX86Nq8f6bJEYoaCX2JGUUkpMxas49F/r6FurRo8dPUAvnNGB30QS+QEKfglJmTm5nHrzHSyt+5ndL823DOmL60aqd2CyMk4bvCb2dPApcAOd+9XzvIJwHVlnq83kBi8+tYGIB8oAYpDvUiAyFcKi0r407ureXLhepo3qM306wcxql/bSJclEtNC2eJ/Bvgb8Fx5C939YeBhADO7DPjNUZdXPN/dd51inRKHPl23m8lpGazfdZDvpXTktot706R+QqTLEol5xw1+d19gZskhPt+1wEunUpDI/sIipr21khcXbSSpeX1euHkow7u1jHRZItVG2Pbxm1l9YBTwizLDDrxjZg484e4zjvH48cB4gKSkpHCVJTFmfvZ27pidwc78w9x8dmd+O6IH9WvrUJRIOIXzf9RlwEdH7eY5291zzawVMN/MVrr7gvIeHPylMAMCF1sPY10SA3YdOMyUOVm8kb6VXm0aMWNcCgM7No10WSLVUjiD/xqO2s3j7rnB7zvMbBYwBCg3+CU+uTuzvsjl3jeyOXS4hN9e1IOfntuV2rV03VuRyhKW4DezJsC5wPVlxhoANdw9P3h7BHBvOF5PqofNew9x26xMFqzeyaCkpjx41QC6t1a7BZHKFsrpnC8B5wEtzWwzcDeQAODu04PTxgLvuPvBMg9tDcwKfrimFvCiu78dvtIlVpWWOs99soGH5q0CYMplfRh3ptotiFSVUM7quTaEOc8QOO2z7Ng6YODJFibVU86OfCamZrDky72c0yORP4ztR4dmarcgUpV0uoRUiSPFpTzxn7X89b0c6tepySPfHcjY09ur3YJIBCj4pdIt37SPianprNyWz6UD2nL3ZX1JbFQn0mWJxC0Fv1SagiMlPDJ/FU99uJ7ERnX4xw9SuKhP60iXJRL3FPxSKT7O2cWktAw27jnEtUOSmHxxLxrXVbsFkWig4Jewyiso4oG5K3j5800kt6jPSz8expldW0S6LBEpQ8EvYTMvaxt3zs5k14HD/OTcLvzm2z2om6Dr3opEGwW/nLId+YVMmZPF3Ixt9G7bmKduGEz/Dk0iXZaIVEDBLyfN3Zm5ZDP3vbmCgqISJozsyfhzupBQU+0WRKKZgl9OyqY9h7htVgYL1+wipVMzpl01gG6tGka6LBEJgYJfTkhJqfPsxxt4eN4qahhMHdOX64Z2oobaLYjEDAW/hGz19nxunZnOsk37OL9nIveN7U/7pvUiXZaInCAFvxzXkeJSHv8gh8fez6FhnVo8es1pXD6wndotiMQoBb8c0xcb9zIxNZ3V2w8w5rR23HVpH1o0VLsFkVim4JdyHTpSzB/nreafH6+nTeO6PH1jChf0UrsFkepAwS/f8OGaXUxKS2fz3gKuH5bExFG9aKR2CyLVxnFPuDazp81sh5llVrD8PDPLM7Nlwa+7yiwbZWarzCzHzCaFs3AJv7xDRUz43+Vc/9QiateswSvjh3HfFf0V+iLVTChb/M8AfwOeO8ache5+adkBM6sJPAZcBGwGPjezOe6efZK1SiV6K2Mrd76Wxd5DR7jlvK788sLuarcgUk2FcgWuBWaWfBLPPQTICV6JCzN7GRgDKPijyI79hdz5WibzsrbTt11jnrlpMP3aq92CSHUWrn38Z5rZcmAL8Ht3zwLaA5vKzNkMDA3T68kpcndeXbyJ+95cwZHiUiaO6sWPv9WZWmq3IFLthSP4lwKd3P2AmV0MzAa6n+iTmNl4YDxAUlJSGMqSiny5+yCT0zL4eO1uhnRuzrQr+9MlUe0WROLFKQe/u+8vc3uumT1uZi2BXKBjmakdgmMVPc8MYAZASkqKn2pd8k0lpc7TH67nf+avolaNGtw/th/XDk5SuwWROHPKwW9mbYDt7u5mNoTAmUK7gX1AdzPrTCDwrwG+f6qvJydn5bb9TJyZzvLNeVzYqxX3je1H2yZqtyASj44b/Gb2EnAe0NLMNgN3AwkA7j4duBr4mZkVAwXANe7uQLGZ/QKYB9QEng7u+5cqdLi4hMfey+HxD9bSpF4Cf7n2dC4b0FbtFkTimAUyOrqkpKT44sWLI11GzFvy5R4mpmaQs+MAY09vz52X9qF5g9qRLktEKoGZLXH3lFDm6pO71dDBw8U8PG8Vz36ygbaN6/LPmwZzfs9WkS5LRKKEgr+a+c/qndyWlsGWvAJ+MKwTE0b1omEd/TOLyP9RIlQTew8eYeqb2aQtzaVrYgP+9ydnkpLcPNJliUgUUvDHOHfnjfSt3PN6FvsOFfH/LujGz8/vpnYLIlIhBX8M25ZXyB2zM3l3xXYGdGjC8z8aSu+2jSNdlohEOQV/DCotdV7+fBMPzF1BUWkpt1/cm5uGJ6vdgoiERMEfY9bvOsik1HQWrd/DmV1aMO2q/nRq0SDSZYlIDFHwx4jiklKe/HA9f5q/mtq1ajDtyv58b3BHfRBLRE6Ygj8GZG3JY2JqOpm5+xnRpzVTr+hH68Z1I12WiMQoBX8UKywq4a/vrWH6f9bRrH4Cj183iNH92mgrX0ROiYI/Sn2+YQ8TU9NZt/MgV5/RgTsu6U3T+mq3ICKnTsEfZfILi3jo7VU8/+mXdGhWj+d+OIRzeiRGuiwRqUYU/FHk/ZU7uH1WBlv3F3LT8GR+P6InDdRuQUTCTKkSBfYcPMK9r2cxe9kWurdqyMyfnsUZnZpFuiwRqaYU/BHk7sxZvoV7Xs8mv7CIX13YnVvO70qdWmq3ICKVR8EfIVv2FXDH7EzeW7mDgR2b8tBVA+jZplGkyxKROBDKFbieBi4Fdrh7v3KWXwdMBAzIB37m7suDyzYEx0qA4lAvElCdlZY6L3y2kQffWklJqXPHJb25aXhnauq6tyJSRULZ4n8G+BvwXAXL1wPnuvteMxtN4ILpQ8ssP9/dd51SldXEup0HmJSawWcb9jC8WwseGDuApBb1I12WiMSZ4wa/uy8ws+RjLP+4zN1PgQ6nXlb1UlRSyj8WruPP766hbq0aPHT1AL5zRgd9EEtEIiLc+/h/BLxV5r4D75iZA0+4+4yKHmhm44HxAElJSWEuK3Iyc/O4dWY62Vv3M7pfG+4Z05dWjdRuQUQiJ2zBb2bnEwj+s8sMn+3uuWbWCphvZivdfUF5jw/+UpgBgYuth6uuSCksKuHP767hHwvX0bxBbaZfP4hR/dpGuiwRkfAEv5kNAJ4ERrv77q/G3T03+H2Hmc0ChgDlBn918um63UxOy2D9roN8L6Ujt13cmyb1EyJdlogIEIbgN7MkIA0Y5+6ry4w3AGq4e37w9gjg3lN9vWi2v7CIaW+t5MVFG0lqXp8Xbh7K8G4tI12WiMjXhHI650vAeUBLM9sM3A0kALj7dOAuoAXwePBg5VenbbYGZgXHagEvuvvblfAzRIV3s7dzx+xMduQXcvPZnfntiB7Ur62PSYhI9AnlrJ5rj7P8ZuDmcsbXAQNPvrTYsOvAYabMyeKN9K30bN2I6ePO4LSOTSNdlohIhbRJepLcnVlf5HLvG9kcPFzMby/qwU/P7UrtWrrurYhENwX/Sdi89xC3z8rkP6t3MiipKQ9eNYDurdVuQURig4L/BJSWOs9/+iUPvr0SgLsv68MPzkxWuwURiSkK/hCUjB/GAAAIhUlEQVTl7MhnYmoGS77cy7e6t+QPY/vTsbnaLYhI7FHwH0dRSSnTP1jLX9/LoV7tmvzPdwZy5aD2arcgIjFLwX8M6Zv3cevMdFZuy+eSAW2ZcllfEhvViXRZIiKnRMFfjoIjJfzp3dU8uXAdiY3qMGPcGYzo2ybSZYmIhIWC/ygfr93F5LQMvtx9iGuHdGTS6N40qad2CyJSfSj4g/IKinhg7gpe/nwTyS3q8+KPh3JWV7VbEJHqR8EPzMvaxp2zM9l14DA/ObcLv/l2D+om6Lq3IlI9xXXw78wPtFt4M2Mrvds25qkbBtO/Q5NIlyUiUqniMvjdndSluUx9I5uCohImjOzJ+HO6kFBT7RZEpPqLu+DftOcQt83KYOGaXaR0asa0qwbQrVXDSJclIlJl4ib4S0qdZz/ewMPzVlHDYOqYvlw3tBM11G5BROJMXAT/6u353DoznWWb9nFez0TuH9uf9k3rRbosEZGICGmntpk9bWY7zCyzguVmZn8xsxwzSzezQWWW3WBma4JfN4Sr8FAcKS7lz++u5pK/LOTL3Qf58/dO4583Dlboi0hcC3WL/xngb8BzFSwfDXQPfg0F/g4MNbPmBK7YlQI4sMTM5rj73lMpuiKzv8jl4Xmr2LKvgJYN61CrprE1r5DLB7bj7sv60KKh2i2IiIQU/O6+wMySjzFlDPCcuzvwqZk1NbO2BC7ZON/d9wCY2XxgFPDSqRRdntlf5DI5LYOCohIAdh44DMCPv9WZ2y/pE+6XExGJWeE6f7E9sKnM/c3BsYrGw+7heav+G/plzc3YVhkvJyISs6LmxHUzG29mi81s8c6dO0/48Vv2FZzQuIhIvApX8OcCHcvc7xAcq2j8G9x9hrunuHtKYmLiCRfQroIDthWNi4jEq3AF/xzgB8Gze4YBee6+FZgHjDCzZmbWDBgRHAu7CSN7Uu+o/jr1EmoyYWTPyng5EZGYFdLBXTN7icCB2pZmtpnAmToJAO4+HZgLXAzkAIeAm4LL9pjZVODz4FPd+9WB3nC74vTAoYOvzupp17QeE0b2/O+4iIgEWOBEnOiSkpLiixcvjnQZIiIxw8yWuHtKKHOj5uCuiIhUDQW/iEicUfCLiMQZBb+ISJxR8IuIxBkFv4hInInK0znNbCfw5Sk8RUtgV5jKCSfVFbporAlU14mIxpqg+tbVyd1DansQlcF/qsxscajns1Yl1RW6aKwJVNeJiMaaQHWBdvWIiMQdBb+ISJyprsE/I9IFVEB1hS4aawLVdSKisSZQXdVzH7+IiFSsum7xi4hIBWIq+M3saTPbYWaZFSw3M/uLmeWYWbqZDSqz7AYzWxP8uqGK67ouWE+GmX1sZgPLLNsQHF9mZmFtSRpCXeeZWV7wtZeZ2V1llo0ys1XB93JSFdY0oUw9mWZWYmbNg8sq873qaGbvm1m2mWWZ2a/KmVOl61eINVX5uhViXZFYt0Kpq8rXLzOra2afmdnyYF33lDOnjpm9EnxPFlmZa5yb2eTg+CozGxmWotw9Zr6Ac4BBQGYFyy8G3gIMGAYsCo43B9YFvzcL3m5WhXWd9dXrAaO/qit4fwPQMkLv13nAG+WM1wTWAl2A2sByoE9V1HTU3MuA96rovWoLDArebgSsPvpnrur1K8SaqnzdCrGuSKxbx60rEutXcH1pGLydACwChh015xZgevD2NcArwdt9gu9RHaBz8L2reao1xdQWv7svAI51IZcxwHMe8CnQ1MzaAiOB+e6+x933AvOBUVVVl7t/HHxdgE8JXIKy0oXwflVkCJDj7uvc/QjwMoH3tqpruhZ4KRyvezzuvtXdlwZv5wMrgKOv4lOl61coNUVi3QrxvapIZa5bJ1pXlaxfwfXlQPBuQvDr6IOrY4Bng7dnAheamQXHX3b3w+6+nsDFroacak0xFfwhaA9sKnN/c3CsovFI+BGBrcavOPCOmS0xs/ERqOfM4J+gb5lZ3+BYxN8vM6tPIDxTywxXyXsV/DP7dAJbZmVFbP06Rk1lVfm6dZy6IrZuHe/9qur1y8xqmtkyYAeBjYQK1y13LwbygBZU0vsV0qUXJTzM7HwC/znPLjN8trvnmlkrYL6ZrQxuFVeFpQQ+5n3AzC4GZgPdq+i1j+cy4CP/+qU6K/29MrOGBMLg1+6+P5zPfbJCqSkS69Zx6orYuhXiv2GVrl/uXgKcZmZNgVlm1s/dyz3OVRWq2xZ/LtCxzP0OwbGKxquMmQ0AngTGuPvur8bdPTf4fQcwizD8GRcqd9//1Z+g7j4XSDCzlkTB+0VgP+fX/gyv7PfKzBIIBMYL7p5WzpQqX79CqCki69bx6orUuhXK+xVU5etX8Ln3Ae/zzV2B/31fzKwW0ATYTWW9X+E6gFFVX0AyFR+svISvH3z7LDjeHFhP4MBbs+Dt5lVYVxKBfXNnHTXeAGhU5vbHwKgqrKsN//dZjiHAxuB7V4vAAcrO/N8BuL5VUVNweRMCxwEaVNV7Ffy5nwP+fIw5Vbp+hVhTla9bIdZV5etWKHVFYv0CEoGmwdv1gIXApUfN+TlfP7j7avB2X75+cHcdYTi4G1O7eszsJQJnC7Q0s83A3QQOlODu04G5BM68yAEOATcFl+0xs6nA58Gnute//ideZdd1F4H9dY8HjtdQ7IFmTK0J/NkHgf8QL7r721VY19XAz8ysGCgArvHA2lZsZr8A5hE4C+Npd8+qopoAxgLvuPvBMg+t1PcKGA6MAzKC+2IBbiMQrJFav0KpKRLrVih1Vfm6FWJdUPXrV1vgWTOrSWAvy6vu/oaZ3Qssdvc5wFPA82aWQ+CX0jXBmrPM7FUgGygGfu6B3UanRJ/cFRGJM9VtH7+IiByHgl9EJM4o+EVE4oyCX0Qkzij4RUTijIJfRCTOKPhFROKMgl9EJM78f/bJg4M+dC+7AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Data\n",
    "plt.scatter(x_train, y_train)\n",
    "# Best-fit line\n",
    "xs = np.linspace(1, 3, 1000)\n",
    "plt.plot(xs, xs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "jblHIh7Zg_md"
   },
   "source": [
    "## Cost by W"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Q6GXN7TCg_md"
   },
   "source": [
    "$$ H(x) = Wx $$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "nQ44HY6hg_me"
   },
   "outputs": [],
   "source": [
    "W_l = np.linspace(-5, 7, 1000)\n",
    "cost_l = []\n",
    "for W in W_l:\n",
    "    hypothesis = W * x_train\n",
    "    cost = torch.mean((hypothesis - y_train) ** 2)\n",
    "\n",
    "    cost_l.append(cost.item())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "XY-t2-wtg_me",
    "outputId": "915cc4cd-4d78-4eca-97c2-0b57b8e97e1d"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEMCAYAAADeYiHoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xd81fW9x/HXJ5sMAiEBYkgIK+wdloIDQXFU3OJCrwNtq3W1ttra2mHrbaut1tZtRUXcA8WNW1kB2TNAAgkhhEBISMj+3D9y6I00mhByzvecnM/z8TgPzvnlnJz3wZb3+f7G9yuqijHGGHO4ENcBjDHG+CcrCGOMMU2ygjDGGNMkKwhjjDFNsoIwxhjTJCsIY4wxTbKCMMYY0ySvFYSIPCUiu0VkTaNtL4rICs8tR0RWeLani8jBRj97xFu5jDHGtEyYF3/308BDwDOHNqjqRYfui8h9wP5Gz9+iqiO8mMcYY8wR8FpBqOrnIpLe1M9ERIALgclH8x6JiYmant7kWxhjjPkOy5Yt26OqSc09z5sjiO8zCShU1c2NtvUSkW+AUuBXqvpFc78kPT2drKwsb2U0xph2SURyW/I8VwVxMTC30eMCIE1Vi0VkNPCGiAxW1dLDXygis4BZAGlpaT4Ja4wxwcjnZzGJSBhwLvDioW2qWqWqxZ77y4AtQEZTr1fVx1Q1U1Uzk5KaHSEZY4xpJRenuU4BNqhq3qENIpIkIqGe+72BfsBWB9mMMcZ4ePM017nAQqC/iOSJyNWeH83g27uXAI4HVnlOe30FuF5V93ormzHGmOZ58yymi79j+5VNbHsVeNVbWYwxxhw5u5LaGGNMk6wgjDHGNCkoC6Jg/0Humb+OPQeqXEcxxpgj9tSX2/hg7S6vv09QFkRZZS2Pf7GN15fnu45ijDFH5EBVLX/9YCML1u/2+nsFZUFkdItjVFon5i7djqq6jmOMMS02f9VOKqrruHBMqtffKygLAmDG2DS2FpWTlbvPdRRjjGmxF5fuoE9SDKPSOnn9vYK2IM4clkxsZBhzl2x3HcUYY1pkU2EZy7eXMGNMGg1znnpX0BZEdEQYZ404hndWF7D/YI3rOMYY06y5S7YTERrCeaN7+OT9grYgAC4ek0ZlTT3zVtjBamOMf6usqeP1b/I5ZXA3EmIifPKeQV0QQ1I6Mii5I3OX7LCD1cYYv/b+2l2UVNRw8VjfzWId1AUhIlw8NpV1BaWsyf+vmcWNMcZvPL94O2kJ0Uzo3cVn7xnUBQFw1ogUosJDmLvUDlYbY/zT1qIDLN62l4vGpBIS4v2D04cEfUHEdwjn9KHJzFuxk4rqWtdxjDHmv7y4dAehIcIFPjo4fUjQFwTAjDFpHKiq5e1VBa6jGGPMt1TX1vPKsjxOHtCVrh2jfPreVhDAmPTO9EmK4cWlO1xHMcaYb/lofSHF5dU+PTh9iBUEDQerZ4xJY1nuPjYVlrmOY4wx/zF3yXaOiY/i+AzfL7FsBeFx7qgUwkPFRhHGGL+xY28FX2bv4YLMVEJ9eHD6ECsIjy6xkZwyqDuvLc+jqrbOdRxjjOGlrIYvrL6YmK8pVhCNXDQmlX0VNby/ttB1FGNMkKutq+elrB2ckJFESqcOTjJYQTQysW8iaQnRzFmU6zqKMSbIfbqxiMLSKmaM8f3B6UOsIBoJCREuGZfG4m172WwHq40xDj2/ZDtJcZGcPLCrswxeKwgReUpEdovImkbb7haRfBFZ4bmd3uhnd4hItohsFJFTvZWrOReM7kFEaAhzFtuV1cYYN3bsreCTjbu5eEwq4aHuvsd7852fBqY1sf1vqjrCc3sHQEQGATOAwZ7X/EtEQr2Y7Tt1iY3ktKHdeXV5nl1ZbYxx4vkl2xEaFjZzyWsFoaqfA3tb+PTpwAuqWqWq24BsYKy3sjXnsvE9Kaus5a2VO11FMMYEqaraOl5auoOTB3bjGEcHpw9xMXa5QURWeXZBdfZsSwEaX4CQ59n2X0RklohkiUhWUVGRVwJm9uxMRrdYnltku5mMMb713ppdFJdXc9n4nq6j+LwgHgb6ACOAAuC+I/0FqvqYqmaqamZSkneuLBQRLhvfk9X5+1mVV+KV9zDGmKbMWbSdnl2imdQ30XUU3xaEqhaqap2q1gOP8/+7kfKBxleC9PBsc+ackSlER4TynJ3yaozxkY27yliSs5dLx6X5dFrv7+LTghCR5EYPzwEOneE0D5ghIpEi0gvoByzxZbbDxUWFM31ECvNW7mR/ha1ZbYzxvucW5RIRFsIFo91cOX04b57mOhdYCPQXkTwRuRr4s4isFpFVwEnALQCquhZ4CVgHvAf8WFWdz3dx6biGNatfXZ7nOooxpp0rr6rl9W/yOXNoMp19tOZ0c8K89YtV9eImNj/5Pc+/B7jHW3laY0hKPCNSOzFncS7/c1w6Iu6HfMaY9umNFfkcqKrlUj84OH2IXUndjMvG92RLUTmLtrb0jF1jjDkyqspzi7YzMLkjo9I6uY7zH1YQzThzWDLxHcJ5brEdrDbGeMfy7SWsLyjlsvFpfrWnwgqiGVHhoZw/ugfvr9nF7rJK13GMMe3QnEW5xEaGcfaIJi//csYKogUuG9+T2npl7mJbTMgY07b2llfz9qoCzh2VQkyk1w4Lt4oVRAv0SozhhIwk5izOpbq23nUcY0w78lLWDqrr6rl0nP8cnD7ECqKFrjwund1lVby7psB1FGNMO1FbV8+zC3OZ0LsL/bvHuY7zX6wgWuiEfkn0Soxh9tc5rqMYY9qJj9YXkl9ykCuPS3cdpUlWEC0UEiJcPr4ny7eXsDpvv+s4xph24Omvc0jp1IEpA7u5jtIkK4gjcH5mD6IjQnnaRhHGmKO0vqCURVv3MnNCT0L9YN6lplhBHIGOUeGcN6oHb63aSfGBKtdxjDEB7JmFOUSFh3DRGP+Yd6kpVhBH6Ipje1JdW88LS+2UV2NM6+wrr+b1b/I5Z2QKnaL9Y96lplhBHKG+XeOY2DeRZxfmUlNnp7waY47ci1k7qKyp54pj011H+V5WEK1wxbHp7Cqt5IO1ha6jGGMCTONTWwd07+g6zveygmiFyQO6kprQwU55NcYcsY/W7ya/5KDfjx7ACqJVQkOEmePTWZKzl3U7S13HMcYEkNn/ObW1q+sozbKCaKULM1PpEB5qowhjTItt2FXKwq3FzJzQk7BQ///n1/8T+qn46HDOHpnCGyvy7ZRXY0yLzP7a/09tbcwK4ihcdVw6VbX1PL94u+soxhg/V1IRGKe2NmYFcRT6dYvjxP5JzF6YS1Wt8yW0jTF+bM7i7VTW1HPlsb1cR2kxK4ijdM3E3uw5UMW8FTtdRzHG+Knq2npmf53DpH6Jfjlr63fxWkGIyFMisltE1jTa9hcR2SAiq0TkdRHp5NmeLiIHRWSF5/aIt3K1teP6dmFA9zie/HIbquo6jjHGD721cie7y6q4dlJv11GOiDdHEE8D0w7b9iEwRFWHAZuAOxr9bIuqjvDcrvdirjYlIlw9sRcbdpXxVXax6zjGGD+jqjzx5TYyusUyqV+i6zhHxGsFoaqfA3sP2/aBqtZ6Hi4Cenjr/X3prBHHkBgbyRNfbnUdxRjjZxZuKWZ9QSnXTOyNiH/O2vpdXB6DuAp4t9HjXiLyjYh8JiKTXIVqjciwUK6Y0JNPNxaxubDMdRxjjB95/IutJMZGMn3kMa6jHDEnBSEivwRqgTmeTQVAmqqOBG4FnheRJicpEZFZIpIlIllFRUW+CdwCl47vSWRYCE99tc11FGOMn8jeXcYnG4uYOaEnkWGhruMcMZ8XhIhcCZwJXKqeo7qqWqWqxZ77y4AtQEZTr1fVx1Q1U1Uzk5KSfJS6eQkxEZw7qgevLrcL54wxDZ78chuRYSFcOi7NdZRW8WlBiMg04HbgLFWtaLQ9SURCPfd7A/2AgNuhf/XEdKpr65ljF84ZE/SKD1Tx6vJ8zhvdgy6xka7jtIo3T3OdCywE+otInohcDTwExAEfHnY66/HAKhFZAbwCXK+qe5v8xX6sb9c4TuqfxDMLc6issQvnjAlmzy3aTnVtPVcdFzgXxh0uzFu/WFUvbmLzk9/x3FeBV72VxZeumdSbS59YzLyVO7kwMzDmWzHGtK3KmjqeXZTD5AFd6ds11nWcVrMrqdvYsX0aLpx74out1NfbhXPGBKM3V+Sz50A110wK3NEDWEG0ORHhuhN6s6nwAJ9s3O06jjHGx+rrlUc/38qg5I5M6N3FdZyjYgXhBWcOO4aUTh145LMtrqMYY3zsw/WFbC0q5/oT+wTchXGHs4LwgvDQEK6Z1IulOftYlhtwx9qNMa2kqjz86RZSEzpw+pDuruMcNSsIL7loTCqdo8N5+NOAO1vXGNNKi7ftZcWOEmYd3ycgVoxrTuB/Aj8VHRHGzAnpfLS+kE02/YYxQeGRz7aQGBvBBaPbxTRzVhDedMWx6USFh/DoZzaKMKa9W7ezlE83FnHlselEhQfetBpNsYLwooSYCGaMSePNFfnsLDnoOo4xxose/XwLMRGhXD4+3XWUNmMF4WVXT+yF0jAnizGmfdqxt4K3VxVwybg04qPDXcdpM1YQXpaaEM0PhiUzd8l2SiqqXccxxnjB419sJUTg6omBtWJcc6wgfOC6E/pQUV3HswtzXUcxxrSx4gNVvJS1g3NGptA9Psp1nDZlBeEDA5M7cmL/JJ7+OoeD1TaJnzHtyeyvc6iqrWfW8X1cR2lzVhA+8sMT+lBcXs2LS20qcGPaiwNVtcxemMvUgd0CelK+72IF4SPjendhbHoCj36+lapaG0UY0x48tyiX/Qdr+NFJfV1H8QorCB+6YXJfCvZX8uqyfNdRjDFH6WB1HU98sZVJ/RIZkdrJdRyvsILwoUn9Ehme2ol/fZpNTV296zjGmKMwd8l29hyo5sbJ/VxH8RorCB8SEW48qS95+w4yb8VO13GMMa1UWVPHo59vYVyvBMb2SnAdx2usIHzs5IFdGZjckX9+kk2dLShkTEB6eVkehaVV/OTk9jt6ACsInxMRbpzcl617ynlndYHrOMaYI1RdW88jn25hZFonju0T2AsCNccKwoFpg7vTt2ssD32cbcuSGhNg3vgmn/ySg/xkcr+AXxCoOVYQDoSECDec1JeNhWV8uL7QdRxjTAvV1tXzz0+zGZoSz4n9k1zH8TqvFoSIPCUiu0VkTaNtCSLyoYhs9vzZ2bNdRORBEckWkVUiMsqb2Vw7c1gy6V2ieejjbFRtFGFMIHhr1U5yiyu4YXLfdj96AO+PIJ4Gph227RfAAlXtByzwPAY4Dejnuc0CHvZyNqfCQkP40Yl9WZ2/n083FbmOY4xpRl298tDH2fTvFsfUgd1cx/EJrxaEqn4OHL4o83Rgtuf+bODsRtuf0QaLgE4ikuzNfK6dMyqFlE4deHDBZhtFGOPn3l1TwJaicm6Y3JeQkPY/egA3xyC6qeqh03d2AYeqOAXY0eh5eZ5t3yIis0QkS0SyiooC+5t3eGgIN0zuyzfbS2wUYYwfq6tXHlywmd5JMZw+tF1/b/0WpwepteFr8xF9dVbVx1Q1U1Uzk5IC/yDR+aN7kJrQgb99uMlGEcb4qfmrC9hUeICbp2QQGiSjB3BTEIWHdh15/tzt2Z4PpDZ6Xg/PtnYtPDSEGyf3Y1Xefj5av7v5FxhjfKq2rp6/f7SJjG6xnBlEowdwUxDzgCs8968A3my0fabnbKbxwP5Gu6LatXNHppDeJZr7P9xk10UY42fmrdzJ1qJybpmSETTHHg7x9mmuc4GFQH8RyRORq4F7gakishmY4nkM8A6wFcgGHgd+5M1s/iQsNISbpvRjfUEp76/d5TqOMcajpq6eBxZsZlByR04d3N11HJ8L8+YvV9WLv+NHJzfxXAV+7M08/uys4Sk89HE2f/toE6cM7h5U+zmN8VevL88nt7iCJ2ZmBt3oAexKar8RGiLcPCWDTYUHmG9zNBnjXHVtw+hheI94Th7Y1XUcJ6wg/MgZQ5PJ6BbL3z/aRK2tF2GMUy9l7SC/5CC3TM0Iiqumm2IF4UdCQoRbpmSwtaiceSttvQhjXKmsqeOfn2QzumdnTsgI/NPpW8sKws+cOrg7g5I78sCCzbbqnDGOvLBkOwX7K7k1iEcP0MKCEJFnW7LNHL2QEOGWqRnkFlfwyrI813GMCToV1bU89EnDanHtfb2H5rR0BDG48QMRCQVGt30cAzBlYFdGpXXi7x9t4mB1nes4xgSVp77cxp4DVdw+bUBQjx6gmYIQkTtEpAwYJiKlnlsZDVc/v/l9rzWtJyL8fNoACkurePrrHNdxjAka+8qrefSzrUwd1I3RPTu7juPc9xaEqv5JVeOAv6hqR88tTlW7qOodPsoYlMb17sJJ/ZN4+NNs9lfUuI5jTFD45yfZlFfXcvup/V1H8Qst3cX0tojEAIjIZSJyv4j09GIuA9w+bQBlVbU8/NkW11GMaffySw7yzMJczhvVg37d4lzH8QstLYiHgQoRGQ7cBmwBnvFaKgPAwOSOTB9+DP/+ahu79le6jmNMu/a3DzeBwM1TM1xH8RstLYhaz1QY04GHVPWfgFWsD9x2Sn/qVXlgwSbXUYxptzYVlvHa8jxmju9JSqcOruP4jZYWRJmI3AFcDswXkRAg3HuxzCGpCdFcOq4nL2XlsaXogOs4xrRLf3l/IzERYfz4pL6uo/iVlhbERUAVcJWq7qJhrYa/eC2V+ZYbJvclMiyE+z7Y6DqKMe3Osty9fLiukOtO6E3nmAjXcfxKiwrCUwpzgHgROROoVFU7BuEjibGRXDOpN++s3sWKHSWu4xjTbqgq//vuRhJjI7lqYi/XcfxOS6+kvhBYAlwAXAgsFpHzvRnMfNu1k3qRGBvBH+evt6VJjWkjH6wrZEnOXm6a0o/oCK+ufhCQWrqL6ZfAGFW9QlVnAmOBu7wXyxwuLiqcm6dksCRnry0qZEwbqK6t50/vrKdv11guHpPa/AuCUEsLIkRVGy+YXHwErzVtZMaYVPp1jeVP726gutYm8jPmaDy3KJec4gp+efpAwkLtn7OmtPRv5T0ReV9ErhSRK4H5NCwRanwoLDSEO88YSG5xBc8szHEdx5iAVVJRzQMLNjOpXyIn9g/e6byb09xcTH1F5DhV/RnwKDDMc1sIPOaDfOYwJ2YkMalfIv/4OJuSimrXcYwJSP/4OJvSyhruPH1g0E/I932aG0H8HSgFUNXXVPVWVb0VeN3zM+NjIsIvzxhIWWUNDyzY7DqOMQEnZ085zyzM4aLMVAYmd3Qdx681VxDdVHX14Rs929Jb84Yi0l9EVjS6lYrIzSJyt4jkN9p+emt+fzAY0L0jF41J5dmFuWy1i+eMOSL3vruB8NAQbj3FptRoTnMF0el7ftaq69FVdaOqjlDVETSsKVFBw4gE4G+Hfqaqdozje9wyNYPIsBDufXeD6yjGBIzFW4t5b+0ufnhCH7rGRbmO4/eaK4gsEbn28I0icg2wrA3e/2Rgi6rmtsHvCipd46L44Yl9+GBdIYu2FruOY4zfq69X7nlnPcnxUVwzqbfrOAGhuYK4GfgfEflURO7z3D4DrgZuaoP3nwHMbfT4BhFZJSJPiYit1tGMayb15pj4KH731jrq6u3iOWO+z6vL81iVt5+fndqfDhGhruMEhOYWDCpU1WOB3wI5nttvVXWCZ/qNVhORCOAs4GXPpoeBPsAIoAC47zteN0tEskQkq6io6GgiBLyo8FDuPGMg6wpKeX7JdtdxjPFbpZU1/O97GxiV1omzR6S4jhMwWjoX0yeq+g/P7eM2eu/TgOWqWuh5j0JVrVPVeuBxGq7WbirLY6qaqaqZSUl2/vIZQ5OZ0LsL932wkX3ldtqrMU158KPNFJdX89uzhhASYqe1tpTLywcvptHuJRFJbvSzc4A1Pk8UgESEu88aTFllLX+12V6N+S/Zu8t4+uscZoxJZWiPeNdxAoqTgvAsXzoVeK3R5j+LyGoRWQWcBNziIlsg6t89jsvH9+T5JdtZk7/fdRxj/Iaqcve8dURHhPLTU2yd6SPlpCBUtVxVu6jq/kbbLlfVoao6TFXPUtUCF9kC1S1TM0iIjuDueWtttldjPN5fW8iX2Xu47ZT+dImNdB0n4NgMVe1EfIdwbp/Wn6zcfbyxIt91HGOcq6yp4w/z1zGgexyXjktzHScgWUG0IxeMTmV4j3j+9M4GDlTVuo5jjFOPfraVvH0HufuswTZbayvZ31o7EhLScMB6d1kVD9o8TSaIbS+u4F+fZnPmsGTG9+7iOk7AsoJoZ0amdeaizFSe+nIbG3aVuo5jjM+pKne9uYbw0BB+dcYg13ECmhVEO/SL0wbQsUM4d762mnq7wtoEmXdW7+KzTUXcdkoG3eNtvqWjYQXRDnWOieDO0weyfHsJLyzd4TqOMT5TWlnDb99ay5CUjsyckO46TsCzgminzhuVwvjeCdz77nqKyqpcxzHGJ+7/YBNFB6r44zlDCbUrpo+aFUQ7JSL84eyhHKyp457561zHMcbrVuWVMHthDjPH92RYj+9bqcC0lBVEO9a3ayw/PKEPb6zYyZeb97iOY4zX1NUrd76+mqTYSG471a6YbitWEO3cj07qS3qXaO56cw2VNXWu4xjjFc8szGFNfim//sEgOkaFu47TblhBtHNR4aH84eyhbNtTzj8/yXYdx5g2l19ykL++v5HjM5I4Y2hy8y8wLWYFEQQm9kvknJEpPPzpFtbttGsjTPuhqtzx2moUuOfsIYjYgem2ZAURJH595iA6RYdz+6srqa2rdx3HmDbx6vJ8Pt9UxM+nDSA1Idp1nHbHCiJIdI6J4HfTh7Amv5THvtjqOo4xR213WSW/f3sdY9I7c/n4nq7jtEtWEEHk9KHJnDakO3//aDPZuw+4jmNMq6kqd72xhoM1ddx73jBbJc5LrCCCzG+nDyY6IpTbX1lJnU3DYQLUO6t38f7aQm6ZkkGfpFjXcdotK4gg0zUuit/8YBDLt5cw++sc13GMOWL7yqv5zbw1DE2J59pJvVzHadesIILQ2SNSOKl/En95fyPbiytcxzHmiPz2rbWUVNTwv+cNs3UevMz+doOQiPDHc4cSFiL89GXb1WQCxzurC3hjxU5+fFJfBh3T0XWcds8KIkglx3fg7rMGsyRnL0/YWU0mAOwureSXr69mWI94bpjc13WcoOCsIEQkR0RWi8gKEcnybEsQkQ9FZLPnz86u8gWDc0elMG1wd+77YBPrC+wCOuO/VJWfv7qKiuo67r9wBOG2a8knXP8tn6SqI1Q10/P4F8ACVe0HLPA8Nl5yaFdTxw7h3PLiCqpqba4m45/mLtnBJxuL+MVpA+jb1c5a8hXXBXG46cBsz/3ZwNkOswSFhJgI/nz+UDbsKuP+Dze5jmPMf8ktLucP89dxXN8uXGGLAPmUy4JQ4AMRWSYiszzbuqlqgef+LqCbm2jBZfKAblwyLo3HPt/K4q3FruMY8x919cptL60kNET4y/nD7YI4H3NZEBNVdRRwGvBjETm+8Q9VVWkokW8RkVkikiUiWUVFRT6K2v798vSBpCVEc9vLKymrrHEdxxgAHvlsC1m5+/j99CEc06mD6zhBx1lBqGq+58/dwOvAWKBQRJIBPH/ubuJ1j6lqpqpmJiUl+TJyuxYTGcb9F45gZ8lB7npjDQ39bIw7y7fv4/4PN3HGsGSmjzjGdZyg5KQgRCRGROIO3QdOAdYA84ArPE+7AnjTRb5gNbpnZ26ZksEbK3byyrI813FMENt/sIafzP2G5Pgo/nTuUJvG25EwR+/bDXjd8x89DHheVd8TkaXASyJyNZALXOgoX9D60Ul9+XpLMb9+cy0j0zrbGSPG51Qblg8t2F/Jy9dPsBXiHHIyglDVrao63HMbrKr3eLYXq+rJqtpPVaeo6l4X+YJZaIjw9xkj6BARyg3PL7dlSo3Pvbh0B/NXFXDbKRmMSrNLoVzyt9NcjR/o1jGK+y4YzoZdZdwzf73rOCaIbC4s4+631jKxbyLXH9/HdZygZwVhmnTSgK5cO6kXzy7K5b01Bc2/wJijVFlTx41zvyEmIoz7L7RTWv2BFYT5Tj87dQDDe8Rz+yur2LHXZn013qOq/OqNNWzYVcZ9Fw6na8co15EMVhDme0SEhfCPi0ehwA/nLLPjEcZrXli6g1eW5fGTyX05sX9X13GMhxWE+V5pXaL524UjWJNfyq/ftOsjTNtblVfCb95cy6R+idw0JcN1HNOIFYRp1pRB3bhxcl9eysrjhaU7XMcx7ci+8mp++NxykuIieWDGSELtuINfsYIwLXLzlAwm9UvkN2+uZeWOEtdxTDtQV6/c/OIKisqq+Nelo0iIiXAdyRzGCsK0SGiI8OCMkSTFRfLD55axt7zadSQT4B5csJnPNhXxm7MGMTy1k+s4pglWEKbFOsdE8PBlo9hTXs2Nc5dTU1fvOpIJUO+uLuCBBZs5f3QPLhmb5jqO+Q5WEOaIDOvRiXvOHsJX2cX8/u11ruOYALR2535ufWklI9M68Yezh9g8S37M1VxMJoBdkJnK5t0HeOzzrfTrFsfl43u6jmQCRFFZFdfOzqJTdDiPXj6aqPBQ15HM97ARhGmVn08bwMkDunL3vLV8lb3HdRwTAKpq67j+uWXsrajm8ZmZdI2zi+H8nRWEaZVDk/r1SYrhR3OWs21PuetIxo+pKr96fQ3Lcvfx1wuGMyQl3nUk0wJWEKbV4qLCefKKMYSGCFc/vZT9FbYSnWnao59v5WXPldJnDrPFfwKFFYQ5KqkJ0Txy2Wh27Ktg1rNZVNXadBzm295ckc+9727gjGHJ3GxXSgcUKwhz1Mb2SuCvFwxn8ba93PrSSurrbToO02DR1mJ+9vIqxvZK4L4LbIbWQGNnMZk2MX1ECoWllfzxnQ107xjFXWcOch3JOLapsIxZz2SR1iWaxy/PtDOWApAVhGkz107qTcH+Sp78chvJ8VFcM6m360jGkcLSSq58agmR4aE8/T9jiI+2ZUMDkRWEaTMiwl1nDKKwtJI/zF9PUlwk00ekuI5lfKykopqZTy5h/8EaXrxuAj06R7uOZFrJjkGYNhUSItx/4QjGpifw05dX8smwuCjfAAAN8klEQVSG3a4jGR86UFXLFf9eyrY95Tw2M9NOZw1wVhCmzUWFh/L4FZn07x7H9c8t4+stdiFdMKisqePa2Vmsyd/PQ5eM5Li+ia4jmaPk84IQkVQR+URE1onIWhG5ybP9bhHJF5EVntvpvs5m2k58h3CeuWocaQnRXDM7i2W5+1xHMl5UU1fPDc8vZ9G2Yu67YDinDO7uOpJpAy5GELXAbao6CBgP/FhEDp3y8jdVHeG5veMgm2lDCTERzLlmHF3jIrny30tYk7/fdSTjBbV19dz60ko+Wr+b308fwtkj7bhTe+HzglDVAlVd7rlfBqwH7H9R7VTXjlHMuXY8HaPCmfnUEjbsKnUdybSh2rp6bnlpJW+t3MkvThvAZTZxY7vi9BiEiKQDI4HFnk03iMgqEXlKRDo7C2baVEqnDsy5ZhwRoSFc/NgiG0m0E7V19dz04or/lMP1J/RxHcm0MWcFISKxwKvAzapaCjwM9AFGAAXAfd/xulkikiUiWUVFRT7La45OemIML143nuiIMC55fJEtWxrgaurq+ckL3zB/VQF3nm7l0F45KQgRCaehHOao6msAqlqoqnWqWg88Doxt6rWq+piqZqpqZlJSku9Cm6PWs0sML8waT3x0OJc9sdgOXAeo6tp6bnz+G95ZvYtfnTGQWcdbObRXLs5iEuBJYL2q3t9oe3Kjp50DrPF1NuN9qQnRvDhrAl1iI5j55GIWbil2HckcgfKqWq6evZT31u7i12cOsqvl2zkXI4jjgMuByYed0vpnEVktIquAk4BbHGQzPnBMpw68eN0Ekjt14Ip/L+G9NQWuI5kW2FdezaVPLOar7D38+fxhXDWxl+tIxstENXBn3szMzNSsrCzXMUwr7Suv5urZS/lmRwm/mz7Eli71YztLDjLzqSVs31vBQxePtOscApyILFPVzOaeZ1dSG2c6x0Qw55rxTO7flbveWMP9H2wkkL+wtFfZu8s4/+Gv2bW/kmeuGmvlEESsIIxTHSJCefTy0VyY2YMHP87m56+uorq23nUs4/HF5iLO+dfXVNcpL8waz/jeXVxHMj5ks7ka58JCQ/jf84bRPb4DDy7YTE5xBY9cNpqEmAjX0YLac4ty+c28tfTrGssTV2TarKxByEYQxi+ICLdOzeCBGSNYsaOEsx76ko27ylzHCkp19crv317Hr95Yw/H9Enn5epuyO1hZQRi/Mn1ECi9dN4Hq2nrO/ddXfLSu0HWkoLK3vJor/72EJ7/cxpXHpvP4zEziomyxn2BlBWH8zojUTsy7YSK9k2K59tks7vtgI3W2zrXXrdxRwg/+8SWLt+3l3nOHcvdZgwkLtX8igpn91zd+qXt8FC9fP4ELRvfgHx9nc9kTi9ldVuk6Vrukqsxdsp0LHlkIwKvXH8uMsWmOUxl/YAVh/FZUeCh/Pn84fzl/GN/s2McZD35pV163sf0Ha7jphRXc8dpqxvfpwts3TmRoD1sFzjSwgjB+74LMVN788UTiosK45IlF/Ond9VTV1rmOFfCWbNvL6Q98wfzVBdw2NYN/XzmGznbmmGnECsIEhP7d43jrhonMGJPKo59tZfpDX9naEq1UU1fPfR9sZMZjCwkLFV65fgI3ntyP0BBxHc34GSsIEzBiIsP407nDePKKTPYcqOasf3zFI59tobbOLqxrqVV5DQei//FxNueN6sH8n0xiZJotvWKaZnMxmYBUfKCKO19fzftrCxmU3JE/njuUEamdXMfyWwer67j/w408+eU2kuIi+cPZQ5k6qJvrWMaRls7FZAVhApaq8v7aXdw9bx2FZZVcPr4nPz21Px3tvP3/UFUWrN/N795ex/a9FVwyLo1fnDbA/o6CXEsLwqbaMAFLRJg2JJnj+iZy3webeGZhDu+s3sWtUzO4MLNH0J/Dv7mwjN+9vY4vNu+hT1KMzaVkjpiNIEy7sSqvhN+9tY6s3H1kdIvljtMHcmJGEg1rVAWPPQeqeOjjbJ5dlEtMRCi3TM3gsvE9CQ/ywjT/z3YxmaB0aLfTve9uIKe4gmP7dOGmk/sxLgi+OZdUVPPo51uZ/XUOlTV1XDIujVun9rdJD81/sYIwQa26tp7nFuXyr0+3sOdAFeN6JfCTk/txbJ8u7W5EUVRWxTMLc3j6qxwOVNfyg2HHcNOUfvRJinUdzfgpKwhjgMqaOuYu2c4jn22hsLSK4T3iufK4dE4fmkxkWKjreEcle3cZT3yxjde+yaemrp5TB3XnlqkZ9O8e5zqa8XNWEMY0UllTx8vL8vj3l9vYuqecxNgILhmbxgWZqaQmBM5U1pU1dby/dhcvZe3gq+xiIsNCOH90D66e2IveNmIwLWQFYUwT6uuVL7P38MzCHBZs2I0qjE1P4JxRKZw+JJn4aP87/bOuXlmWu4/5q3byxoqd7D9YQ4/OHZgxJpVLxvW0YwzmiAVsQYjINOABIBR4QlXv/a7nWkGYo5G3r4I3V+zkteV5bCkqJzxUGNerCycP7MrJA7qR1sXdyOJgdR1Lcvby4bpdvL+2kKKyKiLCQjh1cHdmjEllQu8uhNjUGKaVArIgRCQU2ARMBfKApcDFqrquqedbQZi2oKqsyS/l7VU7+Wh9IVuKygHolRjDmPTOjElPYGyvBNISor12gHt/RQ1rdu5nWe4+vsrewzfbS6iuq6dDeCgnDUhi2pBkJg/oSmykXbpkjl6gFsQE4G5VPdXz+A4AVf1TU8+3gjDekLOnnAUbdrNwyx6W5uxj/8EaAOIiw8joHkf/7nH06xrLMZ06cEx8B7rHR9ElJuJ7v9GrKqWVtZRUVLOzpJLc4nJy91aQs6ectTtL2b63AgARGHxMR47rk8hxfRMZk55Ah4jAPphu/E+gXkmdAuxo9DgPGOcoiwlS6YkxXD2xF1dP7EV9vZJddICsnH2sLyhlY2EZ81cV/Kc0GusQHkpMZBgxkQ3/oNfVK/X1SlVtPfsP1lB72Kp4YSFCj84dGJLSkRljUxmaEs/QlHg6RdsxBeMf/K0gmiUis4BZAGlptuqV8a6QECGjWxwZ3f7/1FFVpbi8moKSSnbuP0hByUH2VdRQXlVLeXUdFdW1AISKEBIiRISF0Dk6nM7REXSOjqBbxyh6dokmOT4q6KcDMf7N3woiH0ht9LiHZ9t/qOpjwGPQsIvJd9GMaSAiJMZGkhgbaauvmXbN376+LAX6iUgvEYkAZgDzHGcyxpig5FcjCFWtFZEbgPdpOM31KVVd6ziWMcYEJb8qCABVfQd4x3UOY4wJdv62i8kYY4yfsIIwxhjTJCsIY4wxTbKCMMYY0yQrCGOMMU3yq7mYjpSIFAG5rnM0IxHY4zpEG2kvn6W9fA6wz+Kv/P2z9FTVpOaeFNAFEQhEJKslk2IFgvbyWdrL5wD7LP6qvXwW28VkjDGmSVYQxhhjmmQF4X2PuQ7QhtrLZ2kvnwPss/irdvFZ7BiEMcaYJtkIwhhjTJOsIHxIRG4TERWRRNdZWkNE/iIiG0RklYi8LiKdXGc6UiIyTUQ2iki2iPzCdZ7WEpFUEflERNaJyFoRucl1pqMhIqEi8o2IvO06y9EQkU4i8orn/yfrPcsoBywrCB8RkVTgFGC76yxH4UNgiKoOAzYBdzjOc0REJBT4J3AaMAi4WEQGuU3VarXAbao6CBgP/DiAPwvATcB61yHawAPAe6o6ABhOgH8mKwjf+RtwOxCwB31U9QNVrfU8XETDin+BZCyQrapbVbUaeAGY7jhTq6hqgaou99wvo+EfohS3qVpHRHoAZwBPuM5yNEQkHjgeeBJAVatVtcRtqqNjBeEDIjIdyFfVla6ztKGrgHddhzhCKcCORo/zCNB/VBsTkXRgJLDYbZJW+zsNX57qXQc5Sr2AIuDfnt1lT4hIjOtQR8PvFgwKVCLyEdC9iR/9EriTht1Lfu/7Poeqvul5zi9p2MUxx5fZzH8TkVjgVeBmVS11nedIiciZwG5VXSYiJ7rOc5TCgFHAjaq6WEQeAH4B3OU2VutZQbQRVZ3S1HYRGUrDN4uVIgINu2WWi8hYVd3lw4gt8l2f4xARuRI4EzhZA+8c6XwgtdHjHp5tAUlEwmkohzmq+prrPK10HHCWiJwORAEdReQ5Vb3Mca7WyAPyVPXQSO4VGgoiYNl1ED4mIjlApqr680ReTRKRacD9wAmqWuQ6z5ESkTAaDq6fTEMxLAUuCcR1z6Xh28ZsYK+q3uw6T1vwjCB+qqpnus7SWiLyBXCNqm4UkbuBGFX9meNYrWYjCHMkHgIigQ89o6FFqnq920gtp6q1InID8D4QCjwViOXgcRxwObBaRFZ4tt3pWdPduHMjMEdEIoCtwP84znNUbARhjDGmSXYWkzHGmCZZQRhjjGmSFYQxxpgmWUEYY4xpkhWEMcaYJllBGGOMaZIVhDHGmCZZQRjTBkQkRERKGj0e4Vn7I8PzOFZE8kUkzl1KY46MFYQxbUBVD5+J9DpgJ9DR8/gS4C3P1NzGBASbasOYtlPumd5ZgEnA68ChEcO1npsxAcMKwpi2U0pDIZwFvAhE0DA76UigVlVXfN+LjfE3tovJmLZzqCCuomF1tDLP4+uARx3mMqZVrCCMaTulwGQa1gQooKEgkoFpNIwojAkoVhDGtJ1S4FbgEc/jMuAa4A1VPegslTGtZMcgjGk7pTQcoF7geVwGZADTnSUy5ijYehDGGGOaZLuYjDHGNMkKwhhjTJOsIIwxxjTJCsIYY0yTrCCMMcY0yQrCGGNMk6wgjDHGNMkKwhhjTJP+D3Wiitwc1u0SAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(W_l, cost_l)\n",
    "plt.xlabel('$W$')\n",
    "plt.ylabel('Cost')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6Caylv3xg_mf"
   },
   "source": [
    "## Gradient Descent by Hand"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "5pRl-n4rg_mf"
   },
   "outputs": [],
   "source": [
    "W = 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "YCAMFpKjg_mf"
   },
   "source": [
    "$$ cost(W) = \\frac{1}{m} \\sum^m_{i=1} \\left( Wx^{(i)} - y^{(i)} \\right)^2 $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "1ZpkeAeqg_mf"
   },
   "source": [
    "$$ \\nabla W = \\frac{\\partial cost}{\\partial W} = \\frac{2}{m} \\sum^m_{i=1} \\left( Wx^{(i)} - y^{(i)} \\right)x^{(i)} $$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "Vn8RhN7ng_mg",
    "outputId": "c726bdbd-42f7-4ee5-db1a-5a17b631d4b5"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(-14.)\n"
     ]
    }
   ],
   "source": [
    "gradient = torch.sum((W * x_train - y_train) * x_train)\n",
    "print(gradient)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xv6NEYi_g_mg"
   },
   "source": [
    "$$ W := W - \\alpha \\nabla W $$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "f4zRrlSJg_mg",
    "outputId": "7f45ff63-b630-4c0e-fa1e-131c1b1db52a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(1.4000)\n"
     ]
    }
   ],
   "source": [
    "lr = 0.1\n",
    "W -= lr * gradient\n",
    "print(W)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "s6lQjuziKJBh"
   },
   "source": [
    "Q1. 경사하강법(Gradient Descent) W:=W−α∇W에서 -를 쓰는 이유는?\n",
    "\n",
    "=> 경사하강법에서 비용을 가장 적게 하기 위해서는 미분값이 0인 W값을 찾아야하는데 현재의 W값이 양수라면 W에 +인 기울기를 빼서\n",
    " 왼쪽으로 이동해 2차함수의 꼭지점에 더 가깝게 이동하고, 현재의 W값이 음수라면 W에 -인 기울기를 빼서 오른쪽으로 이동해 2차함수의 \n",
    " 꼭지점에 더 가깝게 이동하기 위해서입니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "qi8YmCKTg_mg"
   },
   "source": [
    "## Training"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "gCwk2_NTjr6p"
   },
   "source": [
    "위의 내용을 참고해 코드를 완성시키세요."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "lYYQHd6sg_mh",
    "outputId": "8b250d4e-b233-4f6c-8b6f-c1d854e27bae"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch    0/10 W: 0.000, Cost: 4.666667\n",
      "Epoch    1/10 W: 1.400, Cost: 0.746666\n",
      "Epoch    2/10 W: 0.840, Cost: 0.119467\n",
      "Epoch    3/10 W: 1.064, Cost: 0.019115\n",
      "Epoch    4/10 W: 0.974, Cost: 0.003058\n",
      "Epoch    5/10 W: 1.010, Cost: 0.000489\n",
      "Epoch    6/10 W: 0.996, Cost: 0.000078\n",
      "Epoch    7/10 W: 1.002, Cost: 0.000013\n",
      "Epoch    8/10 W: 0.999, Cost: 0.000002\n",
      "Epoch    9/10 W: 1.000, Cost: 0.000000\n",
      "Epoch   10/10 W: 1.000, Cost: 0.000000\n"
     ]
    }
   ],
   "source": [
    "# 데이터\n",
    "x_train = torch.FloatTensor([[1], [2], [3]])\n",
    "y_train = torch.FloatTensor([[1], [2], [3]])\n",
    "# 모델 초기화\n",
    "W = torch.zeros(1, requires_grad = False) #가중치 W를 0으로 초기화하세요. Gradient Descent by Hand이기 때문에 학습을 통해 값이 변경되는 것이 아닙니다!\n",
    "# learning rate 설정\n",
    "lr = 0.1\n",
    "\n",
    "nb_epochs = 10\n",
    "for epoch in range(nb_epochs + 1):\n",
    "    \n",
    "    # H(x) 계산\n",
    "    hypothesis = W * x_train # H(x)=Wx를 이용하세요.\n",
    "    \n",
    "    # cost gradient 계산\n",
    "    cost = torch.mean((hypothesis - y_train)**2)# #파이토치 코드 상으로 선형 회귀의 비용 함수에 해당되는 평균 제곱 오차를 선언하세요.\n",
    "    gradient = torch.sum((W * x_train - y_train) * x_train)  #Gradient Descent by Hand를 이용하세요.\n",
    "\n",
    "    print('Epoch {:4d}/{} W: {:.3f}, Cost: {:.6f}'.format(\n",
    "        epoch, nb_epochs, W.item(), cost.item()\n",
    "    ))\n",
    "\n",
    "    # cost gradient로 H(x) 개선\n",
    "    W -=  lr * gradient"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9iaJEB1Eg_mh"
   },
   "source": [
    "## Training with `optim`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "KLki9JjIkyEL"
   },
   "source": [
    "위의 내용을 참고해 코드를 완성시키세요."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "ZVBbc9t9g_mh",
    "outputId": "8f8397eb-5c46-4232-8f89-045dad4851f0"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch    0/10 W: 0.000 Cost: 4.666667\n",
      "Epoch    1/10 W: 1.400 Cost: 0.746667\n",
      "Epoch    2/10 W: 0.840 Cost: 0.119467\n",
      "Epoch    3/10 W: 1.064 Cost: 0.019115\n",
      "Epoch    4/10 W: 0.974 Cost: 0.003058\n",
      "Epoch    5/10 W: 1.010 Cost: 0.000489\n",
      "Epoch    6/10 W: 0.996 Cost: 0.000078\n",
      "Epoch    7/10 W: 1.002 Cost: 0.000013\n",
      "Epoch    8/10 W: 0.999 Cost: 0.000002\n",
      "Epoch    9/10 W: 1.000 Cost: 0.000000\n",
      "Epoch   10/10 W: 1.000 Cost: 0.000000\n"
     ]
    }
   ],
   "source": [
    "# 데이터\n",
    "x_train = torch.FloatTensor([[1], [2], [3]])\n",
    "y_train = torch.FloatTensor([[1], [2], [3]])\n",
    "# 모델 초기화\n",
    "W = torch.zeros(1, requires_grad = True)# #가중치 W를 0으로 초기화하고 학습을 통해 값이 변경될 수 있도록 하세요.\n",
    "# optimizer 설정\n",
    "optimizer = optim.SGD([W, b], lr=0.01) # #SGD optimizer를 사용하고 learning rate는 0.15로 적용하세요.\n",
    "\n",
    "nb_epochs = 10\n",
    "for epoch in range(nb_epochs + 1):\n",
    "    \n",
    "    # H(x) 계산\n",
    "    hypothesis = W * x_train # H(x)=Wx를 이용하세요.\n",
    "    \n",
    "    # cost 계산\n",
    "    cost =torch.mean((hypothesis - y_train) ** 2) # #파이토치 코드 상으로 선형 회귀의 비용 함수에 해당되는 평균 제곱 오차를 선언하세요.\n",
    "\n",
    "    print('Epoch {:4d}/{} W: {:.3f} Cost: {:.6f}'.format(\n",
    "        epoch, nb_epochs, W.item(), cost.item()\n",
    "    ))\n",
    "\n",
    "    W = optimizer.minimize(cost)\n",
    "    optimizer.zero_grad()  # gradient를 0으로 초기화\n",
    "    cost.backward()#  # 비용 함수를 미분하여 gradient 계산\n",
    "    optimizer.step() # W와 b를 업데이트"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "Lab03.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}