4-NN-Tuning.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Neural Nets :Tuning"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "%load_ext tensorboard"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib as mpl\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import os\n",
    "import pandas as pd\n",
    "import sklearn\n",
    "import sys\n",
    "import tensorflow as tf\n",
    "from tensorflow import keras  # tf.keras\n",
    "import time\n",
    "import seaborn as sns\n",
    "sns.set()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "python 3.7.1 (default, Dec 14 2018, 13:28:58) \n",
      "[Clang 4.0.1 (tags/RELEASE_401/final)]\n",
      "matplotlib 3.0.2\n",
      "numpy 1.15.4\n",
      "pandas 0.23.4\n",
      "sklearn 0.20.1\n",
      "tensorflow 2.0.0-beta0\n",
      "tensorflow.python.keras.api._v2.keras 2.2.4-tf\n"
     ]
    }
   ],
   "source": [
    "print(\"python\", sys.version)\n",
    "for module in mpl, np, pd, sklearn, tf, keras:\n",
    "    print(module.__name__, module.__version__)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "assert sys.version_info >= (3, 5) # Python ≥3.5 required\n",
    "assert tf.__version__ >= \"2.0\"    # TensorFlow ≥2.0 required"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### A neural net for regression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load the California housing dataset using `sklearn.datasets.fetch_california_housing`. This returns an object with a `DESCR` attribute describing the dataset, a `data` attribute with the input features, and a `target` attribute with the labels. The goal is to predict the price of houses in a district (a census block) given some stats about that district. This is a regression task (predicting values)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.datasets import fetch_california_housing\n",
    "housing = fetch_california_housing()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ".. _california_housing_dataset:\n",
      "\n",
      "California Housing dataset\n",
      "--------------------------\n",
      "\n",
      "**Data Set Characteristics:**\n",
      "\n",
      "    :Number of Instances: 20640\n",
      "\n",
      "    :Number of Attributes: 8 numeric, predictive attributes and the target\n",
      "\n",
      "    :Attribute Information:\n",
      "        - MedInc        median income in block\n",
      "        - HouseAge      median house age in block\n",
      "        - AveRooms      average number of rooms\n",
      "        - AveBedrms     average number of bedrooms\n",
      "        - Population    block population\n",
      "        - AveOccup      average house occupancy\n",
      "        - Latitude      house block latitude\n",
      "        - Longitude     house block longitude\n",
      "\n",
      "    :Missing Attribute Values: None\n",
      "\n",
      "This dataset was obtained from the StatLib repository.\n",
      "http://lib.stat.cmu.edu/datasets/\n",
      "\n",
      "The target variable is the median house value for California districts.\n",
      "\n",
      "This dataset was derived from the 1990 U.S. census, using one row per census\n",
      "block group. A block group is the smallest geographical unit for which the U.S.\n",
      "Census Bureau publishes sample data (a block group typically has a population\n",
      "of 600 to 3,000 people).\n",
      "\n",
      "It can be downloaded/loaded using the\n",
      ":func:`sklearn.datasets.fetch_california_housing` function.\n",
      "\n",
      ".. topic:: References\n",
      "\n",
      "    - Pace, R. Kelley and Ronald Barry, Sparse Spatial Autoregressions,\n",
      "      Statistics and Probability Letters, 33 (1997) 291-297\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(housing.DESCR)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(20640, 8)"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "housing.data.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(20640,)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "housing.target.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Split the dataset into a training set, a validation set and a test set using Scikit-Learn's `sklearn.model_selection.train_test_split()` function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "X_train_full, X_test, y_train_full, y_test = train_test_split(housing.data, housing.target, random_state=42)\n",
    "X_train, X_valid, y_train, y_valid = train_test_split(X_train_full, y_train_full, random_state=42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(11610, 3870, 5160)"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(X_train), len(X_valid), len(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Scale the input features (e.g., using a `sklearn.preprocessing.StandardScaler`). Once again, don't forget that you should not fit the validation set or the test set, only the training set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "scaler = StandardScaler()\n",
    "X_train_scaled = scaler.fit_transform(X_train)\n",
    "X_valid_scaled = scaler.transform(X_valid)\n",
    "X_test_scaled = scaler.transform(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now build, train and evaluate a neural network to tackle this problem. Then use it to make predictions on the test set.\n",
    "\n",
    "**Tips**:\n",
    "* Since you are predicting a single value per district (the median house price), there should only be one neuron in the output layer.\n",
    "* Usually for regression tasks you don't want to use any activation function in the output layer (in some cases you may want to use `\"relu\"` or `\"softplus\"` if you want to constrain the predicted values to be positive, or `\"sigmoid\"` or `\"tanh\"` if you want to constrain the predicted values to 0-1 or -1-1).\n",
    "* A good loss function for regression is generally the `\"mean_squared_error\"` (aka `\"mse\"`). When there are many outliers in your dataset, you may prefer to use the `\"mean_absolute_error\"` (aka `\"mae\"`), which is a bit less precise but less sensitive to outliers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = keras.models.Sequential([\n",
    "    \n",
    "    keras.layers.Dense(30, activation=\"relu\",\\\n",
    "                       input_shape=X_train.shape[1:]),\n",
    "    keras.layers.Dense(1)\n",
    "    \n",
    "])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.compile(loss=\"mean_squared_error\",\\\n",
    "              optimizer=keras.optimizers.SGD(1e-3))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "callbacks = [keras.callbacks.EarlyStopping(patience=10)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 11610 samples, validate on 3870 samples\n",
      "Epoch 1/100\n",
      "11610/11610 [==============================] - 0s 41us/sample - loss: 1.4373 - val_loss: 4.9912\n",
      "Epoch 2/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.8371 - val_loss: 0.8625\n",
      "Epoch 3/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.7555 - val_loss: 0.7340\n",
      "Epoch 4/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.7094 - val_loss: 0.7095\n",
      "Epoch 5/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.6720 - val_loss: 0.6471\n",
      "Epoch 6/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.6392 - val_loss: 0.6335\n",
      "Epoch 7/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.6105 - val_loss: 0.5858\n",
      "Epoch 8/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.5852 - val_loss: 0.5641\n",
      "Epoch 9/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.5626 - val_loss: 0.5456\n",
      "Epoch 10/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.5425 - val_loss: 0.5173\n",
      "Epoch 11/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.5253 - val_loss: 0.5070\n",
      "Epoch 12/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.5096 - val_loss: 0.4918\n",
      "Epoch 13/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4964 - val_loss: 0.4793\n",
      "Epoch 14/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4848 - val_loss: 0.4752\n",
      "Epoch 15/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4743 - val_loss: 0.4597\n",
      "Epoch 16/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4658 - val_loss: 0.4685\n",
      "Epoch 17/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4586 - val_loss: 0.4433\n",
      "Epoch 18/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4520 - val_loss: 0.4364\n",
      "Epoch 19/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4464 - val_loss: 0.4511\n",
      "Epoch 20/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4418 - val_loss: 0.4285\n",
      "Epoch 21/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4377 - val_loss: 0.4347\n",
      "Epoch 22/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4341 - val_loss: 0.4288\n",
      "Epoch 23/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4309 - val_loss: 0.4265\n",
      "Epoch 24/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4281 - val_loss: 0.4219\n",
      "Epoch 25/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4256 - val_loss: 0.4368\n",
      "Epoch 26/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4233 - val_loss: 0.4404\n",
      "Epoch 27/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4210 - val_loss: 0.4438\n",
      "Epoch 28/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4192 - val_loss: 0.4358\n",
      "Epoch 29/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4172 - val_loss: 0.4398\n",
      "Epoch 30/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4154 - val_loss: 0.4463\n",
      "Epoch 31/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4141 - val_loss: 0.4247\n",
      "Epoch 32/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4126 - val_loss: 0.4308\n",
      "Epoch 33/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4112 - val_loss: 0.4330\n",
      "Epoch 34/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4099 - val_loss: 0.4273\n"
     ]
    }
   ],
   "source": [
    "history = model.fit(X_train_scaled,\\\n",
    "                    y_train,\n",
    "                    validation_data=(X_valid_scaled, y_valid),\\\n",
    "                    epochs=100,\n",
    "                    callbacks=callbacks)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5160/5160 [==============================] - 0s 15us/sample - loss: 0.4072\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0.40716409780258356"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.evaluate(X_test_scaled, y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.6690421],\n",
       "       [1.6140628],\n",
       "       [3.5537164],\n",
       "       ...,\n",
       "       [1.504305 ],\n",
       "       [2.528902 ],\n",
       "       [3.6941016]], dtype=float32)"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.predict(X_test_scaled)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_learning_curves(history):\n",
    "    pd.DataFrame(history.history).plot(figsize=(8, 5))\n",
    "    plt.grid(True)\n",
    "    plt.gca().set_ylim(0, 1)\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAesAAAE3CAYAAACKD2n1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xl8VOXd///XObMkM9kJ2dhX2TdFRFaXCsriXqVaaEtLl1tvq9/W1tultrUutba0vWv7K71brVVa6abiAqioVUEQZJNFCEjClpWQjUlmOef3x4RgJBgCCTPJeT8fj3nMnDknM9cnF+Q913XOnGPYtm0jIiIiccuMdQNERETksymsRURE4pzCWkREJM4prEVEROKcwlpERCTOKaxFRETinMJaREQkzimsRURE4twph3VNTQ2zZs1i//79J6zbvn071157LdOnT+eee+4hHA63aSNFRESc7JTCetOmTXzhC19g7969za6/8847+cEPfsDy5cuxbZslS5a0ZRtFREQc7ZTCesmSJdx///1kZ2efsO7AgQPU1dUxevRoAK699lqWLVvWtq0UERFxMPepbPTggw+edF1JSQlZWVmNy1lZWRQXF595y0RERARogwPMLMvCMIzGZdu2myyLiIjImTmlkfVnyc3NpbS0tHG5rKys2enyz1JRUYtltc3FvzIzkykvrznp+pIjAR776wbmfG4gIz2F1K/+K76Z38dMzTrpz3QELdXdGTmxZnBm3U6sGZxZtxNqNk2DjIykVv3MGYd19+7dSUhIYP369Zx33nk8//zzTJkypVWvYVl2m4X1sdc7Ga/bpKQiQGlFgEh6kHBlKZFQENrw/WOlLX+HHYUTawZn1u3EmsGZdTux5pac9jT4ggUL2LJlCwCPPfYYDz/8MJdffjlHjx5l3rx5bdbAtpaU6MYAagMhMBvKt62YtklEROSztGpkvXLlysbHf/jDHxofDx48mH/84x9t16p25DJN/IluqgMhMBTWIiIS/854GrwjSvJ5oiPrxrDWlIuIyMlEImEqKkoJh4Pt/l4lJSaW1TkGUG63l4yMLFyuM49aR4Z1is9DTSB0/Kh1jaxFRE6qoqKUxEQ/SUm57f5tH7fbJBzu+H+TbdumtraKiopSunbNO+PXc+S5wZMawlrT4CIiLQuHgyQlpeprua1gGAZJSaltNhvhyLBO/nRYd5IpFxGR9qKgbr22/J0prAFbI2sREYljjg3rYMiicbeIDjATEekwPvhgHbfe+vVYN+OscmxYAwSCDWmtkbWIiMQxRx4NfiysjwYjpILCWkSkAyosLODRRx+kurqKxEQft9/+XYYMGcaKFctYvPgpTNOkW7du3HffA1RWHuHHP76PQCCAaRp8+9t3Mnz4iFiXcMocHdZ1QUthLSLSCu9uOcQ7mw+1y2tPGpnH1DHdT3n7Bx64jy9+8ctMnXoJH364hXvv/T5//eu/+MMffseiRU+QkdGFxx//FYWFe3n77beYMGESN900j/feW8XmzRsV1vHu+Mha0+AiIh1RIBDg4MEDTJ16CQDDh48gNTWVwsICJk6czLe+9VWmTLmIqVMvYeDAQQQCAe6553vs3PkREyZM4rrrbohxBa3jyLBOOhbW9ZHoEzrATETklEwckcfEEWd+ko8z1dy3eGwbIpEIt9/+XfLzr2L16nd44IH7mD//60yfPoOnn17CqlXv8PrrK3j55aX88pe/jUHLT48jw7pxZF0f7Wx9dUtEpGPx+5Po1q07b721snEa/PDhcvr168+cOdfwm98sYu7crxAOh9m58yN2795F167Z3HDDFxgzZizz598c6xJaxZFh7XGbJHhd1B4bWeukKCIiHc4PfvAAP/vZQ/zxj7/H4/Hy4IOP4vF4+OpXv8Htt99CQkICGRkZ3HPPDwkGg/zoR/fy8stLMU2Te+/9Uayb3yqODGuA5EQPtfWh6IKmwUVEOoxzzx3LueeOBeA3v1l0wvrLLrucyy67/ITnf/vb/2v3trUXR37PGqJT4bX1OsBMRETin3PD2u+htu7YAWYKaxERiV/ODWuf5/g+a02Di4hIHHNuWCd6qNbIWkREOgDnhrXfQyAYDWt9dUtEROKZc8Pa58GyG641qq9uiYhIHHNsWCf53NgcuzC49lmLiEj8cmxYp/i8WGhkLSIi8c+xYZ3s8xwPa+2zFhHpdB588Ie8/PLSz9xm0qSxZ6k1Z8axZzBrMg2usBYROSWhne8S+ug/7fLankFTcA+d3C6v3dE5NqybHGCmsBYR6RDuvvtOpk27nIsuuhSA+fO/yH//9x0sWvRb6uvrqK6u4bbb7mDy5Ita9bp1dXX89Kc/IT9/J6ZpMmfOF7niilnk5+/i0UcfJBKJ4PV6ufvu+8nL68bDD/+IPXt2A3DNNZ/nyiuvaetSm3BsWCd4XJguFwC2TooiInJKPOdMxHPOxJi9//TpM3j11Ve46KJL2bevkGAwyD//+Sx33XUfvXv3Yf369/nVrx5rdVj/6U+/Jy0tjb/8ZQlHjhxhwYIvMXDgIJYsWcycOV/kkks+xyuvvMjWrVsoKyulqqqKJ55YTFlZKb/73f8qrNuLYRiN17XWyFpEpGOYMGESCxc+ytGjtbz22nKmT7+CG264iVWr3uaNN15j69YtBAKBVr/u+vXruOuu+wBIT09n8uQpbNiwngsvnMgvfvEoa9asYuLEKUycOJmammoKCwv4f//vVsaPn8gtt3y7rcs8gWMPMAPwJ3qjDxTWIiIdgsfjYeLEybzzzn9YufJVLrvscm65ZQHbt29l0KDBzJs3/7RmSz99cizbhkgkzMUXf44//elphgwZxpIli3nssYdJS0vnL39ZwnXX3UhhYQHz53+R6urqtiqxWY4O6yRfQ1jrq1siIh3G9Okz+NvfniYtLR2/38++fQV89avfZPz4ibz99ltYp/E3/dxzz+ell54H4MiRI7z99puMGTOWH/zgf9i+fRtXX30dX/vaN/noox28885bPPDAD5gwYRK33/5dfD4fJSXFbV1mE46dBgdI9nuxa9HIWkSkAxk5cjQ1NTVcffX1pKamMWvWVcydewNut5tzzz2furq6Vk+Ff+UrX+PnP/8p8+bdiGVZzJs3n0GDBjN37lf46U9/wpNP/gG328N3v3sX55wzmDffXMncuTfg9XqZPn0G/fsPaKdqoww7Do6uKi+vwbLaphlZWSmUlp7adMRTy3ZwZcFP8Y2ZScK469vk/WOlNXV3Fk6sGZxZtxNrhvipu6iogNzc3mflvdxuk3C48wygmvvdmaZBZmZyq17H4SPr6IlRbCsc66aIiEg7qK+v4xvfmN/suq997RtMmjT1LLfo9Dg7rBM9HIxk0Hv/NhJj3RgRkThm2zaGYbS8YZxJSEjkyScXx+S923Li2uEHmHlYX98XDhcSOXIw1s0REYlLbreX2toqnZOiFWzbpra2Crfb2yav5+yRtc/DhmAfrklaTzh/Da6x7fuldhGRjigjI4uKilJqao60+3uZpnlaR3PHI7fbS0ZGVtu8Vpu8SgeV7PdQZfsJZAzAzF+N97yrO+Q0j4hIe3K53HTtmndW3iteDqqLN46eBk9uOINZeZeR2FUlWKUfx7hFIiIiJ1JYA4f8g8DlJpS/OsYtEhEROZGjw9qX4MZlGhTXgLvnKMK712B3kn0lIiLSeTg6rE3DYPTArry9+RDBHmOxA1VEDm6PdbNERESacHRYA1x/UX/CEYsXClLA4yOU/16smyQiItKE48M6J8PPxed2583NJQTzRhH+eB12OBjrZomIiDRyfFgDXDmxL4leN68f7gahAOHCTbFukoiISCOFNdGjwmdP6MPyQj8RbwphTYWLiEgcUVg3uPS87mSm+dkY6kt43ybs+tpYN0lERARQWDfyuF1cN7U/bx7pAZEw4b0fxLpJIiIiwCmG9dKlS5kxYwbTpk3jmWeeOWH91q1bue6667jyyiv5xje+QVVVVZs39GwYNyQbV1ZfDtup1O9cFevmiIiIAKcQ1sXFxSxcuJDFixfz3HPP8eyzz5Kfn99kmwcffJDbbruNF154gb59+/LHP/6x3RrcngzD4MZLB7K2rg/Woe1YR9v/pPUiIiItaTGsV61axfjx40lPT8fv9zN9+nSWLVvWZBvLsqitje7jDQQCJCZ23KtDn9MznUDeuRhA9XaNrkVEJPZavOpWSUkJWVnHL/GVnZ3N5s2bm2xz1113MX/+fB566CF8Ph9LlixpVSMyM5NbtX1LsrJSzujnv3DDxez67QukbH2HAZff2Eatan9nWndH5MSawZl1O7FmcGbdTqy5JS2GtWVZTS4badt2k+W6ujruuecennzySUaOHMkTTzzB97//fRYtWnTKjSgvr8Gy2uai5m1xeTUPUJU9hp6HX2fz+5vJ69O3TdrWnpx4WTkn1gzOrNuJNYMz63ZCzaZptHqQ2uI0eG5uLqWlpY3LpaWlZGdnNy7v3LmThIQERo4cCcCNN97I2rVrW9WIeDRs6jQsG3b+Z3msmyIiIg7XYlhPmDCB1atXc/jwYQKBACtWrGDKlCmN63v37k1RURF79uwB4PXXX2fEiBHt1+KzJCUrh6rkPnSv3ca2j8tj3RwREXGwFsM6JyeHO+64g3nz5nH11Vcza9YsRo4cyYIFC9iyZQtpaWk8/PDD3H777cyePZt//vOfPPTQQ2ej7e2u6+gp5LiqeHPlaiy7babpRUREWsuw7dinULztsz7Grq+l6qnbeOvoIDIvmcvEEXlt8rrtwQn7eT7NiTWDM+t2Ys3gzLqdUHO77LN2MiMhCU+vkZzvK+Df/8mnPhSJdZNERMSBFNYt8Ay4kGRq6RLYx6vv74t1c0RExIEU1i1w9x4FnkSmZx3kpfcKqKzVta5FROTsUli3wHAn4O5zLgOt3djhEC+883GsmyQiIg6jsD4FngHjMUIBPj8wwFsbD3KwTJfPFBGRs0dhfQpc3YdhJKZwvm8vCV6TZ17dSThixbpZIiLiEArrU2CYLtz9x2Ec2MwXpvRke0EFi17YSsRSYIuISPtTWJ8iz4ALIRLigpSDzLlkAOs+KuUPS7cpsEVEpN21eCEPiTKz+2OkdCWU/x7TZkwiYtn8/c3duEyDr84cimkaLb+IiIjIaVBYnyLDMPD0H09w00tYRyu5YnxvwpbNv/+zB9M0+MqMIZiGAltERNqepsFbwT3gQrBtwnuiVxWbPaEPV07sw7tbinhq2Uc6f7iIiLQLjaxbwdWlO2aXngQ3vYKZkoWr1yiumtSXiGXz0uoCXKbBF6ed0+R63yIiImdKYd1KCZPmUvfmHwks/yWuvEEkXHAD107pR8SyWbamEJfL4AuXDlRgi4hIm1FYt5I79xySbniQ0PY3Ca5/nqPPPYC771iuO/86IpGevLpuH27T5PMX91dgi4hIm1BYnwbDdOMd9jk8AycS3LyM4OZlhPdu4JohU/GMGs7La6Mj7Gun9FNgi4jIGVNYnwHD6yNh7DV4hl5M8IMXCG1/k+nuVfTqO5Y/rQ7hMg2untwv1s0UEZEOTkeDtwHTn07ipHkkff5B3D2GM6TyHR7o+gLl65bz4jv5sW6eiIh0cBpZtyEzPQ/fZbcSKc7HXLOEG6w1FG/Zzns1VzD+8iti3TwREemgNLJuB66cAfhn/w8Jl32bxAQPwwqf5Y3nntOpSUVE5LQorNuJYRh4+44hb94jVCR0Y2jRS/zf396mti4U66aJiEgHo7BuZy63hx7X3EGCx+TCIy/y4FPvc6hc18MWEZFTp7A+C8zULJKmfoW+7lLGh9fyk6fW8+HH5bFuloiIdBAK67PEM2A87nMmcZF7E2NSylm4ZBOvvr8PW+cTFxGRFiisz6LEiV/ETMvmxsS3GN8/ib++vosnX9lBOKIDz0RE5OQU1meR4UnEd+m3oK6am1NWM+vC3ry9+RA/++sGqmqDsW6eiIjEKYX1Webq2oeEcTcQKdjIrK4f840rh7G3qJoH/ryOfSU1sW6eiIjEIYV1DHhGXIar50jq3/sbY3OC3HXzuUQsi4f+sp4PdpbGunkiIhJnFNYxYBgmiRd9DcPrp27l7+iTlcB9Xzqfbl2T+M2/trB01V4deCYiIo0U1jFi+lJJvPjrWBUHqV/9VzJSEvj+TWMYPyyHf/9nD7/51xZqAjqBioiIKKxjyt1jON5RMwhtf5PQnvfxelwsmDWUOZcOZPPucu7/01p2FFTEupkiIhJjCusY8469FjOrL3X/eQKrphzDMJh2fk/unTcWr8fFz/66gX/9Z4/OKy4i4mAK6xgzXO7o17lsi7qVv8e2IgD0zk3h/i+PZeKIPF5ctZefPrOBsiOBGLdWRERiQWEdB8zUbBInzSNStJPgBy80Pp/odTN/5hC+fuVQDpTVcP8T7/P+jpIYtlRERGJBYR0nPAMn4B44keCGFwgf+qjJuvFDc7n/K+PIy/Tzu+c+5MlXtlMfjMSopSIicrYprONI4sQvYqRkU7fy91hHipqsy073cdfN5zLzwt68vekQP/7z+xQWV8eopSIicjYprOOI4fXhu/Rb2HVV1C65i9old1O/Zgnhop3YloXbZXLd1P58Z85ojtaH+clT63htnS4GIiLS2blj3QBpypXVh6QbHiG89wPChRsJbl4Om17GSEjG1Wsk7t6jGdJjBD+aP44/vbSdxa/tYuvHh7lz3vmxbrqIiLQThXUcMlO64h0xDe+IadjBo4T3fUi4YAPhwk2Ed60C04U7bzDfGjyKtd1y+cuqcm752UpuvGQAFwzJwTCMWJcgIiJtSGEd5wyvH0//cXj6j8O2IkSK8wkXbCRSsIHg6sWMBkb0yOPN4DAWvRBk1YdFzJs2iK7pvlg3XURE2ojCugMxTBfuvEG48wbB+BuxKosIF2wktHsNl1a/Rt/RV/H/bXNx7x/XcPWkflx2fg9cpg5LEBHp6PSXvAMz03Lxjrwc/5X34Os3hn6FL/DQZSZDemWw5I18fvLn9RQU6YhxEZGOTmHdCRguNznX34kr7xzc7z3Jf51v8a2rh1NRU88Df17HkpX5+l62iEgHprDuJExPAr7pt2N27UXd648zJqWMBxdcwKSReSxbW8h9f1zDh3vKY91MERE5DQrrTsTw+vBf8R3M1FwCy39FYmUBX75iMN+/aQxul8kvlmxi0dKtVNUGY91UERFpBYV1J2MkJuOb+V0MfzpHX/kFkbICBvXK4Efzx3HlxD68v72Ee/7wHm9vPoilk6mIiHQIpxTWS5cuZcaMGUybNo1nnnnmhPV79uxh7ty5XHnllXz1q1+lsrKyzRsqp870p+OfeSeGJ5HAy48ROXIQj9vk6sn9+OH8ceR1TeKJl3fwwJ/X8VGhrpctIhLvWgzr4uJiFi5cyOLFi3nuued49tlnyc/Pb1xv2zbf+ta3WLBgAS+88AJDhgxh0aJF7dpoaZmZ0hX/zO+BYRB46WdY1aUAdO+axF03n8uCWUOpqg3y08Ub+N9/bqbo8NEYt1hERE6mxbBetWoV48ePJz09Hb/fz/Tp01m2bFnj+q1bt+L3+5kyZQoA3/zmN7n55pvbr8Vyysz0XHwz7sQOBzn64qNYtdFRtGkYXDg8l4e/Pp5rp/RjW0EF9/3fGp55dSc1gVCMWy0iIp/W4klRSkpKyMrKalzOzs5m8+bNjcuFhYV07dqVu+++m+3bt9OvXz/uu+++VjUiMzO5Vdu3JCsrpU1fr6Notu6sodSl3MuhxT8iuPzndJv7AC5/auPqr1yVztWXDGTx8o9Y8d5e3ttaxA2fG8TsyX3xuF1nsfWnR33tHE6sGZxZtxNrbkmLYW1ZVpNzTdu23WQ5HA6zdu1ann76aUaMGMEvf/lLHnnkER555JFTbkR5eQ2W1TYHO2VlpVBa6rwTgXxm3d48Eqd9m8Arv2DfX36If9b3Mbz+JpvcMLUfE4dms+SN3Tzx4laWvr2bz188gLGDsuL2XOPqa+dwYs3gzLqdULNpGq0epLY4DZ6bm0tpaWnjcmlpKdnZ2Y3LWVlZ9O7dmxEjRgAwa9asJiNviQ/ubkPwXXYrVvl+Aq8sxA7Vn7BN96xk7rhhFN+5cTSJXhe/e+5DHnp6PfkHdMCgiEgstRjWEyZMYPXq1Rw+fJhAIMCKFSsa908DjBkzhsOHD7Njxw4AVq5cybBhw9qvxXLa3L1GkXjpN4iU5BNYtpBIxYFmtxvWtws//Mo4vnzFYMqO1PHQX9bzu+c+1EFoIiIx0uI0eE5ODnfccQfz5s0jFApx/fXXM3LkSBYsWMBtt93GiBEjePzxx7n33nsJBALk5uby6KOPno22y2nw9BsH4RB17/yZo3+/F3efc/GOmY0rq0+T7UzTYMqobowbks2yNYUsW1vIuo9KGD80l9kT+5Dbxd/8G4iISJszbDv2Z8bQPusz19q6rbpqQh++SvDD1yB4FFeP4XjHzI5e0asZlbVBlq0p4I0PDhCKWHER2upr53BizeDMup1Q8+nss1ZYdxKnW7cdDBDctpLQluXYgSpcuefgHT0LV88RzR5YFk+hrb52DifWDM6s2wk1n05Y63rWDmd4fSSMnol3+OcI7fgPwU2vEFj2C8zM3njHzMLd9zwM4/ihDWlJXm68ZCCXX9C7MbTf21YUFyNtEZHOSmEtABjuBLzDL8Mz5GLCu1ZRv+kl6l57HDM9D+/ombgHjMcwj/9zUWiLiJw9mgbvJNq6btuyCH/8PsENL2Id3oeRnIl32KW4B03GTDzxhAXNTY/PmtCbvMykNmvTp6mvncOJNYMz63ZCzZoGlzZjmCae/hfg7jeOSOEmgptepn7NEurf/xfufufjGXoJrpwBjfu1mxtpr95axMj+mUw7vydDemfE7clVRETincJaPpNhGLh7j8bdezSRwwcIbV9JaOcqwvmrMTN64Bl6EZ6BEzG8PuB4aF9xQW/e2HCANz7Yz2N/20iPrGSmnd+TC4bm4HHryqwiIq2hafBO4mzWbYfqCe1+j9C2lVhlBeBOwDNgfHS03bV3k21D4QjvbS1mxbp9HCitJTXJyyXndueiMd1J9XvPqB3qa+dwYs3gzLqdULOmweWsMDwJeAdPxTt4KpHSjwltW0lo12pCO97CzOqHd+jFuPuPw3An4HG7mDyqG5NG5rGtoIIVa/fx3Nsf89LqAi4clstl5/eke9f2268tItIZaGTdScS6bru+ltCuVYS2vYF15CB4/XhHTMc7egaGy9Nk24Nltby6bh+rPiwiFLYY3q8L087vybA+XVq1XzvWNceKE+t2Ys3gzLqdULNOioIzOro58VK3bdtEinYS2rKC8N71mGm5JEz+Eu5uQ07YtvpokDc3HGDlBweorA2Sl+ln8shuXDg8l7SklqfI46Xms82JdTuxZnBm3U6oWWGNMzq6OfFYd3jfFureeQq7uhT3OZNJHH8jRuKJ/0BDYYu124t5c+MBdh+owmUajOyfyaQReYzon4nb1fwBafFY89ngxLqdWDM4s24n1Kx91hJX3D1HkPT5nxD84AWCm5ZRW7iRhPFzcA+c0GS62+M2mTgij4kj8jhYVsu7Ww6x6sMiNuwqI9Xv4cLhuUwa2U37tkXEsTSy7iTive7I4X3Uvf1nrOJ8XN2GkDj5S5hpuSff3rLYsucw72w+xKb8MiKWTb9uqUwakce4ITn4E91xX3N7cWLdTqwZnFm3E2rWNDjO6OjmdIS6bdsitP1N6tf+HSIhvGNm4x114gFon1ZVG+S9rUW8veUQB0pr8bhNzhuUxaxJ/clNS8A0nXWylY7Q123NiTWDM+t2Qs2aBpe4Zhgm3qGX4O5zLvWrFhNc92/C+WuiB6Cd5NKcAKlJXqaN68Vl5/dkb1E172w+xHvbinlvazFpyV7GDc5h3NBs+uWl6ixpItIpaWTdSXTEusOFm6IHoNWU4xk8hYRxNzR7AFpzgqEIe4preXXNXrbsKSccsclKT2TckBwuGJpDj6zWfWrtSDpiX58pJ9YMzqzbCTVrZC0dirvXKJI+/xD1658jtGU5oZ2rMLP64MoZgCt3IK6cgZi+1GZ/1utxMXlMdwb3SOVoXYj1O0tZu62Yl98r4KXVBXTvmsS4oTlcMCSb7AxdAUxEOjaNrDuJjl53pHwfoV3vEinOxyrdC1YYACM15xPhPQAzo1vj9bWbq7mqNsj7O0pYs72Y/P2VAPTNS+GCITmcPySHjJSEs1pXe+jofX06nFgzOLNuJ9SsA8xwRkc3pzPVbYeDWGUFRIp3ESnOJ1K0C7uuoTavLxreOQPIHDSKal+PJtfZ/qTyyjrW7ihmzbZiCotrMIB+3VIZ2T+Tkf270isnuUPu4+5MfX2qnFgzOLNuJ9SssMYZHd2czly3bdvYVSWNwR0pzseqOADYkJCEp+95uPuNw9Vt8EmD+1B5Le9vL2HT7jI+PhT9PaUlexnZLxrcQ/tk4EvoGHuFOnNfn4wTawZn1u2EmhXWOKOjm+O0uu36WpJqCyjf+Dbhgg0QqsNISMbdJLhdzf5sZW2QD/eUs2l3OVs/LidQH8FlGgzqlR4N7wFdye0Sv/u5ndbX4MyawZl1O6FmhTXO6OjmOLHuYzXb4SDh/VsI73mfcMHGaHAnpuDucx7u/uNw5Q06aXCHIxa7D1SyaXc5m3eXc7CsFoDsDB8j+2cyol8mA3ukkeiNn1G3k/vaaZxYtxNq1tHg4kiG24unz3l4+pwXDe59WwjvWUsofzWhHW9Gg/vYiPtTwe12mQzqlcGgXhnccPEAyo4E2LwnGtxvbTzIa+v24zIN+nZLZUivDAb3zmBA91Q87ubDX0SkPWhk3Uk4se6WarbD9YQLN0dH3IUbIRwErw93t6G4egzH3XM4ZkrWSX++PhQh/0AlOwoq2F5QwceHqrDtaMAP7JHG4N4ZDOmVQZ+8lJNebKQ9qK+dw4l1O6FmjaxFPsFwJ+Dpdz6efuc3Bndk3xbC+z8kvHc99YCRlou7xzDcPUZE93N7Eht/PsHjYlifLgzr0wWAo3Vhdu4/wo6CCnYUVPDv/+zh3w3bndMzncG90xncK4NeOcm4zLMX3iLS+SmsxRGaBLdtYx05RGT/FsL7txLa8Tahra+D6cIsuMwqAAAZTklEQVSVe0501N1jOGZmz8bvdAP4E92MHtCV0QO6AtHrcX9UeITthdHw/vsb5UA0vPt1S2VA9zQG9kijX7c0/In6ryYip09/QcRxDMPAldENV0Y3vCOmY4eDRIp2Ed7/IZH9HxJc+3eCa/+O4UvFzOqL6U/D8DXc/KkYvjRMXxrJ/lTOG5TF2MHZABypqeejwiPk769k14EjvLh6L7YNBtA9K5mBPdIY0CONgd3TyExL7JDf8RaR2FBYi+MZbm/DVPgw4Easo0eI7P+Q8L4PsSoOEC79OHpSluYO73C5G4I8Fa8vjVEpmZw39FxcnzuPupDNnkNV5O+vJH//EVZvLeKNDQcASE/2MqBHOgO7p9EnL4UeWckd5nveInL26a+DyKeY/nTMcybhOWdS43O2ZWHX12AHKrGPVmIHqrADlVhHqxof27XlhA5uJ7T1dQxfKu6+5zOo/ziGThyIYfTFsmz2l9awa38l+QeiAb5uRwkQHX1nZ/jomZNCr+xkeuWk0CsnmfTkjn96VBE5cwprkVNgmCaGLxV8qdCl50m3i351bDPh3WsIffQ2oW2vYyRl4O43Dk//C+iZ3ZdeOSlcel4PACqq6ykormZfcTWFxTUUFFU1BjhAapKHYZkRBqVU0dN1mPRQMXb3XthDr8D0p7d73SISHxTWIm3IcHvx9B2Lp+9Y7FAd4YKN0eDe+hqhLcsxUrLw9L8Ad/8LMLv0ICMlgYyUBEYP6IptWViVhwgc2kP1vnys8gISaw/irQlCDURsg+JIGkbRDiIfrGRb0jgqekwhNzud3C5+crv48Xr0/W+RzkhhLdJODE8ingHj8QwYj11fS3jvB4R2ryG46WWCG1/ETM/D3ec87GCASHkBVnlh9LvgQLLLg5nZE1evSZhde0NGT4oi6ZSU1bGvopjsPS8xuvZdjmzfwEvrR/N+sB9g0jU9kbzMJPIy/U3uk32e2P4yROSMKKxFzgIjIQnPoMl4Bk3GClQR/ngd4d1rCG58CTwJuDJ74Rk8FVfXPphde2Om551witReQK9ukJU1iNLSMYSLdmKs+is3l63iuqS9bM+8lG31qRwqP8r2ggpCYavxZ5MS3eR28ZOd4Se3i4+cLn5yMvzkdPHF1alURaR5+l8qcpaZvlS8Qy/BO/QS7GAAPAlNvs99qty555B0zX2Ed6/FWPt3xhT+hbG9RpFwzQ0Yad0or6rjUHkth8qPUlwRoPjwUT7aV8HqrUVNXict2Utuhj8a4F185GT46ZqWSNe0RPyJLY/IbcsCw9BX0UTakcJaJIYMr+/Mft4w8QwYj7vPuYS2vkb9hqUc/ce9eAZPJfO8a8jq35WR/Zv+TH0oQklDeBdXHKXo8FGKDwfYsKuU6qOhJtv6EtxkpkaDO7MhwDNTE+maDJlVO3Ed2Ehk32bMlGy8512Ju+/Y0/rgISKfTWEt0gkYbi/eUTNwD5pMcP3zhLa9QSj/PbyjZuAdOR3DffwrYAkeFz2zk+mZfeK5iWvrQpRUBCirrKO8so6yygDllXWUVgbYW1jEQApI8RaQ6jkIhkWF5eNjsz+9KovIeO23VHuzKOpxKXaP0aSn+EhPTiAt2XtWz53uVLYVBsvCcHtj8/62jX30CFZVCVZlEXZVCVZlMXh8eEdOw/UZ36KQlulCHp2EE+t2Ys1wanVbR4qoX/t3wnvXg9eHmdEdMy0PMz0PMz0XV3oeRmoWhvnZn9etumrCez8g/PE6Ige2gRUhkphOVeZwDvgHszecRVlVPZXVAbof3cFk4wNyXJUcDKezLDCKzaFe2Bik+D2kJSWQnuIlPTmB9GQvaUkJpCZ5SWu4pSZ5SfS6mp1Od1Jf27YNoTrsuhrSfBYVRcXYdTXY9bUN9zXYdbUN98eXCQUAAyM1G1eXHpgNN1eXnhip2RhtcL76kwWyVVmMVVXceIAkAKYLMyUL6+gRCNXh6jWahDGzcOUM+Mz3cEJf63rWOKOjm+PEup1YM7Su7vChjwjnr8Y6cgjrSBF2oPL4SsOFmZoVPZgtrSHA0/MwfamED2yNBvTBHWBbGClZuPueh6ff+ZhZfU861R0JR6j5aDX2pqW4aooJ+HLYnTmFXUY/jtSGOFJTz5Gaeiprg82eEM7rNqMBnuwl1e8lLTmBtCQv3XNSIGKR7POQ7PeQ4vOQ5PN0mBG7bdvRYKs9jF1XHQ3ZQHXD44bluqqG+2qIhE/+Yl4/RmIyRkJyw31S9D4xGWwb6/D+6K2q+PhZ91xezC7dMTN6HA/yzJ6YvlRs24p+EAg0nOCn8aQ/DSf++eTzdVVN29YQyEZaDmZqDmZaNmZaLmZqDkZyFwzThV1fS3DrawS3rID6WlzdhuAdPQtX96GO/WCmsMYZHd0cJ9btxJrhzOq2g0exjhQ1hHfDrbIoOl1pNQ0IIy0XT9+xuPuNxczs3aoDyGzLIrxnLcEPnsc6cggzowfe867C3fc8DMPEsmxqAiGqaoNU1gYb7ytr66mqDRKoqcJVW4a37jDJ4SOkmAEqLT/lVjKHI8mUWckctRPwJUSDO9nvIdnX9HGSz0NyYvQ+KdEdfS7Rg9djnpWD4WzbxiovJLznfUIfr8OuLDpxI68fIzGlIWxTMBJTMH3Hl9Oys6iud0WDODEZw5t0yiNkOxzEqjiIdXgfkWMBfngfdqDqE+/vg1AQ7MiJL2BETwR0/BY9ra6ZktkQyNkYyZknfGvhpO0J1RHa/hbBza9gHz2CmdUX75hZuHuPafLhr73+X9uRcHTGIlQHLk+0phgdFKmwRn/AncSJNUP71G1bEeyacqwjB7FqDuPKPQczo/sZ/zE7aWj3Off4dGpVCXZVafRxdfSe+tqmL+RJhFBdk6fCppdadzrVRioVdgpl4SSKQn4O1PkorvcTpPkj2d0ukySfOxrkie5omDcEuj/BjT/Rgz/Bja9x+fh9gqf5afrGem0bq2zv8YCuKgHDxNVtMO6+Y6OzGI3BnNziboj26GsrUHV89F1ZjOH1HQ9kfxpGYmr0gjUJSe1ysKAdCRHa+S7BjS9hV5diZnTHO3om7v4XYJiulq9Tb1nYteVYDf9m7Oqy6MxAqC4axuF67GAdhOuwQ/WNz2N96gOJy4uZ0hUjpWv0Pjl6f+w5IzGl3cJcYY3+gDuJE2uGjln3p0MbwwT7+PfAMczoH83UbMyUrOioLTU7upyaRXa3LEoOlGBVl2FVl2JXl0YfN/yxtqpLm+4vBWyXB9uVgOVKIGx6CZtegniptz3U2W4CETdHIy5qwy6qQyYVIQ+HQ4lUWn6qLB+hZo6/NQ2jMbx9iW58Xhc+r4vuZil9g7vofnQHvnAllmFSmzaAQO4orG4jSUhJJzHBTWLD9m7XqY3uO2JfnyrbikT/TWx4CatiP0ZKFt5RM8ibeDllJVVY1Z/6INfwYc6uLmsavKYLIyEJ3AkY3kQMd2L065CeRPAkYngSMDy+xucMd0J01qGmLPraNWVY1WUnfkB0extD3JU3iITRM9usdoU1nfsf92dxYt1OrBk6dt3HQtsqLzwezqcwndriaMu2o/t+q0ujI66asoaDruqOj7iOPQ4Gjj/+1Gj9kyyPn7A3lZAnhTpXMkddSdSQRLXt50jERygYonvdLvqF80mlhrBtsjOcx8b63mwJ9eSoffKLsLhMgwSPi8QEF4neaIhHb+5PPO8iM8NPJBQh0eNqDPvG7Roetyb845FtW0QKNlG/YSlW6R4Mtxf7Ux+88Pob/61EP8hlHX/sz2ibg+eCAazqsuiHv4YAP/ZB0EhMxj/ze2f8HscorOnYf8jOhBPrdmLN4My6220/pm1Bw1SpXV+DXXskOj1/9MgnHlc0PK48cd+u6cbVYziefufj7j0avH6CYYu6+jCBYIRAfbjhFn1cH4pQFwxTF4xQV9/wOBSJLjc8Xx88vhyOnNrfxU+Gv9ftIsHjwusxSfA0fextuCV4zIb7T95MvN5PPed14XWfvX38kYPb8RRvoc74VDgnJLX7+59NpxPW+p61iDiWYZjg9UVPTpOU8dlXVLOt6NHax4LcCuHuNvSEE9scC7q0NmhfekYS+w8eOR7wx0K9/vjj+mNhXx+hLhQmGLKoD0UIhiLU1oWpqKmnPhghGG54PhihNUMjA5oGvPfYBwLzePi7G4LeHf1gcOyDgNdt4vGYeFwmHreJ2xW9edyfeu7Ycs4gMkeNo6yspg1+e52LwlpE5BQYxicuk5rZ66y8p8dtRr+u1oYXYrFtm3DEoj5kEQxFqD92Cx57bDU+bm59MGRRH46GfmVtkGDjdhbBcHT9mXK7zCZB7/W48LgbnnObeN3R5WPPud1NPwB43C48LuMTzzds7zKi69wm7mPrP/lhwW3iMuPz1LmnFNZLly7ld7/7HeFwmC996UvcfPPNzW735ptv8uMf/5iVK1e2aSNFRKRtGMaxwHJBO1yNzbJtQuHjHwRCYSt6i1iEwxbhSHR9OPKJ5xsehyMWHq+HyqoAwbBFKBz9+eCx12iYHagJhBreo2GbT7zmmTIAt/uTMwAGPbKS+fbnR535L+cMtBjWxcXFLFy4kH/96194vV7mzJnDBRdcwIABTc9CU1ZWxk9/+tN2a6iIiMQ/0zAadwWknMbPn8nxCZZtE2n8EGA3hv0nPywEw9H7ph8S7CbbfPo+My3xtNrTlloM61WrVjF+/HjS09MBmD59OsuWLePWW29tst29997Lrbfeys9//vP2aamIiMhnMA0D89isQSfT4vHuJSUlZGVlNS5nZ2dTXFzcZJunnnqKoUOHMmpUbKcJREREOqMWR9aWZTXZ2W7bdpPlnTt3smLFCp588kmKipo5nd4paO0h7C3JyjqdyZeOz4l1O7FmcGbdTqwZnFm3E2tuSYthnZuby7p16xqXS0tLyc7OblxetmwZpaWlXHfddYRCIUpKSrjppptYvHjxKTdC37M+c06s24k1gzPrdmLN4My6nVDz6XzPusVp8AkTJrB69WoOHz5MIBBgxYoVTJkypXH9bbfdxvLly3n++edZtGgR2dnZrQpqERER+WwthnVOTg533HEH8+bN4+qrr2bWrFmMHDmSBQsWsGXLlrPRRhEREUfT6UY7CSfW7cSawZl1O7FmcGbdTqi5XabBRUREJLYU1iIiInFOYS0iIhLnFNYiIiJxTmEtIiIS5xTWIiIicU5hLSIiEucU1iIiInFOYS0iIhLnFNYiIiJxTmEtIiIS5xTWIiIicU5hLSIiEucU1iIiInFOYS0iIhLnFNYiIiJxTmEtIiIS5xTWIiIicU5hLSIiEucU1iIiInFOYS0iIhLnFNYiIiJxTmEtIiIS5xTWIiIicU5hLSIiEucU1iIiInFOYS0iIhLnFNYiIiJxTmEtIiIS5xTWIiIicU5hLSIiEucU1iIiInFOYS0iIhLnFNYiIiJxTmEtIiIS5xTWIiIicU5hLSIiEucU1iIiInFOYS0iIhLnFNYiIiJxTmEtIiIS5xTWIiIicU5hLSIiEucU1iIiInFOYS0iIhLnFNYiIiJx7pTCeunSpcyYMYNp06bxzDPPnLD+tdde46qrruLKK6/kv/7rv6isrGzzhoqIiDhVi2FdXFzMwoULWbx4Mc899xzPPvss+fn5jetramr44Q9/yKJFi3jhhRcYNGgQ//u//9uujRYREXGSFsN61apVjB8/nvT0dPx+P9OnT2fZsmWN60OhEPfffz85OTkADBo0iEOHDrVfi0VERBzG3dIGJSUlZGVlNS5nZ2ezefPmxuWMjAwuu+wyAOrq6li0aBFz585tVSMyM5NbtX1LsrJS2vT1Ogon1u3EmsGZdTuxZnBm3U6suSUthrVlWRiG0bhs23aT5WOqq6u55ZZbGDx4MNdcc02rGlFeXoNl2a36mZPJykqhtLS6TV6rI3Fi3U6sGZxZtxNrBmfW7YSaTdNo9SC1xWnw3NxcSktLG5dLS0vJzs5usk1JSQk33XQTgwYN4sEHH2xVA0REROSztRjWEyZMYPXq1Rw+fJhAIMCKFSuYMmVK4/pIJMI3v/lNrrjiCu65555mR90iIiJy+lqcBs/JyeGOO+5g3rx5hEIhrr/+ekaOHMmCBQu47bbbKCoqYtu2bUQiEZYvXw7A8OHDNcIWERFpI4Zt222zs/gMaJ/1mXNi3U6sGZxZtxNrBmfW7YSa22WftYiIiMSWwlpERCTOKaxFRETinMJaREQkzimsRURE4pzCWkREJM4prEVEROKcwlpERCTOKaxFRETinMJaREQkzimsRURE4pzCWkREJM4prEVEROKcwlpERCTOKaxFRETinMJaREQkzimsRURE4pzCWkREJM4prEVEROKcwlpERCTOKaxFRETinMJaREQkzimsRURE4pzCWkREJM4prEVEROKcwlpERCTOKaxFRETinMJaREQkzimsRURE4pzCWkREJM4prEVEROKcwlpERCTOKaxFRETinMJaREQkzimsRURE4pzCWkREJM4prEVEROKcwlpERCTOKaxFRETinMJaREQkzimsRURE4pzCWkREJM4prEVEROKcwlpERCTOKaxFRETi3CmF9dKlS5kxYwbTpk3jmWeeOWH99u3bufbaa5k+fTr33HMP4XC4zRsqIiLiVC2GdXFxMQsXLmTx4sU899xzPPvss+Tn5zfZ5s477+QHP/gBy5cvx7ZtlixZ0m4NFhERcRp3SxusWrWK8ePHk56eDsD06dNZtmwZt956KwAHDhygrq6O0aNHA3Dttdfy61//mptuuumUG2Gaxum0/ay9XkfhxLqdWDM4s24n1gzOrLuz13w69bUY1iUlJWRlZTUuZ2dns3nz5pOuz8rKori4uFWNyMhIatX2LcnMTG7T1+sonFi3E2sGZ9btxJrBmXU7seaWtDgNblkWhnH8U4Bt202WW1ovIiIiZ6bFsM7NzaW0tLRxubS0lOzs7JOuLysra7JeREREzkyLYT1hwgRWr17N4cOHCQQCrFixgilTpjSu7969OwkJCaxfvx6A559/vsl6EREROTOGbdt2SxstXbqU3//+94RCIa6//noWLFjAggULuO222xgxYgQ7duzg3nvvpaamhmHDhvHwww/j9XrPRvtFREQ6vVMKaxEREYkdncFMREQkzimsRURE4pzCWkREJM4prEVEROJcpwrrli440hnNnTuXmTNnctVVV3HVVVexadOmWDepXdXU1DBr1iz2798PRE+HO3v2bKZNm8bChQtj3Lr28ema/+d//odp06Y19vmrr74a4xa2vd/85jfMnDmTmTNn8uijjwKdv6+bq9kJff2rX/2KGTNmMHPmTJ544gmg8/f1abE7iaKiIvviiy+2Kyoq7NraWnv27Nn2rl27Yt2sdmVZlj1p0iQ7FArFuilnxcaNG+1Zs2bZw4YNs/ft22cHAgF76tSpdmFhoR0Khez58+fbb775Zqyb2aY+XbNt2/asWbPs4uLiGLes/bz77rv2jTfeaNfX19vBYNCeN2+evXTp0k7d183VvGLFik7f12vWrLHnzJljh0IhOxAI2BdffLG9ffv2Tt3Xp6vTjKw/ecERv9/feMGRzmzPnj0AzJ8/nyuvvJKnn346xi1qX0uWLOH+++9vPEPe5s2b6d27Nz179sTtdjN79uxO1+efrjkQCHDw4EHuvvtuZs+eza9//Wssy4pxK9tWVlYWd911F16vF4/HQ//+/dm7d2+n7uvmaj548GCn7+tx48bx1FNP4Xa7KS8vJxKJUFVV1an7+nR1mrBu7oIjrb2gSEdTVVXFhRdeyOOPP86TTz7J3/72N959991YN6vdPPjgg4wdO7Zx2Ql9/umay8rKGD9+PA899BBLlixh3bp1/OMf/4hhC9vewIEDG6/it3fvXl555RUMw+jUfd1czZMnT+70fQ3g8Xj49a9/zcyZM7nwwgsd8f/6dHSasHbiBUXGjBnDo48+SkpKCl26dOH666/nrbfeinWzzhon9nnPnj15/PHHyc7OxufzMXfu3E7b57t27WL+/Pl873vfo2fPno7o60/W3K9fP8f09W233cbq1as5dOgQe/fudURft1anCeuWLjjSGa1bt47Vq1c3Ltu2jdvd4lVPOw0n9vlHH33E8uXLG5c7a5+vX7+eL3/5y3znO9/hmmuucURff7pmJ/T17t272b59OwA+n49p06axZs2aTt/Xp6PThHVLFxzpjKqrq3n00Uepr6+npqaGf//731x22WWxbtZZM2rUKD7++GMKCgqIRCK8+OKLnb7PbdvmoYceorKyklAoxLPPPtvp+vzQoUPccsstPPbYY8ycORPo/H3dXM1O6Ov9+/dz7733EgwGCQaDvP7668yZM6dT9/Xp6jQf03JycrjjjjuYN29e4wVHRo4cGetmtauLL76YTZs2cfXVV2NZFjfddBNjxoyJdbPOmoSEBB555BH++7//m/r6eqZOncrll18e62a1q8GDB/P1r3+dL3zhC4TDYaZNm8asWbNi3aw29cc//pH6+noeeeSRxufmzJnTqfv6ZDV39r6eOnUqmzdv5uqrr8blcjFt2jRmzpxJly5dOm1fny5dyENERCTOdZppcBERkc5KYS0iIhLnFNYiIiJxTmEtIiIS5xTWIiIicU5hLSIiEucU1iIiInFOYS0iIhLn/n+mxlKEy9YyHQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 576x360 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_learning_curves(history)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Hyperparameter search"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Try training your model multiple times, with different a learning rate each time (e.g., 1e-4, 3e-4, 1e-3, 3e-3, 3e-2), and compare the learning curves. For this, you need to create a `keras.optimizers.SGD` optimizer and specify the `learning_rate` in its constructor, then pass this `SGD` instance to the `compile()` method using the `optimizer` argument."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 11610 samples, validate on 3870 samples\n",
      "Epoch 1/100\n",
      "11610/11610 [==============================] - 1s 46us/sample - loss: 5.4083 - val_loss: 4.2292\n",
      "Epoch 2/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 3.7043 - val_loss: 3.6743\n",
      "Epoch 3/100\n",
      "11610/11610 [==============================] - 0s 33us/sample - loss: 2.6746 - val_loss: 4.1190\n",
      "Epoch 4/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 2.0397 - val_loss: 4.4212\n",
      "Epoch 5/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 1.6337 - val_loss: 4.5097\n",
      "Epoch 6/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 1.3677 - val_loss: 4.2815\n",
      "Epoch 7/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 1.1891 - val_loss: 3.9109\n",
      "Epoch 8/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 1.0654 - val_loss: 3.4672\n",
      "Epoch 9/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.9781 - val_loss: 2.9750\n",
      "Epoch 10/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.9146 - val_loss: 2.5285\n",
      "Epoch 11/100\n",
      "11610/11610 [==============================] - 0s 33us/sample - loss: 0.8676 - val_loss: 2.1245\n",
      "Epoch 12/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.8319 - val_loss: 1.7892\n",
      "Epoch 13/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.8042 - val_loss: 1.5134\n",
      "Epoch 14/100\n",
      "11610/11610 [==============================] - 0s 35us/sample - loss: 0.7823 - val_loss: 1.2938\n",
      "Epoch 15/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.7646 - val_loss: 1.1226\n",
      "Epoch 16/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.7500 - val_loss: 0.9907\n",
      "Epoch 17/100\n",
      "11610/11610 [==============================] - 0s 36us/sample - loss: 0.7377 - val_loss: 0.8902\n",
      "Epoch 18/100\n",
      "11610/11610 [==============================] - 0s 34us/sample - loss: 0.7272 - val_loss: 0.8149\n",
      "Epoch 19/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.7180 - val_loss: 0.7574\n",
      "Epoch 20/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.7098 - val_loss: 0.7162\n",
      "Epoch 21/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.7025 - val_loss: 0.6860\n",
      "Epoch 22/100\n",
      "11610/11610 [==============================] - 0s 33us/sample - loss: 0.6958 - val_loss: 0.6657\n",
      "Epoch 23/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.6897 - val_loss: 0.6500\n",
      "Epoch 24/100\n",
      "11610/11610 [==============================] - 0s 33us/sample - loss: 0.6839 - val_loss: 0.6394\n",
      "Epoch 25/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.6786 - val_loss: 0.6314\n",
      "Epoch 26/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.6734 - val_loss: 0.6258\n",
      "Epoch 27/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.6686 - val_loss: 0.6221\n",
      "Epoch 28/100\n",
      "11610/11610 [==============================] - 0s 34us/sample - loss: 0.6640 - val_loss: 0.6193\n",
      "Epoch 29/100\n",
      "11610/11610 [==============================] - 0s 33us/sample - loss: 0.6595 - val_loss: 0.6173\n",
      "Epoch 30/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.6553 - val_loss: 0.6154\n",
      "Epoch 31/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.6512 - val_loss: 0.6134\n",
      "Epoch 32/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.6472 - val_loss: 0.6118\n",
      "Epoch 33/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.6433 - val_loss: 0.6101\n",
      "Epoch 34/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.6396 - val_loss: 0.6085\n",
      "Epoch 35/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.6360 - val_loss: 0.6073\n",
      "Epoch 36/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.6325 - val_loss: 0.6056\n",
      "Epoch 37/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.6290 - val_loss: 0.6041\n",
      "Epoch 38/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.6256 - val_loss: 0.6026\n",
      "Epoch 39/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.6224 - val_loss: 0.6003\n",
      "Epoch 40/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.6192 - val_loss: 0.5986\n",
      "Epoch 41/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.6160 - val_loss: 0.5954\n",
      "Epoch 42/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.6130 - val_loss: 0.5924\n",
      "Epoch 43/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.6100 - val_loss: 0.5901\n",
      "Epoch 44/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.6070 - val_loss: 0.5885\n",
      "Epoch 45/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.6042 - val_loss: 0.5857\n",
      "Epoch 46/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.6013 - val_loss: 0.5834\n",
      "Epoch 47/100\n",
      "11610/11610 [==============================] - 0s 34us/sample - loss: 0.5986 - val_loss: 0.5801\n",
      "Epoch 48/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.5959 - val_loss: 0.5777\n",
      "Epoch 49/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.5932 - val_loss: 0.5749\n",
      "Epoch 50/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.5906 - val_loss: 0.5728\n",
      "Epoch 51/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.5881 - val_loss: 0.5694\n",
      "Epoch 52/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.5856 - val_loss: 0.5663\n",
      "Epoch 53/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.5832 - val_loss: 0.5633\n",
      "Epoch 54/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.5807 - val_loss: 0.5614\n",
      "Epoch 55/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.5784 - val_loss: 0.5595\n",
      "Epoch 56/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.5761 - val_loss: 0.5578\n",
      "Epoch 57/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.5738 - val_loss: 0.5548\n",
      "Epoch 58/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.5716 - val_loss: 0.5518\n",
      "Epoch 59/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.5694 - val_loss: 0.5495\n",
      "Epoch 60/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.5672 - val_loss: 0.5472\n",
      "Epoch 61/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.5651 - val_loss: 0.5446\n",
      "Epoch 62/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.5630 - val_loss: 0.5427\n",
      "Epoch 63/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.5609 - val_loss: 0.5390\n",
      "Epoch 64/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.5589 - val_loss: 0.5361\n",
      "Epoch 65/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.5569 - val_loss: 0.5333\n",
      "Epoch 66/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.5549 - val_loss: 0.5312\n",
      "Epoch 67/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.5530 - val_loss: 0.5285\n",
      "Epoch 68/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.5511 - val_loss: 0.5271\n",
      "Epoch 69/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.5492 - val_loss: 0.5245\n",
      "Epoch 70/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.5474 - val_loss: 0.5217\n",
      "Epoch 71/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.5456 - val_loss: 0.5199\n",
      "Epoch 72/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.5438 - val_loss: 0.5180\n",
      "Epoch 73/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.5420 - val_loss: 0.5161\n",
      "Epoch 74/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.5403 - val_loss: 0.5140\n",
      "Epoch 75/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.5386 - val_loss: 0.5117\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 76/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.5369 - val_loss: 0.5099\n",
      "Epoch 77/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.5353 - val_loss: 0.5079\n",
      "Epoch 78/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.5336 - val_loss: 0.5062\n",
      "Epoch 79/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.5320 - val_loss: 0.5043\n",
      "Epoch 80/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.5305 - val_loss: 0.5026\n",
      "Epoch 81/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.5289 - val_loss: 0.5010\n",
      "Epoch 82/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.5274 - val_loss: 0.4988\n",
      "Epoch 83/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.5259 - val_loss: 0.4967\n",
      "Epoch 84/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.5244 - val_loss: 0.4952\n",
      "Epoch 85/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.5230 - val_loss: 0.4931\n",
      "Epoch 86/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.5215 - val_loss: 0.4911\n",
      "Epoch 87/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.5201 - val_loss: 0.4896\n",
      "Epoch 88/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.5187 - val_loss: 0.4880\n",
      "Epoch 89/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.5173 - val_loss: 0.4864\n",
      "Epoch 90/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.5160 - val_loss: 0.4846\n",
      "Epoch 91/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.5146 - val_loss: 0.4829\n",
      "Epoch 92/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.5133 - val_loss: 0.4816\n",
      "Epoch 93/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.5120 - val_loss: 0.4802\n",
      "Epoch 94/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.5107 - val_loss: 0.4786\n",
      "Epoch 95/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.5095 - val_loss: 0.4772\n",
      "Epoch 96/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.5082 - val_loss: 0.4759\n",
      "Epoch 97/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.5070 - val_loss: 0.4743\n",
      "Epoch 98/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.5058 - val_loss: 0.4728\n",
      "Epoch 99/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.5046 - val_loss: 0.4715\n",
      "Epoch 100/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.5034 - val_loss: 0.4702\n",
      "Train on 11610 samples, validate on 3870 samples\n",
      "Epoch 1/100\n",
      "11610/11610 [==============================] - 0s 39us/sample - loss: 4.2273 - val_loss: 2.7904\n",
      "Epoch 2/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 2.0515 - val_loss: 2.1264\n",
      "Epoch 3/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 1.3341 - val_loss: 1.4705\n",
      "Epoch 4/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 1.0088 - val_loss: 1.0015\n",
      "Epoch 5/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.8489 - val_loss: 0.7896\n",
      "Epoch 6/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.7662 - val_loss: 0.7104\n",
      "Epoch 7/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.7200 - val_loss: 0.6770\n",
      "Epoch 8/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.6912 - val_loss: 0.6601\n",
      "Epoch 9/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.6708 - val_loss: 0.6464\n",
      "Epoch 10/100\n",
      "11610/11610 [==============================] - 0s 33us/sample - loss: 0.6548 - val_loss: 0.6345\n",
      "Epoch 11/100\n",
      "11610/11610 [==============================] - 0s 39us/sample - loss: 0.6411 - val_loss: 0.6242\n",
      "Epoch 12/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.6291 - val_loss: 0.6284\n",
      "Epoch 13/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.6183 - val_loss: 0.6130\n",
      "Epoch 14/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.6082 - val_loss: 0.5974\n",
      "Epoch 15/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.5988 - val_loss: 0.6021\n",
      "Epoch 16/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.5903 - val_loss: 0.5907\n",
      "Epoch 17/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.5823 - val_loss: 0.5736\n",
      "Epoch 18/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.5747 - val_loss: 0.5723\n",
      "Epoch 19/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.5676 - val_loss: 0.5652\n",
      "Epoch 20/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.5610 - val_loss: 0.5589\n",
      "Epoch 21/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.5547 - val_loss: 0.5514\n",
      "Epoch 22/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.5487 - val_loss: 0.5474\n",
      "Epoch 23/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.5432 - val_loss: 0.5348\n",
      "Epoch 24/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.5379 - val_loss: 0.5247\n",
      "Epoch 25/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.5329 - val_loss: 0.5162\n",
      "Epoch 26/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.5282 - val_loss: 0.5123\n",
      "Epoch 27/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.5236 - val_loss: 0.5157\n",
      "Epoch 28/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.5194 - val_loss: 0.5115\n",
      "Epoch 29/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.5154 - val_loss: 0.5064\n",
      "Epoch 30/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.5116 - val_loss: 0.4988\n",
      "Epoch 31/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.5079 - val_loss: 0.5025\n",
      "Epoch 32/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.5045 - val_loss: 0.4979\n",
      "Epoch 33/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.5012 - val_loss: 0.4884\n",
      "Epoch 34/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4981 - val_loss: 0.4813\n",
      "Epoch 35/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4951 - val_loss: 0.4756\n",
      "Epoch 36/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4922 - val_loss: 0.4775\n",
      "Epoch 37/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4895 - val_loss: 0.4719\n",
      "Epoch 38/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4869 - val_loss: 0.4719\n",
      "Epoch 39/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4844 - val_loss: 0.4638\n",
      "Epoch 40/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4821 - val_loss: 0.4610\n",
      "Epoch 41/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4798 - val_loss: 0.4614\n",
      "Epoch 42/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4777 - val_loss: 0.4602\n",
      "Epoch 43/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4756 - val_loss: 0.4555\n",
      "Epoch 44/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4735 - val_loss: 0.4536\n",
      "Epoch 45/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4716 - val_loss: 0.4511\n",
      "Epoch 46/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4697 - val_loss: 0.4465\n",
      "Epoch 47/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4679 - val_loss: 0.4472\n",
      "Epoch 48/100\n",
      "11610/11610 [==============================] - 0s 33us/sample - loss: 0.4662 - val_loss: 0.4427\n",
      "Epoch 49/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4645 - val_loss: 0.4379\n",
      "Epoch 50/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4629 - val_loss: 0.4380\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 51/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4613 - val_loss: 0.4363\n",
      "Epoch 52/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4598 - val_loss: 0.4352\n",
      "Epoch 53/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4583 - val_loss: 0.4353\n",
      "Epoch 54/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4569 - val_loss: 0.4346\n",
      "Epoch 55/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4555 - val_loss: 0.4300\n",
      "Epoch 56/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4541 - val_loss: 0.4291\n",
      "Epoch 57/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4529 - val_loss: 0.4283\n",
      "Epoch 58/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4516 - val_loss: 0.4252\n",
      "Epoch 59/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4503 - val_loss: 0.4226\n",
      "Epoch 60/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4492 - val_loss: 0.4231\n",
      "Epoch 61/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4479 - val_loss: 0.4219\n",
      "Epoch 62/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4468 - val_loss: 0.4204\n",
      "Epoch 63/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4456 - val_loss: 0.4184\n",
      "Epoch 64/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4445 - val_loss: 0.4180\n",
      "Epoch 65/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4434 - val_loss: 0.4167\n",
      "Epoch 66/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4424 - val_loss: 0.4151\n",
      "Epoch 67/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4413 - val_loss: 0.4144\n",
      "Epoch 68/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4403 - val_loss: 0.4138\n",
      "Epoch 69/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4393 - val_loss: 0.4112\n",
      "Epoch 70/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4383 - val_loss: 0.4103\n",
      "Epoch 71/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4373 - val_loss: 0.4097\n",
      "Epoch 72/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4364 - val_loss: 0.4079\n",
      "Epoch 73/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4355 - val_loss: 0.4084\n",
      "Epoch 74/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4345 - val_loss: 0.4061\n",
      "Epoch 75/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.4337 - val_loss: 0.4057\n",
      "Epoch 76/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4328 - val_loss: 0.4049\n",
      "Epoch 77/100\n",
      "11610/11610 [==============================] - 0s 33us/sample - loss: 0.4320 - val_loss: 0.4044\n",
      "Epoch 78/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.4311 - val_loss: 0.4028\n",
      "Epoch 79/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4304 - val_loss: 0.4026\n",
      "Epoch 80/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4296 - val_loss: 0.4016\n",
      "Epoch 81/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4288 - val_loss: 0.4008\n",
      "Epoch 82/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4280 - val_loss: 0.3999\n",
      "Epoch 83/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4273 - val_loss: 0.3992\n",
      "Epoch 84/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4265 - val_loss: 0.3986\n",
      "Epoch 85/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4258 - val_loss: 0.3975\n",
      "Epoch 86/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4251 - val_loss: 0.3971\n",
      "Epoch 87/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4244 - val_loss: 0.3967\n",
      "Epoch 88/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4237 - val_loss: 0.3961\n",
      "Epoch 89/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4230 - val_loss: 0.3950\n",
      "Epoch 90/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4224 - val_loss: 0.3948\n",
      "Epoch 91/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4217 - val_loss: 0.3940\n",
      "Epoch 92/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4211 - val_loss: 0.3933\n",
      "Epoch 93/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4204 - val_loss: 0.3941\n",
      "Epoch 94/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4198 - val_loss: 0.3928\n",
      "Epoch 95/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4192 - val_loss: 0.3917\n",
      "Epoch 96/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4186 - val_loss: 0.3913\n",
      "Epoch 97/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4180 - val_loss: 0.3908\n",
      "Epoch 98/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4174 - val_loss: 0.3902\n",
      "Epoch 99/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4169 - val_loss: 0.3894\n",
      "Epoch 100/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4163 - val_loss: 0.3892\n",
      "Train on 11610 samples, validate on 3870 samples\n",
      "Epoch 1/100\n",
      "11610/11610 [==============================] - 0s 38us/sample - loss: 2.4134 - val_loss: 1.3718\n",
      "Epoch 2/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.7545 - val_loss: 0.6598\n",
      "Epoch 3/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.6376 - val_loss: 0.5935\n",
      "Epoch 4/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.6023 - val_loss: 0.5817\n",
      "Epoch 5/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.5779 - val_loss: 0.5530\n",
      "Epoch 6/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.5579 - val_loss: 0.5253\n",
      "Epoch 7/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.5411 - val_loss: 0.5445\n",
      "Epoch 8/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.5265 - val_loss: 0.5110\n",
      "Epoch 9/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.5136 - val_loss: 0.5042\n",
      "Epoch 10/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.5024 - val_loss: 0.4798\n",
      "Epoch 11/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4927 - val_loss: 0.4648\n",
      "Epoch 12/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4840 - val_loss: 0.4726\n",
      "Epoch 13/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4764 - val_loss: 0.4456\n",
      "Epoch 14/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4700 - val_loss: 0.4380\n",
      "Epoch 15/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4644 - val_loss: 0.4392\n",
      "Epoch 16/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4593 - val_loss: 0.4401\n",
      "Epoch 17/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4548 - val_loss: 0.4267\n",
      "Epoch 18/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4508 - val_loss: 0.4354\n",
      "Epoch 19/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4472 - val_loss: 0.4276\n",
      "Epoch 20/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4437 - val_loss: 0.4317\n",
      "Epoch 21/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4407 - val_loss: 0.4150\n",
      "Epoch 22/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4379 - val_loss: 0.4173\n",
      "Epoch 23/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4351 - val_loss: 0.4187\n",
      "Epoch 24/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4328 - val_loss: 0.4153\n",
      "Epoch 25/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4304 - val_loss: 0.4099\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 26/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4283 - val_loss: 0.4065\n",
      "Epoch 27/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4261 - val_loss: 0.4083\n",
      "Epoch 28/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4242 - val_loss: 0.4039\n",
      "Epoch 29/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4224 - val_loss: 0.3960\n",
      "Epoch 30/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4207 - val_loss: 0.3992\n",
      "Epoch 31/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4188 - val_loss: 0.3920\n",
      "Epoch 32/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4174 - val_loss: 0.4008\n",
      "Epoch 33/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4159 - val_loss: 0.3919\n",
      "Epoch 34/100\n",
      "11610/11610 [==============================] - 0s 34us/sample - loss: 0.4143 - val_loss: 0.4006\n",
      "Epoch 35/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.4129 - val_loss: 0.3990\n",
      "Epoch 36/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4116 - val_loss: 0.3882\n",
      "Epoch 37/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4104 - val_loss: 0.3860\n",
      "Epoch 38/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4090 - val_loss: 0.3875\n",
      "Epoch 39/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4079 - val_loss: 0.3806\n",
      "Epoch 40/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4068 - val_loss: 0.3814\n",
      "Epoch 41/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4056 - val_loss: 0.3827\n",
      "Epoch 42/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4045 - val_loss: 0.3786\n",
      "Epoch 43/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4035 - val_loss: 0.3812\n",
      "Epoch 44/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4024 - val_loss: 0.3789\n",
      "Epoch 45/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4015 - val_loss: 0.3787\n",
      "Epoch 46/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.4005 - val_loss: 0.3805\n",
      "Epoch 47/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.3995 - val_loss: 0.3754\n",
      "Epoch 48/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3987 - val_loss: 0.3753\n",
      "Epoch 49/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.3977 - val_loss: 0.3772\n",
      "Epoch 50/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.3967 - val_loss: 0.3706\n",
      "Epoch 51/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3958 - val_loss: 0.3808\n",
      "Epoch 52/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.3950 - val_loss: 0.3865\n",
      "Epoch 53/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.3942 - val_loss: 0.3874\n",
      "Epoch 54/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3933 - val_loss: 0.3936\n",
      "Epoch 55/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.3927 - val_loss: 0.3897\n",
      "Epoch 56/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.3919 - val_loss: 0.3940\n",
      "Epoch 57/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3912 - val_loss: 0.3777\n",
      "Epoch 58/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3905 - val_loss: 0.3831\n",
      "Epoch 59/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.3898 - val_loss: 0.3797\n",
      "Epoch 60/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.3892 - val_loss: 0.3834\n",
      "Train on 11610 samples, validate on 3870 samples\n",
      "Epoch 1/100\n",
      "11610/11610 [==============================] - 0s 39us/sample - loss: 1.3072 - val_loss: 5.4533\n",
      "Epoch 2/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.6424 - val_loss: 2.8452\n",
      "Epoch 3/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.5206 - val_loss: 2.5899\n",
      "Epoch 4/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4927 - val_loss: 6.1177\n",
      "Epoch 5/100\n",
      "11610/11610 [==============================] - 0s 29us/sample - loss: 0.4958 - val_loss: 7.5540\n",
      "Epoch 6/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4943 - val_loss: 3.3703\n",
      "Epoch 7/100\n",
      "11610/11610 [==============================] - 0s 29us/sample - loss: 0.4485 - val_loss: 0.7394\n",
      "Epoch 8/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4171 - val_loss: 0.4349\n",
      "Epoch 9/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4102 - val_loss: 0.4125\n",
      "Epoch 10/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4055 - val_loss: 0.3867\n",
      "Epoch 11/100\n",
      "11610/11610 [==============================] - 0s 29us/sample - loss: 0.4017 - val_loss: 0.3774\n",
      "Epoch 12/100\n",
      "11610/11610 [==============================] - 0s 29us/sample - loss: 0.3982 - val_loss: 0.3820\n",
      "Epoch 13/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3944 - val_loss: 0.4447\n",
      "Epoch 14/100\n",
      "11610/11610 [==============================] - 0s 29us/sample - loss: 0.3926 - val_loss: 0.3701\n",
      "Epoch 15/100\n",
      "11610/11610 [==============================] - 0s 29us/sample - loss: 0.3900 - val_loss: 0.3717\n",
      "Epoch 16/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3876 - val_loss: 0.3667\n",
      "Epoch 17/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3851 - val_loss: 0.3589\n",
      "Epoch 18/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3829 - val_loss: 0.3570\n",
      "Epoch 19/100\n",
      "11610/11610 [==============================] - 0s 29us/sample - loss: 0.3807 - val_loss: 0.3977\n",
      "Epoch 20/100\n",
      "11610/11610 [==============================] - 0s 29us/sample - loss: 0.3802 - val_loss: 0.4129\n",
      "Epoch 21/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3783 - val_loss: 0.4629\n",
      "Epoch 22/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3772 - val_loss: 0.3535\n",
      "Epoch 23/100\n",
      "11610/11610 [==============================] - 0s 29us/sample - loss: 0.3749 - val_loss: 0.3909\n",
      "Epoch 24/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3742 - val_loss: 0.3567\n",
      "Epoch 25/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3726 - val_loss: 0.3504\n",
      "Epoch 26/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3710 - val_loss: 0.4323\n",
      "Epoch 27/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3716 - val_loss: 0.3593\n",
      "Epoch 28/100\n",
      "11610/11610 [==============================] - 0s 29us/sample - loss: 0.3697 - val_loss: 0.3478\n",
      "Epoch 29/100\n",
      "11610/11610 [==============================] - 0s 29us/sample - loss: 0.3688 - val_loss: 0.3642\n",
      "Epoch 30/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3681 - val_loss: 0.3518\n",
      "Epoch 31/100\n",
      "11610/11610 [==============================] - 0s 29us/sample - loss: 0.3669 - val_loss: 0.3702\n",
      "Epoch 32/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3662 - val_loss: 0.3608\n",
      "Epoch 33/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3656 - val_loss: 0.3444\n",
      "Epoch 34/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3649 - val_loss: 0.3676\n",
      "Epoch 35/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3645 - val_loss: 0.4251\n",
      "Epoch 36/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3644 - val_loss: 0.6289\n",
      "Epoch 37/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3664 - val_loss: 0.4964\n",
      "Epoch 38/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3626 - val_loss: 0.9082\n",
      "Epoch 39/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3679 - val_loss: 0.7426\n",
      "Epoch 40/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3645 - val_loss: 0.8972\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 41/100\n",
      "11610/11610 [==============================] - 0s 29us/sample - loss: 0.3665 - val_loss: 0.6627\n",
      "Epoch 42/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3618 - val_loss: 0.8439\n",
      "Epoch 43/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3624 - val_loss: 0.8183\n",
      "Train on 11610 samples, validate on 3870 samples\n",
      "Epoch 1/100\n",
      "11610/11610 [==============================] - 0s 38us/sample - loss: 0.7428 - val_loss: 0.5310\n",
      "Epoch 2/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4772 - val_loss: 0.7309\n",
      "Epoch 3/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4394 - val_loss: 0.4341\n",
      "Epoch 4/100\n",
      "11610/11610 [==============================] - 0s 29us/sample - loss: 0.4377 - val_loss: 0.9035\n",
      "Epoch 5/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4206 - val_loss: 2.1735\n",
      "Epoch 6/100\n",
      "11610/11610 [==============================] - 0s 29us/sample - loss: 0.4003 - val_loss: 3.6124\n",
      "Epoch 7/100\n",
      "11610/11610 [==============================] - 0s 29us/sample - loss: 0.4402 - val_loss: 5.5876\n",
      "Epoch 8/100\n",
      "11610/11610 [==============================] - 0s 29us/sample - loss: 0.4307 - val_loss: 1.7575\n",
      "Epoch 9/100\n",
      "11610/11610 [==============================] - 0s 29us/sample - loss: 0.3921 - val_loss: 0.3940\n",
      "Epoch 10/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3885 - val_loss: 0.4234\n",
      "Epoch 11/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3743 - val_loss: 0.4203\n",
      "Epoch 12/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3730 - val_loss: 0.4551\n",
      "Epoch 13/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3709 - val_loss: 0.3796\n",
      "Epoch 14/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3668 - val_loss: 0.3914\n",
      "Epoch 15/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3675 - val_loss: 0.3904\n",
      "Epoch 16/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3647 - val_loss: 0.4041\n",
      "Epoch 17/100\n",
      "11610/11610 [==============================] - 0s 29us/sample - loss: 0.3670 - val_loss: 0.4419\n",
      "Epoch 18/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.3622 - val_loss: 0.3666\n",
      "Epoch 19/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3594 - val_loss: 0.3740\n",
      "Epoch 20/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3613 - val_loss: 0.3728\n",
      "Epoch 21/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3586 - val_loss: 0.4143\n",
      "Epoch 22/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3552 - val_loss: 0.3474\n",
      "Epoch 23/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3556 - val_loss: 0.3595\n",
      "Epoch 24/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3694 - val_loss: 0.3725\n",
      "Epoch 25/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3554 - val_loss: 0.3636\n",
      "Epoch 26/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3517 - val_loss: 0.3741\n",
      "Epoch 27/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3532 - val_loss: 0.3486\n",
      "Epoch 28/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3495 - val_loss: 0.3586\n",
      "Epoch 29/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3587 - val_loss: 0.3759\n",
      "Epoch 30/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3484 - val_loss: 0.3318\n",
      "Epoch 31/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3463 - val_loss: 0.3511\n",
      "Epoch 32/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3460 - val_loss: 0.3682\n",
      "Epoch 33/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.3453 - val_loss: 0.5032\n",
      "Epoch 34/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3471 - val_loss: 0.3380\n",
      "Epoch 35/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3419 - val_loss: 0.3651\n",
      "Epoch 36/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3407 - val_loss: 0.3318\n",
      "Epoch 37/100\n",
      "11610/11610 [==============================] - 0s 29us/sample - loss: 0.3433 - val_loss: 0.3625\n",
      "Epoch 38/100\n",
      "11610/11610 [==============================] - 0s 29us/sample - loss: 0.3430 - val_loss: 0.3544\n",
      "Epoch 39/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3370 - val_loss: 0.3249\n",
      "Epoch 40/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3412 - val_loss: 0.3564\n",
      "Epoch 41/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3378 - val_loss: 0.3569\n",
      "Epoch 42/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3343 - val_loss: 0.3182\n",
      "Epoch 43/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3381 - val_loss: 0.3800\n",
      "Epoch 44/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3325 - val_loss: 0.3294\n",
      "Epoch 45/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3336 - val_loss: 0.3423\n",
      "Epoch 46/100\n",
      "11610/11610 [==============================] - 0s 29us/sample - loss: 0.3324 - val_loss: 0.3299\n",
      "Epoch 47/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3344 - val_loss: 0.3148\n",
      "Epoch 48/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3298 - val_loss: 0.3498\n",
      "Epoch 49/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3413 - val_loss: 0.3286\n",
      "Epoch 50/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3271 - val_loss: 0.5932\n",
      "Epoch 51/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.3313 - val_loss: 0.3845\n",
      "Epoch 52/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.3261 - val_loss: 0.6172\n",
      "Epoch 53/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.3282 - val_loss: 0.6108\n",
      "Epoch 54/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3302 - val_loss: 0.3378\n",
      "Epoch 55/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3258 - val_loss: 0.4014\n",
      "Epoch 56/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.3244 - val_loss: 0.3276\n",
      "Epoch 57/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3222 - val_loss: 0.4102\n",
      "Train on 11610 samples, validate on 3870 samples\n",
      "Epoch 1/100\n",
      "11610/11610 [==============================] - 0s 38us/sample - loss: 0.6626 - val_loss: 1.9500\n",
      "Epoch 2/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4895 - val_loss: 0.4488\n",
      "Epoch 3/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4463 - val_loss: 4.8228\n",
      "Epoch 4/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3891 - val_loss: 8.5697\n",
      "Epoch 5/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4137 - val_loss: 0.3434\n",
      "Epoch 6/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.3676 - val_loss: 0.3476\n",
      "Epoch 7/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3674 - val_loss: 0.4051\n",
      "Epoch 8/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.4190 - val_loss: 0.3482\n",
      "Epoch 9/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3666 - val_loss: 0.3420\n",
      "Epoch 10/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3602 - val_loss: 0.3352\n",
      "Epoch 11/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3579 - val_loss: 0.3361\n",
      "Epoch 12/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3557 - val_loss: 0.3494\n",
      "Epoch 13/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3565 - val_loss: 0.3343\n",
      "Epoch 14/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3527 - val_loss: 0.3270\n",
      "Epoch 15/100\n",
      "11610/11610 [==============================] - 0s 29us/sample - loss: 0.3491 - val_loss: 0.3259\n",
      "Epoch 16/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3452 - val_loss: 0.3255\n",
      "Epoch 17/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3875 - val_loss: 0.3663\n",
      "Epoch 18/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3572 - val_loss: 0.3324\n",
      "Epoch 19/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3551 - val_loss: 0.3254\n",
      "Epoch 20/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3532 - val_loss: 0.3350\n",
      "Epoch 21/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3404 - val_loss: 0.3183\n",
      "Epoch 22/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3382 - val_loss: 0.4119\n",
      "Epoch 23/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3356 - val_loss: 0.3146\n",
      "Epoch 24/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3334 - val_loss: 0.3138\n",
      "Epoch 25/100\n",
      "11610/11610 [==============================] - 0s 29us/sample - loss: 0.3315 - val_loss: 0.3144\n",
      "Epoch 26/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3301 - val_loss: 0.3158\n",
      "Epoch 27/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3263 - val_loss: 0.3053\n",
      "Epoch 28/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3251 - val_loss: 0.3817\n",
      "Epoch 29/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3245 - val_loss: 0.3011\n",
      "Epoch 30/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3224 - val_loss: 0.3064\n",
      "Epoch 31/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3211 - val_loss: 0.3338\n",
      "Epoch 32/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3206 - val_loss: 0.2942\n",
      "Epoch 33/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3185 - val_loss: 0.3047\n",
      "Epoch 34/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3182 - val_loss: 0.2964\n",
      "Epoch 35/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3240 - val_loss: 0.2974\n",
      "Epoch 36/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3160 - val_loss: 0.2945\n",
      "Epoch 37/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3143 - val_loss: 0.2956\n",
      "Epoch 38/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3138 - val_loss: 0.2969\n",
      "Epoch 39/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3122 - val_loss: 0.2938\n",
      "Epoch 40/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3106 - val_loss: 0.2912\n",
      "Epoch 41/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3121 - val_loss: 0.3569\n",
      "Epoch 42/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3137 - val_loss: 0.2975\n",
      "Epoch 43/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3093 - val_loss: 0.2939\n",
      "Epoch 44/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3078 - val_loss: 0.2913\n",
      "Epoch 45/100\n",
      "11610/11610 [==============================] - 0s 29us/sample - loss: 0.3082 - val_loss: 0.2980\n",
      "Epoch 46/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3084 - val_loss: 0.2925\n",
      "Epoch 47/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3104 - val_loss: 0.2883\n",
      "Epoch 48/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3075 - val_loss: 0.2887\n",
      "Epoch 49/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.3050 - val_loss: 0.3013\n",
      "Epoch 50/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3027 - val_loss: 0.3024\n",
      "Epoch 51/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3049 - val_loss: 0.3493\n",
      "Epoch 52/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3023 - val_loss: 0.3955\n",
      "Epoch 53/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3038 - val_loss: 0.2910\n",
      "Epoch 54/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3037 - val_loss: 0.3976\n",
      "Epoch 55/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3027 - val_loss: 0.2860\n",
      "Epoch 56/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3010 - val_loss: 0.2943\n",
      "Epoch 57/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3011 - val_loss: 0.2864\n",
      "Epoch 58/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3035 - val_loss: 0.3168\n",
      "Epoch 59/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3008 - val_loss: 0.2943\n",
      "Epoch 60/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.3017 - val_loss: 0.2994\n",
      "Epoch 61/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3009 - val_loss: 0.3375\n",
      "Epoch 62/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3075 - val_loss: 0.3327\n",
      "Epoch 63/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3022 - val_loss: 0.3012\n",
      "Epoch 64/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3009 - val_loss: 0.3123\n",
      "Epoch 65/100\n",
      "11610/11610 [==============================] - 0s 30us/sample - loss: 0.3008 - val_loss: 0.2923\n"
     ]
    }
   ],
   "source": [
    "learning_rates = [1e-4, 3e-4, 1e-3, 3e-3, 1e-2, 3e-2]\n",
    "histories = []\n",
    "for learning_rate in learning_rates:\n",
    "    model = keras.models.Sequential([\n",
    "        keras.layers.Dense(30, activation=\"relu\", input_shape=X_train.shape[1:]),\n",
    "        keras.layers.Dense(1)\n",
    "    ])\n",
    "    optimizer = keras.optimizers.SGD(learning_rate)\n",
    "    model.compile(loss=\"mean_squared_error\", optimizer=optimizer)\n",
    "    callbacks = [keras.callbacks.EarlyStopping(patience=10)]\n",
    "    history = model.fit(X_train_scaled, y_train,\n",
    "                        validation_data=(X_valid_scaled, y_valid), epochs=100,\n",
    "                        callbacks=callbacks)\n",
    "    histories.append(history)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Learning rate: 0.0001\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAesAAAE3CAYAAACKD2n1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xl8VOWhN/DfObMvmS0zkwlJSNh3FNxYFFyqyOKG3latct9Sae3VUrlvvVpxaa/F7bWX7n1Lr9W3FVpp9WqxClSprQLWAmqQPUAI2Sf7TDLJLOe8f5xkQiSQ7czkJPl9P598Zs7MmXOePE74+TznOc8jyLIsg4iIiDRLHOwCEBER0fkxrImIiDSOYU1ERKRxDGsiIiKNY1gTERFpHMOaiIhI4xjWREREGsewJiIi0rheh3U4HMayZctQWlp61nuHDh3C8uXLsWjRIqxduxbxeFzVQhIREY1kvQrrTz/9FHfccQeKi4u7ff/BBx/E448/jm3btkGWZWzevFnNMhIREY1ovQrrzZs344knnoDf7z/rvbKyMrS2tuLCCy8EACxfvhxbt25Vt5REREQjmL43O61bt+6c71VXV8Pn8yW3fT4fqqqqBl4yIiIiAqDCADNJkiAIQnJbluUu20RERDQwvWpZn08gEEAwGExu19TUdNtdfj719c2QJHUW/8rMtKO2NqzKsfqqqj6CH/z+Y3zpmvG4aGLf6mAgIn9+DoIzAPPlK1Q75mDW43DCelQH61EdrEd1DLQeRVGA223r02cGHNY5OTkwmUzYu3cvLrroIrzxxhtYsGBBn44hSbJqYd1xvMHgthtR19SKotONmDXe1/MHVJKADnJFEYwq/96DVY/DDetRHaxHdbAe1ZHueux3N/iqVauwf/9+AMDzzz+Pp59+Gtdffz1aWlqwYoV6LbyhRK8TEfDYUFbTnNbzis4sSI2V4NLkRETDU59a1jt27Eg+/9WvfpV8PnnyZPzxj39Ur1RDWK7fhmOnG9J6TtEZAOJtkFsaINjcaT03ERGl3oC7wamrXJ8dHx6oQjgSg91iSMs5RVc2AEBqqIDIsCYilSUScdTXBxGPRwe7KJpQXS1CkqQe99PrjXC7fdDpBh61DGuVjc9xAgCOlTZg1oT0XLcWnVkAAKmxEsiZmpZzEtHIUV8fhNlshc0W4N0+APR6EfH4+cNalmU0Nzehvj4Irzd7wOfk3OAqG5OdAb1OwLHTjWk7p2BzA3ojpIbKtJ2TiEaOeDwKm83BoO4DQRBgszlU641gWKvMoNehINuBY6Xpu24tCGJykBkRUSowqPtOzTpjWKfAxFwXiitDaIsl0nZO0ZkNqZEzxxERDUcM6xSYmOdEQpJxorwpbecUnVmQQ0HICa54RkTD2759e3D//V8b7GKkFcM6BcbnOCEAab2FS3QGAFmCFKpO2zmJiCg9OBo8BaxmA3L9dhxN43Xrztu3KqFzjUrbeYmIBktJySk899w6hEJNMJsteOCBb2PKlGnYvn0rNm36DURRxKhRo/DYY0+isbEB//mfjyESiUAUBXzrWw9i+vQZg/0r9BrDOkUm5Dqxc38lEpIEnZj6DoyO27dkDjIjohTaub8CHxRWpOTYl8/MxvwZvb/N6cknH8Ndd/0vLFx4NT77bD8effQh/O53r+FXv/oFNmx4EW63Bz/72Y9QUlKM99//G+bNuxx33rkCH364C4WFnzCsCZiY58KOfWUoqQpjTLYj5ecTTDYIFgdv3yKiESESiaC8vAwLF14NAJg+fQYcDgdKSk5h/vwr8I1vfBULFlyJhQuvxoQJkxCJRLB27X/g6NEjmDfvctx66xcH+TfoG4Z1ikzIdQFQrlunI6wB5bo1b98iolSaP6Nvrd9UkeWzJyWRZSCRSOCBB76NoqKbsHv3B3jyycewcuXXsGjRErz88mbs2vUB3n13O956awt++MOfD0LJ+4cDzFLEnWGCz2XG0dL0TY7CsCaikcJqtWHUqBz87W/KmhWffbYfdXW1GDt2HG6//Ra4XC7cffdXcP31S3H06BH8/Oc/wrZtb2Px4mVYs+YhHD16ZJB/g75hyzqFJua6UHiiFrIsp2VCAcEZgBxpghxtgWC0pvx8RESD6fHHn8T/+T9P4YUXfgmDwYh1656DwWDAV7/6dTzwwH0wmUxwu91Yu/a7iEaj+N73HsVbb22BKIp49NHvDXbx+4RhnUIT8lzY+VklKutakJ3Zt4XG+0N0BQC0jwj3j035+YiIBsPs2Rdj9uyLAQA//emGs96/9trrce2115/1+s9//t8pL1uqsBs8hSbkdizqkZ6ucNHZHtbsCiciGlYY1ikU8FjhsBpwNE2To4gOHyAIDGsiomGGYZ1CgiBgQq4rbWEt6AwQMny8fYuIaJhhWKfYhDwXahpbUdvYmpbzcUQ4EdHww7BOsZnjMgEAHx8LpuV8HWEty3JazkdERKnHsE6xgMeKHK8N+46mKaxdASAehdxcn5bzERFR6jGs02D2RB+OnG5AU0s05efiiHAiouGHYZ0GF03yQZaBT47VpPxcDGsiouGHYZ0GeX47vE5zWrrCBZsL0Bs5IpyIRrx1676Lt97act59Lr/84jSVZmA4g1kaCIKAiyb58O7eUrS0xmE1p67aBUHkiHAiSpnY0Z2IHfl7So5tmLQAhonzU3LsoY5hnSYXTfRj20enUXiiBnOmBlJ6LtGVjURVUUrPQUQ0GB555EFcd931uPLKawAAK1fehW9+cw02bPg52tpaEQqFsXr1GlxxxZV9Om5rayueffb7KCo6ClEUcfvtd2Hx4mUoKjqG555bh0QiAaPRiEceeQJ5eTl48snv4sSJ4wCAW275F9x44y1q/6pdMKzTZGyOA06bEfuOBFMe1jpvPuLH/wG5NQzBbE/puYhoZDFMnD+ord9Fi5bgL395G1deeQ1Ony5BNBrFq6++gocffgz5+QXYu/ef+NGPnu9zWP/617+E0+nEb3+7GQ0NDVi16l8xYcIkbN68CbfffheuvvoLePvtN3HgwH7U19eiqakJL764CTU1QfziFz9JeVjzmnWaiIKA2RN9KDxRi2gskdpzeQsAAIma4pSeh4go3ebNuxyffbYfLS3NeOedbVi0aDEee+xJnDhRhJde+m/8/vcvIxKJ9Pm4e/fuwdKlNwEAXC4XrrhiAT7+eC/mzp2P9eufw9NP/ydsNjuuvfZ6jBs3DiUlp/Dv/34/dux4B/fd9y21f82zMKzTaPYkH6IxCQdO1qX0PDpvPgAgESxO6XmIiNLNYDBg/vwr8MEHf8eOHX/Btddej/vuW4VDhw5g0qTJWLFiZb8mhZJl6XPbQCIRx1VXfQG//vXLmDJlGjZv3oTnn38aTqcLv/3tZtx665dQUnIKK1fehVAopNav2C2GdRpNynPBZtZjb4pHhQsmGwSHH1LwZErPQ0Q0GBYtWoLf//5lOJ0uWK1WnD59Cl/96r2YM2c+3n//b5AkqeeDfM7s2Zfgz39+AwDQ0NCA999/D7NmXYzHH/8ODh06iJtvvhX33HMvjhw5jL///W948snHMW/e5XjggW/DYrGgurpK7V+zC16zTiO9TsSF4734+FgN4gkJel3q/l9J5y1Aovp4yo5PRDRYZs68EOFwGDfffBscDieWLbsJd9/9Rej1esyefQlaW1v73BX+la/cgx/84FmsWPElSJKEFStWYtKkybj77q/g2We/j5de+hX0egO+/e2HMXXqFOzY8Q7uvvuLMBqNWLRoCcaNG5+i31YhyBqYRLq2NgxJUqcYPl8GgsHUdkcMxCfHavDjVwvxzVtnYNYEX8rOE/30LbT9YzPsK37ar0FmWq/HoYL1qA7Wozr6W4+VlacQCOSnoERDk14vIh7vXeu9u7oTRQGZmX37d5kt6zSbPtYDd4YJO/aVpTSszxxkps+dnrLzEBFpWVtbK77+9ZXdvnfPPV/H5ZcvTHOJ+odhnWZ6nYgrLxyF/3n/JCpqm5GdaUvJeToHmZ1kWBPRgMmyDEEQBrsYfWYymfHSS5sG5dxqdlxzgNkgWHBhDnSigL/uK0vZOZRBZlmQOCKciAZIrzeiubmJS+/2gSzLaG5ugl5vVOV4bFkPAqfNiEum+PHB/grcsmAsLKbU/GfQefM5yIyIBszt9qG+PohwuGGwi6IJoij2asS5Xm+E263O5U6G9SC55qJcfHigCrsPVOLq2bkpOYfOV4D4iY84kxkRDYhOp4fXmz3YxdCMwRjwyG7wQTI224GCQAbe3Vuasq4lzmRGRDQ8MKwHiSAIuOaiXFTUtuDQqfqUnOPMQWZERDR0MawH0aVT/LBbDHh3b2lKjs9BZkREwwPDehAZ9DosvHAUPimqQU1D3yee7w2dN5/d4EREQxzDepBdNUu5jWvLruKUHF/nK4AcroXUytmfiIiGKob1IPM4zLhqVi4+2F+BsmBY9eOLvjEAwK5wIqIhjGGtATfML4DZqMcf31P/nujkIDN2hRMRDVkMaw2wWwxYMmc0Pj1eiyMl6o4MF4xWDjIjIhriGNYace3FeXBnmLD5r8dVv+9a583n7VtEREMYw1ojjAYdbr5iDE5WNGHPkaCqx9YFJkJuroPUpO5xiYgoPXoV1lu2bMGSJUtw3XXXYePGjWe9f+DAAdx666248cYb8fWvfx1NTU2qF3QkmD89Gzk+G17923HEE71bK7U3dDlTAQDx8oOqHZOIiNKnx7CuqqrC+vXrsWnTJrz++ut45ZVXUFRU1GWfdevWYfXq1fjTn/6EMWPG4IUXXkhZgYczURTwL1eOQ3V9BO/sUW+iFNGVDcHqQqL0gGrHJCKi9OkxrHft2oU5c+bA5XLBarVi0aJF2Lp1a5d9JElCc3MzACASicBsNqemtCPAjLGZmDXBi9f+fkK1W7kEQYBu1BQkyg9BltVrsRMRUXr0uOpWdXU1fL7OJb78fj8KCwu77PPwww9j5cqVeOqpp2CxWLB58+Y+FSIzU90VoXy+DFWPl27//uWLcf/zO/Di1iN4fvUCGPQDH1oQmjwbwaLdcMr1MPkLevWZoV6PWsF6VAfrUR2sR3Wkux57DGtJkiAIQnJbluUu262trVi7di1eeuklzJw5Ey+++CIeeughbNiwodeFqK0NQ5LUGQE9GEuXpcKK6ybhJ6/tx4tv7MctC8YO+HiSQzlGzWd7YBQze9x/uNTjYGM9qoP1qA7WozoGWo+iKPS5kdpjky0QCCAY7BxFHAwG4ff7k9tHjx6FyWTCzJkzAQBf+tKX8NFHH/WpEHS2WRN9mD8jgD/vPoXj5Y0DPp5oz4TgDHCQGRHRENRjWM+bNw+7d+9GXV0dIpEItm/fjgULFiTfz8/PR2VlJU6cOAEAePfddzFjxozUlXgEueOaiXBnGPHfbx5CWywx4OPpc6YiUXEEshRXoXRERJQuPYZ1VlYW1qxZgxUrVuDmm2/GsmXLMHPmTKxatQr79++H0+nE008/jQceeAA33HADXn31VTz11FPpKPuwZzXrsXLpVFTVteDlbUcGPFmKbtQUINaKRDUnSCEiGkoEWe3psvqB16zP7/X3T+BPO4tx68KxWDq3oN/HkVvDCP/mmzBedDNMF9103n2HYz0OBtajOliP6mA9qkOT16xp8N10+RhcNjULr/7tBPYcru73cQSzHaI3HwletyYiGlIY1kOAIAhYuWQyxuc48as3D+JEef9niNPnTEWiqghyrE3FEhIRUSoxrIcIg16H+2+dAafNiB+/Woiaxki/jqPLmQpICSQqj6pcQiIiShWG9RDisBrxwL9cgFhcwvO/+wS1ja19PoYuMAEQ9YiXcepRIqKhgmE9xIzy2vDvX7wAoUgMz2zch+qGvrWwBb0JuqxxSJQdSlEJiYhIbQzrIWhcjhMP3nEhWqNxPLtxHypqm/v0eV3ONEi1JZDCdSkqIRERqYlhPUQVBBz4jztnI56Q8Oymj1Hah0U/DOMuAyAjVvRh6gpIRESqYVgPYXl+Ox66czYEAXj65b0oPF7Tq8+JziyIWeMRP/bBgCdaISKi1GNYD3GjvDasvfsi+JwW/OgPhXj7w1O9CmDDhHmQ6ssh1Z5KQymJiGggGNbDgNdpwXfuuggXT/bjD+8dx4YtB3ucS9ww9lJA1CN2dFeaSklERP3FsB4mTEYd7r1pGm5dOBYfHazCU7/di7LzXMcWzHbo8y9E/PiHXNiDiEjjGNbDiCAIWDq3AKtvm4mGcBu+99Ie/OWfpyGdo1vcMGE+5EgTEqWfpbmkRETUFwzrYeiC8V7851cvw9QCN3737jGsf+UT1IfOnl5UlzcDgsnOrnAiIo1jWA9TTpsR37ptJlYsmoRjZY149L//gXf3lnZZ3UzQ6aEffxnip/ZBbuvbvdpERJQ+DOthTBAEXDkrB9/7yqUoCGRg41+O4snf7MHJis6FQAwT5gOJOGIn9wxiSYmI6HwY1iNAlseKb99+Ib5+4zQ0hNrw/f+3B7/ZdgRNzVGIvjEQXdmIH9052MUkIqJz0A92ASg9BEHAZVOzMGNsJl5//wR27CvD7gOVWHzpaFwzdi4S+15Dor4cOveowS4qERF9DlvWI4zVrMed107Ek/dciuljPHj9g5NYt8uMhGhA694/DXbxiIioGwzrESo704b7bpmBtXdfBIcnE39tnoj48X/g/Q/29TihChERpRfDeoQbl+PEQ1+ejUnXfxEJQYf4x1vw0C924dUdx9DSyslSiIi0gNesCYIgYOqkfLTWX4uLCrfiuD2Gl/58EL8z6nDFzGx84eI8+F2WwS4mEdGIxZY1JRkvWAxBb8CXA8ewfs1CzJ7gxV/3leE7/3c3fvrafhwsruMqXUREg4Ata0oSLQ4Ypl6D2P6tGP2FO7Dqhmm47crx2LGvFO99XIZ9R4PI8lhx1YWjMG9GNuwWw2AXmYhoRGDLmrowXrAY0BnQ8MEfAQDuDBNuXTgOP7hvPu5ZNgV2ix6/31GE//2zndiw5QAOFtedc+5xIiJSB1vW1EVH6zq8fxusE6+BzpsPADAadJg3PRvzpmejpCqEv31Sjn8crMKHB6qQ6TBj/owA5k0PwO+2DvJvQEQ0/AiyBi5C1taGu8xZPRA+XwaCwZAqxxqppNYQWl97HLLOBOvy70IwmLvdLxZPYN/RGnxQWI6DxfWQAYwb5cCcaQFcMsUPh9WY3oJrEL+P6mA9qoP1qI6B1qMoCsjMtPfpMwxr6pa95RQqNn4X+vFzYb5yFQRBOO/+dU2t+MfBKuw+UInSYDNEQcDUMW5cOjkLsyd6YTWPzOvb/D6qg/WoDtajOgYjrNkNTt2y5E+HcfZNiO59HfFRU2CYdMV59/c4zFg8Jx+L5+SjtDqM3Qcr8c9D1fj1W4fw/7YKmD7Gg4sn+3HhBC9sIzS4iYj6i2FN52ScdSMSFUfQuvO3EP1joXPn9OpzuX47/sU/HrctHIeTFSF8dKgK/zxcjU+P10InCpic78ZFk3yYNcEHp41d5UREPWE3OHWrox6llga0/PExCBYHrLc8DkFv6tfxJFnGyYom7D0SxN4j1Qg2tEIAMDbHgVkTfJg1wYvsTJu6v4QG8PuoDtajOliP6uA1axXwy6iOM+sxXvoZIm/9ALrcabAs+hYE3cC6sWVZxunqMD45VoOPj9XgVJVyniy3BTPHeXHB+ExMzHNBrxv6dxby+6gO1qM6WI/q4DVr0iR97nSYF3wFrX//NVrf+TnM194HQez/V0cQBIzOysDorAzcePkY1DW14pOiGnxaVIu/flyGv+w5DbNRh2kFHkwf68GMsZnwOLofkU5ENBIwrKlXDJMXQE5E0bbzZbTu2ADz1fdCENVp+XocZlw9OxdXz85FWzSBQ6fq8enxGhQer8Xeo0EAQI7XhuljPZg2xoOJuS4YDTpVzk1ENBQwrKnXjNO+AMRjaPvHK2jVG2Be+FUIgrpd1SajDhdO8OLCCV7IsozymmbsP1GHz07W4t29pdj20WnodSIm5TkxdYwHU/M9yMuyQ+zh1jIioqGMYU19YrxgMeR4FNG9/4NIJATzwpUQra6UnEsQBOT47Mjx2XH9ZaPRFk3gyOkGHCyuw4GTdfjDX48DOA67xYDJ+W5MzXdjSr4bfrelx/vCiYiGEoY19Zlx9o0QzDa0ffgKWv74GMwLV0KfPyvl5zUZdZg5LhMzx2UCAOpDbTh8qh4Hi+tw8FQ99hyuBqDMZz55tBuT812YMtoNL5f3JKIhjmFNfSYIAozTvgDdqClo3fFLRLb9CIYpV8I05/ZzTk2aCu4ME+ZOD2Du9ABkWUZVfQSHT9Xj0Kl6fHayFrsPVAIAMh0mTMxzY9JoFyaNdsHvYsubiIYWhjX1m86dA+vNjyG6538Q/fRtxIv3wTj7RhgmXwlBl96vliAICHisCHisuHJWDmRZRllNM46UNOBISdfwdtqMmJjnwsQ8FybkOpHrs0MUGd5EpF0MaxoQQWeA6bIvQl8wG20f/QFtO19GtHAbTBffAv34OaoPQOt1uQQBuT47cn12XHNRrjJYrbYFx0434OjpBhw53YB/tnebW0w6jMtxYkKuCxNynBiT7YDJyNHmRKQdnBSFutWfepRlGYnS/Wj76FVItacg2DNhmDAPhgnzIboCKSpp/8iyjNrGVhwrbcSx0gYcK2tEWbAZAKATBeT57Rif68T4HOWnv/d58/uoDtajOliP6uCkKDSkCYIAfd5M6HKnI35yL2KH/4boJ28i+vEWiFnjYRhzEXTZUyBmjlbtHu2BlNXrssDrsmDudOV/JMKRGE6UN+JYaSOKShvx90/K8c6eUgDK9fFxoxwYl+PEuFFOjM6y815vIkobhjWpThBEGMZeAsPYSyA11yNetBuxo7vQ9uEryg5GC3SBidBlTYAuM08Jb6tr0Ad92S0GzBznxcxxXgBAPCHhdHUYRWWNOF7WiONlTdhzRJmkRScKyPXbMW6UA2OyHRg7yoEsj5X3exNRSrAbnLqVinqUmuuRqDiMRPkRxCsOQ26sTL4nmDMgunMgOvwQHH6IDh/EDB8Ehw+CyT7oQd6hMdyGE+VNOF7ehBPljThZEUJbLAEAsJr0GJOdgYJsJcDHZDswcayX30cV8O9aHaxHdbAbnIY10eaGOH4uDOPnAgDktmYk6koh1ZZAqj2NREM54iWfQI40df2gwQwxwwvB7lVC3O6FkOFVXrO5IZgz0hbmTrsJsyb6MGuiDwAgSTLKa5txsrwJJyuacKK8CW9/WAKp/f+BPQ4TRvszUBDIQEF2BgoCDji4LCgR9RHDmgaNYLJBnz0JyJ7U5XU51gqpKQgpVA05VAMpVAOpKQg5VINYxWEg1tr1QDo9BKsbos0NweaBaPcoIW7zKK/ZPRDMjpRcJxfFzlHnV1wwCgAQjSVQUh3GyYomVNRHcKS4Dp8W1aCj78jjMCE/KwP5ASXE87My4LT3b+lRIhoZehXWW7ZswS9+8QvE43H867/+K7785S93ef/EiRN44okn0NjYCJ/Ph//6r/+C0+lMSYFp+BMMZugy86DLzDvrPVmWgbZmJcBDQcjN9ZCa6yE310NurkOi+jjiJ/cAUvxzB9VBsDqVH7Nd6Vo3Z0CwupRwt3sg2jwQbK4BrSgGAEaDLjmKvKO7LNIWR0lVCCcrQjhVFUJxZQgfH6tJfsZlNyK/fSWy0VkZyM+yI9Np1kz3PxENrh7/VaqqqsL69evx2muvwWg04vbbb8dll12G8ePHA1D+8fzGN76BtWvXYsGCBXj++eexYcMGPPjggykvPI08giAAZjt0Zjt0voJu95FlCXJrGHK4TgnzlnrI4TrlMRKC3BqC1FAJuTV0disdAgRLBgSrG4LNBdHqUga/Wduf2z0QnQEIhr61hC0mPSaNdmPSaHfytY4AP1UVxqnKEEqqQig8UYuOUSRWkx6js+wYnZWBPL/ymJ1pHRbrfBNR3/QY1rt27cKcOXPgcimLNSxatAhbt27F/fffDwA4cOAArFYrFixYAAC499570dTUdM7jEaWaIIgQLA7A4gDOEegd5GgEUnOdEubhWqWF3tLQ3lqvQzx4EnIkBKDrAEjB5oHoyobozDqjVe5pv7ae2asWcXcB3hZLoDQYxumqcDLI3/u4DNG4BADQ6wSMyrQhz2/v/MnKgN1i6HM9EdHQ0WNYV1dXw+fzJbf9fj8KCwuT2yUlJfB6vXjkkUdw6NAhjB07Fo899lifCtHXUXE98fkyVD3eSDUy6jEDgP+8e8iJOBLNjYiH6xFvrEasthyx2jLl58Q/ILU2d9lftGTAlD0OplHj0Zw9Hu7scdBneHpdotxRLsw5YzshySgPhnGyvBEnyhpxsrwJB0/VY+dnnaPpPQ4TCrKdKMh2ID/bgYJsB3L9w+te8JHxfUw91qM60l2PPYa1JEldWgmyLHfZjsfj+Oijj/Dyyy9jxowZ+OEPf4hnnnkGzzzzTK8LwVu3tIf1+HlGwJAFeLMA7wwIyiswQhkQJzfXQwrXQWqqghQsRlvwJCInP0VHn7ZgcUL0FUDnzVduUetolet7151uFoEpuU5Mye0cC9LUHMXp6jBKg2GUVodxOhhGYVEQ8YRyTlEQkOWxIMdnR67Xpjz6bPC5LENuLnR+H9XBelSHJm/dCgQC2LNnT3I7GAzC7+9sifh8PuTn52PGjBkAgGXLlmH16tV9KgTRUCYYzBBc2RBd2QCmJV+XY21wJIKoLTqERLAYUk0xoqcLkwEOCEoXeoa3/Rq5G2L7NXExczRE6/kHaTpsRkwb48G0MZ2t9oQkobIugrJgGKXBZpQFwzhV2YS9h6uTHfkGvYjsTCtyvDaM8tqQ47VjlM8Gr9PMSV2INKrHsJ43bx5+8pOfoK6uDhaLBdu3b8eTTz6ZfH/WrFmoq6vD4cOHMXnyZOzYsQPTpk07zxGJRgbBYIJ51BQYzbnJ1+R4VGl9N1S0/1Qqo9iDJyAX1wGJzlHsgsWhhHayJR6A6ApAsDjPeU1cJ4rI8dqQ47Xh0imdr7dFEyivbUZpMIyyYDPKa5pxuKQBuw9UJfcx6kVkZ9owymvFKK+t/bkNPpcZukGeHpZopOsxrLOysrBmzRqsWLECsVgMt912G2bOnIlVq1Zh9eq9PElTAAAZk0lEQVTVmDFjBn72s5/h0UcfRSQSQSAQwHPPPZeOshMNOYLeCJ0nDzpP97elya0hSPVlkGpLkKg9Dam2BLGKI0Ai1nkMiwM6/ziI/nHQZY2DzlsAwWg573lNRl1yVrUztbTGUV6rhHfHz+dDXK8TkOW2IjvTiuxMW/Ix4LFydTKiNOF0o9Qt1qM61KhHWZaU0eqNlZAaKpCoKYZUdRzSmdO1Wl1K67vjJ3M0dJl5EIzWfp0z0hZHRW0LKtqDvKK2BeW1zQg2RHDmvxiZDjOyM5V1xDseA5k2uOxGVe8R5/dRHaxHdWjymjURDS5BEJPTqyJ3evJ1uTWMRPUJJOpOK13qjZWIHf8H0NY5Ol3I8EHnyW2fb739xxmAkOE9b5haTHqMHaUsUHKmWDyBqroIKuqUIK9sD/GjpQ2IxqTkfiajDgG3FVkeixLgHisCmVZkua2wmPjPDlFf8a+GaIgSzHboR8+EfvTM5GuyLCv3ideWIFFbosy7XlcGqXT/WdfDdVnjocsaDzFrPHTe/F6NTDfodcj125Hr79oqkGUZ9aE2VNS1oLK2BZV1Laiqa8GJ8ib881B1l7vUnTYjstwW+D1WZLktyHJbkeWxwu+2wDSMbjUjUhPDmmgYEQRBGVVuc0M/+oLk67IsQW5phNRUDam+DImq40hUFSFevK/9gyJEdw50vgKIvjHQeQsgZuZB0PVushVBEOBxmOFxmDGtoOs95bF4AtX1EVTWRVBVr4R4ZV0LCo/Xoqk52mVfd4ZJCXK3tctjhvP81+SJhjuGNdEIIAhiMsSRPQmYejUAQIo0Qao6jkTNSSSCJxEv/hjykfeVD4k6iO5c6Hz5ya7zjhXP+rLSmUGvQ47Pjhzf2dfoIm3x9gBXgry6XnncdzSIcCTWZV+X3Qi/2wq/ywK/W/nxtT+3mTmDGw1vDGuiEUy0OCAWzIK+YBaA9m70UI0yiK2mGIlgMWIn93a5Dg4A0JuU+8PbA1zMyIRg80CwZyoLo1jdvVrlzGLSoyDgQEHAcdZ7La0xVDdEUF0fQbgtgZNlDaiuj2D/yVo07u/aIreZ9fC6LPC7lAD3ucztjxZ4HCbeekZDHsOaiJIEQYDg8EF0+ICxlyRfl9uaIYVrIYVqIIeCkEK1kMPK8qWxyqNANNL1QHqjMiLdm690qXvzIbpH9WlFM6vZgIKAAQUBx1mjb9uiCQQbIskwDzYoPyVVIew7GkTijLtLREFAptMEr7MzyL1OC7wuM3xOCzKsBq5uRprHsCaiHgkmG3QmG3SZo7t9//MLokj15ZBqihE7uhOxA+8qO+kMEDPzOsPbNQo6VzYEc9/XBjAZux/oBgCSJKOuqRXBxlYEGyKoaVQCvaaxFR8fCyLU0rV73WTQwes0I9OphHem05wM9EynGTaznmFOg45hTUQDJhgt0BlzAHdOl9dlWYLcWIVEe5e6VFOM2LFdwMEdnZ81Z7TPzpYN0RWA6FTuFRccPghi30eHi6IAr8sCr8uCKfnus95vjcZR09CKmsZWBBsj7c+VMD9W2oBIW6LL/iajDl6HEuaZTnPn8/ZHh83IaVop5RjWRJQygiAm5003jJ8LoD3AQzWQGirPmHa1HPGSTyAfOWN5XZ1eCXB3LupzxyAmOiA6fBDsXggWR79bu2aj/pytcgBobo0lA7y2sRU1Ta2obVR+jpc1ork13mV/vU6Ex2FKhnemwwyPwwSvwwyP0wxPhgkGPW9Jo4FhWBNRWgmCmJykBWfcIw60XxvvmKmtrky5zaziCOqLdnc9iN4EnX8sdNmToAtMhC5rXK9XMOuJzWyALWBAfqD7JRAjbfGuIX7G4/4TtWgMR8/6jMNqSN7a5skwtT83JbdddtOQWwmN0othTUSaIZhs0PnHQecfhzNvxsp06lF9srh9cFsNpMZKJCqLEN37BgAZEHQQPTnKEqTe9qVIPXkQDOoE+JkspvO3zGNxCfUhJcDrQm2obWpFXZPyWFHbjAPFdWiLdu1qFwUBrgwjPBlmuDNM8DhMcGcoQe5u/3HajRzVPoIxrIlI80SjBTpPLuDJ7fK6HG1BorIIicqjSNQUI37qk877xCFAdGYpK5d5Rytd6o4spStdpVZ4dwx6Ubkf3N39vOyyLCut86Y21DUpgV7XHuj1oVaUVIXwaVENonGpy+cEQZn9zd0e6F1+7MqjK8PEWeCGKYY1EQ1ZgtHaZcpVWZaVJUdrTilTrdaWKJO9nPio6+c6Fj7x5EL05ELnyYXoyAJMtpSP/BYEAVazAVazAXnnaJ3Lsozm1jjqQ0qAK4HehoZQG+rDbaisa8GhU3VnDYYDAKtJrwS33QhXexd7x8+YlhgQT8BhM0KvYyt9KGFYE9GwIQhC+8QsmUDB7OTrcrQFUmOV8tNU3T7tajlih/8OxNs6DyDqIVidECxOiHaPspa4JxeiJweiI6tfo9P7+3vYLQbYLecOdEAZ2V4f6gxx5Xk0+by8th6N4Sikzy2uKADIsBrgspvgtCtd7EqgG9tfM8JlUx4Z6trAsCaiYU8wWqHzjYHON6bL6x0j0xN1pZBDQWX+9JZGyC0NSNSWIH5yL9CxDIlOr8yfnjla6Vr35CqzuNk8aQvxzzMb9cjO1CM703bOfSRJRigSQ0OoDZJORElZAxrCUTSElZBvaI6ipDqEpuYoulsw2W4xtIe3MRnsTpsS7M6O12xGmI063o+eQgxrIhqxuoxM74Ycb1NuLasrU5YirT39ueviADrmXc/wKq36M+ZQV17z9GnmNrWJoqCEqs0Iny8DY3zdB7skyWhqiaKxI8jDbcrz5igaw21oCEdRUae01M+cIa6DQS8mz+M4I8Q7th1nPOd19b5jWBMRnYOgN0HnLYDOW5AcnS7LMuRIo7L0aLhGuWc8XKu00CuOIF60G12aqB1h7vArg9sc/vaBbn6IziwIBvOg/G6fJ4pC8tp2Prq/bQ3ovJ7eEeZNzVE0NkfR2KxsNzZHUd0QwbHSxrMWY+lgMurgtHaGuMNmhMNqaH/sDPYMqxEWE1vsAMOaiKhPBEFQBqhZXd2+L0txyOF6JcibgpBCHT81Sqs80tT1eFaXMoObM6DMy27PTF53F6yuXi2Ikk5nXk/P9Z1/33hCQqgl1iXQm5qjaGqOtbfilcFyR083oDkSQze98NDrRDhsBmRY24PcakCGzYgMqwEOq/KYYe3cNg7TVjvDmohIRYKo71wMZdSUs96Xo5HOQW6NVcokMI2ViJ/cA7kt3HVnUad0pzv8EDN8ndfI7ZkQbW4INvegXS/vDb1OTN5e1pOE1BnsTS1RhNoDPbnd/l5ZTRhNzTHEE1K3xzEZdF0CvMtzixH2M1+zGIbMtXaGNRFRGglGS/tqZPlnvde5IEqtsrJZR6u8KYhY1XEg2vK5g515vbxzqVLR7oFga1+u1GhJ0282MDpRTHbD90SWZbRGEwi1RNHUEkOoPcxDLUqrPRRRthvCbThdHUaoJYp4ort2u/I/FBlWpacg+dge6h2v5frsGOU99yC+dGBYExFpxLkWROkgRyOQwnWQm2shNdd3vV5efgjx5nrg853JBnOyFV6dmYWoyZPsdhed/pROEJMqgiDAYtLDYtLDf/ZaLWdJhntECfRwSwzhSEwJ+PZgD7c/r21sRaglhpa2zjngbWY9fvLAghT+Rj1jWBMRDRGC0QKdJwfwnCPMpTjk5obkcqVyc50S6s31kJrrETnxKRLhuq7HTA5+8yuD32wepXvd6oJocwEGy5DoJj6fLuHu6l1PQ0KS0ByJIxSJwWIc/EsNDGsiomFCEPXJW8a64/NloLo82OVaudRUDbmxGvGSQsiRxrM/ZLQkwzwZ6BnKNfnBvMc81XSimByprgUMayKiEUQwmM99zTzWBrmlXpkYprleed5UAylUrUwSU7wPkM6Y4lTQQbC5kqPjBauzvUXuVoLc5laeD5Hr5lrGsCYiIgCAYDBBaL+e3R1ZkpSu9aZqSKEg5KZq5Rp6pBFSYwWkisNAW/PZHzSYO7vXbe72AXCe9tvUPMp7DPTzYlgTEVGvCKJ43m52AJDjUcgtDUqIt9RDCistdDlcB6mlHlJpGeItjThrIJzRAtGmhLdgcUC0OCBYnBCsHY9OiBZnWhZb0SKGNRERqUbQG887hStw9kC4jhHucrgOUnMdpLrTiEeauna5dxD1EGwuiFZ3Z0u9ffEVwZLR/uhQng/iNK9qGz6/CRERDQk9DYQDlNutEG1Rrp9HGiGf8dgxwj1RewryqU+ARLT7g5hsSgvd6lK63TsmkrG1X2Nvb7ELOkP3n9cQhjUREWmOIAiAyQadyQa4R51zP1mWgVgr5EgTpEiTEuiRJsiRUPK51NIAqeIw4s0NgNxNa91obQ/19la52dHZOjdnQJeZd87r+OnCsCYioiFLEATAaIFgtEB0Zp13X1mSILc2QW5pUK6rt7S32FsaldcjTZDqSiFFmroOlDNakfG/fp7i3+T8GNZERDQiCKIIweoCzrEIy5lkKQ65NQw5EoJgHPyV0RjWREREnyOI+l4Hezpoa+01IiIiOgvDmoiISOMY1kRERBrHsCYiItI4hjUREZHGMayJiIg0jmFNRESkcQxrIiIijWNYExERaRzDmoiISOMY1kRERBrHsCYiItI4hjUREZHG9Sqst2zZgiVLluC6667Dxo0bz7nfe++9h6uvvlq1whEREVEvlsisqqrC+vXr8dprr8FoNOL222/HZZddhvHjx3fZr6amBs8++2zKCkpERDRS9diy3rVrF+bMmQOXywWr1YpFixZh69atZ+336KOP4v77709JIYmIiEayHsO6uroaPp8vue33+1FVVdVln9/85jeYOnUqLrjgAvVLSERENML12A0uSRIEQUhuy7LcZfvo0aPYvn07XnrpJVRWVvarEJmZ9n597lx8vgxVjzdSsR7VwXpUB+tRHaxHdaS7HnsM60AggD179iS3g8Eg/H5/cnvr1q0IBoO49dZbEYvFUF1djTvvvBObNm3qdSFqa8OQJLmPRe+ez5eBYDCkyrFGMtajOliP6mA9qoP1qI6B1qMoCn1upPbYDT5v3jzs3r0bdXV1iEQi2L59OxYsWJB8f/Xq1di2bRveeOMNbNiwAX6/v09BTUREROfXY1hnZWVhzZo1WLFiBW6++WYsW7YMM2fOxKpVq7B///50lJGIiGhEE2RZVqf/eQDYDa49rEd1sB7VwXpUB+tRHZrsBiciIqLBxbAmIiLSOIY1ERGRxjGsiYiINI5hTUREpHEMayIiIo1jWBMREWkcw5qIiEjjGNZEREQax7AmIiLSOIY1ERGRxjGsiYiINI5hTUREpHEMayIiIo1jWBMREWkcw5qIiEjjGNZEREQax7AmIiLSOIY1ERGRxjGsiYiINI5hTUREpHEMayIiIo1jWBMREWkcw5qIiEjjGNZEREQax7AmIiLSOIY1ERGRxjGsiYiINI5hTUREpHEMayIiIo1jWBMREWkcw5qIiEjjGNZEREQax7AmIiLSOIY1ERGRxjGsiYiINI5hTUREpHEMayIiIo1jWBMREWkcw5qIiEjjGNZEREQax7AmIiLSOIY1ERGRxjGsiYiINI5hTUREpHEMayIiIo3rVVhv2bIFS5YswXXXXYeNGzee9f4777yDm266CTfeeCP+7d/+DY2NjaoXlIiIaKTqMayrqqqwfv16bNq0Ca+//jpeeeUVFBUVJd8Ph8P47ne/iw0bNuBPf/oTJk2ahJ/85CcpLTQREdFI0mNY79q1C3PmzIHL5YLVasWiRYuwdevW5PuxWAxPPPEEsrKyAACTJk1CRUVF6kpMREQ0wuh72qG6uho+ny+57ff7UVhYmNx2u9249tprAQCtra3YsGED7r777j4VIjPT3qf9e+LzZah6vJGK9agO1qM6WI/qYD2qI9312GNYS5IEQRCS27Isd9nuEAqFcN9992Hy5Mm45ZZb+lSI2towJEnu02fOxefLQDAYUuVYIxnrUR2sR3WwHtXBelTHQOtRFIU+N1J77AYPBAIIBoPJ7WAwCL/f32Wf6upq3HnnnZg0aRLWrVvXpwIQERHR+fUY1vPmzcPu3btRV1eHSCSC7du3Y8GCBcn3E4kE7r33XixevBhr167tttVNRERE/ddjN3hWVhbWrFmDFStWIBaL4bbbbsPMmTOxatUqrF69GpWVlTh48CASiQS2bdsGAJg+fTpb2ERERCoRZFlW52LxAPCatfawHtXBelQH61EdrEd1aPKaNREREQ0uhjUREZHGMayJiIg0jmFNRESkcQxrIiIijWNYExERaRzDmoiISOMY1kRERBrHsCYiItI4hjUREZHGMayJiIg0jmFNRESkcQxrIiIijWNYExERaRzDmoiISOMY1kRERBrHsCYiItI4hjUREZHGMayJiIg0jmFNRESkcQxrIiIijWNYExERaRzDmoiISOMY1kRERBrHsCYiItI4hjUREZHGMayJiIg0jmFNRESkcQxrIiIijWNYExERaRzDmoiISOMY1kRERBrHsCYiItI4hjUREZHGMayJiIg0jmFNRESkcQxrIiIijWNYExERaRzDmoiISOMY1kRERBrHsCYiItI4hjUREZHGMayJiIg0jmFNRESkcQxrIiIijetVWG/ZsgVLlizBddddh40bN571/qFDh7B8+XIsWrQIa9euRTweV72gREREI1WPYV1VVYX169dj06ZNeP311/HKK6+gqKioyz4PPvggHn/8cWzbtg2yLGPz5s0pKzAREdFIo+9ph127dmHOnDlwuVwAgEWLFmHr1q24//77AQBlZWVobW3FhRdeCABYvnw5fvzjH+POO+/sdSFEUehP2dN2vJGK9agO1qM6WI/qYD2qYyD12J/P9hjW1dXV8Pl8yW2/34/CwsJzvu/z+VBVVdWnQrjdtj7t35PMTLuqxxupWI/qYD2qg/WoDtajOtJdjz12g0uSBEHo/L8AWZa7bPf0PhEREQ1Mj2EdCAQQDAaT28FgEH6//5zv19TUdHmfiIiIBqbHsJ43bx52796Nuro6RCIRbN++HQsWLEi+n5OTA5PJhL179wIA3njjjS7vExER0cAIsizLPe20ZcsW/PKXv0QsFsNtt92GVatWYdWqVVi9ejVmzJiBw4cP49FHH0U4HMa0adPw9NNPw2g0pqP8REREw16vwpqIiIgGD2cwIyIi0jiGNRERkcYxrImIiDSOYU1ERKRxwyqse1pwhM7tpz/9KZYuXYqlS5fiueeeA6BMNXvDDTfguuuuw/r16we5hEPHs88+i4cffhgAF7nprx07dmD58uVYvHgxvv/97wPg97E/3njjjeTf9bPPPguA38neCofDWLZsGUpLSwGc+/uXtvqUh4nKykr5qquukuvr6+Xm5mb5hhtukI8dOzbYxRoSdu7cKX/pS1+S29ra5Gg0Kq9YsULesmWLvHDhQrmkpESOxWLyypUr5ffee2+wi6p5u3btki+77DL5oYcekmVZlpcuXSp//PHHsizL8ne+8x1548aNg1m8IaGkpES+/PLL5YqKCjkajcp33HGH/N577/H72EctLS3yJZdcItfW1sqxWEy+7bbb5J07d/I72QuffPKJvGzZMnnatGny6dOn5Ugkcs7vX7rqc9i0rM9ccMRqtSYXHKGe+Xw+PPzwwzAajTAYDBg3bhyKi4uRn5+PvLw86PV63HDDDazPHjQ0NGD9+vW49957AXS/yA3rsGd/+ctfsGTJEgQCARgMBqxfvx4Wi4Xfxz5KJBKQJAmRSATxeBzxeBx6vZ7fyV7YvHkznnjiieRsnIWFhd1+/9L5N97jQh5DRU8LjtC5TZgwIfm8uLgYb7/9Nu66666z6rOvC7SMNI8//jjWrFmDiooKAOoscjMSnTp1CgaDAffeey8qKipw5ZVXYsKECfw+9pHdbse3vvUtLF68GBaLBZdccgkMBgO/k72wbt26Ltvd5UtVVVVa/8aHTcuaC4oM3LFjx7By5Ur8x3/8B/Ly8lifffCHP/wB2dnZmDt3bvI1fif7J5FIYPfu3XjqqafwyiuvoLCwEKdPn2Zd9tHhw4fx6quv4q9//Svef/99iKKInTt3sh774Vx/y+n8Gx82LetAIIA9e/Yktz+/4Aid3969e7F69Wo88sgjWLp0KT766KPzLuBCXb311lsIBoO46aab0NjYiJaWFgiCwEVu+sHr9WLu3LnweDwAgC984QvYunUrdDpdch9+H3v2wQcfYO7cucjMzASgdNG+8MIL/E72w7kWtErnQlbDpmXd04IjdG4VFRW477778Pzzz2Pp0qUAgAsuuAAnT57EqVOnkEgk8Oabb7I+z+PFF1/Em2++iTfeeAOrV6/G1VdfjaeffpqL3PTDVVddhQ8++ABNTU1IJBJ4//33cf311/P72EeTJ0/Grl270NLSAlmWsWPHDlx66aX8TvbDuf49TOdCVsOmZZ2VlYU1a9ZgxYoVyQVHZs6cOdjFGhJeeOEFtLW14Zlnnkm+dvvtt+OZZ57BN7/5TbS1tWHhwoW4/vrrB7GUQ9Pzzz/fZZGbFStWDHaRNO+CCy7APffcgzvvvBOxWAzz58/HHXfcgbFjx/L72AeXX345Dh48iOXLl8NgMGDGjBn42te+hmuvvZbfyT4ymUzn/PcwXX/jXMiDiIhI44ZNNzgREdFwxbAmIiLSOIY1ERGRxjGsiYiINI5hTUREpHEMayIiIo1jWBMREWkcw5qIiEjj/j85p7OqKlDrMAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 576x360 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Learning rate: 0.0003\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAesAAAE3CAYAAACKD2n1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3XmYVNWB/vHvvbVX73sDsgqyCbjLoqBGAXGLSBI1SiYYojMaIjOTieOaGUNcxgyZyTbRn4mTRGYkEyNiDKISlc2ouICsEoVm7YVeq2uve39/VHdBy9LdUHQX3e/nefqpukvdPn28+NY599xzDdu2bURERCRjmd1dABERETk2hbWIiEiGU1iLiIhkOIW1iIhIhlNYi4iIZDiFtYiISIZTWIuIiGQ4hbWIiEiG63BYBwIBrr76anbv3n3Yts2bNzNz5kymTZvGfffdRzweT2shRUREerMOhfVHH33ETTfdxI4dO464/Tvf+Q4PPvggr7zyCrZts3jx4nSWUUREpFfrUFgvXryYhx56iNLS0sO27dmzh3A4zFlnnQXAzJkzWbZsWXpLKSIi0os5O7LTggULjrqtqqqKkpKS1HJJSQmVlZUnXjIREREB0jDAzLIsDMNILdu23WZZRERETkyHWtbHUl5eTnV1dWq5pqbmiN3lx1JX14xlpefhX0VF2Rw4EOjw/vt+/zhVQZOxt/yDvmQcorP1KEemekwP1WN6qB7T40Tr0TQNCgqyOvWZEw7rfv364fF4WLduHeeeey5Llixh8uTJnTqGZdlpC+vW43VUJBSGhgaC4Tg+zwlXR4+Szv8mvZnqMT1Uj+mhekyPrq7H4+4Gnzt3Lhs2bADgiSee4JFHHmH69OkEg0Fmz56dtgKebIY3C78RobE52t1FEREROaJONSVXrFiRev/UU0+l3o8YMYL/+7//S1+pupDDm0OWEaGhOUpZob+7iyMiInKYXt/v68rKwW1E2RUId3dRREQyUiIRp66umnhcPZAAVVUmlmW1u5/T6aagoASH48SjtteHtSc7F8OApoZGoLy7iyMiknHq6qrxev1kZZVrIC7gdJrE48cOa9u2aW5upK6umuLiPif8O3v93OCe7FwAwk0N3VwSEZHMFI9HycrKVVB3gmEYZGXlpq03oteHtcObDUAk0NTNJRERyVwK6s5LZ531+rA2PMl73eJBhbWIiGQmhXVLyzoRVliLiJwK3n//Pe6665vdXYwu1evDmpaWtR1p7uaCiIiIHFmvHw3e2g3uiIWwLBvT1HUZEZFTQUXFTh5/fAFNTY14vT7uvvsfGTlyNMuXL2PRol9jmiZ9+/blgQcepqGhnn/91wcIhUKYpsG3v/0dzjxzTHf/CR2msDYdxE0PfiNMUyhGXpa7u4skIpKxVm/Yx6r1+07KsS8a24dJYzp+m9PDDz/ALbf8DVOmXMbHH2/g/vu/y//8z/M89dTPefLJX1FQUMhPf/ofVFTsYOXKN5k48SJuvnk2b7+9hvXrP1RYn2osdxb+UJSGQERhLSJyCgiFQuzdu4cpUy4D4Mwzx5Cbm0tFxU4mTbqYv/3b25g8+RKmTLmMYcOGEwqFuO++f2Lbtq1MnHgRN9zw5W7+CzpHYQ0Ybs0PLiLSEZPGdK71e7LY9uGTktg2JBIJ7r77H9m+/TrWrl3Fww8/wJw532TatBn89reLWbNmFa+/vpyXX17Kj370s24o+fFRWAOmLwu/WUODwlpE5JTg92fRt28/3nxzRaobvLb2AEOGnM6NN17PT37yJLfe+nXi8Tjbtm3lr3/9hOLiUr785Zs4++zzmDPnq939J3SKwhpw+nLwG3vZrbAWETllPPjgw/zbv/2Ap5/+BS6XmwULHsflcnHbbbdz99134vF4KCgo4L77vkc0GuVf/uV+Xn55KaZpcv/9/9Ldxe8UhTXg9OeQbUZoCCisRUQy3TnnnMc555wHwE9+8uRh26+4YjpXXDH9sPU/+9n/O+llO1l0nzXJ27d8RGnQk7dERCQDKawBw5ONadiEAoHuLoqIiMhhFNaA4U1OjBLT/OAiIpKBFNYc8jCPkFrWIiKSeRTWJLvBAZzxZqKxRDeXRkREpC2FNUBLN7jfiGpiFBERyTgKaw62rP1GRBOjiIhIxlFYA4bHDyRb1gprERHJNAprwDCd2E4vWWaEhkCku4sjIiJpsGDB93j55aXH3Oeii87rotKcGM1g1sL0ZpPVrG5wEZFjiW1bTWzrWyfl2K7hk3GdMemkHPtUp7BuYXizyHXG2KWwFhHJWPfe+x2mTp3OJZd8AYA5c27hW9+az5NP/oxIJExTU4B58+Zz8cWXdOq44XCYxx77Ptu3b8M0TW688RauvPJqtm//hMcfX0AikcDtdnPvvQ/Rv38/Hn74e3z66V8BuP76L3Httden+09tQ2HdwvBkk+2o0fzgIiLH4DpjUre2fqdNm8Grr/6JSy75Art2VRCNRvn975/jnnseYODAQaxb9y7/8R9PdDqsf/nLX5CXl8dvfrOY+vp65s79GsOGDWfx4kXceOMtXHbZ5fzpTy+xceMG6uoO0NjYyK9+tYiammp+/vMfK6y7iuHJIsvcq25wEZEMNnHiRSxc+DjBYDOvvfYK06ZdyZe/fDNr1qzkz39+jY0bNxAKhTp93HXr3uOeex4AID8/n4svnswHH6xjwoRJ/Pu/P85f/rKGSZMmM2nSxYTDzVRU7OTv//4uxo+fxJ13fjvdf+ZhNMCsheHNxkuYxmYNMBMRyVQul4tJky5m1aq3WLHiVa64Yjp33jmXzZs3Mnz4CGbPnoNt250+rm1bn1uGRCLOpZdezi9/+VtGjhzN4sWLeOKJR8jLy+c3v1nMDTd8hYqKncyZcwtNTSd3umqFdQvDk4XHSob18fyHFhGRrjFt2gz+939/S15ePn6/n127dnLbbXcwfvwkVq58E8uy2j/I55xzzvn88Y9LAKivr2flyjc4++zzePDBf2bz5k188Ys38I1v3MHWrVt46603efjhB5k48SLuvvsf8fl8VFVVpvvPbEPd4C0MTxYGNg4rSnM4TrbP1d1FEhGRIxg79iwCgQBf/OIscnPzuPrq67j11i/jdDo555zzCYfDne4K//rXv8EPf/gYs2d/BcuymD17DsOHj+DWW7/OY499n2eeeQqn08U//uM9jBo1khUrXuPWW7+M2+1m2rQZnH760JP01yYZdgY0Iw8cCGBZ6SlGSUkO1dWd746IbV1J+M2n+Zf665k/53L6FWelpTynquOtR2lL9Zgeqsf0ON563L9/J+XlA09CiU5NTqdJPN6x1vuR6s40DYqKsjv3Ozu1dw92cMrRKI2BSK8PaxGRniASCXP77XOOuO0b37idiy6a0sUlOj4K61beZFhnaX5wEZHD2LaNYRjdXYxO83i8PPPMom753ensuNYAsxatz7T2mwprEZFDOZ1umpsbNfi2E2zbprm5EafTnZbjqWXdojWscxwxhbWIyCEKCkqoq6smEKjv7qJkBNM0OzTi3Ol0U1BQkpbfqbBu0RrWhZ44+5p0r7WISCuHw0lxcZ/uLkbG6I4Bj+oGb2E4nODyUuJLsKc60N3FERERSVFYH8LwZFHoSbDvQJBYB4fli4iInGwK60MYnmxyHDESls2+A83dXRwRERFAYd2G4c3CbySvV++qUle4iIhkBoX1IQxPFs5ECJfTVFiLiEjGUFgfwvBkQyRAv+IshbWIiGQMhfUhDE8WdqSZ00qSYa0JAEREJBMorA9heLPAthhU7CIQ0uQoIiKSGRTWh2h9mMeAvGS1qCtcREQygcL6EK1hXZ6T7P7erbAWEZEM0KGwXrp0KTNmzGDq1Kk8++yzh23fuHEjN9xwA9deey233347jY2NaS9ol/Ampxz12mEKcz1qWYuISEZoN6wrKytZuHAhixYt4oUXXuC5555j+/btbfZZsGAB8+bN48UXX2Tw4ME8/fTTJ63AJ1Pr/OB2pJn+JdkKaxERyQjthvWaNWsYP348+fn5+P1+pk2bxrJly9rsY1kWzc3JGb9CoRBer/fklPYkaxPWZdmadlRERDJCu0/dqqqqoqTk4CO+SktLWb9+fZt97rnnHubMmcMPfvADfD4fixcv7lQhioqyO7V/e0pKco7rc3bCy2eA3xFj1OklvLRmJ6GETd8+x3e8U93x1qO0pXpMD9Vjeqge06Or67HdsLYsC8MwUsu2bbdZDofD3HfffTzzzDOMHTuWX/3qV3z3u9/lySef7HAhDhwIYFnpuaf5hB9d5vISqK0lr9gBwPqtVeR6HGkp26mkOx4B1xOpHtND9Zgeqsf0ONF6NE2j043UdrvBy8vLqa6uTi1XV1dTWlqaWt62bRsej4exY8cC8JWvfIV33nmnU4XIJIYnCzvUSFmBH7emHRURkQzQblhPnDiRtWvXUltbSygUYvny5UyePDm1feDAgezfv59PP/0UgNdff50xY8acvBKfZGZeOVbdXkzToF9JFrv1bGsREelm7XaDl5WVMX/+fGbPnk0sFmPWrFmMHTuWuXPnMm/ePMaMGcMjjzzC3XffjW3bFBUV8YMf/KAryn5SOIoHEt2wHDsRp39pNu9vqzms619ERKQrtRvWANdccw3XXHNNm3VPPfVU6v2UKVOYMmVKekvWTcyiAWDFser30r80h7c+2kd9IEpBjqe7iyYiIr2UZjD7HEfxQACsmp30L00OANB1axER6U4K688xcsvA6SZxoILTSpL3Xe+q0uhJERHpPgrrzzFME7NoANaBCvxeF0W5XrWsRUSkWymsj8BRNIBETQW2bTH0tDw276wjYWkmMxER6R4K6yMwiwZALITdVMO5Z5TQFIyxbVdDdxdLRER6KYX1EbQOMkvU7GTMkCLcTpN1W6u6uVQiItJbKayPwCzoB4aJdaACj9vBmCFFrNtajWWnZ0pUERGRzlBYH4HhdGMW9CVRsxOAc0eU0NAcZftudYWLiEjXU1gfReuIcIBxpxfjdBis21rdzqdERETST2F9FI6igdjBeqxgAz6PkzMHF7FuWxW2usJFRKSLKayPwiweAJBqXZ87vITaxgif7dMEKSIi0rUU1kfhKEqGdeJA8rr1WcOKcZgG72lUuIiIdDGF9VEYniyMnGKsmmTLOsvrYuSgAt7boq5wERHpWgrrY3AUDUy1rAHOG15KTUOYikpNPyoiIl1HYX0MZvEA7IYq7GgIgLOHFWMa6goXEZGupbA+BkfRQMAmUbsbgBy/m+ED8nl7YyXxhOYKFxGRrqGwPgazZZCZVXOwK3zaBf050Bhm5fp93VUsERHpZRTWx2BkFWB4c7AOuW49ZkgRQ0/L48XVnxGNJbqxdCIi0lsorI/BMAwc5cOIV3yEHY+m1t0weQgNgSgr3t/TzSUUEZHeQGHdDtfoy7FDjcS2r02tGz6ggNGDC3n57Z2EIvFuLJ2IiPQGCut2OPqOxCwaQGz9K9j2wUFlMycPIRCKsfzdXd1YOhER6Q0U1u0wDAP32OlY9XtJ7FqfWj+4Ty7nnlHCK+9U0BSMdmMJRUSkp1NYd4Dz9AswsgqJrn+lzfovXjyYSDTBn96u6KaSiYhIb6Cw7gDDdOIecwWJvZtJVO9Ire9Xks2EM8t5bd0udu7XAz5EROTkUFh3kGvEJeDyEV2/rM36r1w2lBy/m58v+ViDzURE5KRQWHeQ4fbhGjmF+KfvYDXVpNbn+N3cfu1oqutD/PqVrXrIh4iIpJ3CuhPcZ14BGEQ3tL12fUb/fL548RD+sqlSM5uJiEjaKaw7wcwuwjlsIrGNrxHf+UGbbVeNH8ioQQUsenUbe6r1VC4REUkfhXUneSd9FbN4EKHXfk6i6q+p9aZpMPfqUXjdDn72wscEw7FuLKWIiPQkCutOMlxefNPnY2TlE1r2I6z6/altedkebr/uTKrqQvzod+uJRDV3uIiInDiF9XEwfbn4r/wHAIJ/+iFWsCG1beTAAm6/djR/3dvAT55fTyyuR2mKiMiJUVgfJzOvDN/0+dihBkJ/+iFWqDG17bwRpcyZMZKNO+r4xYsbSVgKbBEROX4K6xPgKB2C74q7sOr3E3xxAVZTdWrbpDF9uPnyYby/rZpf/nELlqVbukRE5PgorE+Qs/9Y/Fd9BzvURHDJAhK1Bx/scfl5/bl+8hDWbtzPfy35mFhc17BFRKTzFNZp4Cgfhv/aewEIvvgI8f3bUtuumTiIGy8byntbq1m4+COCYc1yJiIinaOwThNH4Wn4r7sf05dL6I+PE9u6MrVt6gUDmHvNKD7Z3cDji96nIRDpxpKKiMipRmGdRmZOMf7r7sdRfgbhN58mvPIZ7ETyfusJo8v59pfGUlkXYsFv1rGnprmbSysiIqcKhXWaGd5sfFf+A+5xM4htfoPg0kewArUAnDm4iH+6+WyicYvv//o91m2tbudoIiIiCuuTwjAdeC78Mt7L78Sq20vw+YeI70hOTzq4Ty4P/c359C3K4qd/2MAf3voUSw//EBGRY1BYn0SuIefj/+KDydnOlv8HoTf+H3Y0SEGOh3u+ejYXje3Dy2v+yjPPvUkgpOvYIiJyZM7uLkBP5yjoi/+LDxF9fwnRD/9I855NeCbciB1s4EbzY2YWb8bRGOX9/15D2ZW3M3xgUXcXWUREMoxa1l3AcDjxnH8D/uvux3B5CL/2MyJrnsWq3493xEWEBl3M2eY2diz5L37/xnbiCc14JiIiB6ll3YUcpUPwz/wXErs3Yhb2w8wtBcBj2wT/ksPE9S+zcv3veGTH5Xzz2tGUFfq7ucQiIpIJFNZdzHC6cQ46u+06w8B/4ZeIYHPx+j9hNjt46JcBZk4ZyuXnnYZpGN1UWhERyQQK6wxhGAaeC78MVoJJHy8nJ9vH069bvL+1iq9fNZKyArWyRUR6qw5ds166dCkzZsxg6tSpPPvss4dt//TTT7n11lu59tprue2222hoaDjCUaQ9hmHgmXATrlGXMTayjnvH7mFXdTMPPf0Or7xToad3iYj0Uu2GdWVlJQsXLmTRokW88MILPPfcc2zfvj213bZt/vZv/5a5c+fy4osvMnLkSJ588smTWuiezDAMPJNuwTV8MmW7V7BgwgFGDizguRXbefiZ9/h0byN2Io6te7NFRHqNdrvB16xZw/jx48nPzwdg2rRpLFu2jLvuuguAjRs34vf7mTx5MgB33HEHjY2NRz2etM8wTDwX/01yqtL1S7hj/I1sGj6MTav+TPXzSyh078dRdBpZV9yZGqQmIiI9V7thXVVVRUlJSWq5tLSU9evXp5YrKiooLi7m3nvvZfPmzQwZMoQHHnigU4UoKsru1P7tKSnJSevxuov9pflU/QGa3/5fhmIw1GkTdBXwduB0zqnZQeJ3D1J2zZ3kjp54Un5/T6nH7qZ6TA/VY3qoHtOjq+ux3bC2LAvjkNHItm23WY7H47zzzjv89re/ZcyYMfzoRz/i0Ucf5dFHH+1wIQ4cCGBZ6enWLSnJobq6KS3HygTGpNtwufMxXF6cg88hu+A0RlQ28bvl65gc+COOF37Ip+veoe/Ur2E40jdesKfVY3dRPaaH6jE9VI/pcaL1aJpGpxup7V6zLi8vp7r64AMnqqurKS092PVaUlLCwIEDGTNmDABXX311m5a3nBjD4cQ7/it4zr0OR2F/DMNgUHkuf3frJUQv/Xvesc8kd9dKtj7zPXbuqe3u4oqIyEnQblhPnDiRtWvXUltbSygUYvny5anr0wBnn302tbW1bNmyBYAVK1YwevTok1diAZID0c4d2ZfJt/09n/S/nn6J3Xz2/H/yXy9soLI22N3FExGRNGq337SsrIz58+cze/ZsYrEYs2bNYuzYscydO5d58+YxZswYfvrTn3L//fcTCoUoLy/n8ccf74qyC+Bympxz5XUE3jU494Pnadz1Cvc9dS4Xj+vD1RMGUZTn7e4iiojICTLsDLgHSNesT5xt20RW/4bYphWsL5zKf3/WB9uGi8f15arxAzsd2r21HtNN9Zgeqsf0UD2mR3dcs9YMZj2EYRh4Jt6C3VzH2J2v8vgls1i9183qDZt5d/0Ozh8zkOnjB1Ka7+vuooqISCcprHsQwzTxfuEOgn/8N/jgd0wGJucmt9Xv8LN8y0gigy7iiglDGVCm2zdERE4VCusexnB68F/zz1j1e7GDjdihBuxQA7mffcR1lesIVm5g5f8O56WyCUwZP4JRAwva3IonIiKZR2HdAxmmA0dhfyg8uM499koSVZ/C+y8xteJ94g2beH/pIN7wncXYC85l/KgyXE5H9xVaRESOSmHdizhKh5AzfR5W/T7C65dz3rbVXGj9lR2r3uQ3b42mYMxFTDm7P4W5GkEuIpJJFNa9kJnfB//kr+Eb/yWiW1fR76NXGRR8k+pN7/O798cQ638+139hBH3yvXqWtohIBlBY92KG249nzFTcZ15OYudHFL37PDfXraG2ZgMvPzOavb4zOG/cECaN6UN+tqe7iysi0msprAXDMHEOOpvcgWeRqPiIovf+wI0H3sbmbfZ8UMBb7/YhVjKCIedcyJihJTgdHXoMuoiIpInCWlIMw8A58CyyBowjN7afmo/fpc+ODfQ9sBWzaRMfvfoe9yy7hLNHncbEM8sZVJ6jkeQiIl1AYS2HMQwDb78z8Lj74DnnWux4hPDHKxj7zmL6O5fx048m8/q63ZQX+hk/qowLR5VRVujv7mKLiPRYCmtpl+H04DvrSlxF/TBe+zkPlC5n85Cb+fNONy+s+owXVn3G4D45XDCyjPOGl2o+chGRNNPc4HJER6vHRN0eQst+hB2swzPhZgL9JvDOlmre3rSfisoAAKf3zeW8EaWcO7yE4rzePb2pzsf0UD2mh+oxPbpjbnCFtRzRserRCjcRfv2/SOzZiKPfaLyTv46ZU0xlXZD3tlTx7paqVHAPKM3mrGHFnHNGCf1Ls3vdNW6dj+mhekwP1WN6KKzTQCdjerRXj7ZtE9v8BpG/PAeAZ/yNuEZMSYVxVV2Q97fV8MEn1Wzf3YANFOV6GDu0mLOGFjNiQH6vmDFN52N6qB7TQ/WYHgrrNNDJmB4drUerqZrwm78ksXczjvIz8Iy/EUfpkDb7NDZH+Wh7DR9ur2HjjlqiMQuPy8GoQQWcOaSIMwcXUtJDnwam8zE9VI/poXpMD4V1GuhkTI/O1KNtW8S2vEX0veexQ404h07Ac8EszOyiw/aNxRNs3lnPR9tr2PDpAWoawgCUFfo5c3AhowcVMnxAPj5Pzxj7qPMxPVSP6aF6TA89z1pOSYZh4h55Ca7TLyT64R+JblhG/LP3cA65AOdpo3H0G4XpzwfAmQgzyrWH4YWf8KUCi8bcoaxvzGfDjkZWfrSX19ftxjQMhvTNZeTAAkYMLGBov9xe0WUuInI0CmtJG8Ptw3PBLFyjLiXy3gvEd75P/JPVAJgFfQETq24PYINhAgY+O8GFLi+T+o6E6WPY5R7Kx/tjbNpRx0trd7B0zQ6cDpOh/XIZMaCA4QPyGdJX4S0ivYvCWtLOzC7Cd8lt2PbXsQ5UkNizifjezQC4Tz8fR9mw5HVt2ya+dzOJ3R8T37UBe+cH9DFMTus7kqsvuIBYn3P4pDrOlp11bKmoY8mqz7ABp8NgSJ9chvXPZ9hpeQzpm0e2z9W9f7SIyEmka9ZyRF1dj7ZtY9XuJv7XvxD79B3sxipwuHCNugz3uBmY/jyawzE+2d3Atop6tu6qZ+f+RlpPmz5Ffk7vm8fp/XIZ0jePfsVZmGb33yam8zE9VI/poXpMD12zll7LMAwcRf1xFPXHff4NWDU7iW58ldjHy4lt+jOu0ZfhG3EJox27GeHZQiJrK1bRXiJ5g9jtHcr7zX35cHsNqzbsA8DjdjC4PIfBfXMZ0ieXwX1yKcjx9Lr7vEWkZ1DLWo4oU+rRathP5P0XiW9fC62nqsONo3woZn4fEnu3YtXtBsAsPI3Q8Cv5hCF8uq+J7Xsb2F0VINFybuVluxlcnsuAsmwGlecysDyH/Gz3SQ3wTKnHU53qMT1Uj+mhlrXI55h55fgu/SbW2dcQ370RR/FAzJLBGI6Dp67VWEV85wfEtq7Es/Ypzhp4NhdOugUz+wxi8QQVVQE+29vIZ/sa2bG/iY+219D61TDX76J/WQ4DyrIZUJpD/9Jsygp9OEw9BlREModa1nJEp2I92laC2IblRNb9ATDwnDcT15mXY5htR46Ho3F2VQXYsb+JXZUBKqqa2FPdnGqBOx0m/YqzOK00i9NKsulXknzNy+p8K/xUrMdMpHpMD9VjeqhlLXICDNOBe9yVOIecR3jVb4i8/T9EN76Ke+x0XMMnYzjdAHjdToadls+w0/JTn40nLPbWNLO7OsDuqmZ2VQfY8GktqzfsT+2T7XPRtziLfsVZ9C3Oom+Rn77FWeQeR4iLiHSGwlp6HDOnBN/0+SQqPiLywVIiq39LdN0SXGOm4h55KYb38G+0TofJgLIcBpTltFnfGIyytzoZ4ntqmtlT3cxfNlUSjMRT+/g8DsoLsygv9FNe5KdPoZ/yQj+lBT1zClUR6XoKa+mRDMPAOfAsHAPGkdi3lehHfyT67u+JrluCc+BZuIZfjOO0M1Nd5HY0hNVUg+HLxfTnpY6T63eTO9DNiIEFqXW2bdPQHGVPTTP7aprZXxtk34EgWyrqWLvxYEvcAEoKfBTleikr9FNW4KOsIBniJfleTewiIh2msJYezTAMnH1H4Ow7gkTtLmJb3iK+/W3in72H4c/H8OVhBWog0tz6ARynnYlr2CScg85JdZ1//pj52R7ysz2MHlTYZls4GqeyNsT+2iCVtUHqmmNU7G/k3c2VNIcPtsYNoDDXQ2mBn5L8ZHgnX5M/WV6nutZFJEUDzOSIenI92ok48YoPiX+yFjsRw8wpxswpxsguxqrdRWzbauzmWnD5kjOtHRKaZnYx7rOuwswt6dDvOrQeA6EYlbVBqupDVNWFqKoLUlUXoro+RGMw1uZzXreD4jwfxXleivO9FOclW+jFeV6K8ry9Lsx78vnYlVSP6aEBZiJdwHA4cQ0+D9fg8w7fePoFuM+7nsTeLcQ+WY1Vty/ZDAawIbZvNbFtq3CNuhT32df+dl5PAAAYEElEQVRg+nI7/HuzfS6y++Vxer+8w7aFInFqGsJU1YU40BCiuiFMTX0yyDfvrCMSS7TZ3+N2UJybDO7CXC+FOR4Kcz0U5ngpzPVQkOPF5dTtZyI9hcJa5HMMw8TZbxTOfqMO22Y11xFd9wdiG18jtnUl7tFfwFE2DLNoAEZWwXG3dn0eJ/1Ls+lfevi3bdu2aQ7HqWkIUVMf5kBjmAMNB1//uqehTRd7q1y/i4KWIC9o+cnPbvu+pzyKVKSn079UkU4wswrwTp6Da8z05IC1D/+Y2mZ4sjHyy8GywIpBIk7IsLGcPgxvDoY3G8OXi7PvSBz9RmE4OvbwEcMwkq1yn4tB5UduyUdiCWobw9Q2RahtDFPXGKG2KUxtY4SquhBbK+rbjGBv5XU7Wq6/u1PX4fNS7w+u87g1GE6kOymsRY6Do6Avvqnfwo6GSNTuxjqwE+vALqzGKnA5kkHscOLxugk3NmCHGrHq92IH64mtXwYuL87+Y3EOPhdH0QCM7KIjDmbrKI/LQZ+iLPoUZR11n0g0QX0gQl1ThLpAhPpDXuubo2zf00BDc5RY3Dr8+G4H+Vlu8rLc5GV7yM1yk9uynOs/5H2WW93vIieBwlrkBBhuH87yYVA+7IjbPz8QxU7Eko8M3bGO+I4PiH/6zsFj+fMxcoox88pwFPTDbPkxsovSMpjM43YkbyEr9B91H9u2CUbi1Aei1AciNAQi1AeiNASiNDRHaAhEqagK0NgcJXSEljqA3+MkN8tNjt9Frj/5mtPympvlJsfnIifLTY7fTbbPqaldRTpAYS3ShQyHC+eAcTgHjMO+yMKq+Qyrfj9WoAa7qQarqYbEro+Jb1t98DNZhTgHnpW8b7zPiBNqgbdbPsMgy+siy+uiX/HRW+kAsXiCxuYYjcEoDc1RGpujNAQiyffBGIFglP21QbbtjhIIxTjSfScGkOVzkeN3pbr6k+9bQz75fkA4TjwcI8vnwut29KqR8CKgsBbpNoZp4ig9HUfp6Ydts8MBEnV7sGp3k9izkdi2VcQ2rQCnBzO7CDsWxo5HIBbG8ObgHDAWx4CzcPYbheHydkn5XU4HRXkOivLa/32WZRMIx2gKxmhqjtIYjCbft7w2BqM0h2JU1Yf4dG8jgVAsNVf75zkdRjLgW8I92+9uCXon2V4XWb6DwZ/tSy77Pc6MeL65yPFSWItkIMObjbPPcOgzHEZ/ATseJbF3C/GKD7FDjeDyYrg8GE4PVlM1sb++S2zLW2A6cfQbiXPQuTgHnt1mNrbuZJpG8tq23w3ttNgh2R0fisRbAj2G4XKwd38jgXCMQDBGUyhGcyhGIBRjT3WApmCM5vCRW++QbMH7vc5kr4Ev+Zrtcx2yzkXWIdv93tZlp2aak4ygsBY5BRhON84BY3EOGHvE7XYiTmL/NuIVHxHf+QGRlc8QWfnfOMqH4eg7ElqvC9uA6cDMysfIKsTIysf0FyTDP4O6lg3DwO914fe6KCtsufZfduxJJCzbJhiOp0K8ORyjORQnEE4Ge3MoTnM4llquqgvRHI4RDMc51pRMLqeZCnW/x9ny3onf48KXep9cn3xNfgnwe5343GrRS3oorEV6AMPhTN0bbo+/Eat2d3IQ22friL6/pP0DmM7krWWebAyP/+Csba1NVacbw+lpadF7cZQMwjFgXKcmhTnZzENucSvrxOdaQz4YjtEcjqcCvDmUXA4esi4YidMQiLK3pplgOE4ocuygh+Ttca1B7vO0vH5+OfXjSL66Dy57FfiCwlqkxzEMA0dRfxxF/fGc+0Vs69DZzwywYtjN9VjNddjNtVjN9RAJYIcD2JEAdus86QBGskVuR4PYzXXJa+WRILFNrwMGZtnpOAechaPwNHD7MNx+DI8fw+1raa1n/kjvQ0O+syzbJhxJEIzEUuHdGvDBcIxgJBnwoUg8tb0uEGFPTTOhSJxQJIHVgRmfPS4HXo8Dv8eJ130w1L1uBz63E29LqHvdjoPr3A68ntZ1ydd0TessXU9hLdLDtT5ZLMX0YOSVYeZ1pv15kG3bWAd2Et/5IfGdHxJ99/+OvrPLmwzwrIJka7xkMGbxYMz8Phg94JYt0zBSXd4cx/AA27aJxqxUoIeiccKRRDLcI3HCkTihaKIl2JPvwy37NQSiyX2iCcLR+FGv13+epyXQWwPc1/L+4PqW5ZYvCF63A6+rNfwdyfVuJx6Xicft0K13XURhLSKdYhgGjuJBOIoH4Tn3i1jBeuxAbbL1HQ1hR4MQDWJHwwfXNVUT27aa2MbXkwdxuDEL+mIW9sNReBpGdjEkYtiJGMSjYFnJbnlvNoY3h5irHDvuPKm3rXUHwzDwuB143A4KcjzHfZzW0A9HW8M7GeChltdwNEE4ksB0OqitDya3RZL7RaJxahvDhGOty4nD5qI/FqfDxOt2pMLd4zrkx+1IhrrLicdtfm59a/A7cH/uM26nictpZtQ4iu6msBaRE2L688Gf3+5+tmVhNezHqv6MxIGK5G1pn7un/GiCrW88WZj+PAx/QfJJabklmDmlyfduHzhc4HBhOF2nTDd8Ohwa+sdq4Hf0aVGWZROJHQz9Q78ARGItgd6yLhJLEI4dshyNE4lZBEKxg/vGkj+decajYZAKca8rGejelvB3tyy7neYhQZ/8MuB2O/A4HbhatrXu4275nOeQ5VOpV0BhLSJdwjBNHAV9cRT0xcWk1Hor3ITdXIfhcIPTlRzMZpjYkWbscBN2uIlsZ4yGyv3YwfrU9fb4jvexw8cIHsNMzcdueHMw/PmYWQXJB65kFSTXuTwYTi+4PBguT/J3m/rfomkaqUFvcPwt/kPZtk0sbhGOJYgeEvLJ91Yq0COxBNHUq0U4enA5HE0QiiZoaI4SjVst65OvR7sv/1icDgOPq22wu5xtvxB4nCb9SrKZfuGAtNTD8dJZKSLdyvTmgDfnsPWGJwtySwHIKckhfIQWoR0NYTUlZ3+z42FIxFNd6XakGTvUiB1uwgo1YlV+Qry5HqwjT5N68BebydD25mAW9D049WteWbJMrQPpeliX/MlmGEYqADn6jLfHLZ5IhnY4miAWT4Z/rCXQo3HrkHBPfgmIxhJE4gmiUYtI/HP7xlq+ELR8cahpCJ8aYb106VJ+/vOfE4/H+drXvsZXv/rVI+73xhtv8K//+q+sWLEirYUUETkSw+3DUdQfivp3aH/btpKj3pvrkqPeYxHseBg7lpwNzo5Hk0GfiGEH67Hq9hDd/TFYR7iGazqTLXJnS6vc5Uvdv25mF2L488G2k18eEsmnsBnZRZj5fZLBr7BPK6fDxOkw8Xs7P6r/VNBuWFdWVrJw4UKef/553G43N954IxdeeCFDhw5ts19NTQ2PPfbYSSuoiMiJMgwTw5cLnbg/3LbiWA1V2E1VycFykeSgOaJB7HgkGfTxSLKV37Afa88miIXbKwlGTnHynnbTieFwJl+d7pYR9D5o+RJgON3gcCdfPf5kV76/AMOfd/hIf+mx2g3rNWvWMH78ePLzkwNIpk2bxrJly7jrrrva7Hf//fdz11138cMf/vDklFREpBsYphNHQV8o6Nvhz9jRIHawAUxHcsCbwwWmidV0AKt+L1b9Pqz6/dixECTiYMWT97CHGg628lu3HbVgBrj9ycA2zIO/K3Wve/In2W2f/GkqKiAWNlqWW++HT/YOJK/XK/wzVbthXVVVRUlJSWq5tLSU9evXt9nn17/+NaNGjWLcuHHpL6GIyCkmFZSf4yjyJ7vtO8i2LEhED3bPR5qxm+uSE9oE67EjgeQsc1Y8uW88ih1raf031bTcOheGRBSA6vZ+4aFd+4e08g1PVuoHl7flee0ucLZ+EXGCo6WHwOE6GP6tx/Bk6YvACWo3rC3LanOvm23bbZa3bdvG8uXLeeaZZ9i/f/9xFaKo6Nhz/nZWScnhg1Wk81SP6aF6TA/V4/GzEzGsSBgr0owVCSZ/wslXOxbBikVaXpPX761oBDsWxoqGk/s27ceqCpAIBdofoHcUhsePw5uN6fElu/zNZG+A4XBiur2YLa1/05Oc0tZ0ezFdXgynCzseTZYlGsKORTE9Phz+XEx/Hg5/Lg5/DqYvB4cvJ/mFoQt09fnY7l9VXl7Oe++9l1qurq6mtLQ0tbxs2TKqq6u54YYbiMViVFVVcfPNN7No0aIOF+LAgUDapsHr6H2Ecmyqx/RQPaaH6jE9SkrKk/Xo55gjso2WnyPdhWxbcYjHDhk4F8NOJMCKHTIaP3KwN6D1On/LVLZWNIRtW8lBe7YFkSh2oCHZAxALY8fCyeMejcOd6ik4opb565MtfRe0tvYdLbcFOlwtYwCSyzjcybEBLZcGUp83zOSlBsPEzCnGzCk+pB5P7Hw0TaPTjdR2w3rixIn8+Mc/pra2Fp/Px/Lly3n44YdT2+fNm8e8efMA2L17N7Nnz+5UUIuIyKnDMJ3gdmLgO2m/I9mlH0k+sz0eTYasy3vwHvxEPHUPvh1qSn4RaJ3bPhxIfjYRT43Cb/1iYYebsRNR7HiszeWFY345APBkkfO1n560v7cj2g3rsrIy5s+fz+zZs4nFYsyaNYuxY8cyd+5c5s2bx5gxY7qinCIi0ksYppkaGHfE7Q4nRlYBZBWk5ffZlgWxUGp6XGIRbGywLLAtzOzCtPyeE2HYdmcmgDs51A2eeVSP6aF6TA/VY3qoHtOjO7rBT52JUUVERHophbWIiEiGU1iLiIhkOIW1iIhIhlNYi4iIZDiFtYiISIZTWIuIiGQ4hbWIiEiGU1iLiIhkOIW1iIhIhlNYi4iIZDiFtYiISIZTWIuIiGQ4hbWIiEiGU1iLiIhkOIW1iIhIhlNYi4iIZDiFtYiISIZTWIuIiGQ4hbWIiEiGU1iLiIhkOIW1iIhIhlNYi4iIZDiFtYiISIZTWIuIiGQ4hbWIiEiGU1iLiIhkOIW1iIhIhlNYi4iIZDiFtYiISIZTWIuIiGQ4hbWIiEiGU1iLiIhkOIW1iIhIhlNYi4iIZDiFtYiISIZTWIuIiGQ4hbWIiEiGU1iLiIhkOIW1iIhIhlNYi4iIZDiFtYiISIZTWIuIiGQ4hbWIiEiGU1iLiIhkOIW1iIhIhutQWC9dupQZM2YwdepUnn322cO2v/baa1x33XVce+21/N3f/R0NDQ1pL6iIiEhv1W5YV1ZWsnDhQhYtWsQLL7zAc889x/bt21PbA4EA3/ve93jyySd58cUXGT58OD/+8Y9PaqFFRER6k3bDes2aNYwfP578/Hz8fj/Tpk1j2bJlqe2xWIyHHnqIsrIyAIYPH86+fftOXolFRER6GWd7O1RVVVFSUpJaLi0tZf369anlgoICrrjiCgDC4TBPPvkkt956a6cKUVSU3an921NSkpPW4/VWqsf0UD2mh+oxPVSP6dHV9dhuWFuWhWEYqWXbttsst2pqauLOO+9kxIgRXH/99Z0qxIEDASzL7tRnjqakJIfq6qa0HKs3Uz2mh+oxPVSP6aF6TI8TrUfTNDrdSG23G7y8vJzq6urUcnV1NaWlpW32qaqq4uabb2b48OEsWLCgUwUQERGRY2s3rCdOnMjatWupra0lFAqxfPlyJk+enNqeSCS44447uPLKK7nvvvuO2OoWERGR49duN3hZWRnz589n9uzZxGIxZs2axdixY5k7dy7z5s1j//79bNq0iUQiwSuvvALAmWeeqRa2iIhImhi2bafnYvEJ0DXrzKN6TA/VY3qoHtND9ZgeGXnNWkRERLqXwlpERCTDKaxFREQynMJaREQkwymsRUREMpzCWkREJMMprEVERDKcwlpERCTDKaxFREQynMJaREQkwymsRUREMpzCWkREJMMprEVERDKcwlpERCTDKaxFREQynMJaREQkwymsRUREMpzCWkREJMMprEVERDKcwlpERCTDKaxFREQynMJaREQkwymsRUREMpzCWkREJMMprEVERDKcwlpERCTDKaxFREQynMJaREQkwymsRUREMpzCWkREJMMprEVERDKcwlpERCTDKaxFREQynMJaREQkwymsRUREMpzCWkREJMMprEVERDKcwlpERCTDKaxFREQynMJaREQkwymsRUREMpzCWkREJMMprEVERDKcwlpERCTDKaxFREQyXIfCeunSpcyYMYOpU6fy7LPPHrZ98+bNzJw5k2nTpnHfffcRj8fTXlAREZHeqt2wrqysZOHChSxatIgXXniB5557ju3bt7fZ5zvf+Q4PPvggr7zyCrZts3jx4pNWYBERkd7G2d4Oa9asYfz48eTn5wMwbdo0li1bxl133QXAnj17CIfDnHXWWQDMnDmT//zP/+Tmm2/ucCFM0ziesnfZ8Xor1WN6qB7TQ/WYHqrH9DiRejyez7Yb1lVVVZSUlKSWS0tLWb9+/VG3l5SUUFlZ2alCFBRkdWr/9hQVZaf1eL2V6jE9VI/poXpMD9VjenR1PbbbDW5ZFoZx8FuAbdttltvbLiIiIiem3bAuLy+nuro6tVxdXU1paelRt9fU1LTZLiIiIiem3bCeOHEia9eupba2llAoxPLly5k8eXJqe79+/fB4PKxbtw6AJUuWtNkuIiIiJ8awbdtub6elS5fyi1/8glgsxqxZs5g7dy5z585l3rx5jBkzhi1btnD//fcTCAQYPXo0jzzyCG63uyvKLyIi0uN1KKxFRESk+2gGMxERkQynsBYREclwCmsREZEMp7AWERHJcD0qrNt74Igc3U9+8hOuuuoqrrrqKh5//HEgOdXsNddcw9SpU1m4cGE3l/DU8dhjj3HPPfcAesjN8VqxYgUzZ87kyiuv5Pvf/z6g8/F4LFmyJPXv+rHHHgN0TnZUIBDg6quvZvfu3cDRz78uq0+7h9i/f7996aWX2nV1dXZzc7N9zTXX2J988kl3F+uUsHr1avsrX/mKHYlE7Gg0as+ePdteunSpPWXKFLuiosKOxWL2nDlz7DfeeKO7i5rx1qxZY1944YX2d7/7Xdu2bfuqq66yP/jgA9u2bfuf//mf7WeffbY7i3dKqKiosC+66CJ73759djQatW+66Sb7jTfe0PnYScFg0D7//PPtAwcO2LFYzJ41a5a9evVqnZMd8OGHH9pXX321PXr0aHvXrl12KBQ66vnXVfXZY1rWhz5wxO/3px44Iu0rKSnhnnvuwe1243K5OP3009mxYwcDBw6kf//+OJ1OrrnmGtVnO+rr61m4cCF33HEHcOSH3KgO2/fqq68yY8YMysvLcblcLFy4EJ/Pp/OxkxKJBJZlEQqFiMfjxONxnE6nzskOWLx4MQ899FBqNs7169cf8fzryn/j7T7I41TR3gNH5OiGDRuWer9jxw7+9Kc/ccsttxxWn519QEtv8+CDDzJ//nz27dsHpOchN73Rzp07cblc3HHHHezbt49LLrmEYcOG6XzspOzsbL797W9z5ZVX4vP5OP/883G5XDonO2DBggVtlo+UL5WVlV36b7zHtKz1QJET98knnzBnzhz+6Z/+if79+6s+O+F3v/sdffr0YcKECal1OiePTyKRYO3atfzgBz/gueeeY/369ezatUt12Ulbtmzh97//PX/+859ZuXIlpmmyevVq1eNxONq/5a78N95jWtbl5eW89957qeXPP3BEjm3dunXMmzePe++9l6uuuop33nnnmA9wkbZefvllqqurue6662hoaCAYDGIYhh5ycxyKi4uZMGEChYWFAFx++eUsW7YMh8OR2kfnY/tWrVrFhAkTKCoqApJdtE8//bTOyeNwtAdadeWDrHpMy7q9B47I0e3bt48777yTJ554gquuugqAcePG8dlnn7Fz504SiQQvvfSS6vMYfvWrX/HSSy+xZMkS5s2bx2WXXcYjjzyih9wch0svvZRVq1bR2NhIIpFg5cqVTJ8+XedjJ40YMYI1a9YQDAaxbZsVK1ZwwQUX6Jw8Dkf7/2FXPsiqx7Ssy8rKmD9/PrNnz049cGTs2LHdXaxTwtNPP00kEuHRRx9Nrbvxxht59NFH+da3vkUkEmHKlClMnz69G0t5anriiSfaPORm9uzZ3V2kjDdu3Di+8Y1vcPPNNxOLxZg0aRI33XQTQ4YM0fnYCRdddBGbNm1i5syZuFwuxowZwze/+U2uuOIKnZOd5PF4jvr/w676N64HeYiIiGS4HtMNLiIi0lMprEVERDKcwlpERCTDKaxFREQynMJaREQkwymsRUREMpzCWkREJMMprEVERDLc/weZgf6WiO9BhwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 576x360 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Learning rate: 0.001\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAesAAAE3CAYAAACKD2n1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3XmYFOW99/93VfU6+zALwybIIiCLiBuCQjQKCihGjUGNnCcYHs3Rw5E8R+NRo8kxxiXmkJiYc6K/RJMoiWRTMRGJEuMCEUEURZR9h5kBhll7rarfHz3Tw9jIMMPA9Ex/Xtel3dVV03P39+rhU3fVXXcZruu6iIiISNoyO7sBIiIicmQKaxERkTSnsBYREUlzCmsREZE0p7AWERFJcwprERGRNKewFhERSXMKaxERkTR31GFdV1fH9OnT2blzZ8q6devWceWVVzJlyhTuvvtu4vF4hzZSREQkkx1VWH/wwQdce+21bN269bDrb7/9du69915eeeUVXNdl4cKFHdlGERGRjHZUYb1w4ULuu+8+SktLU9bt2rWLcDjMmDFjALjyyitZvHhxx7ZSREQkg3mOZqMHHnjgc9dVVFRQUlKSXC4pKaG8vPzYWyYiIiJABwwwcxwHwzCSy67rtlgWERGRY3NUPesjKSsro7KyMrm8b9++wx4uP5Kqqnocp2Nu/lVUlMP+/XUARFb8AXvXR6w79V95bulG7rx+LD3yAh3ye7qSQ2siCapJKtUklWrSkuqRqj01MU2DwsLsNv3MMYd1nz598Pv9rFq1ijPOOIMXXniBiRMntuk9HMftsLBuej8AOxomdmAv4ahNRVWISNTu0N/TlWTq5z4S1SSVapJKNWlJ9Uh1ImrS7sPgc+bM4cMPPwTg0Ucf5cEHH+SSSy6hoaGBWbNmdVgDj4Xh8YEdxTITh+VtfclERKQLalPPeunSpcnnTz75ZPL5sGHD+MMf/tBxreoolg8cG5NESGuPUEREuqJjPgyezgyPFwCPm5ikRT1rEZG2s+04VVWVVFTswHGczm5OWqmoMD+3Jh6Pj8LCEizr2KO2W4c1li/xQCKsHVdhLSLSVlVVlQQCWeTnF2Db+nf0UB6PSTyeGtau61JfX0NVVSXFxb2O+fd077nBm3rWqGctItJe8XiU7Ow8XZbbBoZhkJ2dRzwe7ZD369ZhbXgae9aNh8F1zlpEpH0U1G3XkTXr1mGdPAyuc9YiItKFdeuwbu5ZxwD1rEVEuoP33lvJrbf+385uxgnVrcOaxrA2ncaw1gAzERHpgrr1aHDDSgwwM3UYXESk29m+fRuPPPIAtbU1BAJBbrvtPxg+fARLlixmwYJfY5omvXv35tvfvp/q6oP81399m1AohGka/Pu/387IkaM6+yMctW4d1k09a8vRADMRkY7w9od7eGvNnuPy3ueN7sWEUUd/mdP993+br371/zBp0oV89NGH3HPPt/jtb//Ek0/+D0888RSFhT14/PEfs337Vt588x+MH38e1103i3/+cxlr1ryvsE4XTeesTTcGGAprEZFuIhQKsXv3LiZNuhCAkSNHkZeXx/bt25gw4Xy+8Y0bmTjxC0yadCFDhgwlFApx9913sH79p4wffx5XXXVNJ3+CtunWYU3TYXAnBvh0GFxE5BhNGNW23u/x4rqHm4gEbNvmttv+g40bZ7B8+Vvcf/+3mT37/zJlylSeeWYhy5a9xWuvLeGvf13Ej370s05oeft067BO9qydOOBTz1pEpJvIysqmd+8+/OMfS5OHwQ8c2M/AgYOYOfNL/PSnT3DDDV8jHo+zfv2nbNq0geLiUq655lpOP/1MZs++vrM/Qpt067Bu2bPWADMRke7k3nvv5wc/+D6/+MXP8Xp9PPDAI3i9Xm688SZuu+0W/H4/hYWF3H33d4hGo3z3u/fw178uwjRN7rnnu53d/DbJgLA2MHTplohItzF27JmMHXsmAD/96RMp6y+++BIuvviSlNd/9rP/77i37Xjp1tdZG4YBllc9axER6dK6d8+axvPWtmYwExGRrqtb96wB8PiaD4MrrEVEpAvKgLD2QlyHwUVEpOvq/ofBLR84ifuJaoCZiIh0RZnRs7bVsxYRka6r24e1YWmAmYiIdG3dPqxpHA1uGOpZi4hI19Ttw9qwvBCPYpm6kYeISCZ54IHv8Ne/LjriNuedd+YJas2x6fYDzPD4cONRTIW1iMgxi61/m9inbxyX9/YOnYj3lAnH5b27um4f1kbjADPLNHQYXESki7vrrtuZPPkSvvCFLwIwe/ZX+bd/m8cTT/yMSCRMbW0dc+fO4/zzv9Cm9w2Hwzz88PfYuHE9pmkyc+ZXufTS6WzcuIFHHnkA27bx+Xzcddd99OrVmwcf/C6bN2/CMAyuuOJqLr/8S8fh0zbr9mGNx5/oWRvqWYuIHCvvKRM6tfc7ZcpU/va3l/nCF77Ijh3biUaj/PGPz3Hnnd+mf/8BrFr1Lj/+8aNtDutf/vLn5Ofn85vfLOTgwYPMmfMvDBkylIULFzBz5le58MKLePnll1i79kP27aukpqaGp55aQFXVfn760x8rrI+VYXnBbjxnreusRUS6tPHjz2P+/EdoaKjn1VdfYcqUS7nmmutYtuxN/v73V1m79kNCoVCb33fVqpXceee3ASgoKOD88yeyevUqzj13Av/934/wzjvLmDBhIhMmnE9dXS3bt2/jm9+8lQkTzuOWW/69oz9mim4/wCwxGjyOZWo0uIhIV+f1epkw4XzeeusNli79GxdffAm33DKHdevWMnToMGbNmo3bjo6Z6zqfWQbbjnPBBRfxy18+w/DhI1i4cAGPPvog+fkF/OY3C7nqqq+wbds2Zs/+KrW1tR31EQ+r+4e15QPAZ9o6DC4i0g1MmTKV3/3uGfLzC8jKymLHjm3ceOPNjBs3gTff/AeO47T+Jp8xduxZ/OUvLwBw8OBB3nzzdU4//Uzuvfc/WbfuY6644iq+/vWb+fTTT3jrrX9w//33Mn78eXzzm7cTDAapqCjv6I/ZQvc/DO7xAuAzHPWsRUS6gdGjx1BXV8cVV1xNXl4+06fP4IYbrsHj8TB27FmEw+E2Hwr/2te+zg9/+DCzZn0Fx3GYNWs2Q4cO44YbvsbDD3+Pp59+Eo/Hy3/8x52ccsowXn99KTfccA0+n58pU6YyaNDg4/RpEwy3PccLOtj+/XUd1ustKcmlsrL5cET0k38QeeMpfuJeT1Gv3tx0+YgO+T1dyWdrIqrJ4agmqVSThL17t1FW1h+PxyQeb3uvtTtrrSZNtTuUaRoUFeW07fe0q3VdiGE19axt9axFRDJMJBLmpptmH3bd179+E+edN+kEt6h9un1Y40mcs/brnLWISLulwUHYdvH7Azz99IJO+d0dWbNuP8DMaBpgZiisRUTaw+PxUV9f02UDuzO4rkt9fQ2exg7jscqAnnXjYXDTpkFhLSLSZoWFJVRVVdLQUNOukdbdmWman1sTj8dHYWFJh/yebh/WRuNejdew9SUTEWkHy/JQXNxLA+4O40TVpNsfBm86Z+3DRh1rERHpirp9WDeds/ZqNLiIiHRR3T6skz1rI64BZiIi0iV1+7Buus7ao561iIh0Ud0+rJvPWatnLSIiXVP3D+umnjXqWYuISNfU7cPaMAywvHiJ637WIiLSJXX7sAbA41PPWkREuqyMCGvD8uIhrklRRESkS8qIsE70rDXATEREuqajCutFixYxdepUJk+ezLPPPpuyfu3atVx11VVcfvnl3HTTTdTU1HR4Q4+F0XgYXFktIiJdUathXV5ezvz581mwYAHPP/88zz33HBs3bmyxzQMPPMDcuXN58cUXOfnkk/nFL35x3BrcLpYPy43rnLWIiHRJrYb1smXLGDduHAUFBWRlZTFlyhQWL17cYhvHcaivrwcgFAoRCASOT2vbyfB48bg6DC4iIl1Tq3fdqqiooKSk+RZfpaWlrFmzpsU2d955J7Nnz+b73/8+wWCQhQsXtqkRRUU5bdq+NSUluS2W9wSz8Jq1uK6bsi5TZOrnPhLVJJVqkko1aUn1SHUiatJqWDuOk7hWuZHrui2Ww+Ewd999N08//TSjR4/mqaee4lvf+hZPPPHEUTdi//66Duv1Hu52ZTHHxLRjxG03I2/vptvapVJNUqkmqVSTllSPVO2piWkabe6ktnoYvKysjMrKyuRyZWUlpaWlyeX169fj9/sZPXo0AF/5yldYsWJFmxpx3Hm8mDpnLSIiXVSrYT1+/HiWL1/OgQMHCIVCLFmyhIkTJybX9+/fn71797J582YAXnvtNUaNGnX8WtwORuMAM52zFhGRrqjVw+A9e/Zk3rx5zJo1i1gsxtVXX83o0aOZM2cOc+fOZdSoUTz44IPcdtttuK5LUVER3//+909E24+ex4vlxHBcN+UwvoiISLprNawBLrvsMi677LIWrz355JPJ55MmTWLSpEkd27KOZPkw3TgAjutiKaxFRKQLyYgZzAxP4jA4uDoULiIiXU5GhHXTPa292Gh6cBER6WoyIqyNprA2dOctERHpejIirLG8QGPPWve0FhGRLiYjwrq5Z61rrUVEpOvJiLBu6ln7DFsDzEREpMvJiLBO9qyJU1Mf7eTWiIiItE1GhDWHDDDbvLu6kxsjIiLSNhkR1kbjYfD8AGzeXdPJrREREWmbjAjrpp513x5+NimsRUSki8mIsDasRFj3KvCy90AD9eFYJ7dIRETk6GVEWDf1rHvmJaZC36LetYiIdCEZEdZNo8GLcywMdN5aRES6lowIazyJAWYebHoXZ+u8tYiIdCmZEdaNo8GxowzsnceWPTW4mnZURES6iIwIa8MwwfJAPBHWdaEYFQdDnd0sERGRo5IRYQ2A5cONRxnUOx+Azbt0KFxERLqGjAlrw+MDO0rv4mz8PkuDzEREpMvImLDG8uLGY5imwclluWzStKMiItJFZExYGx4fxBM38RjYO58dFXVEY3Ynt0pERKR1GRPWeHy4dmLmsoG987Adl+3ldZ3cKBERkdZlTFgblveQnnUegO7AJSIiXULGhHWiZ50I64IcP0V5uqmHiIh0DRkT1olz1s038BjYO18jwkVEpEvImLDGau5ZQ+JQ+P6aMNV1kU5slIiISOsyJqwNj7dFzzo5OYp61yIikuYyJqwTM5g196JP6pmDZRo6by0iImkvc8L6Mz1rn9eib2mORoSLiEjay5iwbppu9NC7bQ3qnceWvbU4ju7AJSIi6StjwhrLl3i0Dx0RnkckarN7X30nNUpERKR1GRPWhqfpntaHGWS2R+etRUQkfWVMWDf1rN1Y8yCz0sIg2QEPm3bpvLWIiKSvjAlrs0dfAOzdHydfMwwjMTmKetYiIpLGMiasrZ6DMfJ7Evv0zRavD+ydx+7KekKReCe1TERE5MgyJqwNw8B7yvnYez7FqS5Pvj6odx4usFW9axERSVMZE9YA3lMmgGEQW/9W8rWTG+/ApclRREQkXWVUWJvZhVh9RxFb/xau4wCQHfDSs0eWph0VEZG0lVFhDeAdej5ufRX2rrXJ1wb1zmPT7mrittOJLRMRETm8jAtrT/8xGP6cFgPNzh5eSm1DjD+/sbkTWyYiInJ4GRfWhuXFM+Rc4lvfww3XATB6UDETT+vN4ne2s27rgU5uoYiISEsZF9aQOBSOEye28Z/J16794hB69sjiyZc+pi4UO8JPi4iInFgZGdZW0UmYxf1bHAr3+yxuunwEtQ0xnvrruhY3/BAREelMGRnWkOhdO/u3Ye/blnytf1kuV00axOoN+/jHB7s7sXUiIiLNMjesB40Dy5Myo9nks/tx6oBCfvfqBvbs1924RESk82VsWBuBHDz9xxLbuBz3kDtxmYbBjdNOxee1+PkLa4nFdTmXiIh0rqMK60WLFjF16lQmT57Ms88+m7J+8+bN3HDDDVx++eXceOONVFd3jbtYeYdNhEg98W2rW7xemOvna1OHsb2ijj+9samTWiciIpLQaliXl5czf/58FixYwPPPP89zzz3Hxo0bk+td1+Ub3/gGc+bM4cUXX2T48OE88cQTx7XRHcXqfSpGdo+UQ+EApw8p4YLT+/DKih18tGV/J7ROREQkodWwXrZsGePGjaOgoICsrCymTJnC4sWLk+vXrl1LVlYWEydOBODmm2/m+uuvP34t7kCGaeI9ZQL2jo9w6lKvr77mwsH0Ls7miRc/Ztve2k5ooYiICHha26CiooKSkpLkcmlpKWvWrEkub9++neLiYu666y7WrVvHwIED+fa3v92mRhQV5bRp+9aUlOQe9baxcy9hx+pF+HavpHDCVSnrvzPnXO7+37d59Ln3+e6ccQzt36Mjm3rCtKUmmUI1SaWapFJNWlI9Up2ImrQa1o7jYBhGctl13RbL8XicFStW8MwzzzBq1Ch+9KMf8dBDD/HQQw8ddSP276/DcTrmuuaSklwqK9vSC87G6jWMg+8uJpzTD6vXsBafzwPcPnMMj/72fe7532Xc9uXTOKVfQYe09URpe026P9UklWqSSjVpSfVI1Z6amKbR5k5qq4fBy8rKqKysTC5XVlZSWlqaXC4pKaF///6MGjUKgOnTp7foeXcFvjO/BE6c0EsP0/DC/cS2rMJ1m0eBF+cH+db1YynM9fPfC9/nY01JKiIiJ1CrYT1+/HiWL1/OgQMHCIVCLFmyJHl+GuD000/nwIEDfPLJJwAsXbqUESNGHL8WHweeXkPJvvZR/OfNwg3VEv7bT2hYeBexT95IXtZVmOvnjuvGUloQ5Ee/X8OaTfs6udUiIpIpWg3rnj17Mm/ePGbNmsUVV1zB9OnTGT16NHPmzOHDDz8kEAjw+OOPc8899zBt2jTeeecd7rzzzhPR9g5leHz4Tr2Q7K88ROCL3wCPn/Abv6T+t7cT/eQfAORn+7jjurH0KcnmJ3/8kFWfVrbyriIiIsfOcNNgEuzOPWd9eK7rYu9aS/S9F7H3rsc/8Wv4hk0CoCEcY/7vP2DL7lpunDacc0eWHfPvO550nimVapJKNUmlmrSkeqRKm3PWmcowDDx9RxKcfgdWv1FE3vwV8e0fAJAV8PLNa8ZwSr98nnzpY55bugHb0UxnIiJyfCisW2GYHoIX3YJZ1I/Qq49jV24BIOj38M2vjOGLY/vyyood/PB371NTH+3k1oqISHeksD4KhjdA8JJ5GME8Qovn49RUAOCxTK6ffAo3ThvOpt01fPfpd9m8u6aTWysiIt2NwvoomVkFBC/9Jq5j0/DyD3HCzecoJozqxV1fPQPLNHjo2VX84/1dndhSERHpbhTWbWAV9CY45Tbcuv2EXvkxbjySXNe/LJd7/89ZDD2pkF8t/pSnX16nO3aJiEiHUFi3kadsCIELbsIp30R46c9xDxlYlhP0Mu/LpzHt3P688cEe7v/VSs0pLiIix0xh3Q7egWfhH38d8a3vEVn2DIde/WaaBldNGsTcq0dT2xDle79eyZ/f2EzcVi9bRETap9W5weXwfCMvxqnbT2zNYoysfPxjZ7RYP2ZwMYO/fg6/fXUDi5ZtZfWGSm6cdir9yzQJvoiItI161sfAf841eIZMILryz0Q/XpqyPifoZc5lpzL3qtHUhmLc/6uV/Okfmwhvfg+n4WAntFhERLoi9ayPgWGYBCZ9jVCkjshbv8EI5OIdeFbKdmOGFDOk3zn8YclHlHz4K2Kfbqchrw+FV9+H4fF1QstFRKQrUc/6GCUmTflXzJ6DCC/9OfFdHx92u0DdLq6qX8Bp/p2ssk/BW7OL93/3OFW1kcNuLyIi0kRh3QEMj5+sS+Zh5pcRWvIYduXW5DrXdYl+vJSG578Hjk32Zf/JObPvYEuPcxnc8AHP/fJZFr29hWjM7rwPICIiaU1h3UEMfzbBqf8Pw59N6OUf4hzcixsNEV76v0Te+jVWn1PJuuq7WGVDCPo9jLryRuziQVwTXM7bb7/P3U++w8pPKkiD+6qIiEiaUVh3IDO7kKyptwPQ8NcfUP/n7xLfvALfWVcTvOQ2zEDzSHDD9JA35VZ8gQB39F1Brs/lZ89/xCMLVmvKUhERaUFh3cHMgrLEtKSReoiFCU77Fv7Tp2MYqaU2swsJXHgT3rq93D54HbOmDGXXvnq+9+uV/Pj3H2hCFRERATQa/LiwSk4m+8sPYHgDGP7sI27r6TsS39jLiL73IhMmDeWcm8/ltVU7eWXFdr779LucPqSYK84fSL/Stt37VEREug+F9XFi5hQd9ba+sVdg791A+K3fkFVyMtPHD+DCsX15deUOXnl3O/f9cgVnDitlxoQB9ClRaIuIZBodBk8DhmkSuPAmDF+A8N9+ilNTQVbAw+Xnncwj3xjP9PED+HDzfu79xQp+8sc1fLq9SgPRREQyiHrWacLMKiDwxW8Q+ssPqP/dHRi5JXj6jsDXZwRfOns4F5/Zl7+t3Mnrq3exesM++pflMuWsfpw5rBSPpX0uEZHuzHDToIu2f38djtMxzSgpyaWysusOzHJqKohvX4O9ay3x3esgFgYMzJIBePqNwh0+mX9uqGbJih3sPdBAYa6fi87oy8QxvckOeA/7nl29JseDapJKNUmlmrSkeqRqT01M06CoqG2nNNWzTjNmXim+kRfByItwnTh2xRbsXWuxd64lunoR5s6PmHTp/2Piab35aPN+Xlmxg9+/vokX3t7CWUNLOW90L07pV4BhGJ39UUREpIMorNOYYXrwlA3BUzYEzriC2Nb3CL/6MxpeepjgtNsZPaiY0YOK2V5ey9L3drJiXQVvf7SX0sIg543qxfiRZfTIC3T2xxARkWOkw+BdTHzHGkJLfoKZV0pw2u2YWQXJdZGozcpPK3j7wz18sv0ghgEjTu7B1PEDOblnNn6v1YktTy/d/XvSHqpJKtWkJdUj1Yk6DK6w7oLiuz4m9MqPMbILyZp2B2ZOj5RtKqoaeOvDvSz7aA8HaiL4vRZjhhRz9vBSRp5chNeT2YPSMuF70laqSSrVpCXVI5XCup0y5csU37uB0Ms/xAjkkjX9DszcksNu5zgu5bUR/rZ8Kys/raQuFCPo9zD2lGLOHt6T4f0LM3I0eaZ8T9pCNUmlmrSkeqTSADM5Ik/ZELKm3UHDyz+k4cUHE4GdX5aynWkajB5cQq/8ANddfArrtlWxYl05763fx9sf7iU74GHUwCJOG1zMqIE9yPqcEeUiItJ5FNZdmFU6kKzp3yL0lx/Q8OL38U+4Ac/JZ37uSHCPZTJqYBGjBhYxa4rDR1v28976StZs2s8/Py7HNAxO6ZfPaYOLGTO4mJ49sk7wJxIRkcPRYfBuwK7aTfi1/8E5sAOrz6n4x1+PVdgnub61mjiOy+Y9NXywcR/vb9zHrsp6AEoLgwzvX8iwkwoZ1r+Q/Gzfcf8sJ0omfk9ao5qkUk1aUj1S6Zx1O2Xql8l1bGLr/k7k3T9BLIJ31MX4x87A8AVTauLU7iO+bTXxbe+DYRC86BYMXzC5vvJgiA827uPjrVV8uqOKUMQGoHdxNsNPKmRY/wKG9CsgL6vrhnemfk+ORDVJpZq0pHqkUli3U6Z/mZxQDdF3/0jskzcwgnn4z7mGXuMnU75ubWNAv4ezfwcAZn4ZTk0lVtlggpf+PwxPavjajsP28jo+2VbFum1VbNhZTSSWCO/SgiAD++QxqHc+A3vn0a80p8sMVsv078nhqCapVJOWVI9UCut20pcpwa7YTHjZMzgVmzF8QdxoCAwDq+cQPANOx9P/dMz8MmIb/0l46c+xThpNcPK/YZhHHsYQtx227Klh465qNu+qYePuaqrrogB4PSb9y3IZ2CuPAWW5DOiVR2lhEDMNZ1PT9ySVapJKNWlJ9Uil0eByTKzSgWTNuIf4+rfxHNhEvMcgrJNOwwzmtdjOO3gcbjRE5K1fEf77kwQuuAnD/PzesccyGdK3gCF9E5OxuK5LVW2ETbtr2LSrmk27q/n76l3E4g4AQb8nEdyN4X1SaQ4lBUFMM/0CXEQkXSmsuzHDMPEOPZ+SkqlH3PPznXoBbjREdMVCIr4g/vP+5ajnFjcMgx55AXrkBThrWCmQOHS+e18DW/bUsHVvLVv31LDk3R3YjUdPfB6T3sXZ9C3NoW9JDn1LsulbkkNeNxrAJiLSkRTWAoB/zFSINhB9/yUMfzb+s7/c7veyTJN+pTn0K81h4mmJ12Jxh1376thRUceuynp2VNSxZuM+3lqzJ/lzOUEvZUVZ9C7KoqxHNr2LsygryqY4L6CeuIhkNIW1JPnOugo32kD0/b+ALwv/mGkd9t5ej8mAsjwGlLU8DF9TH2VnZR07K+vZs7+ePfsbWL1hH7UNzSHusUxKC4P0LAxSWhiktDAr+bxHroJcRLo/hbUkGYaBf8JXE4G94vcQC+M9ZcJhZ0brKHnZPk7N7sGpA1rOb14XiiXDe8/+eiqqQlRUhfhoy4Hk+XAAj2VQnN8U4kFKC5rDvCg/0GVGp4uIHInCWlowDJPAF75O2LGJrl5EdPUijPwyPCedhqf/GKyyIa2OGP8s13XBtcGxMTz+o/qZnKC3xUC2Jo7rcrA2QnlViIqqBsqrQlRWhSivCvHp9oPJy8oADCA/x0dR4zn1orwAhXl+ivICDIrYGLZNbtCre3+LSNpTWEsKw/QQvOgWnNpK4ts/IL7tfWJrXyP24SvgC+LpOwoztxg3FsGNhSEWxo2FceORxPN4FOwYxGO4duPzxisErX6j8J91NVZx/3a1zTxkQNvw/oUt1rmuS019lIqDiV545cEQB2oi7K8Js728ltUb9hG3nRY/4/WYFOb66ZHrb3xfPz1yAxTm+hOv5wXIDngU6CLSqRTW8rnM3BJ8Iy7CN+Ii3FiY+K612Ns+SAT41vcwvAHw+jG8wcbHAEYwHzw+DMsLlhc83ubn8SjRdX+n4U/34Rl4Nv6zrvzcQ+yu6+Ic2EF84z+xK7dglQ3BM2AsZlH/zw1OwzDIz/GTn+NP6ZE3vWdtQ4z9NWFsw2DLzoMcqAlzoCbCgdow67ZVcbAuwmdnHvA1Bnphrp+CHD952T7ysn3kf+YxN8uLdYTL3kRE2kthLUfF8AbwDjgD74Azjul9fGOmEv3gZaIfvkJ8y0q8Q8/HN3ZG8p6W9mfEAAAabUlEQVTcTvVeYhvfIb7pnzgH94BhYhb2ThySf+9FjOweiUldBpyB1euU5CF5NxbGObgHp2o3TtUu7KrdGB4f/nO+nLx9qGEYyaAtKcllcFluSvtsx6G6LkpVbYSq2ggHaiNU1YYTz2sibNpdTU19rMXh9mSNgNwsL3nZfvJzfORl+cjPaQzzrESY5x7ymOn3FBeRo6ewlhPK8GXhP+sqvCMuIrp6EbF1fye2YRnewedi79+Gs28bYGD1OgX/yIvxDDwLM5CLE6rBbuzRxz55g9ja18CXhVXcH6emArduf/MvMS3M/F7YdfuI71iD/5xr8A7/AobRejhappk8zH4k4WicmvooNfUxqusjVNdHqamPUl0fpbou8bh3fz3V9VHi9uFn5wv6LXKzEkGel+0jrzHEm3YocoNecoJesoNecoIevB6rLaUWkW5EYS2dwszKJzDhq/hGTSGy6nli69/ELB6Af9xMPAPPTva0k9sH8zCHno936Pm4sQjxXR8R37oap2oXVtkQzIJJmIV9sAp7Y+SVYpgWTu0+wm88ReStXxPftILApNmYeaUd0v6Az0PA56G08Mjbua5LQyRObUOMmvootQ0xahui1DZEqUk+j1F+oIENO6PUNcT4vIl3fV6T7EAiwHOCXrIDnsYg95IdSCw3hXt2wENWQCEv0l1obvAM0BVq4jo2htnxoeK6LrFP3yCy/Hfg2vjP/jLeEV+ktDS/1Zq4Tjwxp3o0hBttwI2GMPzZmLklLe5S1pEcx6Uu1BTsUerCcepDMeoa/6sPx6gPxQ95HqM+HE/ODnc4Po9JVmOwN4X9ocHetNyrZx6RUJQsv4eA30PQZ+H1mBk9uK4r/O2cSKpHKs0NLhnleAQ1JM5T+4ZNwtN3JOE3f0Vk2bPEN79L/YQZxCr344RqcEPVuKGa5v8i9bjRBohHP/99A7kYeaWYeSWYeaWYuSWYxf0xC/sc02cxzebz6kfLdV3CUTsZ3E1B3hCON4f7IcFeXtWQ2CYU+9xD9E0s0yDo9xD0WwT9HrL8nubHQOKx6Xl2wJtYbno9kDj6oElrRI6dwloygplTRPCSecTXv0V4+W8p/+MPmld6AxjBPIxgHmZ+GUYgG3xZGL4gRuMjviCGN4gbqUucI6+pwKmpxN67gfimd5KXpuHxYZWcjFU6CLN0YOIxu5Vj5cfIMJoC1UNxG37OdV2icSfZc/cFfewtryUUiROKxhOPETu53BBOvFZ5MERDpHl9a/w+i6DPSrYx6LMae+4eAj6LgN9qPK1gNf7nwd/03Gs1Pvfg91oalCcZ66jCetGiRfzP//wP8Xicf/mXf+H6668/7Havv/46//Vf/8XSpUs7tJEiHcEwDLxDz8fT/3TyzFqqI55ESB/lRC2fx7XjuLWV2JVbsCs2Y1dsIvrhK+AkgszILsQs6JU4fJ5Xgpnb2CPPLQF/Nrg2brg+0aOP1EOkDjdch+vYmAW9sHr0xfBnd0QJWjAMA7/Xwu+16JEXoKQkl7K8ttXCcVzCjUHeEGl+rA/HCIXjhKKNYR9pfh6OxKmqixKOJsI+HI2nXC73eZp6+oGm8G8M/qaQD/gSn8ff+PjZZb/Xwuc1W7ymWe6kK2g1rMvLy5k/fz5/+tOf8Pl8zJw5k3POOYfBgwe32G7fvn08/PDDx62hIh3FCOQQKOlFbQedezMsD0ZBL8yCXniHjAfAjUdx9m/HrtiEXbkVp6ac+Nb3cMOf+Z2WB+x4678juwdmj75YPfpi9uiLWTIAM79Xp59PNk2DrICXrIC33e/R1MMPRxPBHY7YRGJ2cjkStQkfshyO2oQjzUFfXRdl7yHbRaP25w7SOxzLNFICvel5Xq4fHIeA14PPZyZ6+o3rfYeEv8/Tckcgsc7EY2X2OX/pOK2G9bJlyxg3bhwFBYlJJqZMmcLixYu59dZbW2x3zz33cOutt/LDH/7w+LRUpAsxPD6snoOxerbcqXWjIZzafTi1Fbg1lTgNBxOH2/3ZGP6cxGMg8Yhh4FTtxj6wE6fxv+iutc099ryeiWvO+5+O1XPIEe9D3l5uLIy9ax1GQU+sgt5H/3OReuy968HyJj6TLwv8jacWPjNd7aE9/PwOuE1qU/g3hXckmgj/SGOQhw95nnjdSTwesl0kalPTEKWqLkJ9KEa0cWfhSAP5DscwSAS3x8TXGOJej4mvadljNi43hr439QhAU/B7Pc07Bi0ftVOQCVoN64qKCkpKSpLLpaWlrFmzpsU2v/71rzn11FM57bTTOr6FIt2I4QtiFfXDKup3VNubuSV4Tmr+u3KdOE51OfaeT4lvW03so1eJrVmMEcjFOmkMngGnYxaU4dbux6nbj1u7L7FzULcPt3Yfhj8Hq99IPP1GJ+Z5t1J7xG48SnzHh8Q3vUN82/tgJwbamUX98Q4eh2fQOSmX1kFiRyS+bTWxTe9g7/wouVORwhvAzCrAc8p5eIdPwgykTk5zLA4N/7zWNz+iz470jdvNwd60QxCN20QbAz/atCMQc4jGEzsC0eTrjc/jDrF44pRAdV3ieTTuJLf57JS4R6tpJ8D72R0AT3Pg+z4T/ofuBPg9jTsSTTsRTT/b9H5ei9xoHMd1MbVjcMK1GtaO47TYY3Ndt8Xy+vXrWbJkCU8//TR79+5tVyPaOoS9NSUlHfvH3x2oJqm6bE16FsIpw4AZOJEGGja/T8OnK2jYuIrw+jdbbmtaePKK8RWU4uk5FrumklBTwHsDBAeMImvQGIInn0bD5k24a9+i/tN3cCMNmFl55J12AVlDzyG2bwd1H71J5J3niLyzkMBJw8kZcT5Zg88gvGs99R+/RcPG93DjUazcIvLPmkbWKWcCBk64HidSjxNuSDwP1xEp30r43T8Qe+8FskecT/5Zl+IvG9gZ1WzVif6e2I5LJBpP7hSEozaRaDy5Q9DiKEDUJhZPPI/FnOTOQizevPMQjtqEYjYH66PJ9w1HE+vae+GuxzKTOwLez/T2m5abAj65A3DIkQKfxyLLrSevbhteYrh5ZRj5vbCCOXgbjxQ07XR4G3/OY7iYteW4jTMUBgeMItBveMcWv51OxHek1eus//znP7Ny5UoeeOABAB5//HFc100eBn/sscf4y1/+QiAQIBaLsX37dkaPHs2CBQuOuhG6zvr4Uk1SdceauE4ce8963IaDGLnFmDlFGFmFKYfH3VgYe/e6RO95xxrc2n3NK71BPCePxTtoHFafU1MuQ3Oqy4lt+ifxjY3TwTYygnl4Bp6FZ9A4rJ6Djmq2OPvALmJrXyW24W2IR7HKTsE78iI8A8a2+c5ux0t3/J40cV2XuO009uqbe/2H9v6jMYdYPHGUIBp38Pm9HKwOEYsnXo/Zie2Syy1eP/Q/G79dx0nsYZC1l0Hecsqs6pQ21ToByu08Kux8Kpw8PNj0sg7SyzpIqVWDx2h51GFFfBivuedgW/5EwDeGvMcy8VhG4vGQZa9ltnjNaxmHPP/MzzU+tyyTkoIAxfmHn1vhRF1n3WpYl5eXc+211/KHP/yBYDDIzJkzuf/++xk9enTKtjt37mTWrFltHg2usD6+VJNUqkmC67q41eXEd62loKwX9fmDMTytnzd2XRdn/3biO9dilQzA6jW03deXu5F6Yp++SXTta7i1lYlL6bIKMIN5ievZs/ITj8E8DMuLe8gkNW60ITlpjRHMx9PnVKy+IzBzitrVls/S96Slz9bDdV3cun3Y5RtxqnYn77jnxqOJeQrsGK4dw6mtxK0uT/yQN4hROhinZAix4sHEzCDU7IWavZi15Zi15Xjqy7FiDQBE/T0IBUup85dS5y+h2ltCnZHDgMo3Obl6BWEzm3fzL2abZyBx203sPNgutp3YaYjbiZ2SuO0QjzvEmpbjzlEPRMzye/jpvIlHVZOjcVwmRenZsyfz5s1j1qxZxGIxrr76akaPHs2cOXOYO3cuo0aNatMvFJH0YRgGRkEZvoIysktyaTjKf3QMw8Aq7t/uW522eC9/Nr7Rl+AdORl7xxriOz9KTlDjVO/B3bseN1wHn/2n1RtocR28s3sd8U3/BMDML8PqOxJP3xFYvYYlZ5xL3ls9nggR7FjzbV6jYdxYKPkcO0pdrz7YVmHi+vuj3InBjiUG1nXD87pOPIq9dwN2xUbsvRuxyzfihhp7yIYBHv8hd9zzYXgSz62C3ljDv4DVazhmUb/D7NidnPq7wrUYpucIswWOxK6YgvXGU0w88OfEnfzGX4+Zld9iK9d1cKr34lRswanak5zQyMgtxs0uxrb8jSHvYNsOcdvGrqvCrdmLUVuBWVuOP6vjL51sK003mgFUk1SqSap0ronr2InL3uxYYmS5N5h6eN91cap2Yu9cS3znR9h71icGxxkW+AKJED3k3uptYhgYuaWJ694b5593I/W4DQdx66twGg42Pj8ITjwRVMH8xEQ7wbzE86z8xlH+ZiLYDAMwmp87duL6+kh942PiOeE68AawigdglgzAKjk5MVOedWJOFbiui73nU2LrXsfeuiqxkwMYuSWJKx7KhmD1HIxZ2Pe4XJHQavuceOJOfqteAK8f/znXYARycSo2Y1duxq7YArFQYmPDSuysHSI5G2EwD6duH051ecvZCz0+rN6nEpzy74fdAUubw+AngsL6+FJNUqkmqbpbTdx4FLt8I/aujxNzvB9yb/WmHh+WN3Efdm8g0VP3BhI9OW8Aw/JS4A2xb/OGxK1XD+5OPFbvbR7p7g1iZhdgZCX+M7MLwRdMhG3T9LUN1YkpbQ93dOBwvIHmS/kC2Rj+7MSlcJVbINoYOqYHs6gfVvEAjKzGe7cbyf817giQOEoQrmsO/3B94jHagNmjL57ep2L1ORWrZ+rpDzdcR2zD28TWvZ4Yn+DLIm/URKJFQ7BKB6f0YDubfXA3kTeeTlwyCGBYmEV9sUoGYpUOxCwZiFnQC2IhnJpKnJqKxksoE7MRuqGaxFiP/DLM/J6Nj2UY2YVHPEqisG6n7vYPTkdQTVKpJqlUk1SHq4nr2IlBfP7sRMgfJdexE+fXcRO9e9dp7uW7DphW4j0PczkdNJ4fbpopr3Irzr6t2Pu2Ngf44RhWIvADOY3hn/gPjx+7cgtOxebE77a8iR5y71OxevQhtnkl8c0rwI5hlg7Cd+oFeAaeRWmv4rT+jriug73jIwx/FmbRSUd16uJYpc05axERaWaYFkY7BrAZpgWBHNp7JtswjMabx5TiHXQO0HiOHLexw/6Z567b6rlzNxrC3vsp8V3rsHetJfruHxIrvAG8Q8/HO/wLWEUntbPFJ55hmHhOSh383B0orEVEuigjed67nT/vC+I5aQyek8YA4IRqcPZvxyoddNxuAyvto7AWEREAzGAeZt+Rnd0MOQzdbkZERCTNKaxFRETSnMJaREQkzSmsRURE0pzCWkREJM0prEVERNKcwlpERCTNKaxFRETSnMJaREQkzSmsRURE0pzCWkREJM0prEVERNKcwlpERCTNKaxFRETSnMJaREQkzSmsRURE0pzCWkREJM0prEVERNKcwlpERCTNKaxFRETSnMJaREQkzSmsRURE0pzCWkREJM0prEVERNKcwlpERCTNKaxFRETSnMJaREQkzSmsRURE0pzCWkREJM0prEVERNKcwlpERCTNKaxFRETSnMJaREQkzSmsRURE0pzCWkREJM0prEVERNKcwlpERCTNKaxFRETSnMJaREQkzSmsRURE0pzCWkREJM0dVVgvWrSIqVOnMnnyZJ599tmU9a+++iozZszg8ssv51//9V+prq7u8IaKiIhkqlbDury8nPnz57NgwQKef/55nnvuOTZu3JhcX1dXx3e+8x2eeOIJXnzxRYYOHcpPfvKT49poERGRTNJqWC9btoxx48ZRUFBAVlYWU6ZMYfHixcn1sViM++67j549ewIwdOhQ9uzZc/xaLCIikmE8rW1QUVFBSUlJcrm0tJQ1a9YklwsLC7n44osBCIfDPPHEE9xwww1takRRUU6btm9NSUluh75fd6CapFJNUqkmqVSTllSPVCeiJq2GteM4GIaRXHZdt8Vyk9raWm655RaGDRvGl770pTY1Yv/+OhzHbdPPfJ6SklwqK2s75L26C9UklWqSSjVJpZq0pHqkak9NTNNocye11cPgZWVlVFZWJpcrKyspLS1tsU1FRQXXXXcdQ4cO5YEHHmhTA0REROTIWg3r8ePHs3z5cg4cOEAoFGLJkiVMnDgxud62bW6++WYuvfRS7r777sP2ukVERKT9Wj0M3rNnT+bNm8esWbOIxWJcffXVjB49mjlz5jB37lz27t3Lxx9/jG3bvPLKKwCMHDlSPWwREZEOYriu2zEni4+BzlkfX6pJKtUklWqSSjVpSfVIlTbnrEVERKRzKaxFRETSnMJaREQkzSmsRURE0pzCWkREJM0prEVERNKcwlpERCTNKaxFRETSnMJaREQkzSmsRURE0pzCWkREJM0prEVERNKcwlpERCTNKaxFRETSnMJaREQkzSmsRURE0pzCWkREJM0prEVERNKcwlpERCTNKaxFRETSnMJaREQkzSmsRURE0pzCWkREJM0prEVERNKcwlpERCTNKaxFRETSnMJaREQkzSmsRURE0pzCWkREJM0prEVERNKcwlpERCTNKaxFRETSnMJaREQkzSmsRURE0pzCWkREJM0prEVERNKcwlpERCTNKaxFRETSnMJaREQkzSmsRURE0pzCWkREJM0prEVERNKcwlpERCTNKaxFRETSnMJaREQkzR1VWC9atIipU6cyefJknn322ZT169at48orr2TKlCncfffdxOPxDm+oiIhIpmo1rMvLy5k/fz4LFizg+eef57nnnmPjxo0ttrn99tu59957eeWVV3Bdl4ULFx63BouIiGQaT2sbLFu2jHHjxlFQUADAlClTWLx4MbfeeisAu3btIhwOM2bMGACuvPJKHnvsMa677rqjboRpGu1p+wl7v+5ANUmlmqRSTVKpJi2pHqnaWpP21LDVsK6oqKCkpCS5XFpaypo1az53fUlJCeXl5W1qRGFhdpu2b01RUU6Hvl93oJqkUk1SqSapVJOWVI9UJ6ImrR4GdxwHw2jeC3Bdt8Vya+tFRETk2LQa1mVlZVRWViaXKysrKS0t/dz1+/bta7FeREREjk2rYT1+/HiWL1/OgQMHCIVCLFmyhIkTJybX9+nTB7/fz6pVqwB44YUXWqwXERGRY2O4ruu2ttGiRYv4+c9/TiwW4+qrr2bOnDnMmTOHuXPnMmrUKD755BPuuece6urqGDFiBA8++CA+n+9EtF9ERKTbO6qwFhERkc6jGcxERETSnMJaREQkzSmsRURE0pzCWkREJM11q7Bu7YYjmaKuro7p06ezc+dOIDFl7GWXXcbkyZOZP39+J7fuxPvpT3/KtGnTmDZtGo888gigmvz4xz9m6tSpTJs2jaeeegpQTZo8/PDD3HnnnYBuUnTDDTcwbdo0ZsyYwYwZM/jggw8y+t/ZpUuXcuWVV3LppZfyve99DziBfzduN7F37173ggsucKuqqtz6+nr3sssuczds2NDZzTrh3n//fXf69OnuiBEj3B07drihUMidNGmSu337djcWi7mzZ892X3/99c5u5gnz9ttvu1/5ylfcSCTiRqNRd9asWe6iRYsyuibvvPOOO3PmTDcWi7mhUMi94IIL3HXr1mV0TZosW7bMPeecc9xvfetbruu67rRp09zVq1e7ruu6//mf/+k+++yzndm8E8pxHPe8885zY7FY8rVM/nd2+/bt7nnnnefu2bPHjUaj7rXXXuu+/vrrJ+zvptv0rA+94UhWVlbyhiOZZuHChdx3333JWeTWrFlD//796devHx6Ph8suuyyj6lJSUsKdd96Jz+fD6/UyaNAgtm7dmtE1Ofvss/n1r3+Nx+Nh//792LZNTU1NRtcE4ODBg8yfP5+bb74ZOPxNijKpJps3bwZg9uzZXH755TzzzDMZ/e/s3/72N6ZOnUpZWRler5f58+cTDAZP2N9Ntwnrw91wpK03FOkOHnjgAc4888zkcqbXZciQIcl/bLdu3crLL7+MYRgZXRMAr9fLY489xrRp0zj33HMz/nsCcO+99zJv3jzy8vKAjrlJUVdWU1PDueeey+OPP87TTz/N7373O3bv3p2x35Nt27Zh2zY333wzM2bMYMGCBSf076bbhLVuKHJ4qkvChg0bmD17NnfccQf9+vVTTYC5c+eyfPly9uzZw9atWzO6Jr///e/p1asX5557bvK1TP/bOf3003nkkUfIzc2lR48eXH311Tz22GMZWxPbtlm+fDnf//73ee6551izZg07duw4YfVo9RaZXUVZWRkrV65MLn/2hiOZqrUbsWSCVatWMXfuXO666y6mTZvGihUrMrommzZtIhqNMnz4cILBIJMnT2bx4sVYlpXcJtNq8te//pXKykpmzJhBdXU1DQ0NGIaR0TcpWrlyJbFYLLkD47ouffr0ydi/neLiYs4991x69OgBwEUXXXRC/266Tc+6tRuOZKrTTjuNLVu2JA/hvPTSSxlVlz179nDLLbfw6KOPMm3aNEA12blzJ/fccw/RaJRoNMprr73GzJkzM7omTz31FC+99BIvvPACc+fO5cILL+TBBx/M6JsU1dbW8sgjjxCJRKirq+PPf/4zP/jBDzL239kLLriAt956i5qaGmzb5s033+SSSy45YX833aZn3bNnT+bNm8esWbOSNxwZPXp0Zzer0/n9fh566CH+7d/+jUgkwqRJk7jkkks6u1knzC9+8QsikQgPPfRQ8rWZM2dmdE0mTZrEmjVruOKKK7Asi8mTJzNt2jR69OiRsTX5PI8++miLmxTNmjWrs5t0wlxwwQV88MEHXHHFFTiOw3XXXccZZ5yRsf/OnnbaaXz961/nuuuuIxaLMWHCBK699loGDhx4Qv5udCMPERGRNNdtDoOLiIh0VwprERGRNKewFhERSXMKaxERkTSnsBYREUlzCmsREZE0p7AWERFJcwprERGRNPf/A8DwDAjxAOe4AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 576x360 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Learning rate: 0.003\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAesAAAE3CAYAAACKD2n1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzs3Xl8XHW9//HXWWbLvnSStCnQhS50SVmltNAiCoVCke0qolQFe11vlfvTK1cQFwSUqz+u1+0KV+Wq4IWfeoGKlLIvtiJ7SxdK9yZtksm+zXrO9/fHJGnSJs1MMsmcmfk8H9bMZJZ8Zw7Je77L+Xw1pZRCCCGEEI6lp7sBQgghhDg+CWshhBDC4SSshRBCCIeTsBZCCCEcTsJaCCGEcDgJayGEEMLhJKyFEEIIh5OwFkIIIRwu4bDu6urisssuo7a29pjbtm/fzlVXXcWKFSu45ZZbiMViKW2kEEIIkcsSCuu3336bj370o+zbt2/I27/61a9y22238eSTT6KU4uGHH05lG4UQQoicllBYP/zww3zzm9+koqLimNvq6uoIhUKceuqpAFx11VWsX78+ta0UQgghcpiZyJ3uuOOOYW9rbGzE7/f3X/f7/TQ0NIy9ZUIIIYQAUrDAzLZtNE3rv66UGnRdCCGEEGOTUM/6eKqqqggEAv3Xm5qahhwuP57W1m5sOzWbf5WXF9Dc3NV//f4ndtDaGeamDy9KyfOnm920n+BTP8a7/NMYU+amuzlDOvoYiIknxyC95P1PPycfA13XKC3NT+oxYw7r6upqPB4Pr7/+OmeccQaPPvooy5YtS+o5bFulLKz7nq9POBKjLtCV0udPJ8uyiLUHsMJBNAe/pmx5vzOZHIP0kvc//bLpGIx6GHzNmjVs2bIFgB/84AfcddddXHzxxfT09LB69eqUNXCsvG6TUMRKdzNSR4sfMqWy6DUJIYQ4rqR61s8++2z/5fvuu6//8ty5c/nDH/6QulalkNdjEIpk0Xnfeu/nK9tObzuEEEJMmKyvYOZzm8QsRTSWJeHW27NGZcnrEUIIMaKsD2uv2wAgmCW9a02Lvx4JayGEyB1ZH9Y+T3ykP2vmrWUYXAghck7Wh7XX3RvW4ezoWcswuBBC5J7sD2tP7zB4loW1srNkpEAIIcSIsj6sfe4sHQaXnrUQQuSM7A9rT3YtMOsfBpc5ayGEyBlZH9ZH5qyzo2etyZy1EELknBwI63jPOnuGweXULSGEyDVZH9Yet4FGNi4wk7AWQohckfVhrWsaXo+RPXPWssBMCCFyTtaHNfRu5pElc9ZHzrPOktcjhBBiRDkS1lm0mYesBhdCiJyTE2Ht85gEs2SBmaZpoGkyDC6EEDkkJ8La6zayp9wogGZIWAshRA7JibD2uc3sOXULQNOl3KgQQuSQnAjrrFoNDvEV4UqluxVCCCEmSG6EtdskmC2rwSG+yEx61kIIkTNyIqx9nvhqcJUlvVFN02XOWgiREeyeduye9nQ3I+PlRli7TZSCSDRLAk6XsBZCZIbQ8/cReubn6W5GxjPT3YCJ4PXEX2YwEsPTWys8o2m6nGcthMgIdmcTqqsJZcXQjJyInHGREz3rbNzMQ0nPWgiRAVSwA6wYdsvBsT+XHcNqrUtBqzJPToS1r3ebzKzazEMWmAkhHE5ZMYj0AGA17h7z80W3PkPPH27DDnWO+bkyTW6Etae3Z50tYS2nbgkhMoAaEKpW454xP591aAcoC7v10JifK9PkRFh7e3vW2TIMHl8Nnh2vRQiRvVSwI37BcI05rJVSWA27ALDb68fatIyTG2Hd27POmsIomiELzIQQjqeC8VO2jOp5qPZ6VLh79M/VXt/fU7fbDqekfZkkJ8L6yJx1lvRGddnIQwjhfCoYD1fzxFOBsQ2FW/XvxS+4vKj2hjG3LdPkRFgfWQ2ePT1rWQ0uhHC6vmFw88RFgDa2sG54Dzz5mNXzpWedrVymjqFrWTNnLavBhRCZwA52gOFCyy9FL5mMFRhbz9qoPBm9ZDJ2RwBlZ0nnK0E5EdaapuF1G9lz6pasBhdCZAAV6kDzFaFpGnrFDOzGPaMq+2yHOrHb6zEqZ6GXVIGyUB1N49Bi58qJsAbwebJnMw9NetZCiAyggp1oviIAjIoZqFAnqjOQ9PPY9fFV4EbVLPTiqvj32nNrKDxnwtrrNrJozlpqgwshnE8FO9C8hQAYFTOB0S0ysxreA93A8E9HL5kM5N7pW7kT1h4ze+aspdyoECIDqGBHf89aL5sKhntUlcyshl3ok05CM91onnw0byF2m4R1VvK5zeyZs9Y0Oc9aCOFoSilUqAO9N6w13cDwT0u6Z62sKFZgD0blrP7v6cVV0rPOVvFh8OzpWcswuBDC0aIhsGJovsL+b+kVM7Cb98drhifIbtoPVgyjakBYl1Tl3OlbORPWPo+RNRXMpNyoEMLp+qqXad6i/u8ZFTPiO3A1H0j4efqKoRiVJ/d/TyuejAp2oHo3CckFORPWXrdJKEtWg8fPs5ZTt4QQzmX3Vi/rm7OG0S0ysxreQyv0o+eV9H9PL+ldET6GeWsVixB86idEdrwwqtPJJloOhbVBOGphZ0PI6dKzFkI4W1/1soFhreWXofmKE15kppSKF0MZMAQOA8J6DPPWVtM+YntfI/zirwk9/VNUqGvUzzURciasfZ4s2nlLyo0KIRxuyLDWNIyKGQlXMlMdDahQ56DFZQB6YQVo+pjmre2m/QC4F60ktu9Nuv94G7HD7476+cZbDoZ1Fsxby2pwIYTDqVBvWHsLB31fr5iJam9IqCfbtyXm0T1rzTDRCv1j7llrvmI8Z3+YvA/dAoaL4J+/R/jVPzqylGnOhHXfZh7BbOhZ61IURQjhbCrYAZ58NMMc9H2jYgZAQr1rq/49cOehl0455rb4ivDRh7Ud2I8+6aT+NuVf/W3MWecSeXMdPY/dhd3ROOrnHg85FNa9PessONda0wwpNyqEcDQV7EQ/qlcNYPink+gOXFZDfPMOTTs2quLnWjeMakpQxcLYbXUYvWENoLm8+M6/Ee8Fn8VuO0T3H28jumtT0s89XnImrH2evp515oe1lBsVQjjdwOplA2luH3rplBHDWoW6sFsPDTplayC9ZDJYEVR3a9Jts1tqQSl0/7RjbnOdvJj8q7+DUXYCoWd/QfC5e1GRYNI/I9VyJ6z7e9ZZ0COVXbeEEA7Xt+PWUIwEduCyGoeer+6jF1cCjGqRmdW7uMyYNG3o5y7041t1M+4zriC2axM96+5M+6LenAnrI3PW2dGzVjIMLoRwsIE7bh1Nr5iJCnehjjMvbNXvAk3H8M8Y+jn6NvQYxby13bQPzVOAll827H003cBzxhXkrfo6rtnnAlrSPyeVzJHvkh28nmzqWUu5USGEcynbQoW6jlkJ3qd/kVnj7v4e8tGshvfim3e4PEPervmKweUd1VaZVu/iMk0bOYCNqlnD9u4nUkI963Xr1rFy5UouuugiHnjggWNu37p1K1dffTWXX345n/nMZ+jo6Eh5Q8cqu3rWcuqWEMK5VKgTUMP3rEurwXQPO2+trBhW455h56shfs62XjI56Z61sqLYrbWDFpdlghHDuqGhgXvuuYcHH3yQRx55hIceeohdu3YNus8dd9zB2rVreeyxx5g+fTq//OUvx63Bo2UaOi5Tz4qiKJr0rIUQDqaGKDU6kNa7N/VwYW03HwArOmKPdjS7b9mtdWBbQy4uc7IRw3rjxo0sXryYkpIS8vLyWLFiBevXrx90H9u26e7uBiAYDOL1esentWPkcxtZceoWspGHEMLBhqpedjTdPwO7+QDKih5z25HNO0YOa9XVjIpFEm7bSIvLnGrEOevGxkb8fn//9YqKCjZv3jzoPjfffDM33HADd955Jz6fj4cffjipRpSXFyR1/5H4/UPPk+TnubE1bdjbM0VLgY+IUkyaVJDQnEs6ZPp7nA3kGKRXLr//XQ1RgkB59WTc5UO/D12z5tO4+QmKrGa8R/WgG9r2YhVXUDntxOP/nBOn0/g6FOudeIboKQ91DAKv1RH15FExY4Zj/34OZcSwtm170AtSSg26HgqFuOWWW7j//vupqanh17/+NV/72te49957E25Ec3NXyjbY8PsLCQQ6h7zNZWi0d4SGvT1ThIPxT6KBxvb4kLjDHO8YiIkhxyC9cv39jzQ2ANAWNNCGeR9sT7wqWfPOLbjdVf3fV0rRs387RvW8Ed9DSyuOP8e+3bj08kG3DXcMug/uQis7kaam9G3coeta0p3UEYfBq6qqCAQC/dcDgQAVFRX913fu3InH46GmpgaAj3zkI/z9739PqhETxec2s6PcaF81H5m3FkI4kAp2gmaAO2/Y++gFZWh5JcfMW6vOACrYntAKbL04ua0ylR3DbjmQcfPVkEBYL1myhE2bNtHS0kIwGGTDhg0sW7as//aTTjqJ+vp69uyJv+HPPPMMCxcuHL8Wj4HPY2bJnHVvb1pWhAshHChevaxwxGFmo2LmMWF9ZL56+JXgfTSXBy2/LOHCKHbbYbBiGbcSHBIYBq+srOSmm25i9erVRKNRrrnmGmpqalizZg1r165l4cKF3HXXXXz5y19GKUV5eTl33nnnRLQ9aV6PkSWrwXt/AaRnLYRwIHuYUqNH0ytmENv3eu852fFhYavhPXD50EunJvSz9JLJ2O0NibWrd3GZno1hDbBq1SpWrVo16Hv33Xdf/+Xly5ezfPny1LZsHHjdZpacZ93Xs878Dx5CiOwzXF3wox0pjrIH88T4VKpVvwujciaanliBTb24kuiuTcespxqKFdgHpge9qOq493OinCk3CvFTt4LZUMGsd8463bVqhRBiKCrUMWz1soHip09pWI27448Ld2O31o14ytZAeslkiAT7Txc7HrtpP0b5iQl/EHCSzGvxGHg9JjHLJmZleMjpssBMCOFcifas4ztwVffvbR0PbZVUec/+RWYjFEdRto3VnJmLyyDXwrq35GjGz1v3rQaXBWZCCIdR0TDEIgmFNcSHwq3eHbis+vfim3dUDL15x1D0kr4V4cdfZGZ31EMsnJGLyyDHwrpvm8xghq8I7z+3WnrWQgiH6RuO1hMMa71iBoS7UR0NWA270MtPQHMlXgVTKygHwxyxZ53Ji8sg18La07uZR4aHtZxnLYRwKhUaudToQEbFTCB+ypbVuDup+WoATdPRi6pGPNfaCuwDw4VeMiWp53eKnAprb2/POuOHwXUZBhdCOFMidcEHiu/A5SGy4wWIRRI6v/qY5ygZeUMPu2l/vNfuwKqPicitsPb0zVlnR89ayWYeQgiHsZMMa03XMfzTsBviuzmOZu9ovbgK1RFA2UP/bVfKxmran3GbdwyUU2F9ZM46w0NOhsGFEA7Vvz1mAqdu9ekbCtfyy9ALyke497H0ksmgLFRHYMjbVWcTRIMZO18NORbWR1aDZ3jPWobBhRAOpYId4PKime6EH6P3rv4eTa8aBq4IH3oo3Arsiz+/hHVm8Hmyo2etSc9aCOFQiZ5jPZBReTLoJmb1/FH9zJHOtbab9oFuxOfHM1RC5UazhSdbetZG/BOriobT3BAhhBhMhZIPaz2vhPyP/htaXvGofqbmyUfzFmK3D32utdW0H71sKprhGtXzO0FO9ax1TcPrzvzNPLSCUgBUd0uaWyKEEIOpYAd6EvPVffT80iOjhqOgl0wechhcKRUvM5rBQ+CQY2EN8XnrTD/PWs+PL8CwuySshRDOMpph8FTQi4c+fUt1t6DCXegZvBIccjCsfR6TYKb3rF0eNE8Bqqs53U0RQoh+StmoUGd6wrqkChXsQIW7B30/GxaXQQ6GtddtZv6cNfESe7aEtRDCQVS4G5RKS1hr/YvMBu9tbTftA01HLzthwtuUSjkY1gahDF8NDqAXlKFkGFwI4SDJVi9LpeE29LCa9qOXTknqVDInyrmwjg+DS89aCCFSLa1hXVgBmn7MvLXdtD+ji6H0yb2wdhuEMnyBGcR71kSDqEhPupsihBDAwOplaRgGN0y0Iv+gnrXd3YoKtmd0mdE+ORfW8TnrzB8G1wr6VoRL71oI4QxHetbJn7qVCkevCM/0bTEHyr2w9hgEwxZKqXQ3ZUz66ufKvLUQwilUsB00Dc1TkJafHw/rBlRvdUeraT+gYZSfmJb2pFLOhbXPY2IrRSSW2aU6pWcthHAaFexE8xai6emJFr1kMljR/k6M3bQPvaQKzeVNS3tSKffCur/kaGYPhWu+YtAM6VkLIRxDhTrSMl/d5+ga4VaWLC6DHAxrb+82mZm+yEzTdbSCUulZCyEcww52pG2+GgafvmV1t6O6WzK+GEqf3AtrT7xnnQ2nb+kF5VLFTAjhGCqYnuplfTRfMbh82G31hOv3AGR8mdE+ORfWvv6edWYPg0N8o3bpWQshnCJddcH7aJqGXhJfER6u3wuQFYvLIAfDOut61t2tKDuzF8sJITKfikUgGkxrWMOR07ci9XvQiirQPPlpbU+q5FxYZ1XPuqAMlI3qaUt3U4QQOU6FeguipDusS6pQXc2EandkzXw15GBYez29YZ0lPWuQfa2FEOnXV71MT+NqcAC9eDIAVldr1qwEh1wMa3ffMHg29KzlXGshhDOku3pZn74V4UBWlBntk3Nh7TZ1dE0jmOGnbsHAKmYS1kKI9FLBdsABw+DFlUcuZ1HP2kx3Ayaapmn4PEbGF0UB0Nw+cPukZy2ESDs76Iw5a830oOWXYRg6uje9vfxUyrmwhr49rTO/Zw1951rLnLUQIr1UqAMMN5iedDcFc+b7yC/MJ/O7ZEfkZlh7zKyYswbZ11oI4Qyqt3qZpmnpbgrexddS5i8kEOhMd1NSJufmrCF++lY2zFlDvGctYS2ESLd0F0TJdjkZ1t4smbOG3nOtw92oaDjdTRFC5LB0lxrNdrkZ1m4zK86zBtDzywCwu6V3LYRIHxXqQJewHjc5GdZ5HoPOnihKqXQ3Zcy0/tO3ZJGZECI9lFIyDD7OcjKsp00uoisY5VBTd7qbMma6FEYRQqRbpAdsK617WWe7nAzrmhnxgNu8O/MDTssvAU2TwihCiLRxSvWybJaTYV1W5GWqvyA7wlo30fJKpWcthEiZ2KHtqFjii1bt/rCWnvV4ycmwBlh0cjnv1bbTE4qmuyljphWUyZy1ECIl7K4Wgn/+PpHNTyb8GCVhPe5yNqxrZpZjK8U7ezM/5ORcayFEqtgtBwCwat9J+DFO2R4zm+VsWM+cUky+12RLFgyF95UcVcpOd1OEEBnOaqmLf23YjYqGEnpMf8/aWzBu7cp1ORvWuq6xYEY5W/Y0Y2f4KVxafhnYsf5fGCGEGC27pTZ+QVlYh3ck9BgV7ABPPpqekxWsJ0TOhjXEh8I7eqLsO5zZ9WN1OddaCJEidmstxpRTwHARq92W0GNUUAqijLeEwnrdunWsXLmSiy66iAceeOCY2/fs2cP111/P5Zdfzo033kh7e3vKGzoeFs4oRwM2725Kd1PGRCvorWIm89ZCiDFQdgy79TCGfzpG1Wysuq2JPS4kpUbH24hh3dDQwD333MODDz7II488wkMPPcSuXbv6b1dK8bnPfY41a9bw2GOPccopp3DvvfeOa6NTpcDnYkZ1UcafwiU9ayFEKtjtDWDH0EurMarnY7fWYXe3jvg4qV42/kYM640bN7J48WJKSkrIy8tjxYoVrF+/vv/2rVu3kpeXx7JlywD47Gc/y8c+9rHxa3GK1cycxL76Ttq7MngjDE8+mB7pWQshxsTuXVyml03FnDofAKtu5KFwO9gh1cvG2YirARobG/H7/f3XKyoq2Lx5c//1AwcOMGnSJL7+9a+zfft2ZsyYwTe+8Y2kGlFentoVhH5/4lV0zj/zRP73xT3sC3TzwemTUtqOiRQu8eOKtif12seTU9qRy+QYpFcmvv8tWxsJaToVs2ajGSb784owm3fi91887GOUFaMz3E3BpEmUOuw1Z+IxGM6IYW3b9qDNxJVSg67HYjH+/ve/87vf/Y6FCxfy7//+73zve9/je9/7XsKNaG7uwrZTsyLbn+SG4wUujZICNy+/Vcei6WUpaUM62N4SQi2NjthsPdljIFJPjkF6Zer7H6zdjV5cSXNrGAijV82le/fbNDZ2DPq7P1DfMHmP7SHmoNfs5GOg61rSndQRh8GrqqoIBAL91wOBABUVFf3X/X4/J510EgsXLgTgsssuG9TzdjpN06iZWc7WvS3ErMw9Tzl+rrUMgwshRs9qrUMvm9p/3Zg6H9XTht16aNjHSPWyiTFiWC9ZsoRNmzbR0tJCMBhkw4YN/fPTAKeddhotLS3s2BE/H+/ZZ59l/vz549ficVAzcxKhiMV7tZmxin0oWkE5KtiBikXS3RQhRAZS0TCqIzAorM2pCwCw6oavZibVyybGiGFdWVnJTTfdxOrVq7niiiu47LLLqKmpYc2aNWzZsgWv18tPf/pTbr31Vi699FJeeeUVbr755oloe8rMm1aKaWgZfQqX3nv6lkpg5aYQQhzNbq0DFHrpkbDWC8rRiiuJ1Q5/Cldfz1qXBWbjKqFyM6tWrWLVqlWDvnfffff1X160aBF/+MMfUtuyCeR1m8w5oYTNu5v5yAWz0t2cUdEG7GutF1emuTVCiEzTV7nMGNCzBjCr5xPd+VeUFUMzjo0M2R5zYuR0BbOBFs6cxOHmHhrbguluyqgcOdda5q2FEMmzWmrBcKMV+gd935g6H2JhrMbdQz5OBTtAN8GdNxHNzFkS1r0WzYyHXaZu7KHllwLx7e2EECJZdmstelk1mj44FszJc0HThq1mZgfj1cuGWy0uUkPCuldlWR6VpT7eztB5a81wofmKpWcthBgVu6V20Hx1H82Tj+6fMey8tQp1yBD4BJCwHmDhzHJ27G8jHLHS3ZRR0WRfayHEKNjBDlSwA6Osesjbzep52IE9qHD3MbdJqdGJIWE9wKKZk4hZNtsPZOaKar2gTHrWQoik9S0u08uO7VkDGFMXgFLEhtgyU0mp0QkhYT3A7BNK8LiMjN3YI96zbkFl+P7cQoiJFT9t6zhhXTETTA/WUUPhSqnenrUMg483CesBXKbOvGmlbN7dlJGBpxeUgxVBhbvS3RQhRAaxWw6ieQrQfMVD3q4ZJsbkOcSOXmQWDYEVlb2sJ4CE9VFqZpbT0hGmrunYuRmn69vXWrbKFEIkw2qJlxk93opuc+p8VHsDdueRRbhSvWziSFgfpWZmfOetTBwK1wcURhFCiEQoZWMfVRN8KEZ1vPTowN51f0EUmbMedxLWRykt9HBiRQGbd2XeKVyaFEYRQiRJdTZDNDRiWOulU9DySgbNW9uyiceEkbAeQs3J5eyq66A7FE13U5KieQvBMKVnLYRImN06dJnRo2mahlE9D+vQdpSK71AopUYnjoT1EGpmTMJWiq17M2vuV9O0+O5bMmcthEiQ1XfaVunQ51gPZFbPR4U6sZsPArI95kSSsB7CjClFFPhcvL0r83qouhRGEUIkwW6pRSsoR3P7RryvUT0PoL/0qAp1gtuHZrjGtY1CwnpIuq6xYEYZW/Y0Y9uZdQqXll8uc9ZCiITZLSMvLuuj55eil1b3lx5VPe3Sq54gEtbDqJlRTlcwyt7DHeluSlL0gjJUTzvKiqW7KUIIh1NWDLvt8Ijz1QMZ1fOx6neiYhFUqFP2sZ4gEtbDWDCjHE2DJ145QCSaObXC46dvKVRPZpZMFUJMHLv9MCgr4Z41gDl1HlhRrIZdUhd8AklYD6PA5+KqZTN4Y2eAu373Bk0Zss91X2EU2SpTCDESu+X4ZUaHYkyeC7qBVfuOlBqdQBLWx3HpOdNYe3UNjW1Bvn3/q7yz1/lzwbqcay2ESJDdUguagV48OeHHaC4vRuXJxGq3oEJd0rOeIBLWIzh11iRu+8SZlBR6uOeht/nzxn3YDq4bfqRnLWEthDg+q+UgekkVmmEm9Tijel7v6VtKqpdNEAnrBFSW5XHr9WfyvnmV/OnFPfz0T1voCTlzAZdmetC8hdKzFkKMKJEyo0Mxq+f3X5ae9cSQsE6Qx23wj6vm8dEPzGLz7mZu/81r1AWcubuVVlAmc9ZCiONSkSCqsymhYihH0/3Tofe8bJmznhgS1knQNI0LzzqBr370NILhGLf/5jX+vr3huI9RStEdinK4uZv99Z0cbu6mpSNEVzBKNGaPy1aceoGcay2EOL6+PayNshOSfqymG5hTTolflp71hEhuokIAMPuEEr75ybP4+SPv8J+PbmXHgTb8JV46u6O0d0fo7InQ0ROhsydKR3cE6ziFVXRNw+3S8bgM3C4dr9vkksUnsnhe1ajbpxWUY9dtQyl13C3vhBC5q7/M6CiGwQHM6WcSq9uGnl+WymaJYUhYj1JpoYd/ue40Hnp2F8+8Hv+P3mXqFOW5Kcp3UVLg4cSKQory3RTluSjMd+M2DSIxi0jUIhy1e79aRKJ2/GvM4mBjF796fDsVJXnMmDK6T6x6QVl8U/hID3jyU/myhRBZwm6pBdODVlg+qsebJ59DwfQz0ExPilsmhiJhPQamofOxC2ezauk03Ga8dzzWnmxXMMq3f/0qP3tkC9/85FkU5rmTfg4tv3df6+4WDAlrIcQQ4ovLqtG00c2GapoGEtQTRuasU6Aoz43XbaZkyLnA5+ILVy2gozvKvY9tHVVtcr339C2ZtxZCDEUphd1Sm1SZUZFeEtYONK2qiI9fNJut+1p55OW9ST9e6y2MIivChRBDUcH2eF3vUgnrTCFh7VDLFk3h3JrJ/HnjPt7a1ZTUY7W8YtAN6VkLIYY0mjKjIr0krB3s4xfO5sTKAv5r3TYak6hNrmk6Wn6ZVDETQgzJHuNKcDHxJKwdzO0y+PyVCwH42Z+2JLX7l15QhpJhcCHEEKyWWjRfEbqcI50xJKwdrqLEx5pV8zjQ2MXvNuxMuIiKVlAuPescZXc0Ysci6W6GcDC7tVZ61RlGwjoDLDp5EquWTOPlLYd58e1DCT1GLyhHdbeibHucWyecxGrcQ/dD/0rby39Md1OEQyllx0/bGkWZUZE+EtYZ4kPnTmf+9DIeeGonew93jHh/raAclI3qaZuA1gknUNEQwWf/E5TKb6boAAAgAElEQVRF9/a/jkspW5H5VEcAYhHpWWcYCesMoesa/7hqHsX5bn72v+/QFYwe//75cq51rglvfADVEcCctYRoy2Hs1tp0N0k4kNX738VoaoKL9JGwziCFeW4+f+VC2rvDIxZMOXKutYR1LojueZXouy/hPvVSPGd/BNCI7Xkt3c0SDtS/Erx0SppbIpIhYZ1hpk8u4roLZ/PO3hZ++r9bCA+zQryvilkmF0ZRto3VsEuGc0dgd7UQeul+dP903GdegZ5XjPfEU4jtlbAWx7JbatEK/Wgub7qbIpIgYZ2Bzj+1mo9dOJu33mviB79/k86eY1f+am4fuPMyehg8svkJeh79Ltb+t9LdFMdSyib0/H1gRfFd8Bk0PV7uP3/uOditddhth9PcQuE0dkudlBnNQBLWGeoDZ0zl81cu4EBjF3f+9vUhi6boGXz6lt3TTuTNdQCE31wnvethRDevxzq0Hc+Sj6EXH9lWNX/O2fHbpXctBlBWFLu9XhaXZSAJ6wx2xpwKvnLtqXQFo9z5m9eOWSWuFZRhtx1GRUNpauHoRV77E8SiuBauwA7swTq0Pd1NchyraR/hV/+IOe0MXHOWDbrNLCpHr5gpQ+FiELv1EChbwjoDSVhnuFlTS/j69Wfgdhnc/eCbbN59pI64Oe10VEcD3f/vFmIHMmco2Wo+SPTdF3HNvwDPWVej+YqJvPXndDfLUVQsTOiZ/0TzFuJd9qkhd3xzTT8Tu2k/dkdjGlo4euFXHib43L3pbkZWslulJnimkrDOApPL87nl+jOoKsvjP/6wpb9winvucnyXfx3N9BBc/+8En/4ZtsPPu1ZKEf7b78Gdh+f0D6GZbtw1F2PVbcNq3J3u5jlGeNP/YLfX433/P6J5C4a8jzn9TABie1+fyKaNid3TRmTLk8Te24jVsCvdzck6dkst6AZ6cWW6myKSJGGdJYoLPPzLdacxb1op9z+xg0de2oNSCrNqNnlXfxv3mVcS2/cG3Q//K5Htz6OUMyubWQfewqrbhueMK/pDyHXK+eDJJ/Km9K4BYvveJLr9OVw1F2NWzxv2fnqRH33SSRk1bx3d9hzYNrh9hN94LN3NySpKKaz699BLpvQvRBSZQ8I6i/g8JmuvqeHchZN57K/7+PUTO4hZNprhwnP6h8i/5rsY5ScRful+go/dhdU7JOYUyooR+ttD6MVVuOa9v//7mtuHe/4Hie1/E6vFWW2eaHZPG6EXf4VefhKes64e8f7m9DOxG3dnxCl8yooS3f4cxok1uBetxDq4GatpX7qblTVie17FangP19xlI99ZOI6EdZYxDZ1PrZzL5Uun8fLmw9z+36/x+6ffY+M7hzkULcC98qt4l9+I1XaInj/eRvjVP6IcsulDdNszqPZ6POdce8wnf/eCC8H05PTcdfw0rf9CRcN4L/gMmuEa8TGuvqHwfc4fCo/tfgUV7MC94CLc8z8I7jwib6xLd7Oygor0EN70IPqkk3DNuyDdzRGjIGMhWUjTNK44bwb+Eh/Pv1nHC2/VEYnFh71dps4JFQWc7P80i4MvUP7mOqK7/473/Bsxq2anrc0q1EX49UcxqudjnLDomNs1bwGuee8numUD9plXohdVpKGV6WP3tBF5cx1W7Tt4zl2NkWD1Kb1kMnrpVGJ7X4t/4HEopRSRLU+hl1ZjVM9D0zTcCy4k8sajWC0HpTTmGIVf/ROqpx3fRWvRdCPdzRGjkFBYr1u3jp///OfEYjE+8YlP8LGPfWzI+z3//PN85zvf4dlnn01pI8XoLF04maULJ2PZNvXNPRxo6GJ/Qyf76zt56d0ONoRrmGNO4iPW3yh99C7e9J7NwcrlTCrNw1/sw1/iw1/iJc87cg9urMKvPwLRIJ5zPjrkymYA98IVRN95msjbT+A97xPj3iYnsBp3E3nnaWJ7/g62hWvOebhOef/IDxzAnH4GkTcew+5pR88rHqeWjo1VvxO7eT+e8z7Zf/zdCy4ksuVJIm+sw/fBz6e5hZnLCuwluvUZXPMvwKiYke7miFEaMawbGhq45557+NOf/oTb7ebaa6/l7LPP5uSTTx50v6amJr7//e+PW0PF6Bm6TrW/gGp/AecsiBfOsJWiqS3I/oYu3jp0OtMPruOM8N8o37uX+zcvpdU+ssI432syqdjHpBIvZYVeSgs9g/6VFHhwmaOfUbHaDhHd9iyuucuPW1lJzy/FNefceA3sMz6Enlcy6p/pZMqKEtv9dyJbn8YO7AWXF9e8C3DP+wB6SdXIT3AUc8aZRN54lNi+N3DPSy7oJ0p0ywbw5OOadU7/9zRvAe75HyDy1l+w2g5hlEgt62Qp2yb00v1ovqKE1jgI5xoxrDdu3MjixYspKYn/YVyxYgXr16/ni1/84qD73XrrrXzxi1/khz/84fi0VKSUrmlUlOZRUZrHWXMrgHlE39vItJd/w7d86+msuZZDeXNpag8RaAsSaAtSG+jmnT0tQ9YjL8xzxcO7wMPkikI8hkZxvpviAjclBR6K890U5bsxjWNDPfy3h8D04D7zqhHb7V60kuiOF4hsXo938bWpeCscw+5uJbr9OaLbn0cFO9CLq/As/TiuWUvj5WNHSS+dilZcGR8Kd2BY250BYvvfwL1oJZrpGXSba+EKIu88ReTNx/G9f02aWpi5otuewW7aj/cDn0Nz56W7OWIMRgzrxsZG/H5///WKigo2b9486D6/+c1vmDdvHosWHTvXKDKHa9YSjMqTCT7zcwpf+xUL5i7Hs+S6QX9AlVIEwzFaO8NH/nUdudzSGWZvfScd3UPUKwcK8lwU53soLnBTWuBhpl7L6QffpnnmpbR2aJSoCIU+17BD4XpRBebMxUS3PYfn1MuGPcc4k9g9bYQ3/T6+S5ay46uhF1zYO3c79jWgmqbhmn4Wkbf/ggp1Oe49i2x9BtCGXPik+4pwnfJ+ou88hX3Gh3JurcJY2N2thF/9I8bUBZgz3pfu5ogxGjGsbdse9IdTKTXo+s6dO9mwYQP3338/9fX1o2pEeXlq/3j4/YUpfb6c4i9ETbuLlhf+h/ZNj0LTbiquuAlP5bRBdztphKeJWTZtnWFaOkK0doRo6QzT1vu1tSNES0eI7fubWao9ThMF3PlqCdarrwLxRXBlRV4mlfgoK/KS5zXxeUy87vjX4rKlTN+1iYZX/oI69XJ8HpN8r4vCfDf5Xhe6PnTQO5EdCXLo0R9hNddR/L6VFJ1xMa7S5Ie6hzLw9yB8+jLq3vozvpbtFC5yzmpgOxLkwM6XyJ+7mMrp04a8T+z9/8DBbc+i79iA/9LPTWj7xiLdf4caXvoFmrKZcvnncJUWpbUt6ZLuY5BKI4Z1VVUVr712pKhCIBCgouLIp9v169cTCAS4+uqriUajNDY2ct111/Hggw8m3Ijm5q7j7s2cDL+/kECgMyXPldMWXoGvdBah5+6l7tdfw3P2R3DN/+CwPd6B/P5CWlu6ASj1mZT6CphReewHssi25wi/3EZs6RpuLl1Aa8eRXnpbby995/4WQhGLUMQaNPx+Y8EJzNz+NN/ZVEyYIwvgNCDPa1Lgc1Hgc5Hf99XrosBnku9zxW/3usjzusj3muT1/jP0iT2TUdkWwSd/hNW4H9+KL2GfuIi2GJCC/36P/j1Qhh+tcBItm18mNOWshJ8nuvOv2D3tuBddktCxT1Zk6zPYoW7UrPcf5/fWxJyzjM7Nz2HPuwS9d692J0v336HYgc0Et2/CfeZVtMXyU/LfVKZJ9zE4Hl3Xku6kjhjWS5Ys4cc//jEtLS34fD42bNjA7bff3n/72rVrWbt2LQC1tbWsXr06qaAWzmVOnU/eNbcTeuGXhDc+QOzgFlxzl2NOmYvmyR/Tc6tID5HX/oRRNZuCeUso1TQYYf2QrRTh3uCO1FeQ99wPuOWcbpqmLKMnHKU7GKMrGKUrFKU7GKUrGKW9K0JdoJuuUJRwZOi9v/v4PAZ5Hhf5vnhPPd/nosAbD/j83mCPXz7yvTyPidulJx1kSinCL/8W6+BmPOd9EvPE8Z1C0jQNc9oZRLc+jYr0JDR/Gdn+POGX7o+3tzOA59zrUzIs30cpm+g7T6H7p6NXnnzc+7pPXUl0x/NE3noc77mrU9aGbKRiYUJ//S16yWTciy5Jd3NEiowY1pWVldx0002sXr2aaDTKNddcQ01NDWvWrGHt2rUsXLhwItop0kT3FeFb8WWiW58m/OofsQ5uBk1DnzQNc8opGNXzMKpmHbMwaDjKtlE9rUTeehwV6sJzznUJB52uafg88aFwChfQs3MeJQdfpHrpZWime8THR2M2PeEY3cEoPaEY3aFo77/YkevB+NeeUIy6QBfdwfjt1nFGfgw93q48j4nPG/866HLvMH7f5TyPSemB5/DteB5t4UrMucsTev1j5ZpxFtEtTxLb/xauWUuOe9/oexsJv/TfGCfUoJdWE938BMqK4F12I1qKRiCs2neO1Dcf4b8BvaAc1+xzib77Iu7TVqHnl6akDdko8sY6VGcA72U3J1Q4R2QGTTlgo2AZBs8MyophNe6Ob6pxaDtWw25QFugmRuVMjOp5mFPmMWnaNJr27cPuasLuCKA6m45c7mqJPwYwZ5+H7/wbR92eWN02go/fjefcT4zrKmelFKGINSjMu0Px0A+GY/T0/guGhrgcih2zev4M9x5WF7zM6+Fp/Lb7PNA0vG4TQ9fQdQ1dY8Dl+Fej97LL1PF6THxuo/eric9j4HXHPwh43fHLFf4CurtCuAwdl6njMnRMA4xH/hXdPwPfin9CHyYgo3tfJ/T0TzGqZuO75J/BcBF54zEir/8v5oz34b3gH1NSW7rniR9iNx8k/6M/QDNGfj67o5Huh27GteBCvOd8dMw/fzyl6++Q1VpHzx9vwzx5Mb7zc3v1vJOzYFyGwYXooxkm5uQ5mJPnAFeioiGs+p3E6rZh1W0n8tojRPhfDhz9OF8RWuEkjIoZ6DPPRiuchF7ox5gyd0ztMaacgl4xg8jbf8E1d9m4VWbSBvToJ42ipkjMsglFLHpCUSJ12ynatImeohkYsz/JP0SgJxwjFI5hKYWyFZatsG2FrXovK+LXbUUkZhEMx2jpCBGKxC+HRhjeH+jqvEoW97zN//n+BmzD3dvbd/XP25+s1bK89U90eCezo/xqPFub8LgMzIJzKJsZZdLuPxPo7KbttE9huF2Yuo6haxiGhsdl4HYZeN3GkKfoDWS1HcI6uAX3mVcmFNTQeybAyecQ3fYc7lMvRffl5qKp4SilCL/03+Dy4jn7I+lujkgxCWsxaprLi3lCDeYJNUC8ZGjs0HbyjTA9WgFaoR+9cFLCQ+RJ/3xNw3PqKoIbfkR058vxdkRDqFgYFQ1DNIyKheJfo2FULILuK0QrqkAvqkDLKxmXRVNHMw2dAp+OLxSg57VfohdXUPGh/0PlGOf9+/TN5QfDMYIRi1A4Rl6Bh6bmbqIxm2jMJmbFv/paTdy73uUTNVFq82YSDMXi0wDhGCXd+zk3so56VcJP6s+j+9DBo35SGed63sc/8Hf2Pf5Dftn5fqLD/Akx9Hh4e9xG/OugyzrndD/DdAzWN09De3kv7t7bvb2BbxgaltX3gcXG7v0Q4/KexTxrI+9t+H/sn3whLpcxYJTB6D1r4Miow1iK9YyFshP/AJUqsZ0vY9XvxLPsU/JBJgtJWIuU0bwFuGacRbG/kMgEDT8ZJy1CL51K+MVfE072waYbvS+4iyrQiyrRi+P/tPyylAa53dNG8Ikfohkmvov/ecwL9AYaNJffy+8vJFDsPea+yp5Md+3/41T3Ac45/7L+71uNe+h5/An00gpmrbqZ//AW9o4GxAhFLSzLxrIVln0GrftnMPedh7h9xiscWvBJYrqHaO+HgXDEIhS1iESPrOAPD/ja0RNBi/RwonqHt6wZPP5WKzGrOanXuzr/JObXb+LHOybTo459jQOZvT1+09QxdR3T0DANHaP3q6lrGIYe/56uYRrx6/2XB4wc9N2nb1rBZeq4TKP3a/xfftdBinavpzPwLlbJCdiVc1BVp2BUzsJwewb8HD2lpxjaPe2E//YQRuUsXHPOS9nzHo/qHfkZaRTleCzbpqk9RH1zD4G2IB630V9EqaTAQ0Gea9jpmuO1K2bZWJYztwEeLQlrkdE0Tcf7gc9h1b4DLg+aywOmB83ljS86c3njPXuXB81woUKd2O0N2B0N2B2B+OX2euyDm8GKHXleXxFGxUz0ypkYFTMx/DPizz0KKhomuP7fUaFO8lb9K3qRf+QHjRNN1+Orwnf/DRWLoJlurOaD9DzxQzRvIb5Lv9rfKzv6A0C/6ouJVpSgPXcvs977b/IuvimpDx+Rt58g/EqM8z78cc6fdBKWbROJxqcKItF4sFu2ioekPmDOXo+Hp94xHf3xb/O95T3EFnygfyog1Duy0Hc9PtIQv2xZCsuyidl9f8jjX+MjDhb+0D4MK8xudQJhWydmxXv0lq2OXLbUsAsNpxgtrPS9xTR3LV22h79F5lAdaGVa6zO43n2KmNLZGatgZ7SKndHJHLTKUcQ/NAxck2DoGtpR1/tuN3o/bBj6kQ8Uhq4xPbabxd3P4LZDPGWfS+cT7/Y/vm/Ng67TfzmefRoa0JeDfR9Mtf7/g3DUIhjufR+P+WcRjMRQCgp8LkoKPJQVxUsPH12OuLTQQyxmU9/SQ0NrkPrmnt7LPTS2Bo+7eFPXNIryXRQXeCjprYiY53UdGUkaMJrUM+C4W7aKzwsXefCX+JhUHN/nIL7fQfxfvteckJG1VJGwFhnPKKvGKKtO6L6a29dbBWvwWQxK2ajuVuyORuzWQ1iNe7AadxPb/2bvAzX0shPiwV05E6PiZLTiyhF/2ZVtEXzmZ9jN+/Fd9CUM//TRvMSUMmecSXTH81i1W9FLqgj+5d/QTDd5l/1LwqusXScvBsMk9MzP6Xn8bvJWfjWhymjKtohsfRpj8hyMSfHSOoau4/PoQ38wGErhdILTziC24xmKT19JScHoRimUHSO251Uib/8Fu7l3yN/lxZx2Oq6ZizGmzjtmIV2816aIxqz49ELLIbR3/oyr9nVs00f7tJW0Vi9hTlExrW097IiE8LXtJb9tF9Udu5gdfAt4i5juoSVvGvX5szlUsICY0uND/Uph2za2Te96BTVgVOPI5ahlo4W7WRZ9gTlqN/WUs85aweHDJrZq6Z82UGrA2geb/uvE/3dcpnFkxKZvIaO/xEeex4xPM3hMTF2jrTvSWxchxL7DHXT0REd4Xp3KUh9TyvM5bZafyjIfVWV5VJT4CMds2rvCtHdFaO+O0NYVpr07QntXhNbOMPvqO+kJx/C6DXxuE68n/rWsyEu1Jz79kdc7FWK4TPYfaifQFuTN9wJ0HtWu+BoUL7o+YMpliPfashWVZXl885OJ1ycYD7IaXKRcNh0DFerqDe5dWA27sRr3QDQYv1HTwe2L9+JdPnB745d7v4fLh+oMENv/Jp6l1+Oe/4EJa/fxjoGyY3T99ksYk6Zhtx0GOxbv8ZdMTvrnxA68TfCpH6MXVeE551r0sqlovuJhP8RE97xK6Omf4r3wn3BNPyPpn9fHatpPz5++iZZXgjntdMzpZ2JMnpPQIkMVDRN990Uim9ejuprRS6bEi77klxLb/QrRva9BJIjmLcSccRbmzLPjpycOOMfc7ggQfuNRYu/9FQw37oUX4a65uH+EYbj33w52YB3agVW3lVjdVlRnE1p+Ke4FF+E65fyEasArpYjt/hvhvz6AioZwn3457lNXjnqFfl8EqN7/U70xPtoiQX3VCweWITZ0jaryPKpK8ygr8k5IlcGjj0EwHBu010GgLUhzewgFR43i9E6TDLg+uTyP809LrEOQiNGsBpewFimXzcdAKRu77TBWwy5URwAVDaGiQYiE4pcjwfgit2gQFQmBFcF96qV4EtikJJVGOgbB5/+L2M6XwZNP3mU3Y5SPfr/oWN02gk/+CGLxVQOatxC9bGr/P6PsBPTSajSXh57H7sTubiH/I3eP+Xzt2IG3iO54idjBLWBF4uF60mnx4K6ed8wqczvYQXTr0/Fa5OFujKrZuBddgnHiokFBrKwosYNbiO36G7H9b8WfO78Mc+b7ME9cRGz334nueBF0Dde8Dwy5Mj2R3wGlFFbdViJvPY51aDu4fbjnXYBrwYXD7ihnd7cSeul+rANvo1fMwLv8RozS1IVINnHy3yEJa5x9gHKFHIMjjq6lP1FGOgZW425CL/wq/sc+BXscq3A3VtN+7JZa7JZarJZa7NZaiPVt6KKhFU6KV0JbfC3umovH/DP7f3Y0TOzgZmJ7Xyd24C2IhsDtwzzxVMzpZ6IXVxHd9gzRd18CK4Y57TTcNZdgVM1K4LlDxPa9QXT3K1gH3+mtK2Dgmrv8uMVZkv0dsAJ7ibz9F2J7XwPNwDV7Ke6aS/q3RFVKEX33RcKb/gdsC89ZV+FacFHKCtRkIyf/HZKwxtkHKFfIMUg/JxwDpWxUZxNWy8H+EFeRIL4Pfn7ctmtUsQhW3Taie1+LrzcIx2vUo5u4Zi/BVXPxqPfFVqEuYnXbMPzTR1wkONr3325vILJ5PdGdL4FlxefP55xH5J2nsOq2Ykyeg3fZDejFlaN6DbnECb8Dw5GwxtkHKFfIMUg/OQbxuXnr0A7s1kOYM9837NDyeBjr+2/3tMeH7Lc9G//A4fLiOfvD8XntFNZnz2ZO/h2QCmZCCNFL003MqQtg6oJ0NyVpel4xnrOuxn3qpcQObMaonJkRu42J8SNhLYQQDqW5vLhmvi/dzRAOIOMpQgghhMNJWAshhBAOJ2EthBBCOJyEtRBCCOFwEtZCCCGEw0lYCyGEEA4nYS2EEEI4nIS1EEII4XAS1kIIIYTDSVgLIYQQDidhLYQQQjichLUQQgjhcBLWQgghhMNJWAshhBAOJ2EthBBCOJyEtRBCCOFwEtZCCCGEw0lYCyGEEA4nYS2EEEI4nIS1EEII4XAS1kIIIYTDSVgLIYQQDidhLYQQQjichLUQQgjhcBLWQgghhMNJWAshhBAOJ2EthBBCOJyEtRBCCOFwEtZCCCGEw0lYCyGEEA4nYS2EEEI4nIS1EEII4XAS1kIIIYTDSVgLIYQQDpdQWK9bt46VK1dy0UUX8cADDxxz+9NPP82HPvQhLr/8cj7/+c/T3t6e8oYKIYQQuWrEsG5oaOCee+7hwQcf5JFHHuGhhx5i165d/bd3dXXxrW99i3vvvZfHHnuMOXPm8OMf/3hcGy2EEELkkhHDeuPGjSxevJiSkhLy8vJYsWIF69ev7789Go3yzW9+k8rKSgDmzJnD4cOHx6/FQgghRI4xR7pDY2Mjfr+//3pFRQWbN2/uv15aWsqFF14IQCgU4t577+X6669PqhHl5QVJ3X8kfn9hSp9PJE+OQfrJMUgvef/TL5uOwYhhbds2mqb1X1dKDbrep7Ozky984QvMnTuXK6+8MqlGNDd3YdsqqccMx+8vJBDoTMlzidGRY5B+cgzSS97/9HPyMdB1LelO6ojD4FVVVQQCgf7rgUCAioqKQfdpbGzkuuuuY86cOdxxxx1JNUAIIYQQxzdiWC9ZsoRNmzbR0tJCMBhkw4YNLFu2rP92y7L47Gc/yyWXXMItt9wyZK9bCCGEEKM34jB4ZWUlN910E6tXryYajXLNNddQU1PDmjVrWLt2LfX19Wzbtg3LsnjyyScBWLBggfSwhRBCiBTRlFKpmSweA5mzzi5yDNJPjkF6yfuffk4+BuMyZy2EEEKI9JKwFkIIIRxOwloIIYRwOAlrIYQQwuEkrIUQQgiHk7AWQgghHE7CWgghhHA4CWshhBDC4SSshRBCCIcbsdxoulhWjNbWALFYJKnHNTbq2LY9Tq1yPtN0U1rqxzAce2iFEEIkybF/0VtbA3i9eeTnVyW1OYhp6sRiuRnWSim6uztobQ0wadLkdDdHCCFEijh2GDwWi5CfXyS7eCVB0zTy84uSHo0QQgjhbI4Na0CCehTkPRNCiOzj6LAWQgghhIR1wt544zW++MV/THczhBBC5CAJayGEEMLhHLsafKC/bjnMy5sPJ3RfTQOlEn/uc2sms3Rh4iunDxzYz91330FnZwder48vf/krnHLKfDZsWM+DD/4GXdeZMmUK3/jG7bS3t/Gd73yDYDCIrmt86UtfZcGChYk3TgghhCBDwtpJbr/9G3z8459k+fILeOedLdx669f4/e//xH33/Zx77/01paVl/PSnP+LAgX289NILLFlyLtddt5q//W0jmze/JWEthBAiaRkR1ksXJt77Hc/zrIPBIIcO1bF8+QUALFiwkKKiIg4c2M/Spefxuc/dyLJl57N8+QXMmjWHYDDILbf8Czt3vsuSJedy9dUfHpd2CSGEyG4yZ50EpY79EKAUWJbFl7/8Fb773bspLCzi9tu/wZNP/oWamlP53e8e5uyzz+GZZzbwta/dlIZWCyGEyHQZ0bN2iry8fKZMqeaFF57tHwZvaWlmxoyZXHvtlfzkJ/dy/fWfIhaLsXPnu+ze/R6TJlXw4Q9/lNNOO5MbbvhYul+CEEKIDCRhnaTbbrudf/u3O/nlL3+By+XmjjvuxuVyceONn+HLX/4CHo+H0tJSbrnlW0QiEb797Vv5y1/Woes6t9767XQ3XwghRAbSlEpm7fT4aG7uwrYHN6O+fj9VVScl/Vy5XBu8z2jfu1Tx+wsJBDrT9vOFHIN0k/c//Zx8DHRdo7y8ILnHjFNbhBBCCJEiEtZCCCGEw0lYCyGEEA4nYS2EEEI4nIS1EEII4XAS1kIIIYTDSVgLIYQQDidhnWJ33PEt/vKXdce9z7nnnjlBrRFCCJENJKyFEEIIh8uIcqPRnX8l+u6LCd1X0zSSKcrmmrMM1+ylx73P17/+VS666GLOP/8DANxww8f5p3+6iXvv/RnhcIjOzi7WrqyQqQQAAAhLSURBVL2J8847P+GfCxAKhfj+97/Lrl070XWda6/9OJdcchm7dr3H3XffgWVZuN1uvv71bzJ58hTuuuvb7NmzG4Arr/wHLr/8yqR+nhBCiMyUEWGdbitWrOSpp57g/PM/wMGDB4hEIvzxjw9x883f4KSTpvH666/yox/9IOmw/tWvfkFxcTG//e3DtLW1sWbNJ5g1aw4PP/wg1177cS644IM88cSf2bp1C01NATo6Ovj1rx+kqSnAz3/+YwlrIYTIERkR1q7ZS0fs/fYZj9rgS5acyz333E1PTzdPP/0kK1Zcwoc/fB0bN77Ec889zdatWwgGg0k/7+uvv8bNN38DgJKSEs47bxlvvvk655yzlP/7f+/mlVc2snTpMpYuPY+urk4OHNjPP//zF1m8eClf+MKXUvoahRBCOJfMWSfA5XKxdOl5vPzyizz77FNceOHFfOELa9i+fStz5sxl9eobkhp673P0/tjxvbFjvP/9H+RXv/odp5wyn4cffpAf/OAuiotL+O1vH+bqqz/CgQP7ueGGj9PZ6cwi9UIIIVJLwjpBK1as5H/+53cUF5eQl5fHwYP7ufHGz7J48VJeeukFbDv53vzpp5/F448/CkBbWxsvvfQ8p512Jrfd9q9s376NK664mk9/+rO8++4OXn75BW6//TaWLDmXL3/5K/h8PhobG1L9MoUQQjhQRgyDO0FNzal0dXVxxRXXUFRUzGWXfYjrr/8wpmly+ulnEQqFkh4K/9SnPs0Pf/h9Vq/+CLZts3r1DcyZM5frr/8U3//+d7n//vswTRdf+crNzJ49l+eff5brr/8wbrebFStWMnPmyeP0aoUQQjiJ7GedhWQ/ayHHIL3k/U8/Jx+D0exnLT3rcRAOh/jMZ24Y8rZPf/oznHvu8glukRBCiEwmYT0OPB4v99//YLqbIYQQIks4eoGZA0boM468Z0IIkX0cG9am6aa7u0PCJwlKKbq7OzBNd7qbIoQQIoUcOwxeWuqntTVAV1dbUo/TdX1Up1FlC9N0U1rqT3czhBBCpJBjw9owTCZNmpz045y8AlAIIYQYjYSGwdetW8fKlSu56KKLeOCBB465ffv27Vx11VWsWLGCW265hVgslvKGCiGEELlqxLBuaGjgnnvu4cEHH+SRRx7hoYceYteuXYPu89WvfpXbbruNJ598EqUUDz/88Lg1WAghhMg1Iw6Db9y4kcWLF1NSUgLAihUrWL9+PV/84hcBqKurIxQKceqppwJw1VVX8R//8R9cd911CTdC17XRtH3Cnk8kT45B+skxSC95/9PPqcdgNO0aMawbGxvx+48sWKqoqGDz5s3D3u73+2loSK5mdWlpflL3H0mylWFE6skxSD85Bukl73/6ZdMxGHEY3LZtNO3IpwCl1KDrI90uhBBCiLEZMayrqqoIBAL91wOBABUVFcPe3tTUNOh2IYQQQozNiGG9ZMkSNm3aREtLC8FgkA0bNrBs2bL+26urq/F4PLz++usAPProo4NuF0IIIcTYJLTr1rp16/jFL35BNBrlmmuuYc2aNaxZs4a1a9eycOFCduzYwa233kpXVxfz58/nrrvuwu2WKlpCCCFEKjhii0whhBBCDM+xtcGFEEIIESdhLYQQQjichLUQQgjhcBLWQgghhMNlVViPtOGIGB9dXV1cdtll1NbWAvEStatWreKiiy7innvuSXPrst9PfvITLr30Ui699FLuvvtu4P+3d/cgjaxhFIBPSKJoYSGIWgQLsRBBG1FSDbEI4mfQzlQpUgliII1GEa3ECAHRXgKCjVZiUBvBRkVRlDQWQYw/EEHEH9CQCeG93XDvZUGWXWdmZ8/TzZDiJYeZly9FDjMw2/LyMgYGBqCUQjqdBsAMrLC4uIhEIgHAgQVT4hCPj48SCATk5eVFPj4+JBQKSS6Xs3osx7u8vJTBwUHp6OiQ+/t7KRaLomma3N3dSblclmg0KgcHB1aP6ViHh4cyMjIipVJJdF2XSCQi29vbzMBEJycnEg6HpVwuS7FYlEAgIFdXV8zAZEdHR9Lb2yuTk5MiIqKUkouLCxERmZqakvX1dSvH+2WOOVn/u3CktrbWKByh77WxsYG5uTnjX+uy2SxaWlrg8/ng8XgQCoWYwzdqaGhAIpFAVVUVvF4vWltbkc/nmYGJenp6sLa2Bo/Hg+fnZ1QqFby/vzMDE72+vmJpaQmjo6MAflww9ad//45Z1j8qHPnZQhH6efPz8+ju7jaumYO52trajBdSPp/H7u4uXC4XMzCZ1+vFysoKlFLw+/18Dkw2OzuLeDyOuro6AL+nYMpuHLOsWShiD8zBGrlcDtFoFBMTE/D5fMzAArFYDMfHxygUCsjn88zAJJubm2hubobf7zfuOfE99GVF5p+iqakJZ2dnxvX/C0fIHF8Vv9Dvd35+jlgshunpaSilcHp6ygxMdH19DV3X0d7ejpqaGgSDQezt7cHtdhufYQbfZ2dnB09PTxgaGsLb2xs+Pz/hcrkcVzDlmJP1V4UjZI6uri7c3Nzg9vYWlUoFmUyGOXyjQqGAsbExpFIpKKUAMAOzPTw8YGZmBrquQ9d17O/vIxwOMwOTpNNpZDIZbG1tIRaLoa+vDwsLC44rmHLMybqxsRHxeByRSMQoHOns7LR6rL9OdXU1kskkxsfHUSqVoGka+vv7rR7LsVZXV1EqlZBMJo174XCYGZhI0zRks1kMDw/D7XYjGAxCKYX6+npmYKFUKvWfgqlIJGL1SL+ERR5EREQ255ifwYmIiJyKy5qIiMjmuKyJiIhsjsuaiIjI5risiYiIbI7LmoiIyOa4rImIiGyOy5qIiMjm/gHzeiohGC3XuQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 576x360 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Learning rate: 0.01\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAesAAAE3CAYAAACKD2n1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzs3XmcXFWZ+P/PvbVXb9Wd7k5n3/eFhACGJCTsQSAoi4gocURxGwdhRmccBZ0ZRNRRmfmOywg/BRFQoo4oAiFAWBKSkIUlIXsT0tl732q/t+75/XG7q9PpparTlXR11/N+vfLqWm7fOnWT1FPnnOc8R1NKKYQQQgiRtfSBboAQQggheifBWgghhMhyEqyFEEKILCfBWgghhMhyEqyFEEKILCfBWgghhMhyEqyFEEKILCfBWgghhMhyaQfrYDDItddey5EjR7o8t3v3bm644QaWL1/Ot771LUzTzGgjhRBCiFyWVrB+9913+cQnPsHBgwe7ff7rX/863/72t3nhhRdQSrFq1apMtlEIIYTIaWkF61WrVvGd73yH8vLyLs8dPXqUaDTKvHnzALjhhhtYvXp1ZlsphBBC5DBnOgfdf//9PT5XU1NDWVlZ8n5ZWRnV1dX9b5kQQgghgAwkmFmWhaZpyftKqU73hRBCCNE/afWse1NRUUFtbW3yfl1dXbfD5b1pbAxhWZnZ/GvYsHzq64MZOVc7ZVmEn/pnXHOW4559RUbPna74rrUY7z6H/+YH0Byufp3rTFyjoUSuT2pyjXon1ye1XL5Guq5RXJzXp9/pd7AeNWoUHo+Hbdu2sWDBAv7yl7+wdOnSPp3DslTGgnX7+TJJmQZmcy16PJbxc6fLikcxm2uxDANN6/df24C9j8FCrk9qco16J9cnNblG6TvtYfA77riDHTt2APCjH/2IBx54gKuuuopwOMzKlSsz1sCsYNlL0TTdMXBt0NsCtJUYuDYIIYQYEH3qoq1duzZ5++GHH07enj59On/84x8z16oso9oD5IAGa0dbW0wkI0AIIXJL/8dTc0EyWA/g5Wr/oiA9ayHEWZZImDQ21mKa8Yyds6ZGx7KsjJ0vGzmdboqLy3A4+h87JFinIzHww+CaBGshxABpbKzF6/WTl1eRsdU+TqeOaQ7dYK2UIhRqobGxltLSEf0+n9QGT0cWDYNLsBZCnG2mGScvr1CW5faBpmnk5RVmbDRCgnUaknPWGRjKOG3JOWsJ1kKIs08Cdd9l8ppJsE5HWza49KyFEEIMBAnW6WgLkDJnLYQQA++tt7byla98fqCbcVZJsE5Hor1nPZDD4PZryzC4EELkHskGT0M2rbNODskLIUSOO3Soih/+8H5aW1vwen3cddfXmDFjFmvWrObJJx9D13VGjhzJvffeR3NzE//xH/cSiUTQdY2vfvXrzJ49Z6DfQtokWKcjixLMZBhcCDGQ3thxnPXbj/f7PJoG6pRqo0vmjmDxnPSXOd1337186lN/x7Jll/Leezu4555/4Xe/+z8efvgXPPTQIxQXl/Czn/03hw4dZN2611i0aAm33rqSTZs2sH37OxKsh5wsKDcqc9ZCCNEhEolw7NhRli27FIDZs+dQWFjIoUNVLF58EV/60mdZuvRili27lClTphGJRPjWt/6Zffv2smjREm688eYBfgd9I8E6Ddk0DC5z1kKIgbR4Tt96vz3pb1EUpbr+rlKQSCS4666vUVn5ETZuXM99993L7bd/nuXLr+bxx1exYcN6Xn55Dc899wz/9V8/789bOKskWKdDyo0KIURW8fvzGDlyFK+9tjY5DN7QUM/EiZO45Zbr+elPH+K22z6DaZrs27eX99/fT2lpOTff/Anmzz+P22//5EC/hT6RYJ2OLBgGl2AthBCdffvb9/Gf//k9fvWrX+Jyubn//h/icrn47Ge/wF13/T0ej4fi4mK+9a1/Ix6P8+//fg/PPfcMuq5zzz3/PtDN7xMJ1unIgmFwTbbIFEIIAM499zzOPfc8AH7604e6PH/FFVdxxRVXdXn85z///854284UWWedBpUV66w7tsgUQgiRWyRYpyMLetYyDC6EELlLgnU6sqDcqARrIYTIXRKs09E+9DyARVFknbUQQuQuCdZpkHXWQgghBpIE63RkUbCWnrUQQuQeCdbpSJig6WjaAF4uTYK1EELkKgnWaVBGFFyeAW2Dpml2wJZgLYQQOUeCdRqUEUFz+we6GaA7ZJ21EEKk6f77/43nnnum12OWLDnvLLWmf6SCWTriETSXb6BbYc9bS89aCDGAjH1vYOx9vd/n0TQNdcoema5pS3FNXdzvcw9FEqzToOIRNPfAB2tNgrUQIsd985tf58orr+Liiy8D4PbbP8U//MPdPPTQz4nForS2Brnzzru56KKL+3TeaDTKD37wXSor96HrOrfc8ik+/OFrqazczw9/eD+JRAK32803v/kdRowYyQMP/DsHDrwPwPXXf4zrrrs+02+1EwnWaVDxCJqvcKCbIT1rIcSAc01dnJHe7+lukbl8+dW8+OLzXHzxZRw+fIh4PM6f/vQU3/jGvYwbN55t27bw3//9oz4H61//+pcUFRXx29+uoqmpiTvu+DRTpkxj1aonueWWT3HppZfz/PN/Y+fOHdTV1dLS0sIjjzxJXV0tv/jF/0iwzgbKiKAXDR/oZrTNWUuwFkLkrkWLlvDggz8kHA7x0ksvsHz5h7n55lvZsGEdr7zyEjt37iASifT5vNu2beUb37gXgEAgwEUXLeXtt7dx4YWL+clPfsibb25g8eKlLF58EcFgK4cOVfGP//gVFi5czN///Vcz/Ta7kASzdMictRBCZAWXy8XixRexfv3rrF37IldccRV///d3sHv3TqZNm87Klbd3mQtPh1LWKfchkTC55JLL+fWvH2fGjFmsWvUkP/rRAxQVBfjtb1dx440f59ChKm6//VO0trZm6i12S4J1GlQ8AjJnLYQQWWH58qv5/e8fp6gogN/v5/DhKj772S+ycOFi1q17Dcvq+/D6ueeez7PP/gWApqYm1q17lfnzz+Pb3/5Xdu/exUc/eiOf+9wX2bt3D+vXv8Z9932bRYuWcNddX8Pn81FTU53pt9mJDIOnoBImJIysSDBDd0qwFkLkvLlz5xEMBvnoR2+isLCIa6/9CLfddjNOp5Nzzz2faDTa56Hwz3zmc/z4xz9g5cqPY1kWK1fezrRp07ntts/wgx98l0cffRin08XXvvYNpk6dzquvruW2227G7XazfPnVTJo0+Qy9W5umTme8IMPq64NYVmaaUVZWQG1t5oYjrGgrocf+Ac+iT+KefUXGzns6Qn/6DlpeAP9Vd/frPJm+RkONXJ/U5Br1bqhdnxMnqqioGJfRc55ugtlg092103WNYcPy+3Qe6VmnEre/nWVLURTpWQshRPpisShf+MLt3T73uc99gSVLlp3lFp0eCdYpqLZgjds7sA1B5qyFEANHKWWXPR5kPB4vjz765IC8diYHriXBLAUVDwPSsxZC5C6n000o1JLR4DPUKaUIhVpwOt0ZOZ/0rFOJRwGyZumWMoyBboUQIscUF5fR2FhLMNiUsXPqun5aWduDidPppri4LDPnyshZhrCOnnV2BGvpWQshzjaHw0lp6YiMnnOoJeGdaTIMnkLHnPXAB2uZsxZCiNwkwToFZbRngw98sJaetRBC5CYJ1qnEI6A70RyugW4J6E6pDS6EEDlIgnUK2bI9JtDWszYHuhVCCCHOMgnWKWRLXXCQOWshhMhVEqxTUEa29awlWAshRK6RYJ1KtmyPCbKftRBC5CgJ1ilk35y1BGshhMg1EqxTUPEwZEOpUWTOWgghcpUE6xSUEUXLgk08AOlZCyFEjkorWD/zzDNcffXVXHnllTzxxBNdnt+5cyc33ngj1113HV/4whdoaWnJeEMHglLKnrPOkp41uhNQqCFeT1cIIURnKYN1dXU1Dz74IE8++SRPP/00Tz31FJWVlZ2Ouf/++7nzzjv561//yoQJE/jVr351xhp8VplxUBZkUYIZIGuthRAix6QM1hs2bGDhwoUEAgH8fj/Lly9n9erVnY6xLItQKARAJBLB682SYeN+yqpNPGibswYZChdCiByTctetmpoayso6tvgqLy9n+/btnY75xje+we233873vvc9fD4fq1at6lMjhg3L79PxqZSVFWTkPHGtmRBQVFpMfobO2R/NhX5iwLASHw5f/9qTqWs0VMn1SU2uUe/k+qQm1yh9KYO1ZVlompa8r5TqdD8ajfKtb32LRx99lLlz5/LII4/wL//yLzz00ENpN6K+PohlZWZT80xuu5aoqQOgNaoRyYKt3OJhe/i7rqYZvR/T6LI1Xe/k+qQm16h3cn1Sy+VrpOtanzupKYfBKyoqqK2tTd6vra2lvLw8eX/fvn14PB7mzp0LwMc//nE2b97cp0Zkq2zaHhM4ac5ahsGFECKXpAzWixYtYuPGjTQ0NBCJRFizZg1Lly5NPj9u3DhOnDjBgQMHAHj55ZeZM2fOmWvxWdQerGXOWgghxEBKOQw+fPhw7r77blauXIlhGNx0003MnTuXO+64gzvvvJM5c+bwwAMPcNddd6GUYtiwYXzve987G20/45IJZlmXDS7BWgghcknKYA2wYsUKVqxY0emxhx9+OHl72bJlLFu2LLMtywbxKJA9PWt7nTVSH1wIIXKMVDDrRXvPWtZZCyGEGEgSrHuhjCi4vGh6dlwmmbMWQojclB1RKFvFw2iuLCrwIsFaCCFykgTrXqhsqgsOyWAtc9ZCCJFbJFj3QsUj2bPGGqRnLYQQOUqCdS+UEcmeTHBkzloIIXKVBOvexLMrWLcv3ZJgLYQQuUWCdS9UPJI9BVHgpDlrWbolhBC5ZEgFa6UUe6saMnc+mbMWQgiRBYZUsD5UHeRr/28dlUeb+30uZSXAjGXVMLjMWQshRG4aUsHa7bLfTm1TpP8nM7Ks1ChIz1oIIXLUkArWgXwPAM3BeLfPK8tCqfT2zc66TTxA1lkLIUSOGlLB2ut24HU7aArGujynLJPQk/+IuXddWudSbZt4yJy1EEKIgTakgrWmaZQUersP1sEGVLiJRMPhtM6V7FlnUbCWOWshhMhNQypYAxQXemlq7RqsrZZaAFQ4zeQzw573zq5yo7JFphBC5KIhF6yHFXppCnWds7Za24J1pCWt86h4W5KaOxs38pB11kIIkUuGXLAuKbKHwU9NJFMtNfbPPgbr7OpZyzC4EELkoiEXrIsLvMQNi0isc0Cz2oK1FUlvGLwjWGfRnLWmgaZLsBZCiBwz5IJ1SZE9bN0c6jxv3T4MTiyUXrnOeMQOjA53ppvYP7pD5qyFECLHDLlgPazQDtanJplZLbXgcAGgIq0pz9NealTTtMw3sj90h/SshRAixwy5YF1caBdGaTqpMIqKhSAexlE2wb6fxlB4tm2PmSTBWgghcs6QC9Yl7T3rk4bB25dtOYZPBkCF00gyy7btMdtoEqyFECLnDLlg7fe68LgdNLV29KytVju5TB8+CUizZ51t22O2050yZy2EEDlmyAVrsGuEn1zFrD0T3FFu96ytNJZvZd32mO10h6yzFkKIHDM0g3Wem+aTgrVqqUXzFqD7i8DpSWuttYqHs3IYXOashRAi9wzNYF3g6ZRgZrXWohWWAaD5CtMaBseIZldBlDYyZy2EELlnaAbrfHenKmZWSy16QTkAmr8oZc9aKWX3rF1ZVGq0nayzFkKInDMkg3VRnoe4aRGJmSjLRAXr0dt61rqvMHU2eMKwe69Z2LNGd4CSYC2EELlkSAbrQIFddawpGEcFG0BZ6IVtPes0hsE7So1mZ89ahsGFECK3DMlgXZzfXhgllswE1wra56yLUNFg70PJ2bg9ZhuZsxZCiNwzJIN1UadgbRdE0U9KMAOFivZccjTZs5Z11kIIIbLA0AzWefYweHMwjmqtBd2J5i8G2oN171tlduxlnY3BWnrWQgiRa5wD3YAzwedx4nU7aAzGsFQNWkEpmm5/L9H8RUB6wTob11lruiO9XcOEEEIMGUMyWIM9FN4UjGNZtckhcLCzwQFUuJckMyN7g7X0rIUQIvcMyWFwgOJ8u4rZyWuswU4wg3R71tmXYCbrrIUQIvcM2WAdyPcQDbZAPNypZ43LCw4XVi/LtzrmrGXplhBCiIE3pIO1M1wPkCw1CqBpWtta69561mFwuNH07JslkKVbQgiRe4ZssC7Kd1OEHZBPHgaHtrXWvZUcjUeysyAKSM9aCCFy0JAN1oF8D6W6vZZaLyjt9FyqKmb29phZOF8NMmcthBA5aAgHazfD9CAJV36XrG7d33t9cGVEsjMTHEB3Ss9aCCFyzBAO1h6GOVqJeoq7PGeXHG1FKavb31Xx7A3W9py1rLMWQohcMmSDdVFbzzrkDHR5TvMVgrJQ0WD3vxyPZGepUZA5ayGEyEFDNlh7nFCsh2iisMtzqdZa23PWWRyslepxVEAIIQYzZcawgvUopQa6KVkl+9YmZYgKNuDQFHWJ/C7PpaoPns3D4OgO+6eVAMeQ/a4lhMhR0fWPYe57Ay2vBEfFVBwjpuKomIpePBJNy93PvCEbrNu3xjwez+vynOZvD9ZdM8KVssCIZm2w1joFa9fANkYIITLMajyGVlSBY9hYEsf3YL6/yX7Ck4dj+BSc4+bhnnHxgLZxIKQVrJ955hl+8YtfYJomn/70p/nkJz/Z6fkDBw7wne98h+bmZsrKyvjJT35CUVHRGWlwutq3xjwS7rpeWm8fBu8uI9yIASprg3WnnrUQQgwxKliPc9w8vEtvRymFaq0lcWIfieP7MI/uJHboHZzj5qH7u+YjDWUpxxSqq6t58MEHefLJJ3n66ad56qmnqKysTD6vlOJLX/oSd9xxB3/961+ZMWMGDz300BltdDpUay0WDg4HnV3nPtx+e1/o7nrW7aVGsznBDGSttRBiyFEJAxVpQcsfBtgVJ/XCclxTl+BddjueRbfax4WbBrKZAyJlsN6wYQMLFy4kEAjg9/tZvnw5q1evTj6/c+dO/H4/S5cuBeCLX/xil573QLBaaoh5ijESEI51XurUXnLU6mbOOqs38QB7nTVIz1oIMeSoUCMAel5Jt8+396ZzMVinHAavqamhrKyjtnZ5eTnbt29P3j906BClpaV885vfZPfu3UycOJF77723T40YNqxrElh/lJUVcCRcj1ZYDsdBczkpKyvodEysoBhHItTl8WhMIwwEykrwn/JcNmg9lkctUBLw4gqcfvtOfd+iM7k+qck16p1cn9ROvUaRcBUhoHjUaHzdXD/TPYpDQJ4eozDHrm/KYG1ZFpqmJe8rpTrdN02TzZs38/jjjzNnzhz+67/+i+9///t8//vfT7sR9fVBLCszafplZQXU1LQQbzhBYsQCAD441IjfoXU6LuHKw2xuoLa2tdPjZk0dAC0RCJ3yXDYwQgYADXXN6MbpDdWXlRV0ed+ig1yf1OQa9S7Xr49KmMS2/BH3OVej+7oun4Xur5Fx5AgALQkfwW6un0rYIaul5gSxQXx9dV3rcyc15TB4RUUFtbW1yfu1tbWUl3dsjFFWVsa4ceOYM2cOANdee22nnveAiIXAiOAqHg5AUzDW5ZCeNvNQ8ah9Q+ashRDitFj1VRjbV2NWvd233ws1AKD1MAyuOZxonnxUuOe9HYaqlMF60aJFbNy4kYaGBiKRCGvWrEnOTwPMnz+fhoYG9uzZA8DatWuZNWvWmWtxGqxW+8uFv6QC6D5Y6357m8xTk89UPAwg2eBCCHGarJA9p6xaalMc2ZkK1qN5C9Cc7h6P0fwBmbPuzvDhw7n77rtZuXIlhmFw0003MXfuXO644w7uvPNO5syZw89+9jPuueceIpEIFRUV/PCHPzwbbe9R+7Itd0kFPk+EpmC8yzGar9AOeLEQeE8ajkgmmGVnsNYkWAshspwK24liVrCuT79nBRvQ8rvvVbfT/EVYEqy7t2LFClasWNHpsYcffjh5+5xzzuGPf/xjZlvWD1arXRBFLyglkH+0x2FwACvSguOkYK2MCKCBy3NW2tpnEqyFEFlOtfWsrda+BWsVakAvKOv1GM0fwDp+4rTbNlgNydptqqUWzVeI5vYRyPfQ3FPPmq5VzFQ8Ai5v9pa1a1u6JXPWQohs1d7z7eswuBWsT9mz1v1FqHBzztUOz9KI1D9WSw1a27ezQL671571qUlmWV0XHDp61gnZJlMIkZ3a55RVpBlldu0sdfs78QjEI+htBVF6ovkD9jbBsVC/2zmYDM1g3VqLXmgH66J8D03BWJdvYR31wU/JCM/yYC1z1kKIbKfCTdC2xDfdeetUmeDttLbCKLk2bz3kgrVKmKhgfXLeI5DvwUwoQtFTqph58kDTuywBUEYWb48JMmcthMh6KtSEXjLavt2SXrBWwXqAZKnRnmj+9r0dJFgPamZLHSiFXmivBQ/k20sATh0K1zQdzVc4aIfBlSXD4EKI7KMSBioWxDF8CtCxlDYVK2j3rPWUc9btJUdza631kAvWRmM1wElz1nZWd/fz1oVY3SSYadlaEAWkZy2EyGrtPV69dBw4nGlnhKtQA2hacpi7JzIMPkSYTXawPrVn3VNGeNc563D2buKBzFkLIbJbe49XzytGyy9Fpd2zrkfzF3d8xvVAc3nA5ZVh8MHOaDwBuhMtz/72VdRrz7pryVEVj4K76x7YWUOCtRAii1ltO2dp/mL0gtL0e9ZpFERpZ1cxk2HwQc1sqkYvKE2uk/a4HPg9Tppae+pZd6zXU5YJiXhW96xlnbUQIpu193g1fxF6QVn6c9ahhh63xjyVvdZaetaDmtFYbW+NeZKifDdNoe7rg5MwwbBLjNK2icdgSDCTnrUQIhupcBPoDjRvPlpBKcRC9hrq3n5HqT73rC3pWQ9eSimMpuou5eoCbWutT5UsjBK2h8KzfhMPZM5aCJHdrFATmj+ApunJz+JUQ+Eq2goJI2VBlHa5uJnHkArWxEKoWDhZEKVdIN/T4zA4kMwIT377GwTZ4DIMLoTIRirclMzY1gtKgdTLt1R7QZQ0e9a6vwjMWMoe+1AypIJ1+z8IrUuwdtMc6qaK2SklR1WW77gFnDQMLuushRDZR4WbkmuhtbZgrVL0rK22gih6Xvo9a/u1cmcofEgFa72ogoJzLsM5ckanx3usYnbqZh6DKlhLz1oIkX2sk3rWmrcAnJ7UPetg33rWubjWekgFa83to+zaL3fJ5g4UtC3faj2lipm3ADSto2dtZH+w1jTdrrkrwVoIkWWUGYdYKLl0VtM09IKyNHrWDeBw2Z/JacjFkqNDKlj3pCivh5Kjuo7mLeiSYJbVc9YAukPmrIUQWSdZveykKmRaQWkaPet6tLwStLbNP1LJxZKjORGskz3rHquYtSeYDYKlW2CvtZZgLYTIMu3LqbS84uRj7YVRett/2go1pKwJ3onbb5cylZ710BLooWcNdpKZ1V7FLB6254QdrrPZvL7THRKshRBZR4Xbq5d19Kz1gjIwor3uP92XNdZgD6/n2vKtnAjW7vYqZj1s5tExZx1Fc/vTHooZKJoEayFEFlKhbobBC3tfvqWsBCrcmHb1suR5/YGO5OAckBPBGuyh8J4382gfBg9n917W7WTOWgxCSqm060SLwcmuXuYET17ysY7CKD0E63ATKJVyH+tT6b7cKjmaO8E6391zFTMzjjKi2b89ZjvdIeusxaBjHthC6Pf/LAF7CLNCjWh5gU6jk3qKtdbp7mN9qlwrOZpDwbr7kqO6v32tdQvEI9mfXAYyZy0GpcSJvaAsrPrDA90UcYacXL2sneb2gyevxy9pqq0gipZmQZTkef1FdtVKs+uI6VCUM8G6KN9NUzDeTRWztmAdbkYZgyNYy5y1GIwSdVX2z6ajA9wScaaocHOn+ep2ei/Lt9pLjfa1Z51cvpUj89Y5E6wD+R4SliIYMTo93l5y1Iq02OVGB0GwljlrMdgoy8KqPwSA1Xh8gFsjzhQr3NilZw2g55f2MgxeD25fnztKuVZyNGeCdXF+92utTy45OnjmrJ2gJFiLwcNqPg5mHNCwmo4NdHPEGaCMmD2VmNc1WGuFZT2utVbBhrRrgnc6Z1sVs1xZa50zwboov4cqZj67vJ3MWQtx5li1BwFwjJyO1XS81wIZYnDqqF5W3OU5vaAUEka3Q9ZWqG9rrNt19KwlWA8pgWTP+tSSo040T749n6KsQTEMLnPWYrBJ1B0Ehxvn+AVgRJPzlGLoaO/hdjsM3rZ8S7V0nbdWwT5WL2uj+QpA02UYfKgJ5LvRNY39h7v+xWr+Qqzmavv2IAjWMmctBhurrgq9dCx6ySj7fpPMWw81qpdg3b5VphXsPG+tzDgq2orWx4IoYG9qpPkKpWc91LicDi4/bzTrdxzn3crO/2A0XxFqkAVr6VmLwUIpi0T9IRyl49ADIwGwGmXeeqhJVi/rZs66fa21dUrPuiMTvO9z1pBba61zJlgD3LhsEqPL8njkud20hDoSzTRfISraat8eDMFak6IoYvBQzdVgRHGUjrcTOj15EqyHICvcaO+rcMoWxQCa02N/zp6SEW71cR/rLuf1504Vs5wK1i6nzuevm0U4luCR53Ynk1zaM8Ltg7I/WMuctRhMEnUHAdBLx6NpGo7ASMkIH4LaC6L0tLeCVlDWZa11e0GU0+1Z6zm0mUdOBWuA0WX5fOziSbz7fj2vvWN/YLSvtYa2ajvZTuasxSCSqKsChwu92B4C14tHyJz1EKRCTeh5XTPB27VvlXkyq20YXOvl93pjb+bRmhOfhzkXrAEuO280s8YX8/uX93O8PoR+Us9ac3sHsGVpOkM9a6WUvVZSiAyyag+il4yxR4QAPTAKFW3t2JpWDAl2z7qox+f1gjJUsAFlWR2/E6xH8xWinea2xPbrqeTOiUNZTgZrXdO4/ZqZuF0OHnpmF5anIPnc4OhZOzMerJVSRF/+BaFV38iZWrvizFPKIlFXhaNsfPIxvXgEIBnhQ43VTV3wk2kFpaASnZbtWcGGPu+21emcOVTFLCeDNUBxgYdPXzWNqhOtvL4v2PFEjs5ZG++twTywGRVqxDy8PaPnFrlLtdRcanQdAAAgAElEQVSCEUEvHZd8TDLChx4Vj4ARReumIEq7jq0yO4bCVaihz/tYdzpnDhVGydlgDbBgWjlL5o5g9bttf9FOD5o+CC5JhuesEyf2E9u0Cue4+Wi+QszKTRk7t8ht7clljtLxyce0/BJweiTJbAhp79l2t2yrXcdWmXaSmVKqrWd9+sE6l0qODoLIdGbdevkUvIVt3wbPcq/aTFiYCSv1gafKYM/airQQefnnaPkleC/+HM6JF2AeegcVD2fk/P1qW7SV2Dt/QyVkmdpgZdVVge5ELx6VfEzTdPTACOlZDyFWuBHoviBKO3u4W0v2rK1YGIzoaVUvS56zLTk4F4bBnQPdgIHmdTu5fcUcws+4CYXhvx/bit/rxO9xkud1JW9XDPMzb3Jpj8sS+spMWNz/2DZqmiKcN62MC2dVMHVsAD2d8+uZWWetLIvo2l+ioq34P3IvmicP15QLMXa+hPnBNlzTLur3a5x225Qi+srDJA5vR88vxTV54YC1RZy+RN1B9JLRaI7OHzV6YASJ43sHqFUi09qDZa/B2uFEyytOLt9KtNhBu6/7WHc5p7cgJ4bBcz5YA0waVURdXhEYTnxuB6GIQU1jhHDUJBw1sdrWY88YV8ynr5pGeXH/k9D+tuEgVdWtzJtcyuY9NazbfpziAg8LZw7nwlkVjC7P7/F3MzVnHX/rLySO7sSz9DM42uYU9bKJaIXlGJWbBjRYG7vWkji8HTQHRuVGCdaDkFKKRF0VrgnndXlOLx6FWbnR3uluMBQiEr1SbT3r3obBwR4Kby+MYjbbP/vTs4bcKYwiwbpN3qjJ5Dtd/NPS+Z0eV0oRjSfYtKuaP7xSybd/tZmPXDSBK88fg+M057erTrTy7MYqLpxVwR0rZhIzEryzv46NO0+wZsthnn/zEKPL8rhswWiWzRvV9QS6A5RCKQtNO702mId3EH/rrzinLsY1bWnycU3TcE1eSPztZ7DCTd1uJH+mJRqPEdv0exyjZ6OXjMbY8SIqGkTz9vwFRmQf1VoHsRD6SfPV7U7OCHeUTzzLLROZZoWawOlOOZWoFZSROLYbALO9Z92PbHDInZKjOT9n3c536RfwLr29y+OapuHzOLlk/ii++7kPMXN8CX945X3u+81Wqk609vl1zITFr57dTb7fxa1XTAHA43LwoZnDuetj5/Djryzmk1dMxenQ+c3qvby1r+suNbStVz3d3rXZUkd07S/RS0bhXbKyy9C+c9JCUArz/TdP6/z9oRIm0Vd+ieby4r34c7gmXwgqgXFg81lvi+ifZHLZScu22jnaM8IlyWxIsNdYF6ecJtQLSlGhRlTCsIO15uhUlOp05ErPWoJ1H5QUevmHG+fw5Y/OpikY577fbGXVK5XEjPSD5t82HORIbZBPL59OnrdrIYBCv5vLFozmm7ctYOzwfB59fg/NoVPWPettAyLdBGulLOJ7XiO+4wWMA1tI1BzACjehlJ3IphIm1f/3Y5Rl4rv8K2hOT5dzOIpHog8bhzEAWeHxbX/GqqvCs/Tv0P0B9GFj0YtHSob6IGTVVYHm6JRc1k4rLAfdIUlmQ4QKN6UcAgfQC8sAhQrWY7bUoeUF+r0CR/cHUJHmIb9HugyD95GmaZw3vZwZ44tZtbaS1W8e4q29tXz+ullMHFnY6++ePPw9b0ppr8c6HTp3rJjFvz+yhd88v4d/uHFO8lur1kPPWilFbNNTGDte6HpC3YGWV4zmcGM1HcN7+ZfRAxU9vr5rykJim57Caj6BXtTzcZlkHt9L/J3ncE1fimv8AsC+3s5JC4lv/T+s1rrk8g+R/ezkslFoTneX5zTdgV5UIYVRhggr3IRj2LiUx2n5bbtvtdahWupPuyZ4p3P6A2AlULEgmrcg9S8MUtKzPk15XhefuXoGX//EfBKW4oHHt/HytiM9frvrbvg7lVGledy0bCLvVNaxbvtJH2ptwfrUtdbG9ucxdryAa9bl5K/8Kf4b/h3f8q/iWXwb7rlX4Rg+Gc1XQMlln8Y18YJeX9s58UOAdtZ61yoWIvrKQ2iF5XguvLXTc67JFwJgvH+W2qJOYzmd6EQphVVXlUxc7I4eGEFCetaDnlIKFWpKq7633bO2g7Xds+5fchl0rLUe6kPhEqz7aca4Yr7zmfOZNaGEJ17cxy//upNIrOuyqmfe6H34uyeXnz+G6WMD/O7l/dQ0RewHu+lZG/vWE3tzFc6JF+BZdCuaNx9H6Tic4+bjnnUZngs+hu/SL+Jf8a8EFl6X8nX1/BIcI6ZhVG46K8NL0TceR4Ua8V36eTRX5/rsemEZ+vDJmPvPfLA2Dmzh4I9uw2o6ccZfayhToQZUtLXb5LJ2evEoVGuNlLcd7IwomLG0klE1f7Fd1KmlBrOlvt+Z4PY5c6PkaFrB+plnnuHqq6/myiuv5IknnujxuFdffZVLL700Y40bLPJ9Lu68aS43LpvIlj013PebrRyp7Shh2pfh71PpmsZnr5mJrsGv/7YLy1InDYPbXwrMQ+8Qfe3XOEbNwnvJHaedIX4q55QLUc0nsNoShc4Uo3ITZuVG3As+gqN8UrfHuCYvxGo8QqLh8Blrh4oGia1/DBWPEt/9yhl7nVzQUbms9541SmG1VJ+lVokzIVkQJY05a03X0fKHkag5AJbZr+pl7c50yVHzyHuEn3lgwOfEU36qV1dX8+CDD/Lkk0/y9NNP89RTT1FZWdnluLq6On7wgx+ckUYOBrqmcc2F4/nnT8wnEjP57m+28saO48nh74K89Ie/TzWsyMutl09l35FmXthyqFPPOlFdSeTFn6MPG4vviq+c9u413XFNOA90xxkdCrdaa4mu/w368Mm4513b43HOiReApmPu33jG2hLd9DtULIxnxGTMfW9Ij68frNqDoOnow8b2eEz7lpmSZDa4qZAdJHsriHIyvaCURM379u1+FERpd6ZLjsbfe7HL1p4DIWWw3rBhAwsXLiQQCOD3+1m+fDmrV6/uctw999zDV77ylTPSyMFk2thi/u0z5zNxZCG/enY3//HoltMa/j7VotkVnDu1jD+/foD6oAFAov4w4dUPouUV4/vwP2a8uITmycM5Zi7m+2922tYuE1QsRGzLnwj94R5QCt8ln+8YMeiG7ivEMXo2xvtvnpE5ZfPIe5j73sA972qKL7kVFQtiHnwr46+TKxJ1VeiBkd0ml7XTiypA0yRYD3LtPdp0azLoBaWQsD/DMtGz1pwecPlSDoMb+zeQaDjSp3OreITEkZ04J5yXseqVpytlsK6pqaGsrCx5v7y8nOrqzsNWjz32GDNnzuScc87JfAsHoaJ8D/90yzyuuXAcR2pDpzX8fSpN01h51TT8XhcvvWUnm0VffRhNd+K/+mud9uTOJOeUC1HhJhLH92TkfCoeIfbWXwj+7mvE334G59hzyLv+O+iF5Sl/1zV5ISpYT+LE/oy0JdkmI0Z03aPoRRW456/AN34OWkEZxu5XM/o6ucJOLjuIXtZ7drDmdKMVlElG+CDXHqzT7VlrBR3xJBPZ4AB6irXWVksN0VceJrbp9306r3noXbBMnN1U4TvbUi7dsiyr0zcKpVSn+/v27WPNmjU8+uijnDhxekk5w4ZltjJVWVl2pO9/8aZ5XHfxZMqL/Tgd/Z9HLgO+est8/vzY+1Bg18Udeeu9eCom9P1caV4jK7CEqtd/jfPIVsrmfajPr5M8TzxKy7bVNG38M1YkiH/qBRQv/Tie4ePTP0fRUqrW/wbnkW2Uzc3cf576F/+Iaq2j4rb78I2wPzyKz7uShleeoEhvxT1sZMZeayjp6d+Q2dpAMNJC0fhpFKX4d5YoH4PRfCJr/s9m0lB8T92pU2Hibi/lo1J/4QYIjhpDzRb7y1rZ6IqM9FiNwDAwgj1e8/p3/w9QJI7uothr4ixInbkOUP36OzjyAlTMnpexXKDTlTJYV1RUsHXr1uT92tpayss7/lJWr15NbW0tN954I4ZhUFNTw6233sqTTz6ZdiPq64NYVmYm78vKCqit7XtlsTPFBTQ2hDJ2vglleYybMpG6o1t4JriU0O+qKA/UUBbwJf+UF/sozOt5+LGv18gxbgGtuzeiFtzSZVhTmXHMyk3E974ORgycbvuYk3/qTsyqt1GRFhxj5uI/73ocZRNoAejj35Vj3Hxad72BOvfmLptDnI5EzQHCW57FNfNSgr4xBGtbKSsrIDbqfNB+T/WGZ/EuvKXfrzPU9PZvyKx6D4CwdwTxFH+/Zt5wjAPvUlPd1Os0yGCTbZ9DZ1KkrgZ8gbTfb0LZnTNnUSl1dcEUR6fHdBaQqD3QbRuUGSP49kvoZROwaj+gevNLuOdelfKcyowRqnwL19Ql1NVl7jMcQNe1PndSU37aLVq0iP/5n/+hoaEBn8/HmjVruO+++5LP33nnndx5550AHDlyhJUrV/YpUIu+u/bDi1j71ljy6kKEGyPsqmqk8b3OoxozxhVz08WTmDCi/8PjrikXYu5/A/Pw9uSmDFakBWPXKxi7XkZFWtCLR6MXltlJWQkDFW5CmQbKjIEZRx82Fs+Cj+KoOL0ku2RbJi/ErNxE4sgOnOPmp/6FXqiESfT1X6P5A3gu+Fin53R/AOf4+Xai2fk3ZjRxb6hL1FUBGo5hY1IeqwdGgGWiWmvRzlLxHZFZqo97CGhthY2chZkrcNRecvTUkV+wV5sQD+NZeAuxTb/H2L8xrWBtHn4PzHhWDIFDGsF6+PDh3H333axcuRLDMLjpppuYO3cud9xxB3feeSdz5sw5G+0UJ/G4HHz4Q53nAw0zQV1zlNqmCFXVQV7ccpj7frOV86aXc8PSiVSUnP5OYY6RM9B8hZiVm9ADIzB2rMHY/wYkTBxj5toFV0bOOCsJGI7Rs9E8+Rj7N/Y7WMfffQ6r4Qi+K7/abXKea8bFmB9stbcLlV2/0paoPYgeGNFlvXx3OjLCj5+1Snkis6xwE46y9Ddj0XxF4HRnNFjr/gCYcXvN90n/l5VSGDvXopeMxlExFdeURcQ2PEGi4SiOkm42STqJ+cFW8OThGDE1Y+3sj7TGEVesWMGKFSs6Pfbwww93OW706NGsXbs2My0TfeJyOhgxLI8Rw/KYO6mUyxeMZvWbh1iz5TBv7a1l6TkjuG7JBAL5XWuBp6LpDpyTPtS2z/VWcLhwTVmCa86VOIrP7nyupjtxTroAY+/6fm2vmGg6Zu86NvECnOO7D/qOUTPtRLM9r0mw7gOrvgrHiOlpHau3beiRaDqKk/59+RoKYtueRh82Ftf4cwe6KWlJVi8b34eetabhu/RLFE2cTHOGFnacXMXs5M8Eq+Z9rPoqPEs+3Va6+EPENv4Os3IDjlNG006mEiZm1Tt2FrieHVW5pYLZEOXzOLl+6US+/8ULuXj+SNZtP843/ncjf3rtfQ5Xt9LYGiMS69irOxXXjEvQh43Dfd715N36Y7xL/+6sB+p2zskXQiLe69IqZcTsDUyMaJelXkpZxF5/FFwePIs+2eM5NE3HNX0ZiWO7B7SimYoGM16QIVH7AbEtf8r4WnIr3IwKNeLopXLZyTS3Dy2vGKtRMsITDYeJb3ua2LpHUUZ0oJuTnngYEvE+b6XrHD8/o4mb7Znop661ju98CVw+XFPsksXJJaD7N/a6BDRxdBcYEVwTFmSsjf2VHV8ZxBlTlOfmU1dO48rzx/DndR/w7MYqnt1Y1ekYj8uBx+3A63bgczvJ97so8LnI97k6bvvdFJ73VcqL/QS8PSevnQ2O4ZPRCkoxKjfimroYaFsuVH8I88gOEod3kDhRCeqk2ulOjz0s6/Ki6TpW03G8yz6L7u99ez7XtCXEt/6Z+J7X8C78+Gm1VxkxjN2vYrVU4xw7z+6xp0iOU2Yc8+A2jN2vkTi+B+fUJXiX3t7vHYqUGSe29c8YO1aDUqDpeM67vl/nPFl7tTu9m20xe6IHRspWmYDx3kt2Kc5IC/Eda/Ccm7os8ECz+rhs60zpruSoFW7GPLAF18xLO03JuKYsIrr2f0kc34tz5Ixuz2d+sBVcXhyjZ53ZhveBBOscUV7s5wvXzeLaC8fRFE1QWxckGk8QjZvEjETb7QSRmEkwYlDbGKE1YnRb59zjdlBR4mfEMH/bzzyGF/twOnTMhIWRsDBNCzOhkrc9bgeBfA/FBR7yvM6U89uxeIKWcJxoPEF5wIfH3ZEprGkarkkLib/7HPE9r5E4vo/EkR2oSAsA+rAxuOcutze1N2MoI2b3VIxo8rZ7/AKcU5ekvG66P4Bz3DzMfetR59/Qp0QzFQ8Tf+8lYttfQIuHMHHi3LUW3H6c48/FNfF8HKNmdQrciYajGHtexdi/AWIhtIIynBMvwNy3nqgZx3vp5097WM48vpfoa79GtVTjmr4MFQsRf/dZXFMWoRcNP61znqp9Dbyjl8plp9IDIzD2re82OShXqGgQY/9GXFOX2MF6+/O4Z16K5s3sstZM62v1sjNF72YzD2PPa2AlcM+8rNOxzvHzweXF3L+x22CtrARm1ds4x87LqsRSCdY5ZlRZPvP6sKzETFiEIgatEYPmYJwTDWH7T32I/Yeb2LSz73WdnQ6dQL6bQIGH4nwPHpeD1nCclrDR9jNO3OgYotKA8hI/Y8ryGF2ez5iyfMYOn49b/Y3Y64+gefJxjJ6Fc8wcHKNn93lILhXXjIsxD27DPPg2rkm971YG9gdv/L01xHa8iGZE2GWM5qXYMhpdwxkZr+LikuNMPrANc9/6tsA9H0fpeIz338SqrgTdYQfz6cvsXrimE393ArE3nyJixvFd/uVeK4N1aU88QmzzHzB2rUUrKMN3zT/jHDUTK9yEeeQ9ohuewHfV3f0KlCoWsjNt967DMWJan3IJ9OKR9hepUIP9BevUcyuL+Nt/w1Eypsf8gsEuvuc1SMRxzb4c0DD/eC/xd5/D86GbB7ppvUpWL0ujLvgZ5faDw5Xs6SsrgbH7VRyjZnXZClhzenBOWIBxYAuexZ/q8n8pcWIfKtqKM4uGwEGCtUjB6dApyvdQlO9hdBnMmtC5PGAsnuBEQ5jqxjBK2ce7nBpOh9522/4ZiydoDMZoao3ZP9tuH6oJEjcSFPhdFPrdVJT4Kcyzbxf43bhdOsfrwxyuCXKoOsjWvbXJ1z7HdwWuvALi3jEE4l4C1R6KwyGKCxIUF3jQNAhFDEJRs+Nn1CAUMTAtRSDfTVGeh0C+x77ddv9UjtGz0PKHYex5tddgbYWbiG9/AWPXWjBj7DDGsSYyhwmzZvOlReMp8Lt45e0J/GZjFZHIuVw9PszSoqNw8C3MfW+gF1Xg+dDHcU5d3KUinfucD4PTTeyN3xJ54b/xXXknmit1sqB56F2i636DCjfimrMcz3k3JH9P9wfwnHe9nXBz8K3Tmp9TSmF+sJXYG79FRYO4z7ka94KP9ukc7UlmVtPxLhWtlLKIrXsMY8+roDvxX/evPW720h1j7zpim/+Ao2IqznHzcYydi55lex4rK4Gx82UcI2fgKLGXuzknLyT+3ou4Zl+BnsbWkwMla4bBNQ3NH0gOg5tVb6NCDbgWf6rb412TF2HuewPz0Dtdtgu2k2jdOMfMPePt7gsJ1qJfPG4H4yoKGFdxdj4AIzGTo3UhjtQEOVI7ioaWGM2tMQ7W1NMSSp0s5dA1exhe12gJxekub6swz01JgYfSgI+yIi+lAR+Thl9A8fvPE288jitQQcJSWJYiYSnMxmOonWvgg01gJXjbmMCa8GzGT5/Gl5dMYHhxx7K55ReMZek5I3lxy2FWbz7E3w4WsGjmEq6bV0DpmPG99m7dsy5Dc3mIvvYrIs//2O4Nd9ODVWYM8+BbGHvXkzi6E714JL4r7sFRPolYPME7u6p5c1c1h2pamTdpLNcUjCS24Qmco2en9QWgnRVqJLb+Mcyqt9FLx+H78D/1ustWTzpt6DF6dsf7UIrYG4/bX5JmX4lZ9TaRNf+D/4Z/S2v0xDzyHtHXH0EPjCRRXWl/CGsajuFTcIydh3P8PPSiEac1omBFW0kc22PvfGcl7L3lk39MgsNHoMrnplX1yjz4VpfA4jnvesz3NxN/6694L/p0n9t3tqhwE7h8aS3TO9O0k0qOGjtfRssfhnPsvG6PdYycgeYPYOzb0ClYK2VhfrAN55g5ffq/cDZIsBaDis/jZPKoIiaP6poYZiasth57nMZgDIA8r5M8r4s8n/3T63YkP5wtS9EajtMUjNMcitEUjNMUjBE1FUdOtHC4upV39tdiJhSFWh7/FtB49rHHeSayAFBMctZwiXcnc9xHMJSDzbFJvBKdyfgpk/jyRRMZVZrX43u4bskELl0wmuc2VfHytiO8sauOAv8Ryot9lAf8DC+2K9GVF/spDXhRyv6iEimYizbnNkp2PE7NH+5n16TbcPkLKQ94KTOP4T2yGfP9zWBE0PKH4T7/JrSZV/BuVQubN77HO5V1xA2L4gIP44YXsG5HNQeZw1cLX2Df808w+srbUm44o5RFy1trCL30GFgJPB/6OK45V552BTLNW4Dmye+0oYdSitiGJzB2rbV76xd8DNf0iwg/fR/RF3+G79p/6TVJL1F/mMiLP0UvHoX/um+Cy4NVV4VZ9Q5m1TvEN68ivnkVWtFwvMs+i7Mi/bW0Khok/PR9qJaaHo+pATyLP4V71uUpz2e896Kdm3BSYNELy3HNWIax+zXc53w4rdr5A0GFGgd+CLyN7g9gNR4j0XiMxLHduC+4qceETE3XcU5eiLHjRaxoa3K0xao5gAo3Zd0QOEiwFkOI06FTWuSjtCi9+VJd15JD/NAxMnByqUhLKZpaY9Q1Rwlu2sXSloMMnzydcQ2bKIwcxXD4OVx+CXXlC/F58vnKiELGDk9vlCHf5+LmSyZzxXlj2LTrBNUNEWoaw+w93MjGnb0vFZvlWsbt1muMfOsXvB0fzwjP+zgdrYSUk33aJI4UzMUqm0LomMlbr75JJGaS73OxePYILphRzpQxAXRNIxQ12LSzmvfeOsj04+t54OcljJ08hSVzRzCmPJ88rwtd7+h5WsF6oq88TPD4HhwjZ+Bd+pl+BxJN09CLOzLClVLENv4OY+dLuOYsx33Bx9A0DUfJGLzLPkf05Z8T2/BEjz1OK9RIZPVP0Ny+TqMPjrIJOMom4DnveqxgPeahd4lvf4HI6gfxX/fN5BB0b1TCIPLi/6BCDfiWf9Uu5KI7kn803Qm6TuL1h4lsWoVz1Owuc6YnS9RVkTixD8/CW7oEFve512HsXU9s65/xXfqFdC9nRiVqDmAF63vcdcoKNw34EHg7zV+EdXQXxs6XQXfimra01+NdUxZhbF+N+f5m3LPsJDTjg612zsjY7NuUSoK1EL3QNY2SQi8lhV5MdRWR1T9hztE/oRUOx71gJflTF1Pi7N9wWXGBp0tFuriRoLY5Sk1jmLqmKE6HhtfjxOd24vM48HnOJ9E0hxHr/5eRzneIFE/mQNEV7NMmcaIlQU1ThNp3j+HQNc6dWsaHZgxnxvhiHKcEhDyvi8sWjMaa+fe0/v5f+IzvHX7yfjGbdtmJgxqQ53NR4Hcx11XFZcZadBT7R17HAc9s9K3N6FoLuq7ZQVezpxqcDh2HQ8fl0HA4dJwO+zFN04jEOnIIwlH754JGN+ONSh7/y3sstTYyqvYNtOmX2UHspCDhmnQBVt1B4u8+h146DveMizu9HxWPEFn9E1Q8gv+6b6L3sAWjnj8M98xLcY49h/DT9xF57sf4P3KPvX1jD5RSRNc9SuL4XryXfrHXCnpl13yJQw/dTeTVh/Bf960eRx3i770ITg+uaRd1baM/gHv25cTffZ7EvKvT+jKRSVZLDeHn/hPiEVzTLsKz+LauewOEm9GHTz6r7eqJ5g9APIyx/w2cky5IuROhY9hY9JLRGPs34J51WVv+xTZ7hYan+1GxgSTBWog0OUbPxn3e9ejFo3COO7ffa55743Y5GFWa1+NQOgDDz8Ua8V3QdAoKSikHTu4PKKVQik49457ovkJ8F9xE+Ru/5UdXu9nFZBpaowTDBpFQiOm1LzIlup3jlPNUbBk1lQUkrONYSmFZbevc214vXQ5dw+914ve6GOMMMIMoU479jVGOPbwencafNoykfPcmxrflRJQUeO2aAOWXUlZ2ALX+t7S4ynCPmIKlFM0tEbwbf4Gv/ijbx32Cg9siNAd3EIraeyfruoaudXyp0HUNh64xZfgtnH/4N4Se/RH5H72nx+VS8Xefxdz3Bq5zP0Jr+Txi9SGGFXpxu7oGYmfhMLyLbyO69n+Jv/Nst2umrUgLZuUmXNOX9hgc3POuIb77FWKb/4T/qrt6vJaJxqNY1e/jHH9uRpZ7KTNO5KWfARqu2VdgvPciiYYj+K74SjIJ0DATWKFGnFnSs07mMRjRZE85FefkRcQ3r8JqrrYLKLXW4px/7Rls5emTYC1EmjRdx3PuRwa6GZ30NgStaRp9yZ1yzbgEY+/rJLY8xbk3P4DmLiNRV0V07ZNY0RO4513DlAXXc6/D2eOuUkrZSXcJS2Em7LX2ifa19wmFshQ+j5M8nxOPqyN/wDycR+T59Zzn2IOasoyxY1dwfXWQQydaef9oC5t3d54f9mvn8I+FR3G/9FO+23wNLcrHx/2bWOTdz+9CF7LpbQd+TzVF+W7yffYcvJGwOn2xsCw74GzbG+FN51K+bLzIvsf+g82jPsX4MaVMHlWEUnC8PkT8wFZmH/4je/Qp/Pq1YmIvbUi2pSjPTWnAS1mRj9KAl9IiH5PHleAqPYe8iRcQ3/YXnGPP6ZJ8Z+x+FSwT1+zLMcwENU1Rahsj5PtcjB2ej9vlQPPk4T7nauJb/kTixP4uG+Ek6g4Sf+sZzIPb7Ac2PIFr+jLcc5f3a6/o2MbfYdVV4bvyq/bSwpHTib7yMOH/+zcSSz7Pa8fy2fT2+/yrx2R9ZYTpE4OMLh/YNeHtJUf1sgnoadYqd3KPs3AAABBjSURBVE1eSHzzH+y6BsoCTev3ngNnigRrIQRgfxnxLllJ+Onv2jWq80uIvfkHNG8+vmu+jnPUzNTn0LS2IW+7Ml669GFjQHPgmrYYz0WfZo6mM2dSx5B0MGLQEooTMxLEjQQxI0Fr0yjGvf1T/nn0mzQXTWX0sf0EJ13BDQtu4O/y3N32eLsTiZkcONbCnt0BZh1ZxfSqVfxyx8VYbdWYxzjquLPwBQ5Tzht5V3LRuCJGDPPjcTuoa45S1xSlrjlC5dFmNu+u6VTCN1+fwDeK3qPh6f/HqyP+jpLiAkoKPLQGI5y7cw21jrE88uQHNLbs4eRBCV3TGF2Wx4SRhUwqn8Mczxqim/+If8U30DSNxIn9xN5+hsTh7eDyoWZfQ6JiJs4D6zF2voyx82WckxfiPufqLhtWnFw7IRg2CEYM8n0uRpfnk+9zYVRuxNj9Cq65H06ua3eNX8CJJYXo635B3os/oSm8gKkVM6AVKhvgd7/ezMJZFXz0ogmUBU6vXn9/6YV2YR/37CvSzvDX80twjJyOUbkRTXfiGDE95fD5QNFUposOn4ahvJ91NpJr1Ltcvz7R1x+xqz8BznHz8Sy7vcva5DNxjaxoK5onv09LqYwDW4i+9DO7rZMX4r3kC/0q7hLf/SqxdY8SH/shdo24Dq8ZZOp7P0N3usi7/tspy9OaCYvG1hgGGpVVDdQ1R3BV7+Lixj+x3pzDH1rs4Heu+wM+nb+Ov7ivJTJsZjL7vyzgoyUU54MTLXxwrIUPjrcSjplc5NnDTXmbeYULGG8dZoJ+nKDl4ZXoTNbHphFVHXPJxXqQS327Wejej1szqWQcWx3nUmmUEYwYRGKJnprPpPwQX3L/lRZvBYfnfp7Rw4uoa47y4tbDVB5pptCV4MvDtzAivA992Fis+kNw5ddZfdDDS1uPYFmKi+eN4trF4ynKO6XYiGVR2xTleH2I6oYIw8vyKfG7GFWWh9Nx+lNKSimi8QTBiEGssZpAxUjyfekXDTL2riP62q8A8Cz6FO7ZqTP4++t09rOWYJ2D5Br1Ltevj4oGibz8C7uK2sxLuw1+2XSN4u8+T6KuCu/Fn81IecjYtr8Q3/ZnXHOW25u4tNTg/8g9OEpGp32OU69PdN2jGLtfw/nhr9OSNx7/qz9CM0Lk3fxAr2uxLaWobYzwwdEGJmz5T/xmMxE9j8qihRwvWYDDbc/ju10OHA6NuGERi5vEDAsr0srYpi1MCr6FV0U45p7A3rLLITAyWe8/3+ciz+ukJRzn2PEGZu75X5xmmB+3rqDe6Fg7XVrk5fIFo1kydyQ+j078nWeJb/k/QJF3yw/RC8tpbI3xzBsf8Pq7x3E5dS6ZPwqHQ+N4fZjj9SFqGiMkuvmcd+gao8vyGVeRz7jhBYytKKDQ7yYYMTr+hNtGApL3452eNxOdz1uU72bkMDvnY+RJf9qnRE6m4hGCv70TEgZ5n3zwrBShkWBNdn2IZCu5Rr2T65PaUL5GSili6x/D2P0KaBq+5XfjHNu3alanXh9lRAn98V5A4V16O5Fnf4hn0Sdxz74i7XMmag9iNR7BOfGCvpWbNWIYu9YSe/uvYMRwzbjYTpQ8abREKUX0lYcwKzfhu+br6CNmUNMU4UhNEI/bwazxJV0SFc3DOzCP7sRzwc2dki2rG8L8ed0BNu+uwaFrlBf7knsIjBjmp6JtTwG3183bu09QdaKVQ9WtHDzRSijadS+Cdppmr17otMFQ2+38tttet5O6pgjH6kIcqw9xrC5MzOgYSXA6/v/27i42qjIB4/gz/ZiWapGP9MN0WS5cUpUviUQpYWnKLq10KAXSxG439KJJIxFt0k1UFALesGAlaax6IYnBNZaleCFQgoVo9GJthUCUbgImIFZBSjuA0hamM0Pn3Yt2ZymWng4U5t2Z/++GnDmnZ16etOeZc2bmvC6lupOU6k5U6tA3K1LdSfqj/wul65pOTv+L0tOSlZ7mDv87MW3wOW79JsXdoKwV2weR8UJGoyMfZ7GekQmF5D+6R4lTf6/kGQsj/vmR8rlx8bR8+/8uJSVLrgQ9+Nf6O56P/U6EfD0KHN83+CIkOUUp81Yoedaf5UpMVuDkF/L/6x9yz181bh+i7PMFlepOvO0l7t+8oDFGV3r86rjYq+v9waFCdofLOC01SQkRvsURMkZXevp14dJ1Xbh0Tb2+wcmB+v03whMX9QcG5AsM6Hr/4Fn6SI2YOXmCtj2XF9Fzj+ZOypoPmAHALVwJCUpdUD6u+0zKniH33GUKnDio5FlL72tRS4Nfz0tdtEbJM5fI/3WT/EeaFDj1hdyP/2nw/um/myX3vJJxe76RLjmPxuVyaepDqZr60PjdujTB5QrfKGnOI86fjg+FjPr6g+q9PnipvXdocqH0tOhOCyxR1gBw37jnr5LrgclK/sP4naVFKnFyjtKW/U03zv1b/q93y//1P+V6YIpSlzw3pnuZx7KEBJcmprk1Mc0tya4bo1DWAHCfuBKTI3qf+l5KmjZbiTmP68bZo0qYOt262cgwHGUNAHHKlZAY1bN8jF18X/MAAOD/AGUNAIDlKGsAACxHWQMAYDnKGgAAy1HWAABYjrIGAMBylDUAAJajrAEAsBxlDQCA5ShrAAAsR1kDAGA5yhoAAMtR1gAAWI6yBgDAcpQ1AACWo6wBALAcZQ0AgOUoawAALEdZAwBgOcoaAADLUdYAAFiOsgYAwHKUNQAAlqOsAQCwHGUNAIDlxlTWzc3NKi4uVmFhoRobG3+z/rPPPlNpaalWrFih559/XlevXh33gQIAEK8cy7qrq0v19fXatWuX9u7dq6amJp05cya8vq+vT6+//rp27Nih/fv3Kzc3V2+//fY9HTQAAPHEsaxbW1u1YMECTZo0SWlpaSoqKlJLS0t4fTAY1ObNm5WVlSVJys3NVWdn570bMQAAcSbJaYPu7m5lZGSElzMzM9Xe3h5enjx5spYuXSpJ6u/v144dO7RmzZqIBjF16oMRbe8kIyN9XPcXi8hodOTjjIxGRz7OyGjsHMs6FArJ5XKFl40xw5b/q7e3V+vWrdOjjz6qVatWRTSIy5f7FAqZiH7mdjIy0uX19o7LvmIVGY2OfJyR0ejIx1k8Z5SQ4Ir4JNXxMnh2dra8Xm942ev1KjMzc9g23d3dqqioUG5urrZs2RLRAAAAwOgcy3rhwoVqa2vTlStX5PP5dPjwYS1evDi8fmBgQGvXrtWyZcu0YcOGEc+6AQDAnXO8DJ6VlaXa2lpVVlYqGAyqrKxMc+bMUXV1tWpqanTx4kWdPHlSAwMDOnTokCRp1qxZnGEDADBOXMaY8Xmz+C7wnvX9RUajIx9nZDQ68nEWzxndk/esAQBAdFHWAABYjrIGAMBylDUAAJajrAEAsBxlDQCA5ShrAAAsR1kDAGA5yhoAAMtR1gAAWI6yBgDAcpQ1AACWo6wBALAcZQ0AgOUoawAALEdZAwBgOcoaAADLUdYAAFiOsgYAwHKUNQAAlqOsAQCwHGUNAIDlKGsAACxHWQMAYDnKGgAAy1HWAABYjrIGAMBylDUAAJajrAEAsBxlDQCA5ShrAAAsR1kDAGA5yhoAAMtR1gAAWI6yBgDAcpQ1AACWo6wBALAcZQ0AgOUoawAALEdZAwBgOcoaAADLUdYAAFiOsgYAwHKUNQAAlqOsAQCw3JjKurm5WcXFxSosLFRjY+Nv1p86dUqrV69WUVGRNmzYoBs3boz7QAEAiFeOZd3V1aX6+nrt2rVLe/fuVVNTk86cOTNsm5deekmbNm3SoUOHZIzRnj177tmAAQCIN0lOG7S2tmrBggWaNGmSJKmoqEgtLS164YUXJEk///yz+vv79cQTT0iSVq9erYaGBlVUVIx5EAkJrjsZ+33bXywio9GRjzMyGh35OIvXjO7k/+1Y1t3d3crIyAgvZ2Zmqr29/bbrMzIy1NXVFdEgJk9+IKLtnUyd+uC47i8WkdHoyMcZGY2OfJyR0dg5XgYPhUJyuf73KsAYM2zZaT0AALg7jmWdnZ0tr9cbXvZ6vcrMzLzt+kuXLg1bDwAA7o5jWS9cuFBtbW26cuWKfD6fDh8+rMWLF4fX5+TkKCUlRcePH5ck7du3b9h6AABwd1zGGOO0UXNzs9577z0Fg0GVlZWpurpa1dXVqqmp0ezZs/Xdd99p48aN6uvr08yZM7V161a53e77MX4AAGLemMoaAABED3cwAwDAcpQ1AACWo6wBALAcZQ0AgOViqqydJhyJV319fVq+fLnOnz8vafAWsiUlJSosLFR9fX2URxd977zzjjwejzwej+rq6iSR0c3eeustFRcXy+PxaOfOnZLIZyRvvPGG1q9fL4nJjW61Zs0aeTwelZaWqrS0VCdOnOB4HSkTIy5evGgKCgrML7/8Yq5du2ZKSkrM6dOnoz2sqPv222/N8uXLzcyZM825c+eMz+cz+fn55qeffjLBYNBUVVWZL7/8MtrDjJqvvvrKPPvss8bv95tAIGAqKytNc3MzGQ05cuSIKS8vN8Fg0Ph8PlNQUGBOnTpFPrdobW01Tz/9tHnllVeMMcZ4PB7zzTffGGOMefXVV01jY2M0hxdVoVDILFq0yASDwfBjHK8jFzNn1jdPOJKWlhaecCTe7dmzR5s3bw7fVa69vV3Tp0/XtGnTlJSUpJKSkrjOKSMjQ+vXr5fb7VZycrIeeeQRdXR0kNGQp556Sh9++KGSkpJ0+fJlDQwMqKenh3xu8uuvv6q+vl5r166VNPLkRvGcz9mzZyVJVVVVWrFihT766COO13cgZsp6pAlHIp1QJBZt2bJF8+fPDy+T03AzZswIH1Q7Ojr06aefyuVykdFNkpOT1dDQII/Ho7y8PH6HbrFp0ybV1tZq4sSJksZncqNY0tPTo7y8PL377rv64IMPtHv3bl24cIHfoQjFTFkzocjYkNPITp8+raqqKr388suaNm0aGd2ipqZGbW1t6uzsVEdHB/kM+fjjj/Xwww8rLy8v/Bh/Y8PNmzdPdXV1Sk9P15QpU1RWVqaGhgYyipDjFJn/L7Kzs3Xs2LHw8q0TjmCQ08Qs8ej48eOqqanRa6+9Jo/Ho6NHj5LRkO+//16BQECPPfaYJkyYoMLCQrW0tCgxMTG8TTznc/DgQXm9XpWWlurq1au6fv26XC4Xkxvd5NixYwoGg+EXNMYY5eTk8DcWoZg5s3aacASD5s6dqx9++EE//vijBgYGdODAgbjOqbOzU+vWrdP27dvl8XgkkdHNzp8/r40bNyoQCCgQCOjzzz9XeXk5+QzZuXOnDhw4oH379qmmpkZLlizR1q1bmdzoJr29vaqrq5Pf71dfX58++eQTvfnmmxyvIxQzZ9ZZWVmqra1VZWVleMKROXPmRHtY1klJSdG2bdv04osvyu/3Kz8/X88880y0hxU177//vvx+v7Zt2xZ+rLy8nIyG5Ofnq729XStXrlRiYqIKCwvl8Xg0ZcoU8hnF9u3bh01uVFlZGe0hRU1BQYFOnDihlStXKhQKqaKiQk8++STH6wgxkQcAAJaLmcvgAADEKsoaAADLUdYAAFiOsgYAwHKUNQAAlqOsAQCwHGUNAIDlKGsAACz3H2rNaxHKORC8AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 576x360 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Learning rate: 0.03\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAesAAAE3CAYAAACKD2n1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzs3Xd8VOed7/HPOWe6ehkVeu/dDdNsXMDGxj3FzprcOCHV67X3Jru5cUmyWcdJNrtONptkF28Sr2OTNXHiggtgG3fAVAOmiyZR1FGfdsr9Y0YDQgK10Wg083u/XrykaWeeR4C+83TFsiwLIYQQQiQstb8LIIQQQoiLk7AWQgghEpyEtRBCCJHgJKyFEEKIBCdhLYQQQiQ4CWshhBAiwUlYCyGEEAlOwloIIYRIcF0O66amJm6++WZOnDjR7rF9+/Zxxx13sHjxYh5++GF0XY9pIYUQQohU1qWw3rlzJ3fffTfHjh3r8PHvfOc7PPbYY6xduxbLsli1alUsyyiEEEKktC6F9apVq/j+979PQUFBu8dOnjyJ3+9nxowZANxxxx2sWbMmtqUUQgghUpitK096/PHHL/hYZWUlXq83etvr9VJRUdH7kgkhhBACiMEEM9M0URQletuyrDa3hRBCCNE7XWpZX0xRURFVVVXR29XV1R12l1/MmTPNmGZsDv/Ky0unpqYJALP6OL43f4Xr6q+gZHjxrX4C5+zPYxt5aY+vHyrZRHDLC7hvexTVnRWTMsfSufVPNalcd0jt+qdy3SG16z8Q666qCjk5ad16Ta/DevDgwTidTrZt28Yll1zCyy+/zIIFC7p1DdO0YhbWrdcDMAwDvb4Kw+9Ddevo9VXYAn7UXryXEfCh11dhhnRwJubporH8WQ40qVx3SO36p3LdIbXrnwp173E3+PLly9m9ezcAP//5z3niiSe44YYbaGlpYdmyZTErYK8o4epZlgGWGbmvl1300sUvhBAizrrVsl6/fn30+6eeeir6/YQJE3jhhRdiV6pYUbXwV9OEmH/wSv5PckIIIRJDr7vBE5oa6TgwDaLhqsimbUII0R2GoXPmTBW6HuzvorRTWalimmZ/F6NDNpuDnBwvmtb7qE3qsFaUSMvaMrGs2LSEFSLd4DG6nhBCJLozZ6pwuTykpRUl3Gofm01F1xMvrC3Lorm5gTNnqsjPL+719ZK7mdlhy1rGrIUQojt0PUhaWmbCBXUiUxSFtLTMmPVGJHdYR1rWlnXOmHXM/rFJy1oIkTokqLsvlj+z5A7rNi3r1m6SGP3wJKuFEELESZKHdQezwaUbXAghBrTt27dy//1f7e9ixFVSh7XSOvP73HXWsWpZS9NaCCFEnCT1bPA2LetWvc5qaVkLIUQiKC09zr/8y49paKjH5XLz4IPfZuLEyaxbt4aVK59BVVUGDRrEo4/+iPr6Ov7pnx7F5/Ohqgp/93ffYcqUqf1dhS5L7rA+dwezSEtYiVVngizdEkKkoI92n+bDXaf75NrzphUzd2rXlzn96EePsmzZl5g/fyGffrqbRx75R/70p7/y1FO/ZcWKP5CTk8uvf/1LSkuP8cEH7zFnzjzuuWcZmzZtYNeuTySsE0abMevWpVu9vKaMWQshRL/z+XycOnWShQuvRddNpkyZSmZmJqWlx5k7dz7f+MaXWbDgaq666hrGjh2Pz+fj4Yf/gYMHDzBnzjzuvPOz/V2FbknusFbOmQ0ebQnLmLUQQvTU3Knda/32FctqvxGKZYUPcHrwwW9TUnIrGzd+yI9+9Cj33fdVFi9ewrPPrmLDhg95++11vP76an7xi9/0Q8l7JqnDWlGUcGCf+5caq5axZLUQQvQbjyeNQYMG8847b0e7wWtraxg1ajSf//zt/Md/rODee7+EruscPHiAw4cPkZ9fwGc/ezczZ17Kffd9ob+r0C1JHdZAeK21GcPZ4NGwl7QWQoj+9NhjP+LnP3+Cp576T+x2B48//jPsdjtf/vLXePDBb+F0OsnJyeHhh39AMBjkhz98hNdfX42qqjzyyA/7u/jdkvxhrWhtu0tkNrgQQgxos2ZdyqxZlwLw298+1W5v8Ouvv4Hrr7+h3et+85v/jkv5+kJSr7MGoi3r6EEeMTp1y5KWtRBCiDhJ+rBWFC2yzjrG4SpZLYQQIk6SPqxR1cgOZjFqWcvSLSGEEHGWAmGttV1nHTPStBZCCBEfyR/WitpmB7Pet4wjr5esFkIIESfJH9atLeuo3i7dav1G0loIIUR8JH1YK4radgezWLWshRBCiDhJ+rBG1cIbosiYtRBCiAEqBcI60rImtuusJauFECKxPf74D3j99dUXfc68eZfGqTS9kzo7mFmtR2T29nrSDS6ESF2hgx8ROvB+n1zbPn4B9nFz++TaA13yh3W7lnWsxqylaS2EEPH2ve99h0WLbuDqq68F4ItfvIf773+IFSt+QyDgp7GxiQceeIj586/u1nX9fj8//ek/U1JyEFVV+fzn/4Ybb7yZkpJD/Oxnj2MYBg6Hg+997/sUFw/iiSd+yJEjhwG4/fbPcMstt8e6qm0kfVgrynlj1jE7dUvCWgiReuzj5vZr63fx4iW8+eYbXH31tZSVlRIMBvnLX57nu999lOHDR7Bt2xZ++cufdzusf//7/yIrK4s//nEVdXV1LF/+RcaOHc+qVSv5/Of/hmuuuY433niVPXt2U11dRUNDA3/4w0qqq6v47W9/JWHda+e3rGO2dEsIIUS8zZkzjyef/BktLc289dZabrhhCXfddTcbNnzAO++8xZ49u/H5fN2+7rZtW/nudx8FIDs7m/nzF7BjxzauvHIu//ZvP+Pjjzcwd+4C5s6dT1NTI6Wlx/n7v7+f2bPn8q1v/V2sq9lOCkwwa93BLHJblm4JIcSAZbfbmTt3Ph9++D7r17/JokU38K1vLWffvj2MHz+BZcvuO3twUze0OZ2RcOepYegsXHgdv//9s0ycOJlVq1by858/QVZWNn/84yruvPNzlJYe5777/obGxsZYVbFDyR/WrTuYWTFqWUdJN7gQQvSHxYuX8L//+yxZWdl4PGmUlR3ny1/+OrNnz+WDD97DNM3OL3KeWbMu47XXXgagrq6ODz54l5kzL+Wxx/4f+/bt5bbb7uQrX/k6Bw7s58MP3+NHP3qMOXPm8eCD38btdlNZWRHraraRAt3g4ZZ19EjLXveCy3ajQgjRn6ZNm0FTUxO33XYXWVlZ3Hzzrdx772ex2WzMmnUZfr+/213hX/rSV/jXf/0py5Z9DtM0WbbsPsaPn8C9936Jn/70n3n66aew2ex8+9vfZdy4Cbz77nruvfezOBwOFi9ewujRY/qotmGK1ZP+ghirqWnCNGNTDK83g6qqs90RvrW/xGyqxnHJbfjX/QrPHT9Eyx/e4+vrx3fgW/tLPLd/H807MhZFjqnz659KUrnukNr1T+W6Q9/Xv7z8OEVFPf+92ZdsNhVd735LOl46+tmpqkJeXnq3rpMyLWvZblQIIVJPIODna1+7r8PHvvKVrzFv3lVxLlHPJH9Yn783eKzCtv87JIQQIm4sy0IZgJtCOZ0unn56Zb+8dyw7rpN/gpka2cEsVpuiDLx/q0II0Ss2m4Pm5oaYhk+ysyyL5uYGbDZHTK6X/C3r1nXW0X9j0g0uhBDdkZPj5cyZKpqa6vq7KO2oqtqj2d/xYLM5yMnxxuZaMblKAovuYBaz7UZbySdMIURq0DQb+fnF/V2MDqXK5MIU6AY//zzr3l6wdemWhLUQQoj4SP6wViKzwWk9dUvGrIUQQgwsyR/WqtrmiExZuiWEEGKgSf6wVtS2p27J0i0hhBADTNKHtaJqkVO3Wu/obTd4+PUS1UIIIeIl6cM6vCnKObPBpRtbCCHEAJP8Ya1qYPXBbHBpWwshhIiTFAjrcBWtaFe4jFkLIYQYWJI/rBUt/NXUI7djM2YthBBCxEvSh7USaVkT65a1EEIIESddCuvVq1ezZMkSFi1axHPPPdfu8T179nDnnXdyyy238LWvfY2GhoaYF7THoi3rSFjHqmUs3eBCCCHipNOwrqio4Mknn2TlypW89NJLPP/885SUlLR5zuOPP84DDzzAK6+8wsiRI/nd737XZwXuttYxayPW3eAS1kIIIeKj07DesGEDs2fPJjs7G4/Hw+LFi1mzZk2b55imSXNzMwA+nw+Xy9U3pe0J9byWtRBCCDHAdHrqVmVlJV7v2SO+CgoK2LVrV5vnfPe73+W+++7jxz/+MW63m1WrVnWrEHl56d16fme83ozo9w2ZHgKAx6URBPLzM9HSMi742s74WtLwAdlZbtzenl+nL3kTtFzxkMp1h9SufyrXHVK7/qlQ907D2jRNlHO6ji3LanPb7/fz8MMP8/TTTzNt2jT+8Ic/8I//+I+sWLGiy4WoqWnCNGPTrXz+cWmh5hAAzU0tAFTXNKG29HxenV4Xvk5dXQtNCXgsW6ocF9eRVK47pHb9U7nukNr1H4h1V1Wl243UTlOrqKiIqqqq6O2qqioKCgqitw8ePIjT6WTatGkAfO5zn2Pz5s3dKkSfUs6bDS5Lt4QQQgwwnYb1nDlz2LhxI7W1tfh8PtatW8eCBQuijw8fPpzy8nKOHDkCwNtvv83UqVP7rsTddd6Yda+PyBRCCCHirNNu8MLCQh566CGWLVtGKBTirrvuYtq0aSxfvpwHHniAqVOn8sQTT/Dggw9iWRZ5eXn8+Mc/jkfZuya6zjrGs8Fl6ZYQQog46TSsAZYuXcrSpUvb3PfUU09Fv7/qqqu46qqrYluyWImss7ZiPhtcwloIIUR8pM4OZkbrmHVvqyzd6EIIIeIr6cO63d7gvb1cTK4ihBBCdF3yh/X5e4P3umUdIWPWQggh4iQFwvq8MeveNo1lu1EhhBBxlvxhff46a+nIFkIIMcAkfVgrah+dZy0NayGEEHGS9GHd2rK2Yt6ylrQWQggRH8kf1uefutXrrJYxayGEEPGVAmHdus66delW8ldZCCFEckn+5FJi3LKWMWshhBBxlvRh3bqDmYxZCyGEGKiSPqzP38FM6fURl9KyFkIIEV/JH9ZtdjCLQatalmkLIYSIsxQI63PGrHvdqj7Lkqa1EEKIOEn+sG5dZ23oxKZZLEu3hBBCxFfSh7XSpmXdv2URQggheiLpw/rs3uB6bLrBZemWEEKIOEv+sG5tWVsmsW1aS1oLIYSIjxQI63OrGMMxaznPWgghRJwkf1gr51QxJt3gvb+EEEII0R1JH9aKop4N6Rgu3RJCCCHiJenDGji7i5ks3RJCCDEApUZYt45bx6RhLbPBhRBCxFdqhHUsW9bSky6EECLOUiOsoy1rWbolhBBi4EmJsG7dxUyRpVtCCCEGoJQI6+jyLVm6JYQQYgBKjbBu3cUsBmEdm9a5EEII0XWpEdZKX1RTusGFEELER2qEdXTLURmzFkIIMfAkVVjrhsmrHx7BMM029ytK7LrBpRdcCCFEvCVVWJdVNvFfL+5mz9EzbR/ok6VbQgghRHwkVVjnZjgBqKrztX2gL7YblW5wIYQQcZJUYZ2Z5sDp0Kg8c15Yx7JlLa1zIYQQcZZUYa0oCkW5nvYt69alW0IIIcQAlFRhDVCUl9YurJXopiixrK50gwshhIiPpA1r69wx5Zi2rGXMWgghRHwlYVh7COom9c3Bs3fKdqNCCCEGsCQM6zTgvBnhMdxuVNJaCCFEvCVhWHsA2s4IV2K5g1mEdIMLIYSIk6QL68JcDwptW9Znj8iMgUjr3JIJZkIIIeIk6cLabtPIyXSe1w3eF7PBhRBCiPhIyvQqyHZTVec/e0d0b/D+KY8QQgjRG0kZ1vnZbio7alnHpLqydEsIIUR8JWVYF2S7aWgOEgga4Tti2bKW7UaFEELEWZfCevXq1SxZsoRFixbx3HPPtXv8yJEj3Hvvvdxyyy18+ctfpr6+PuYF7Q5vths4O8lMiel51kIIIUR8dRrWFRUVPPnkk6xcuZKXXnqJ559/npKSkujjlmXxjW98g+XLl/PKK68wceJEVqxY0aeF7kxBTtuwju066wjpBhdCCBEnnYb1hg0bmD17NtnZ2Xg8HhYvXsyaNWuij+/ZswePx8OCBQsA+PrXv84XvvCFvitxF7S2rKPj1rFcZx0NfAlrIYQQ8WHr7AmVlZV4vd7o7YKCAnbt2hW9XVpaSn5+Pt/73vfYt28fo0aN4tFHH+1WIfLy0rv1/M6MGJpDmstGU8DA682gJs1FPWCza3i9Gb26tt6k0wxkpLvI7OW1+kpv6ziQpXLdIbXrn8p1h9SufyrUvdOwNk0T5ZzuY8uy2tzWdZ3Nmzfz7LPPMnXqVH7xi1/wk5/8hJ/85CddLkRNTROmGZuWqtebQXV1E/lZbo6frqeqqhG/PzzRTDcsqqoae3V9s6UZgMYmP4FeXqsveL0Zva7jQJXKdYfUrn8q1x1Su/4Dse6qqnS7kdppN3hRURFVVVXR21VVVRQUFERve71ehg8fztSpUwG4+eab27S8+4s32xVda630xXnWMmYthBAiTjoN6zlz5rBx40Zqa2vx+XysW7cuOj4NMHPmTGpra9m/fz8A69evZ/LkyX1X4i7y5riprvOFW+xqLE/dkhnlQggh4qvTbvDCwkIeeughli1bRigU4q677mLatGksX76cBx54gKlTp/LrX/+aRx55BJ/PR1FRET/72c/iUfaL8ma7MUyLM40B0lvXWcd06Za0rIUQQsRHp2ENsHTpUpYuXdrmvqeeeir6/fTp03nhhRdiW7JeKjhnRnh6LFvWrSSrhRBCxElS7mAG522MorSeuiVLt4QQQgw8SRvWuZlONFWhqs53dgezGLSsYxL4QgghRDckbVhrqkpepivcsu6L2eBCCCFEnCRtWEN4RnjlGd/ZHcxku1EhhBADUHKHdbb7vJa1jFkLIYQYeJI6rAuy3TT7dYJ6JFhljbQQQogBKKnD2pvtAqDB33qutSzdEkIIMfAkeViHl281+iJhLd3gQgghBqCUCOu65lD4jpi0rKUrXQghRHwldVi7nTYyPHbqW4zOn9xt0rIWQggRH0kd1hBuXde16OEbMmYthBBiAEr6sC7IdlPXFOkGlzFrIYQQA1DSh3V+zFvWMmYthBAivpI+rAuy3RhWDAM2cinZwEwIIUS8JH1Ye7NdmJFqKkosqytpLYQQIj6SPqwLcjwYMe26lm5wIYQQ8ZX0YZ2V7kBp3Rs8ptuNSstaCCFEfCR9WKuKQla6K3JLlm4JIYQYeJI+rAGyM8I7mcWkZS1Lt4QQQsRZaoR1ZiSsY9KyljFrIYQQ8ZUSYZ2TFQ7rkBGD1nC0YS0tayGEEPGRGmGd6QEgEOqLPcKFEEKIvpUSYZ2XHQ5rf8iMwdVkzFoIIUR8pURY52b2RVgLIYQQ8ZESYW132AGobfBjmr1sEUvDWgghRJylRFi3borS5DfYtLc8RleVtBZCCBEfKRHWqOFqprnsvPj+EUJ6byaaSTe4EEKI+EqNsFbCLevhxZnUNAR4Z/vJnl9KkWO3hBBCxFdqhHWkZZ2d7mLyyFxWbzhGi1/v50IJIYQQXZMaYR1pWSsK3HXVaJr9Om98fLyXF5WWtRBCiPhIkbBWol+HF2Uwe1Ihb24p40xjoKcXjFnRhBBCiM6kRFgrihJpXYdD9rYFozBMi5c/PNrDCyJj1kIIIeImJcIaiIxbh8O6INvNwpmD+WDXKU7XNPdvuYQQQohOpFBYa22OyLx57gicdo2/vHekBxeTbnAhhBDxkzphrahtwjrT4+DGK4ax/WAVJSfru3sx6QYXQggRNykT1mp2MWpmQZv7Fl02jMw0By+8e7h7F5OGtRBCiDhKmbBOu+1RHNNuaHOf06GxZPZwDpbVcby8sZtXlJa1EEKI+EiZsL6QuVOLcNhU3v2kO7uaSdNaCCFE/KR8WKe57Fw+sZBNeyrwBbq4q5ks3RJCCBFHKR/WAAtnDSYQMti4J1YncgkhhBCxI2ENjCjKYHhhBu/uOInVpRaz0sXnCSGEEL0nYU14h7OrZw7iRFVzF5dxyZi1EEKI+JGwjrhiUiFup8a7O7ow0UyBWM0GP1ndTH1TT/coF0IIkQokrCNcDhtXTi5iy/4qGluCcXlPX0Dnx3/cxorVe+PyfkIIIQYmCetzXD1zMLph8tHuC0808wV0QrpFWUV312W398HOU/gCOvuOn6E0BtcTHdNP7MHS4/MBTAgh+kKXwnr16tUsWbKERYsW8dxzz13wee+++y7XXHNNzAoXb0O86YwdksW7n5zE7GACWSBk8MsXdmFYFgfL6mjyhXr8Xrph8ubWMkYWZ+C0a6zbUtaboosLMOpO4Xv9Xwjtfbu/iyKEED3WaVhXVFTw5JNPsnLlSl566SWef/55SkpK2j2vurqan/70p31SyHhaOHMwlWd87Dt2ps39Id3k1y/u5lBZHTZNxTRN1m4u7dI1LT1I4JPXsYK+6H1bD1RS0xBg6ZyRzJtWzMd7K3pxvra4EOPUfgD0Uwf6uSRCCNFznYb1hg0bmD17NtnZ2Xg8HhYvXsyaNWvaPe+RRx7h/vvv75NCxtMl4wtId9vbTDQzTJMVr+zh0yO1fPHGCdg0lcJcD29tPUFDF8a3Qwc/Irh5FcE9bwFgWRZrPy6jKNfDtDF5XH/pEEzTYv32E31Wr1TVGtZGxSEsy+zn0gghRM90GtaVlZV4vd7o7YKCAioqKto855lnnmHSpElMnz499iWMM7tNZf60YnYcquZMYwDTsvj9a/vZdrCKu68dy4LpgwCF0YMyCYYM1nx88da1ZVmE9q0HILTvXSzT5EBpHccrGll8+VBURaEgx8PMcV7e3XGSQNCIQy1Tg2VZGKf3g90NgWbMM93ZUlaI1GKZJsF978r8jgRl6+wJpmminHO0pGVZbW4fPHiQdevW8fTTT1Ne3rMdwPLy0nv0ugvxejN69frbrxnHGx+Xsi0S2Bv3lPM3N0zgc9ePB6BZVcjP8XDVrCG8s+Mk99w4kZwMV4fX8p88SFNNGe5R0/Ed2UlawyHWf6KTle7glqvH4rBrAHz2+vF899cfsuv4GZbMGdmr8ve2/gPZuXUPVp+gyddA9tw7qfvoL7gbS8kaP6kfS9f35O8+dZ1bf8syKf/fH+MeMYXsK2/r0utbju6k/IOnycxKI2P6wJp7lAp/952GdVFREVu3bo3erqqqoqDg7FGTa9asoaqqijvvvJNQKERlZSX33HMPK1eu7HIhamqaMM3YrFv2ejOoqurdzGobMHlkLs+/dRDDtLjximEsnF4cva5pWfh8QRZdOoT3dpzg2df28vlrx3Z4Ld+G18DuQpu3HOX0w5x87xW27ruE2+aNpL6u5Wy50+2MLM7gr+sPccmYPFSlZxuvxKL+A9X5dQ/u3QZAaMjlKGnvUH9oJ8Hhc/ureH1O/u5Ts+7Qvv6hI1vwH9lBoL6W0Jhru3SNwMFPAag7vBf/oMv6pJx9YSD+3auq0u1Gaqfd4HPmzGHjxo3U1tbi8/lYt24dCxYsiD7+wAMPsHbtWl5++WVWrFhBQUFBt4I6UV07awiGabFw1mDuunp0m94EUMCyKMr1cOXkIt7ZcZK6DjY2sQLN6Ic/xj5mNoorHfvEq9Aq9lBob2bhrMFtnqsoCosuG0bFGR+7Smr6uHbx9drGY/0yHm+c2o+SloOSWYBWNA6j/KBsEyuSnmWZBLe9DIBZW4YV6trEVaPycPhr1dE+K5vouU7DurCwkIceeohly5Zx2223cfPNNzNt2jSWL1/O7t2741HGfjFjbD6PL7+CL1w/7rygBs65vXTuCAzD4vVNx9tdI3RoAxgh7BMXAuAfNgfLUrhz8CkyPI52z79kvJfcTCfrtnRtlvlAcPR0A3957wjPrjvIJ4eq4/a+rePVWvEEFEVBKx6H1VKH1VgVtzJ0xvQ1yKQ3EXP60a2YZ05gG3MlWCZG1ZFOX2NZFmZFOKzNmjIZt05AXVpnvXTpUl599VXWrl3L8uXLAXjqqaeYOnVqm+cNGTKE9evXx76U/aQ4L+0i3dHhFlphjoc5U4t4d8epNkuvLMsitPcd1IJRaPnDAVi/v4U9oSGMC+zGMtqv0bZpKtddMpT9pXUcLx9Y3TodsSyL59eXkOmxM6wwnf9+dS+Vdb7OXxgDZv1pLF8D2qAJAGhF4fkGxunEWMJl+hpo/tO3Ce2R9d8idlpb1Wr2IJxz7gHAiITwRV/XUIkVaEIbOg0sA7NW9n1INLKDWU+d05u6dM4ILMvi9Y1nW9dG+UHMulM4Iq3qQMhg/fYTnM67DCXQhH50W4eXXTC9GKdDS4rW9SeHqjlYVset80byrdunoijwmxd3E9L7fsZ765ItW3E4rNWcYnCmYZQf7PP37gqjdCfowQv+OxCiJ/QjWzDPnMRxya2orgzUrCKMivb7YpyvtQvcMeW6yO3OW+MiviSse0A579Qtb7abuVOLeW/nSWob/ACE9r4DDje20ZcD8NHu0zT7dabOnYeSWUBob8c9EB6XnfnTitm8r3JAb5KiGyZ/fvcwxXke5k8fhDfbzVdunkRpRRPPvdn3gXnueDWAoqjYisahn06MsNaPbQfAKD+EFWzp5NlCdM4yI63qnMHYRoUniKmFYzArD3c6V8OoKAlPhB08BcWTLePWCUjCuicUhfNP3bp5znAsC1a+dYjNO0oIHtlCbf4stpbUs3V/Jes2lzFqUCZjh+bgmHg1RvlBjNqOJ11dd+lQTMvi7W0Dd5OUD3aeory2hbuuHo2mhj/cTB+Tz01XDuf9naf5cNfpPnvv88erW2nF47AaKjBb6vrsvbvC0oPoJ/ag5g0Hy0A/sadfyyOSg35kc7g375JbUZTwr3atcAyWvxGrofKirzUqD6MVjEJRVTTvSExpWSccCesYyc9yc/WMwWw/WMXB915HtQz+c28ev33pU37z0qdU1vm48YphKIqCbfx80Gzh1ncHCrLdzBrn5Y2Pj/PPz2zlxfePcLCsDt0YGJORfAGdlz48yrih2UwfnYtvzZP43n0Ky7K4ff4oJg7P4Y/rDvTZ4SVWfXl4vLp4fJv7z45b92/r2ji5F4wgzsvuAIcHo2xXv5ZHDHyWaRDc/jIxmCBkAAAgAElEQVRq7hBsIy+N3q8Vjga4aFe4pQcwa8rQCsLPVb0jMevLpccnwXS6zlpcQAfdSndfN5brLxuM8/U3MNxjuP+OxViWhWWBTVMozksDQHVlYBt5GaFDH+G84jMo9vYbqnzxhgkMzk9jz7FaXt14jNUbjuF2akwYlsPkkbkU5LjJTnOSneEkzWVrP2O9H73x8XEaW0I8+JkxGIc/joaRXjwB+/j5fO2WyfzgD5v5zUuf8tgXL8Pjiu0/Q711vDoyuayVmj8MbE6M8gPYI8MT/UE/viPS5TgJ25Ap6KW72m02JER3NO39CLPuNK7rvhVtVQOo2YPB7sKoKME+ruM9BoyqY2CZ0bDWCkZF77cNTu5NhAYSCeueuMAvVVVVyG0+iq+5CtcVd5Kdn3bBSzgmXYNespFQySYcE69u93i6285t80dx2/xRNPtD7Dt2hk+P1rLnaA07zlsCZdMUstKcZKc7KMhLw2VTyUpzkJnmCH9Nd+Bx2tANi5BuEtINQoZJSDcxTYu8LBfFuWk4HVqvfiwAtQ1+1m4u44pJhYzId9K8/s+o+SNQ7E78G55DGzSRzIx8vnHbFH62cgf/8dddzBzrjZY1Kz1cbo+z5x9AjNP7UTzZKJmFbe5XVBta4Zh+nWRmWSb68U+wDZ2GotmxDZse7r6sKY2uGhDdY9aX42spBc+w/i5Kv7BMg7oP/oyaOxTbyEvaPKaoKlrBaIzKC7eszcjkMjUS0lr+CACMqiMS1glEwrrHOp6wEdr7Dooro91/mvOphWNQc4cS2rse+4SrLhpMaS47l04o4NIJBViWRU29n9rGAHVNAeqagtQ3nf3+RGUTZxr8PTq+My/TRXG+h0F5aQzKT6Mo10NxnqfDNeEX8uIHR7AsizsXjCK483Ws5jO4rv0GaloOzS88iv/d/8Z98z8wdkg291w/jj+9dYj9pe3HkG2aQobHQbrbTobHHvnqIMNtZ9zQbCYMz+nw/S3Lwji1H23wxA5/plrxOIJbX8IKNKM4L/xhqq+YVUexfPXYhs8Il2doePmjXrpTwrqH/O8/TUvVEdL+5hcoDk9/Fyfu9JJNhGpP4br+b9u0qltphaMJ7liNFfShONztHjcqDqNkFqC6MwFQXOkomYWYlTLJLJFIWPdUB1ltNp9BP74D+9TFKJr9oi9XFAX7pIUEPnwGs/IwWuGYLr2toijkZ7vJz27/nw7Obr2nGyYNzUHqm4M0V51CqTuBv2AKdrsNu6Zit2nYbAoKClV1Pk7XNHO6poVT1c0cKK0jpJ8dH09z2SjOC4d3UZ6H4lwPg7xpeLPcqOrZQCytaGTD7nIWXz6MXFsLzTvfwDbqcmxF4wBwXXkP/vd/T+jTN3FMXczCmYO5asYgWvw69U2BaHnrm4M0NAdpbAnR5AvR2BKkut5PU0uIloAOwJRRuXzm6jEMLWi7ZV94vLoerbhtF3grrWgcYGFUHMI2bEaXfuaxpB/bAYqKbeg0AFR3Jqp3JHrZLpyzbol7eQY6s6k2snbeCvdSTRpYe1r3lmUaBLa/gqNwJLYRszp8jlYwBiwLo+pou5ayZVnhyWWDJp73mpH9PrdDtCVh3SPh2eBmSx1mTSlGzQnM2rLwWkXL7LBbuyP2MVcS+HgVLa//HK14PLbBk9EGT0bNGdTr8UubppLjNPDseY3QnrfANNDqx+Na8CXUrKI2zx1e1HYTfNOyqK73U17TQnlNM+W1LZyuaWH3kRo+3H26zXsU53kYlB9uie8+UoPHZeOmOcMJbPgdYOK84rNnnz9+Ptqx7QQ2/xltyBS0nMGoikK6O9xyHuylU8GQwTs7TvLqhmP84PebmTO1iNvnjyI3Mzzur0c2PTl/vLqVVjAaVA3j9MH+Cevjn6AVjUNxnf2QYRs6jeD2VzD9jaiu5D+QIJb0w5sACy0zn9D+91IurPWSTVgNFeQs+kdaLvA749xJZu3CurkWq6UuOl4dfY13JHrJJsyWOlRPdt8UXnSLhHVPKAqhA+8T2v/e2bvSclHzhmKfuRQ1q/AiLz7nMg43npv+gdDBD9FP7CFQujN8vzsrOvlIGzo12j3VVZZpENr/HsGtL2L5m7CPn4+aP5zAlhdofuFRnJfeHm79qx2PUauKQkG2m4JsN9NG57V5rMWvc7q2mVPVzZyubuFUTTMlJ+r5eG/42NS7rx2Lq/44LSWbcMxcipqRf86PTcG14Eu0vPAI/neewnPbIyhq9/4JOuwaiy8fxrxpxby24ThvbStj875KFl02lHtvmhxeX93BeHW0DDZHuCVbfhBnt96598yGSswzJ3BeeXeb+23DphPc/jLGiU9Rx1wZ51INbKGSjajeUWTPXEjNut9hVB+Ljrkmu9ZWtZo3DM+4y2ipburweYozDTW7OLrxybla72sN9FaaNzx+bVYeRR0xM8YlFz0hYd0D9imLsOrLUfOGouYORcsd0qal1B1awajo7EuzsQr95F6Mk3sxTnyKXrIRUFALRmEbNh3bsOmoecMu2ur2Hd1Fy5rfYdaeQCsej/PKe6JjobYRswh8+AyBj1cROrIF14L70PKGdqu8HpeN0YOyGD0oq839/qBOTUOA4lwXvpcfR/Fk45hxU7vXq54snPOW4X/r1wS3r8Z56e3dev9WaS47n71mDNfMGsxfPzjCaxuPs25LKd/P2MUp22A+fvFTstJbJ605yckI/8nNcKIVjSO0ey2WHkCxxS+y9eOfAGAb3vaXn+odgeLKQC/dhV3CusuM2pOYNWU453yB9CkLqHn7GUL730ebN6K/ixYXra1q56K/7bQnTiscg35sR7tVB0bFYdDsqLltfw+o+cNAUcOTzBI4rK1AM3qTAfR+cmyik7DuAeeMJX1yXTXDi2PCVTDhKizLxKw+jl66E710J8GtfyW49a8onmy0IZNRNEd4s309cPZroIXG2jKUjHxc130L28hL2/zHVNNycC16AP3oFgIfPUvLX3+AY+ZNOKYs6vGHjVYuh43B+TZCBz/CrDqC6+rlHS5JA7CPugx97ByCO1ZjGzY9+mGlM5YeRD+2HcWThS0yxpaf7earSyez+LJhlBw4RMaBFsptQzld28L+0jM0+/V215nuDnKf2+BP/7uWluwx4UCPzJrPTnPGZEZ6R/TjO1BzBqNmFrS5X1FUtKHTMEp3Ypkmitr97Q+sQDOWFdtz4ROdXrIxPP4/6nI0dzq2UZcROrQR5+zPxfVDWH+wTIPAjnCr2ja847Hqc6mFY7AOfIBVX4GSfXYYzKg8jJY/AkVrGwWKzYmaOyShdzKz9CDNL/4TTQ0VKGm5aN6RqN6RaN6RaN4R/TKBtC9JWCcoRVEj/+hG4rzkNsyWeoyyXeilOzFKd4WXj9kcKDYH2JwoNgdKWja5M64mOGJ++P4Or6tgH3U5tkGT8G9cSXD7KwR3rEYrGodt+CxsI2a2C5OuskIBApv/jOodiW3sxVuIrjlfoPnUfnxv/Rr7pGuxDZ+Omt3xWL1ZV05w3zuEDn4IgWYA7JOujfxSDtdzeFEG40IBqg/AklsXcXPkF1JIN6lvCnCmKcCZxgC1DQEa67xYx98iP3iCN0vzqG8OohvtZwxqqkK6x06mx0FG5Gu6x06ay47bacPt1PA4bbidNjwuG9npTrLSHB3WwQo0Y5w+gGN6xx/0bMOmoR8Kf9Dp6mTDVvrpA/he/xdsVyyFKbd267UDlWWZhEo2og2ehOoJ9/LYJ1yFfmgD+pEt2MfN6+cS9i29ZBNWfQXO6ztvVUNkkhlgVJagRv5vWIaOWX0M++TrOn6NdySho1sTdg+A4I7VWA0VZF15G82V5RhVR9GPnd1rXy0YheeGv+91QyRRSFgPEKonC3X8fOzj51/0edldPIhdcaXjXvhVjKmL0Y9tQz+2ncCmPxHY9KfwLkjDZ0bXRyt2F9jd53zvBEUDRWnznzi48zWsljrc523M0OH7O9NwX/dN/B8+Q3DzKoKbV6FkeMPd/cNnhLvtynYT2rse49Q+UDRsIy/BPvFq9NKdhHavxTi9H9e1X0eLdOH5S/eGx6vPmTNgt6kdzp5v/stQ5rmaWHTTXCzLoiWgU9/UOhs9QENTkEZfKDorvbElyOG6ehpbQviDFz6IxOO0UZzvoTgvLbIEzkNhjgf36e0olok2vONJbbYhU0BR0Mt2dSuszfoK/Ot+BaZB3aZX8Ay5DC17UJdfP1AZFYexmmqwX3pH9D6taBxKVhGhfe8ldViHW9WrUfOGdrmLWs0pBoc7sjlK+Gdj1pSCobebXBZ9TcEo2P8eVkNlm/9TicCsO01w5xvYxlxJ3jX3YkZ+51mBZoyqYxiVhwlufxn/+7+PLGlLvA8b3SVhneK0/OFo+cNxXnoHZkMl+rEd6Me3E/zk1Q53aWtH1cLBraqgB7CNvgKtaGzX3rtwDGl3/hNmU22ku/8TQvvfD89ej1DS83Bcdmd4klxkVqotMvnO/+5TtLz4Q5xXfA775GvxHd/Tbj/wC7530XhCB97HMnUU1UaaK9xiHnSRjWxaGaaJL2DgC+jRPy1+ndrGQHjiXU0zuw7XtNn//Itp7zPG7uL//k8pTscp3E4bLoeG3aZi08J/blOLse3cxBsnxuFyaGR6wt3xmR575KuDjDQHHqeG3aZhBZrxrXkSCwvP0u/hX/skgQ0rcd/4f2P6y8myLKrq/RwoPYNlhc96z+zG2vu+oJdsBM3RZrmSoig4Jiwg8PEqjDOn0HKS80OLfvhjrPrySKu6a0MmihLZHOWcbUejk8suENaad2T4eVVHujxpNh4sy8L/0R/BZsc5+/NtHlOcadiGTMY2ZDKKzUlg058I7XkLx5Tr+6m0sSNhLaLUzAIc0xbjmLYYy9+E2VSDpQcg6McK+bFCPggFwveZJlgGmCaWaYBlomh27FMXdf9903NxTFqIY9JCLD2IcWofRvlBtMKxaEOndTiGaxs6Fc9d/4z/vd8R2PAc+pEtGE21OGeM7+Ad2tOKxxHa8xZmdWmXx8yjr1VV0t0q6e6Lr6Vv8oU4XdNMdW0jEzY/T1XWZBZPHB4JeAN/UCdkmBiGhW6YHGYYc41N1FVWUBNy0dAcxDA7/sDk1Ey+lv42I9RKXrDdSu36Ri6xX8HlJ95h9fMvcco9FpumoqkKTocWHpNvDf/IGH2Gx46mqe3ObLcsi6o6H/tL6zhQeoYDZXXUNpw9AU5dozBxeDaXTSxk1jhvpz+HWLNMHf3wZmwjZrbb5MM2bh6BzX8htP89tPNm3ScDyzTDM8Bzu96qbqUVjCa4/ZXo5ihG5WGUtBzU9NwOn6/mDAbNgVF5NKEmPuqHP8Y4uRfn3HujQyAdsU9dhH5qL4FNz6MVjR3wqwQkrEWHFFc6Wj+M9Sg2R3Tme2dUdybuxQ8S2vs2gU3PA6AN6mJYRzZqMU4f6HZYQzgwrKAPgj6sYAtW0IeantdmvD/dbWfskGxGcgKfGWDU5fMZN7zjVgyAUVNAy1828Z2rHNjHn+2eb4hsEtMQ6Y73+UOMLH2ZoQ2n2Zi1hGZ1BGZAZ5syiVHsZGbDerbU5xMwNXTDxBc0CFyk614hvFWupirRTW5au/ozPHbGD8thyexsxg/NxjAttuyvZPO+Cp5+Yz9/XHuAySNzmTXOS26mk3R3uIci3W3H5dD6pPvRKPsUK9DUYYCo7kxsI2aiH9qAdfldnW5ONNDohzdFWtX3d7lV3So8vGJhVB7BNmQyRsXhC7aqARRVQ8sfjlEV2xO4Qke2hFvAPdjK1Aq2ENj4J1TvSOwTF170uYqi4Lr6K7T85TF8b/2WtDt+0OEObgOFhLUY0BRFwTH5OrRBE0nzl+PLKu7S69TI2HZgywsEd74enqyn2cHmCH+vauEeAyMEhoFlhsDQwQhhBf1gBDsqDbbhM7BPuyE8fhoJKv34DtAcaIMnX7xMuUNQ0nLQS3diHz8fRVGi3fOth8AABD55nWDDJzhmLmXRZXfS2pfh9WZw+pM0fK/9jEcuq8U56+xks0DIOCf0w1+bfCEM08I0rXZfi/M8jB+WQ3Gep13gDivM4I4FozhW3sjmfRVs2V/JrsM17eqjqQppLlt4tzxNCbf0I19tqoLDruFxRYYg3DY8zvDXcJ1tpEWCP81lw2E/uzQnVLIRnOk0ZI2lpbKJloDOkYomTlU00uIP4QyNY5Z/K2teeIUS21iK89IYVpjO0IJ0CnM8bXbdG0gs0yS4/ZXwnJIL7FZ2MdEDOipLUPOGYjVWoXWyiYxaMIrQ3neiw0W9pZftxv/Wb0BRcC1c3u0We2Dri1i+Btw3PNilVROqKwPXNV/H9+pP8H/4P7gWfu2CHyCNMyfDs+DP2RsikUhYi6Sg5QwmwzsBfxcm17VyzV2GXrYL9CCWEQQ9FF4GZ4TA1MPLf5xp4V9SWviPotrB4UJxuFEcnvAndYcbxe7GOL2f0J714SVa3pE4pi7GNurS8MEdQyZfcIZ+K0VRsA2dRujw5gv+cgwd3Upw8ypsoy7H0cEaddvgSdhGXkpwx2vYx81DTQ9vauO0a3iz3XgvsE1tdymKwsjiTEYWZ/KZhWOoqG2hsSVEsy9Ekz9Es0+n2R++HdJNdDPc1W8YFroZ/trs16mq89HsDz/3YlMk7DYVj8uGZgT4f66tbA6M5s//+XGHz1XRGJmdzqC67XyoDmbX4ZrocILDrjLUm87QwgxyIyfWeVx2PC5b9IODw6ZiRD64GIZ59nvTip6id+5XE/D59XO2yY1smdsSDB+Uk+kKT3LMckX+uMnLcuG0d29tsH7kY8z68nYna3WV4kxDzRkUblHnhQ89UQsv3LKGyIxwYy1m7cle711vNtXgX/9fqDmDUVxp+NevAEPvdNJsK6P6GKE9b2GftDA6nn7R97MsVEXBVjwexyW3E9z6V/RBk7BPWNDmeXr5IYI7VodPB7S7cV9/P7YhF/9g3R8krEXKap2IErPrDZ6EY8ZNhA5+RHD3Ovzr/xNlYyaWrwHbrK4tqdKGTSO0/z2M8pLolqlmSx1G+SGMihJCe99BLRiN6+qvXPAXtvPKu9FLdxHY9L+4r/tWh8+x9CDmmVOoeUN63WJSlfDxr8V5nT/3QkzLwh8waPGHouHd7Ndp9oWi37f4Q4zw7cFRbZA5ZT7/J290eOmcy8aQoiwC/iDpLhsup43Q9iZytr3Ijz43GjMtj1PVzZRVNlFa0URZZSOb91ZE95mPJQWiy/0y0xyoCpRVNfNJSU278+jTXDZyMlzkZp7dtCcnPXzsbetmPhluO6qqhFvV215GzRmCbeQl+IPhDwf1TUGafSGGNoVQTYOsdAfaRVqcWsEYQse2oecOAUVDyRuGFfmU1OHBN9HjMo/2KqwtQ8f31m+wTB3P9fejpOfgW/cr/O/9DssIdbpNrGWZ+D94BsWVgfOyOzt8TjBkcKCsjt1Hath9pJbqOh/DizIYMziLMYMuZ1zhHvwfPYtaOBo1exDGid0Ed7yKUX4QxZWBY9at6Me24Xvj33DOXxbe8yKBKJbVlSm/faumpgnzAhNpusvbxaVLySqV659IdbcsE6N0J8FdazFqy0j77BNd2jbWCvpoeub+8Kx2dwZGRQlWY+RIVM2GVjQe18Kvdjix5tz6B7a/THDri7hv+oc2Y4Omv5HQnvWE9ryF5W8MnxA36nLsY69ELRid8EtcWl7/OWbdadLu/pc2H1bO/7s3m2po/tO3ccy4+YK/3EO6QYtfj3wQCH9AaPHrBHUDTQ132Wuq0uZ7VVFAAZVwuLX+uDwuO5keO+kee4dhaVoWDc1Bquv8VNf7qK73c6YxcM4fPw0tIYZrVRRq9TiVEE5Fx6XoZDgM8m0tjLGO8Rfzej5uHkIg1PEcBEWB7PTwTn05mS5UJbxFcEtkxcKE0B5ut39ArZFGk+XiXxva7jKoAHa7isuu4bBrOO0qf2v+gWP2MWzJuiE8TBHZy//cuQkW5/U2WGCaFg0tQeoaAwwve40xTdt4RVvMlpah2DWF/Awbt5jrGBI4zPEhN6KPvZZ0t52QbhDQTYIhg0DIIBgyya34mDEnVnN01GdoLJyFw65FDiRSafDrbNx1igNl4QOI7DaVCZEhnGPljRw93UBIN8lUWvjH7NcwbG6w2ckKVBCwZ1I9eAG+oVfi8nhwWEGydvwBZ9V+mkddS/PEm1EULfphKlZUVSEvr3tzgqRlLUQfUBQV2/CZ7bYW7fR1Djfa4MkYZbvCu9UVjkGbfD1a4WjU/OFdnjDlmHYjoQMfENjwHNqd/4TVWEVw9zpCBz4EI4g2bDr2EZegn/g0vM/93rdRMrzYx8zGNuZK1LSc8Kx/PRjZIS8Y3p5VtaFk5KN4snrUFWuZJuj+8OQ8VUNxZ3X5A4LZUo9xcg+O6Td1+t5qeh7akKmEDnyAbfQVqJkF7YYh7DaNrHSNrPSL/xK2LCt8UE/VUbTsQeHtYbs5cU1VFLLTnWSnOxkzpP0HLaOmFP+mVZgnP21zv4mKrjgIKnYOauMJ5E3nqsjmO5lpDrLTnXhcNlS7jaMnzlDbEOBMQ/gI3bLKJhTCWwR7nDbys1xkqOPh9Afkas205Ezi1mkjo3UEMK1wC7U1KAMhk+ozReTrFZyubQkPc0TmOnTVTMcxrk3fxjZlGlWZk5he7EA3LGob/DzTfDU36TrTT7zBqwfLedMfPjJWwaJQq2e4VsUIWzVDnMc4pBfyH1tdwL5271GU6+HqGYOZOiqXcUOz28xx0A2T0oomSk7Ws+mIxTV1f6U6mMFK35VsDY7CqNBg+4Gzf1dcyp0ek3lH3ubQ/kM82zQPh8vFrx5c0O5940la1kkmleufLHW3IjPMlbTcbrV0z69/6Ng2/Ot+hZo3PLwBhqphHzsH+7TFaDmD27yffmwboUMbMU7t7eL6ehtKei5qRj5qeh6KJxvLCEEovMyPUCCy3M8fvi/oi37fhuZAzcxHyfCiZhagZnhRM7wo6bnhP8706M8g+OmbBDY8h+eux9FyB7e5TEd/93rZLnxvPAlY4Q18MryoWUWo2cXhPxle1Nb3OW97UivYgn5iT3jXwLLdWC3nnLmu2dC8o8LLgYrGohWO7fHWlmZjFYEtf0Uv2QROD84ZN2MbeQnYXeENiDR7l/4NdPXfvmWZNP3P/RBswbXwq9jHzun0NYEtfyH4yWukf+m3KDYnlmURCBk0tQTxnyrBbKgk4J0Y+bs62+OgKgoZeg32N59AzR2K5+bvttvWFMAwdJreXoF6bDNNBdOxGz7s9aWoevjfiuXwoHpHY112N7o7j5BuENRNQpHW95gReWim2e66F2I2n0FxZ2FYhJdRBg18/vB+Cbphhk8/tiwySt8n99BqAplDCM37JoOGdG3yaldIy1qIJBCevNb7iWC24bPQhs3AqDiEY+bN2Cdf2+Fxh4rDjX3cPOzj5mG21KEf2w56MLKdrfOcbW0doIcwm6qxGqsxG6sxm2rQS3dh+RrAZo/scOcK73Znc6E408ItcXvrRDxXdFIeRgizsRqroQqzsZLQ6QMdhLktfKJdWg5mfQVq3tB2QX3B+g+dhuczj2PWlGLWnQ7/qT9N6NS+8CTCcznTUNPCwU3QF948xDLB4cY2ZAq2odNQC0djnjkVnj9QfojgzjXwyWvhn2FmIVrkYB81bwha7rBwvS8QtKa/keD21YT2rgdFwTFjCY7pS/p8P2tFUdEKR2OU7b7osq1zaQWjwDIxqkuxFY3FrD+NcmgjrpJNOBurIhfW0IZMCm9lPGIWijMNKxSg5aWnsDQH7mu/2WFQA2iajczrvk7gIw/pBz5EzRmMNm5O5JCj0ShZhRftSfHmpXXrQ7qalgOEwy/D4yDDc4Enjr6D0MjhKG//F56Nv8K66/F+HSaSsBYiSSmKgnvR3wJWlyeRqZ7sHp0JHYv9oy3Lwgo0hcO7uRar+QxmU/ir1VwLNieOaTd265pazqB2O5lZlonVVBP+oBB9j9rwJkDNtaBoOKYvQRs6Fa1wTJujZLXsQdhHXhq+jh7AqDyCUX4ofK59bRn60W1ApGfC7kJNzwfLCHf/m3r4A4BpYAVbwDSwj5+P45LbowESD/ZRl0MogNLFMwDUyMzr4I7VBHz1mNXHQVHQBk3CfsmtqFlF6Me2EzqyGf97v4MPnkYbMgUsE/PMKdxL/u8FN15ppagqrvn/B+e8ZT0aXukr9hGXoN76MEb5wf4uioS1EMnsQmeWx/x9YtDiUBQFxZUBrgw0ur9RTdffRw13iWd4e3cdmxPboInRE+AArJAf88xJjJoyzJqycPe5qka35VVULfy93Yl9/IJ+2RLV3oUzBs6lerJRMrwYZbtQvSNxXnl3eB7AOb00WuEYHJd/BrPqKKEjm9EPb8ZqrsVxye3hfe+7KJGCulXrlsz9TcJaCCFiRLG70ApGd7mLeaDw3PQPYBmoWUUXfI6iKJGu61FYV3wWq6Gqy6130TkJayGEEBelZnavF0JR1IQ7qWugS7w+ByGEEEK0IWEthBBCJDgJayGEECLBSVgLIYQQCU7CWgghhEhwEtZCCCFEgpOwFkIIIRKchLUQQgiR4CSshRBCiAQnYS2EEEIkOAlrIYQQIsFJWAshhBAJTsJaCCGESHAS1kIIIUSCk7AWQgghEpyEtRBCCJHgJKyFEEKIBCdhLYQQQiS4LoX16tWrWbJkCYsWLeK5555r9/hbb73Frbfeyi233MI3v/lN6uvrY15QIYQQIlV1GtYVFRU8+eSTrFy5kpdeeonnn3+ekpKS6ONNTU384Ac/YMWKFbzyyiuMHz+eX/3qV31aaCGEECKVdBrWGzZsYPbs2WRnZ+PxeFi8eDFr1qyJPh4Khfj+979PYWEhAOPHj+f06dN9V2IhhBAixdg6e0JlZSVerzd6u6CggF27dkVv5+TkcP311wPg9/tZsWIF9957b7cKkZeX3q3nd8brzYjp9QaaVK5/KtcdUqYL85YAAAj4SURBVLv+qVx3SO36p0LdOw1r0zRRFCV627KsNrdbNTY28q1vfYsJEyZw++23d6sQNTVNmKbVrddciNebQVVVY0yuNRClcv1Tue6Q2vVP5bpDatd/INZdVZVuN1I77QYvKiqiqqoqeruqqoqCgoI2z6msrOSee+5h/PjxPP74490qgBBCCCEurtOwnjNnDhs3bqS2thafz8e6detYsGBB9HHDMPj617/OjTfeyMMPP9xhq1sIIYQQPddpN3hhYSEPPfQQy5YtIxQKcddddzFt2jSWL1/OAw88QHl5OXv37sUwDNauXQvAlClTpIUthBBCxIhiWVZsBot7QcasYyeV65/KdYfUrn8q1x1Su/4Dse59MmYthBBCiP4lYS2EEEIkOAlrIYQQIsFJWAshhBAJTsJaCCGESHAS1kIIIUSCk7AWQgghEpyEtRBCCJHgJKyFEEKIBCdhLYQQQiQ4CWshhBAiwUlYCyGEEAlOwloIIYRIcBLWQgghRIKTsBZCCCESnIS1EEIIkeAkrIUQQogEJ2EthBBCJDgJayGEECLBSVgLIYQQCU7CWgghhEhwEtZCCCFEgpOwFkIIIRKchLUQQgiR4CSshRBCiAQnYS2EEEIkOAlrIYQQIsFJWAshhBAJTsJaCCGESHAS1kIIIUSCk7AWQgghEpyEtRBCCJHgJKyFEEKIBCdhLYQQQiQ4CWshhBAiwUlYCyGEEAlOwloIIYRIcBLWQgghRIKTsBZCCCESnIS1EEIIkeAkrIUQQogEJ2EthBBCJDgJayGEECLBSVgLIYQQCU7CWgghhEhwXQrr1atXs2TJEhYtWsRzzz3X7vF9+/Zxxx13sHjxYh5++GF0XY95QYUQQohU1WlYV1RU8OSTT7Jy5Upeeuklnn/+eUpKSto85zvf+Q6PPfYYa9euxbIsVq1a1WcFFkIIIVKNrbMnbNiwgdmzZ5OdnQ3A4sWLWbNmDffffz8AJ0+exO/3M2PGDADuuOMO/v3f/5177rmny4VQVaUnZY/b9QaaVK5/KtcdUrv+qVx3SO36D7S696S8nYZ1ZWUlXq83erugoIBdu3Zd8HGv10tFRUW3CpGTk9at53cmLy89ptcbaFK5/qlcd0jt+qdy3SG1658Kde+0G9w0TRTl/7d3N6FNbGEYgN/QpqWCLoI/kVIExYUULaIo0WKISKidxhQpmBbMIlgoiIFutErRjdoaC8GiCxdSERV/FlJaVBSl4E9QWrTZVChqbIsR4m+MhsyQfndxubm3Fy/xojVxzvvsZk4W55sczjczi3n/vgsQkRnH+caJiIjox+Rt1na7HYlEInecSCSwcOHC/xx/+/btjHEiIiL6MXmb9YYNGxCJRPD+/Xuk02ncunULmzZtyo1XVlaivLwcIyMjAID+/v4Z40RERPRjLCIi+X40MDCA06dPwzAMNDU1obW1Fa2trQgGg1i5ciWePXuGzs5OpFIpVFdXo6urC2VlZb9i/kRERKb3Xc2aiIiICodfMCMiIipybNZERERFjs2aiIioyLFZExERFTlTNet8gSNmlEql0NDQgKmpKQB/fh7W4/HA7XYjHA4XeHaz5+TJk9A0DZqmIRQKAVCndgA4ceIE6uvroWka+vr6AKhVPwAcO3YMHR0dANQKE9q5cyc0TYPX64XX68Xo6Kgye9/du3exfft2bN26FYcPHwag0LoXk3jz5o24XC758OGDfPnyRTwej4yPjxd6WrPq6dOn0tDQINXV1TI5OSnpdFqcTqdMTEyIYRgSCARkaGio0NP86R48eCA7duyQTCYjuq6L3++XgYEBJWoXEXn06JH4fD4xDEPS6bS4XC4ZGxtTpn4RkYcPH8r69etl3759IiKiaZo8efJERET2798vFy5cKOT0Zs309LTU1taKYRi5c6rsfRMTE1JbWyvxeFx0XZfm5mYZGhpSZt2b5sn6n4Ejc+bMyQWOmNmVK1dw6NCh3BfjotEolixZgqqqKpSWlsLj8ZjyGixYsAAdHR0oKyuD1WrFsmXLEIvFlKgdANatW4dz586htLQU7969QzabRTKZVKb+jx8/IhwOo62tDcC3w4TMWvuLFy8AAIFAANu2bcP58+eV2ftu376N+vp62O12WK1WhMNhVFRUKLPuTdOsvxU48n8DRX43R44cwdq1a3PHqlyD5cuX5zbmWCyGGzduwGKxKFH7X6xWK3p7e6FpGhwOhzL/PQAcPHgQ7e3tmDdvHoCfEyb0u0gmk3A4HDh16hTOnj2LS5cu4fXr10r8969evUI2m0VbWxu8Xi8uXryo1Lo3TbNmoIh612B8fByBQAB79+5FVVWVUrUDQDAYRCQSQTweRywWU6L+q1evYvHixXA4HLlzKq371atXIxQKYe7cubDZbGhqakJvb68S9WezWUQiERw9ehSXL19GNBrF5OSkErUD3xGR+buw2+0YHh7OHf87cEQF+UJXzGRkZATBYBAHDhyApml4/PixMrU/f/4cuq5jxYoVqKiogNvtxs2bN1FSUpL7jVnrv379OhKJBLxeLz59+oSvX7/CYrEoEyY0PDwMwzByNysigsrKSiXW/vz58+FwOGCz2QAAW7ZsUWbdAyZ6ss4XOKKCmpoavHz5Mve6aHBw0JTXIB6PY/fu3ejp6YGmaQDUqR0Apqam0NnZCV3Xoes67ty5A5/Pp0T9fX19GBwcRH9/P4LBIDZv3oyuri5lwoQ+f/6MUCiETCaDVCqFa9eu4fjx40rsfS6XC/fv30cymUQ2m8W9e/dQV1enxLoHTPRkvWjRIrS3t8Pv9+cCR1atWlXoaf1S5eXl6O7uxp49e5DJZOB0OlFXV1foaf10Z86cQSaTQXd3d+6cz+dTonYAcDqdiEajaGxsRElJCdxuNzRNg81mU6L+b+np6ZkRJuT3+ws9pVnhcrkwOjqKxsZGTE9Po6WlBWvWrFFi76upqcGuXbvQ0tICwzCwceNGNDc3Y+nSpUqsewZ5EBERFTnTvAYnIiIyKzZrIiKiIsdmTUREVOTYrImIiIocmzUREVGRY7MmIiIqcmzWRERERY7NmoiIqMj9Af0xY2iCriePAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 576x360 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "for learning_rate, history in zip(learning_rates, histories):\n",
    "    print(\"Learning rate:\", learning_rate)\n",
    "    plot_learning_curves(history)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's look at a more sophisticated way to tune hyperparameters. Create a `build_model()` function that takes three arguments, `n_hidden`, `n_neurons`, `learning_rate`, and builds, compiles and returns a model with the given number of hidden layers, the given number of neurons and the given learning rate. It is good practice to give a reasonable default value to each argument."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "def build_model(n_hidden=1, n_neurons=30, learning_rate=3e-3):\n",
    "    model = keras.models.Sequential()\n",
    "    options = {\"input_shape\": X_train.shape[1:]}\n",
    "    for layer in range(n_hidden + 1):\n",
    "        model.add(keras.layers.Dense(n_neurons, activation=\"relu\", **options))\n",
    "        options = {}\n",
    "    model.add(keras.layers.Dense(1, **options))\n",
    "    optimizer = keras.optimizers.SGD(learning_rate)\n",
    "    model.compile(loss=\"mse\", optimizer=optimizer)\n",
    "    return model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a `keras.wrappers.scikit_learn.KerasRegressor` and pass the `build_model` function to the constructor. This gives you a Scikit-Learn compatible predictor. Try training it and using it to make predictions. Note that you can pass the `n_epochs`, `callbacks` and `validation_data` to the `fit()` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "keras_reg = keras.wrappers.scikit_learn.KerasRegressor(build_model)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 11610 samples, validate on 3870 samples\n",
      "Epoch 1/100\n",
      "11610/11610 [==============================] - 0s 41us/sample - loss: 0.9440 - val_loss: 9.4997\n",
      "Epoch 2/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.6070 - val_loss: 34.8291\n",
      "Epoch 3/100\n",
      "11610/11610 [==============================] - 0s 34us/sample - loss: 0.6444 - val_loss: 2.0556\n",
      "Epoch 4/100\n",
      "11610/11610 [==============================] - 0s 33us/sample - loss: 0.4426 - val_loss: 0.4424\n",
      "Epoch 5/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.4072 - val_loss: 0.3828\n",
      "Epoch 6/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.3978 - val_loss: 0.3810\n",
      "Epoch 7/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.3908 - val_loss: 0.3813\n",
      "Epoch 8/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.3859 - val_loss: 0.3877\n",
      "Epoch 9/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.3821 - val_loss: 0.3704\n",
      "Epoch 10/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.3788 - val_loss: 0.3779\n",
      "Epoch 11/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.3761 - val_loss: 0.3823\n",
      "Epoch 12/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.3742 - val_loss: 0.3783\n",
      "Epoch 13/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.3715 - val_loss: 0.3787\n",
      "Epoch 14/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.3689 - val_loss: 0.3870\n",
      "Epoch 15/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.3677 - val_loss: 0.3929\n",
      "Epoch 16/100\n",
      "11610/11610 [==============================] - 0s 33us/sample - loss: 0.3653 - val_loss: 0.3907\n",
      "Epoch 17/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.3632 - val_loss: 0.3841\n",
      "Epoch 18/100\n",
      "11610/11610 [==============================] - 0s 33us/sample - loss: 0.3628 - val_loss: 0.3691\n",
      "Epoch 19/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.3606 - val_loss: 0.3888\n",
      "Epoch 20/100\n",
      "11610/11610 [==============================] - 0s 31us/sample - loss: 0.3593 - val_loss: 0.3602\n",
      "Epoch 21/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.3577 - val_loss: 0.3654\n",
      "Epoch 22/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.3569 - val_loss: 0.3606\n",
      "Epoch 23/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.3551 - val_loss: 0.4164\n",
      "Epoch 24/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.3544 - val_loss: 0.3564\n",
      "Epoch 25/100\n",
      "11610/11610 [==============================] - 0s 33us/sample - loss: 0.3533 - val_loss: 0.3717\n",
      "Epoch 26/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.3522 - val_loss: 0.3663\n",
      "Epoch 27/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.3516 - val_loss: 0.3929\n",
      "Epoch 28/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.3508 - val_loss: 0.3429\n",
      "Epoch 29/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.3493 - val_loss: 0.3521\n",
      "Epoch 30/100\n",
      "11610/11610 [==============================] - 0s 33us/sample - loss: 0.3488 - val_loss: 0.3937\n",
      "Epoch 31/100\n",
      "11610/11610 [==============================] - 0s 35us/sample - loss: 0.3478 - val_loss: 0.3611\n",
      "Epoch 32/100\n",
      "11610/11610 [==============================] - 0s 35us/sample - loss: 0.3467 - val_loss: 0.3792\n",
      "Epoch 33/100\n",
      "11610/11610 [==============================] - 0s 34us/sample - loss: 0.3455 - val_loss: 0.3740\n",
      "Epoch 34/100\n",
      "11610/11610 [==============================] - 0s 34us/sample - loss: 0.3452 - val_loss: 0.3629\n",
      "Epoch 35/100\n",
      "11610/11610 [==============================] - 0s 33us/sample - loss: 0.3443 - val_loss: 0.3792\n",
      "Epoch 36/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.3432 - val_loss: 0.3710\n",
      "Epoch 37/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.3428 - val_loss: 0.3737\n",
      "Epoch 38/100\n",
      "11610/11610 [==============================] - 0s 32us/sample - loss: 0.3420 - val_loss: 0.3575\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x1a3b91d9b0>"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "keras_reg.fit(X_train_scaled, y_train, epochs=100,\n",
    "              validation_data=(X_valid_scaled, y_valid),\n",
    "              callbacks=[keras.callbacks.EarlyStopping(patience=10)])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.76681733, 1.8267894 , 4.295369  , ..., 1.3182094 , 2.648156  ,\n",
       "       4.0545692 ], dtype=float32)"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "keras_reg.predict(X_test_scaled)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use a `sklearn.model_selection.RandomizedSearchCV` to search the hyperparameter space of your `KerasRegressor`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "from scipy.stats import reciprocal\n",
    "\n",
    "param_distribs = {\n",
    "    \"n_hidden\": [0, 1, 2, 3],\n",
    "    \"n_neurons\": np.arange(1, 100),\n",
    "    \"learning_rate\": reciprocal(3e-4, 3e-2),\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import RandomizedSearchCV\n",
    "\n",
    "rnd_search_cv = RandomizedSearchCV(keras_reg, param_distribs, n_iter=10, cv=3, verbose=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 3 folds for each of 10 candidates, totalling 30 fits\n",
      "[CV] learning_rate=0.0023302047292153563, n_hidden=2, n_neurons=41 ...\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 7740 samples, validate on 3870 samples\n",
      "Epoch 1/100\n",
      "7740/7740 [==============================] - 0s 61us/sample - loss: 1.6032 - val_loss: 1.7070\n",
      "Epoch 2/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.6348 - val_loss: 0.5833\n",
      "Epoch 3/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.5404 - val_loss: 0.4916\n",
      "Epoch 4/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.4899 - val_loss: 0.4363\n",
      "Epoch 5/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.4539 - val_loss: 0.4108\n",
      "Epoch 6/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.4299 - val_loss: 0.4061\n",
      "Epoch 7/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.4147 - val_loss: 0.4094\n",
      "Epoch 8/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.4044 - val_loss: 0.4162\n",
      "Epoch 9/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3966 - val_loss: 0.4136\n",
      "Epoch 10/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3907 - val_loss: 0.4069\n",
      "Epoch 11/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3860 - val_loss: 0.4050\n",
      "Epoch 12/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3811 - val_loss: 0.4054\n",
      "Epoch 13/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3777 - val_loss: 0.4089\n",
      "Epoch 14/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3745 - val_loss: 0.4118\n",
      "Epoch 15/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3712 - val_loss: 0.4270\n",
      "Epoch 16/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3682 - val_loss: 0.3820\n",
      "Epoch 17/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3652 - val_loss: 0.3883\n",
      "Epoch 18/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3628 - val_loss: 0.4042\n",
      "Epoch 19/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3605 - val_loss: 0.3741\n",
      "Epoch 20/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3581 - val_loss: 0.3736\n",
      "Epoch 21/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3557 - val_loss: 0.3662\n",
      "Epoch 22/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3535 - val_loss: 0.3517\n",
      "Epoch 23/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3511 - val_loss: 0.3583\n",
      "Epoch 24/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3493 - val_loss: 0.3570\n",
      "Epoch 25/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3477 - val_loss: 0.3631\n",
      "Epoch 26/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3463 - val_loss: 0.3423\n",
      "Epoch 27/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3446 - val_loss: 0.3695\n",
      "Epoch 28/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3420 - val_loss: 0.3397\n",
      "Epoch 29/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3414 - val_loss: 0.3492\n",
      "Epoch 30/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3404 - val_loss: 0.3500\n",
      "Epoch 31/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3385 - val_loss: 0.3407\n",
      "Epoch 32/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3379 - val_loss: 0.3365\n",
      "Epoch 33/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3365 - val_loss: 0.3482\n",
      "Epoch 34/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3353 - val_loss: 0.3435\n",
      "Epoch 35/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3334 - val_loss: 0.3367\n",
      "Epoch 36/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3329 - val_loss: 0.3609\n",
      "Epoch 37/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3318 - val_loss: 0.3485\n",
      "Epoch 38/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3304 - val_loss: 0.3455\n",
      "Epoch 39/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3301 - val_loss: 0.3264\n",
      "Epoch 40/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3291 - val_loss: 0.3259\n",
      "Epoch 41/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3274 - val_loss: 0.3431\n",
      "Epoch 42/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3265 - val_loss: 0.3371\n",
      "Epoch 43/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3258 - val_loss: 0.3332\n",
      "Epoch 44/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3249 - val_loss: 0.3310\n",
      "Epoch 45/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3240 - val_loss: 0.3318\n",
      "Epoch 46/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3236 - val_loss: 0.3268\n",
      "Epoch 47/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3222 - val_loss: 0.3270\n",
      "Epoch 48/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3215 - val_loss: 0.3296\n",
      "Epoch 49/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3204 - val_loss: 0.3274\n",
      "Epoch 50/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3198 - val_loss: 0.3180\n",
      "Epoch 51/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3185 - val_loss: 0.3408\n",
      "Epoch 52/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3177 - val_loss: 0.3154\n",
      "Epoch 53/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3173 - val_loss: 0.3288\n",
      "Epoch 54/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3154 - val_loss: 0.3332\n",
      "Epoch 55/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3148 - val_loss: 0.3334\n",
      "Epoch 56/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3145 - val_loss: 0.3171\n",
      "Epoch 57/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3140 - val_loss: 0.3272\n",
      "Epoch 58/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3129 - val_loss: 0.3203\n",
      "Epoch 59/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3125 - val_loss: 0.3195\n",
      "Epoch 60/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3119 - val_loss: 0.3197\n",
      "Epoch 61/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3101 - val_loss: 0.3084\n",
      "Epoch 62/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3102 - val_loss: 0.3236\n",
      "Epoch 63/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3092 - val_loss: 0.3236\n",
      "Epoch 64/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3089 - val_loss: 0.3137\n",
      "Epoch 65/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3076 - val_loss: 0.3309\n",
      "Epoch 66/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3071 - val_loss: 0.3164\n",
      "Epoch 67/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3066 - val_loss: 0.3116\n",
      "Epoch 68/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3053 - val_loss: 0.3275\n",
      "Epoch 69/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3058 - val_loss: 0.3333\n",
      "Epoch 70/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3058 - val_loss: 0.3057\n",
      "Epoch 71/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3042 - val_loss: 0.3118\n",
      "Epoch 72/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3034 - val_loss: 0.3156\n",
      "Epoch 73/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3023 - val_loss: 0.3021\n",
      "Epoch 74/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3018 - val_loss: 0.3117\n",
      "Epoch 75/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3012 - val_loss: 0.3137\n",
      "Epoch 76/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3009 - val_loss: 0.3042\n",
      "Epoch 77/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3001 - val_loss: 0.3010\n",
      "Epoch 78/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.2996 - val_loss: 0.2999\n",
      "Epoch 79/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.2981 - val_loss: 0.3223\n",
      "Epoch 80/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.2989 - val_loss: 0.3033\n",
      "Epoch 81/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.2978 - val_loss: 0.3024\n",
      "Epoch 82/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.2972 - val_loss: 0.2972\n",
      "Epoch 83/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.2968 - val_loss: 0.3024\n",
      "Epoch 84/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.2964 - val_loss: 0.2988\n",
      "Epoch 85/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.2953 - val_loss: 0.3088\n",
      "Epoch 86/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.2944 - val_loss: 0.3030\n",
      "Epoch 87/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.2944 - val_loss: 0.2996\n",
      "Epoch 88/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.2941 - val_loss: 0.2984\n",
      "Epoch 89/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.2934 - val_loss: 0.3058\n",
      "Epoch 90/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.2928 - val_loss: 0.3072\n",
      "Epoch 91/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.2923 - val_loss: 0.3268\n",
      "Epoch 92/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.2918 - val_loss: 0.3025\n",
      "3870/3870 [==============================] - 0s 14us/sample - loss: 0.3291\n",
      "7740/7740 [==============================] - 0s 16us/sample - loss: 0.2901\n",
      "[CV]  learning_rate=0.0023302047292153563, n_hidden=2, n_neurons=41, total=  27.1s\n",
      "[CV] learning_rate=0.0023302047292153563, n_hidden=2, n_neurons=41 ...\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:   27.3s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 7740 samples, validate on 3870 samples\n",
      "Epoch 1/100\n",
      "7740/7740 [==============================] - 0s 55us/sample - loss: 1.3620 - val_loss: 1.9705\n",
      "Epoch 2/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.5921 - val_loss: 0.6401\n",
      "Epoch 3/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.5145 - val_loss: 0.4716\n",
      "Epoch 4/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.4709 - val_loss: 0.4411\n",
      "Epoch 5/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.4432 - val_loss: 0.4216\n",
      "Epoch 6/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.4248 - val_loss: 0.3981\n",
      "Epoch 7/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.4126 - val_loss: 0.3831\n",
      "Epoch 8/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.4020 - val_loss: 0.3851\n",
      "Epoch 9/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3949 - val_loss: 0.4085\n",
      "Epoch 10/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3881 - val_loss: 0.4479\n",
      "Epoch 11/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3839 - val_loss: 0.4876\n",
      "Epoch 12/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3786 - val_loss: 0.5219\n",
      "Epoch 13/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3754 - val_loss: 0.6181\n",
      "Epoch 14/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3716 - val_loss: 0.6809\n",
      "Epoch 15/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3688 - val_loss: 0.7110\n",
      "Epoch 16/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3663 - val_loss: 0.7682\n",
      "Epoch 17/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3642 - val_loss: 0.8752\n",
      "3870/3870 [==============================] - 0s 14us/sample - loss: 0.3741\n",
      "7740/7740 [==============================] - 0s 16us/sample - loss: 0.3604\n",
      "[CV]  learning_rate=0.0023302047292153563, n_hidden=2, n_neurons=41, total=   5.5s\n",
      "[CV] learning_rate=0.0023302047292153563, n_hidden=2, n_neurons=41 ...\n",
      "Train on 7740 samples, validate on 3870 samples\n",
      "Epoch 1/100\n",
      "7740/7740 [==============================] - 0s 55us/sample - loss: 1.5622 - val_loss: 0.6830\n",
      "Epoch 2/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.6203 - val_loss: 0.5885\n",
      "Epoch 3/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.5482 - val_loss: 0.5091\n",
      "Epoch 4/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.4988 - val_loss: 0.4652\n",
      "Epoch 5/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.4643 - val_loss: 0.4598\n",
      "Epoch 6/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.4398 - val_loss: 0.4594\n",
      "Epoch 7/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.4229 - val_loss: 0.3934\n",
      "Epoch 8/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.4119 - val_loss: 0.5014\n",
      "Epoch 9/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.4040 - val_loss: 0.4281\n",
      "Epoch 10/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3975 - val_loss: 0.3707\n",
      "Epoch 11/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3922 - val_loss: 0.4708\n",
      "Epoch 12/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3880 - val_loss: 0.4576\n",
      "Epoch 13/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3850 - val_loss: 0.3639\n",
      "Epoch 14/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3817 - val_loss: 0.4434\n",
      "Epoch 15/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3793 - val_loss: 0.3625\n",
      "Epoch 16/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3756 - val_loss: 0.4428\n",
      "Epoch 17/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3742 - val_loss: 0.3508\n",
      "Epoch 18/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3716 - val_loss: 0.3632\n",
      "Epoch 19/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3692 - val_loss: 0.4190\n",
      "Epoch 20/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3664 - val_loss: 0.4335\n",
      "Epoch 21/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3653 - val_loss: 0.3813\n",
      "Epoch 22/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3628 - val_loss: 0.4380\n",
      "Epoch 23/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3617 - val_loss: 0.3523\n",
      "Epoch 24/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3600 - val_loss: 0.3579\n",
      "Epoch 25/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3582 - val_loss: 0.4265\n",
      "Epoch 26/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3569 - val_loss: 0.4404\n",
      "Epoch 27/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3556 - val_loss: 0.3602\n",
      "3870/3870 [==============================] - 0s 15us/sample - loss: 0.3564\n",
      "7740/7740 [==============================] - 0s 16us/sample - loss: 0.3544\n",
      "[CV]  learning_rate=0.0023302047292153563, n_hidden=2, n_neurons=41, total=   8.3s\n",
      "[CV] learning_rate=0.015956195942385693, n_hidden=0, n_neurons=97 ....\n",
      "Train on 7740 samples, validate on 3870 samples\n",
      "Epoch 1/100\n",
      "7740/7740 [==============================] - 0s 47us/sample - loss: 0.7604 - val_loss: 5.4232\n",
      "Epoch 2/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.6441 - val_loss: 14.1695\n",
      "Epoch 3/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.6713 - val_loss: 3.7025\n",
      "Epoch 4/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.5019 - val_loss: 4.6624\n",
      "Epoch 5/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.8346 - val_loss: 49.8473\n",
      "Epoch 6/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 3.0755 - val_loss: 0.3787\n",
      "Epoch 7/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4175 - val_loss: 0.3511\n",
      "Epoch 8/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3728 - val_loss: 0.3492\n",
      "Epoch 9/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.5112 - val_loss: 0.3382\n",
      "Epoch 10/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3575 - val_loss: 0.3333\n",
      "Epoch 11/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4000 - val_loss: 0.3305\n",
      "Epoch 12/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.5751 - val_loss: 1.1609\n",
      "Epoch 13/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.4492 - val_loss: 0.3487\n",
      "Epoch 14/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3481 - val_loss: 0.3288\n",
      "Epoch 15/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3440 - val_loss: 0.3263\n",
      "Epoch 16/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3395 - val_loss: 0.3361\n",
      "Epoch 17/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3359 - val_loss: 0.3362\n",
      "Epoch 18/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3359 - val_loss: 0.3213\n",
      "Epoch 19/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3335 - val_loss: 0.3174\n",
      "Epoch 20/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3284 - val_loss: 0.3236\n",
      "Epoch 21/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3275 - val_loss: 0.3134\n",
      "Epoch 22/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3257 - val_loss: 0.3121\n",
      "Epoch 23/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3243 - val_loss: 0.3127\n",
      "Epoch 24/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3218 - val_loss: 0.3214\n",
      "Epoch 25/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3217 - val_loss: 0.3085\n",
      "Epoch 26/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3191 - val_loss: 0.3113\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 27/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3189 - val_loss: 0.3051\n",
      "Epoch 28/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3172 - val_loss: 0.3059\n",
      "Epoch 29/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3154 - val_loss: 0.3059\n",
      "Epoch 30/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3146 - val_loss: 0.3134\n",
      "Epoch 31/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3111 - val_loss: 0.3168\n",
      "Epoch 32/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3134 - val_loss: 0.3356\n",
      "Epoch 33/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3130 - val_loss: 0.3226\n",
      "Epoch 34/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3100 - val_loss: 0.3243\n",
      "Epoch 35/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3097 - val_loss: 0.3224\n",
      "Epoch 36/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3088 - val_loss: 0.3283\n",
      "Epoch 37/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3113 - val_loss: 0.3471\n",
      "3870/3870 [==============================] - 0s 13us/sample - loss: 0.4386\n",
      "7740/7740 [==============================] - 0s 14us/sample - loss: 0.3676\n",
      "[CV]  learning_rate=0.015956195942385693, n_hidden=0, n_neurons=97, total=  10.0s\n",
      "[CV] learning_rate=0.015956195942385693, n_hidden=0, n_neurons=97 ....\n",
      "Train on 7740 samples, validate on 3870 samples\n",
      "Epoch 1/100\n",
      "7740/7740 [==============================] - 0s 46us/sample - loss: 0.7714 - val_loss: 0.7641\n",
      "Epoch 2/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4561 - val_loss: 1.2963\n",
      "Epoch 3/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.4122 - val_loss: 0.3778\n",
      "Epoch 4/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.4151 - val_loss: 0.4568\n",
      "Epoch 5/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3952 - val_loss: 0.7350\n",
      "Epoch 6/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3759 - val_loss: 0.7615\n",
      "Epoch 7/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3707 - val_loss: 0.5305\n",
      "Epoch 8/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3696 - val_loss: 0.3510\n",
      "Epoch 9/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3634 - val_loss: 0.3410\n",
      "Epoch 10/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3602 - val_loss: 0.3868\n",
      "Epoch 11/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3531 - val_loss: 0.8745\n",
      "Epoch 12/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3574 - val_loss: 0.3859\n",
      "Epoch 13/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3542 - val_loss: 0.6109\n",
      "Epoch 14/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3512 - val_loss: 0.4101\n",
      "Epoch 15/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3496 - val_loss: 0.3618\n",
      "Epoch 16/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3458 - val_loss: 0.3277\n",
      "Epoch 17/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3454 - val_loss: 0.4292\n",
      "Epoch 18/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3431 - val_loss: 0.3737\n",
      "Epoch 19/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3434 - val_loss: 0.6402\n",
      "Epoch 20/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3444 - val_loss: 0.3269\n",
      "Epoch 21/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3388 - val_loss: 0.6092\n",
      "Epoch 22/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3431 - val_loss: 0.3750\n",
      "Epoch 23/100\n",
      "7740/7740 [==============================] - 0s 43us/sample - loss: 0.3417 - val_loss: 0.3615\n",
      "Epoch 24/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3409 - val_loss: 0.6085\n",
      "Epoch 25/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3331 - val_loss: 0.4557\n",
      "Epoch 26/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3308 - val_loss: 0.4441\n",
      "Epoch 27/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3303 - val_loss: 0.3698\n",
      "Epoch 28/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3282 - val_loss: 0.3496\n",
      "Epoch 29/100\n",
      "7740/7740 [==============================] - 0s 45us/sample - loss: 0.3314 - val_loss: 0.3630\n",
      "Epoch 30/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3258 - val_loss: 0.3160\n",
      "Epoch 31/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3255 - val_loss: 0.3887\n",
      "Epoch 32/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3229 - val_loss: 0.7654\n",
      "Epoch 33/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3269 - val_loss: 0.3289\n",
      "Epoch 34/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3192 - val_loss: 0.3337\n",
      "Epoch 35/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3304 - val_loss: 0.4950\n",
      "Epoch 36/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3192 - val_loss: 1.9769\n",
      "Epoch 37/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3214 - val_loss: 0.6508\n",
      "Epoch 38/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3190 - val_loss: 0.3250\n",
      "Epoch 39/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3191 - val_loss: 0.7641\n",
      "Epoch 40/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3161 - val_loss: 0.4083\n",
      "3870/3870 [==============================] - 0s 13us/sample - loss: 0.3752\n",
      "7740/7740 [==============================] - 0s 14us/sample - loss: 0.3483\n",
      "[CV]  learning_rate=0.015956195942385693, n_hidden=0, n_neurons=97, total=  11.3s\n",
      "[CV] learning_rate=0.015956195942385693, n_hidden=0, n_neurons=97 ....\n",
      "Train on 7740 samples, validate on 3870 samples\n",
      "Epoch 1/100\n",
      "7740/7740 [==============================] - 0s 46us/sample - loss: 0.7449 - val_loss: 1.8796\n",
      "Epoch 2/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.4798 - val_loss: 0.5772\n",
      "Epoch 3/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.4285 - val_loss: 0.3831\n",
      "Epoch 4/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4161 - val_loss: 1.4244\n",
      "Epoch 5/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4231 - val_loss: 18.9345\n",
      "Epoch 6/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3970 - val_loss: 6.8569\n",
      "Epoch 7/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.6343 - val_loss: 3.4721\n",
      "Epoch 8/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4484 - val_loss: 103.4562\n",
      "Epoch 9/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.5382 - val_loss: 0.3930\n",
      "Epoch 10/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3965 - val_loss: 0.3660\n",
      "Epoch 11/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4080 - val_loss: 0.3647\n",
      "Epoch 12/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3822 - val_loss: 0.3534\n",
      "Epoch 13/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3735 - val_loss: 0.3537\n",
      "Epoch 14/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3672 - val_loss: 0.3453\n",
      "Epoch 15/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3628 - val_loss: 0.3444\n",
      "Epoch 16/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3766 - val_loss: 0.3401\n",
      "Epoch 17/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3711 - val_loss: 0.3460\n",
      "Epoch 18/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3623 - val_loss: 0.3325\n",
      "Epoch 19/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3570 - val_loss: 0.3644\n",
      "Epoch 20/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3536 - val_loss: 0.3517\n",
      "Epoch 21/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3496 - val_loss: 0.3682\n",
      "Epoch 22/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3599 - val_loss: 0.3256\n",
      "Epoch 23/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3469 - val_loss: 0.3334\n",
      "Epoch 24/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3439 - val_loss: 0.3432\n",
      "Epoch 25/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3469 - val_loss: 0.3219\n",
      "Epoch 26/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3416 - val_loss: 0.3368\n",
      "Epoch 27/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3394 - val_loss: 0.3201\n",
      "Epoch 28/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3370 - val_loss: 0.3207\n",
      "Epoch 29/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3346 - val_loss: 0.3202\n",
      "Epoch 30/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3315 - val_loss: 0.3141\n",
      "Epoch 31/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3335 - val_loss: 0.3171\n",
      "Epoch 32/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3306 - val_loss: 0.3172\n",
      "Epoch 33/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3389 - val_loss: 0.3142\n",
      "Epoch 34/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3332 - val_loss: 0.3638\n",
      "Epoch 35/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3315 - val_loss: 0.3186\n",
      "Epoch 36/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3256 - val_loss: 0.3102\n",
      "Epoch 37/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3233 - val_loss: 0.3238\n",
      "Epoch 38/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3291 - val_loss: 0.3299\n",
      "Epoch 39/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3339 - val_loss: 0.3433\n",
      "Epoch 40/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3250 - val_loss: 0.3063\n",
      "Epoch 41/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3228 - val_loss: 0.3480\n",
      "Epoch 42/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3186 - val_loss: 0.3669\n",
      "Epoch 43/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3251 - val_loss: 0.3058\n",
      "Epoch 44/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3247 - val_loss: 0.3364\n",
      "Epoch 45/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3180 - val_loss: 0.3104\n",
      "Epoch 46/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3232 - val_loss: 0.3273\n",
      "Epoch 47/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3166 - val_loss: 0.3027\n",
      "Epoch 48/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3136 - val_loss: 0.3203\n",
      "Epoch 49/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3131 - val_loss: 0.3446\n",
      "Epoch 50/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3147 - val_loss: 0.3076\n",
      "Epoch 51/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3226 - val_loss: 0.3362\n",
      "Epoch 52/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3121 - val_loss: 0.3097\n",
      "Epoch 53/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3116 - val_loss: 0.4066\n",
      "Epoch 54/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3176 - val_loss: 0.3047\n",
      "Epoch 55/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3096 - val_loss: 0.3028\n",
      "Epoch 56/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3083 - val_loss: 0.3614\n",
      "Epoch 57/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3074 - val_loss: 0.3032\n",
      "3870/3870 [==============================] - 0s 13us/sample - loss: 0.3190\n",
      "7740/7740 [==============================] - 0s 14us/sample - loss: 0.3051\n",
      "[CV]  learning_rate=0.015956195942385693, n_hidden=0, n_neurons=97, total=  15.3s\n",
      "[CV] learning_rate=0.0017196854145436866, n_hidden=2, n_neurons=77 ...\n",
      "Train on 7740 samples, validate on 3870 samples\n",
      "Epoch 1/100\n",
      "7740/7740 [==============================] - 0s 56us/sample - loss: 1.5431 - val_loss: 2.1386\n",
      "Epoch 2/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.6880 - val_loss: 0.5965\n",
      "Epoch 3/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.5931 - val_loss: 0.5344\n",
      "Epoch 4/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.5405 - val_loss: 0.5002\n",
      "Epoch 5/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.4996 - val_loss: 0.5025\n",
      "Epoch 6/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.4679 - val_loss: 0.4393\n",
      "Epoch 7/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.4435 - val_loss: 0.4388\n",
      "Epoch 8/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.4256 - val_loss: 0.3981\n",
      "Epoch 9/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.4115 - val_loss: 0.4108\n",
      "Epoch 10/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.4016 - val_loss: 0.4239\n",
      "Epoch 11/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3936 - val_loss: 0.4047\n",
      "Epoch 12/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3870 - val_loss: 0.4046\n",
      "Epoch 13/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3808 - val_loss: 0.3774\n",
      "Epoch 14/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3757 - val_loss: 0.4053\n",
      "Epoch 15/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3706 - val_loss: 0.3574\n",
      "Epoch 16/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3669 - val_loss: 0.3605\n",
      "Epoch 17/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3628 - val_loss: 0.4119\n",
      "Epoch 18/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3604 - val_loss: 0.4028\n",
      "Epoch 19/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3572 - val_loss: 0.3772\n",
      "Epoch 20/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3547 - val_loss: 0.3834\n",
      "Epoch 21/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3522 - val_loss: 0.3464\n",
      "Epoch 22/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3494 - val_loss: 0.3959\n",
      "Epoch 23/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3477 - val_loss: 0.3378\n",
      "Epoch 24/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3456 - val_loss: 0.4059\n",
      "Epoch 25/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3446 - val_loss: 0.3477\n",
      "Epoch 26/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3420 - val_loss: 0.3920\n",
      "Epoch 27/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3410 - val_loss: 0.3325\n",
      "Epoch 28/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3389 - val_loss: 0.3782\n",
      "Epoch 29/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3373 - val_loss: 0.3309\n",
      "Epoch 30/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.3366 - val_loss: 0.3642\n",
      "Epoch 31/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3350 - val_loss: 0.3590\n",
      "Epoch 32/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3333 - val_loss: 0.3305\n",
      "Epoch 33/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3319 - val_loss: 0.3564\n",
      "Epoch 34/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3305 - val_loss: 0.3445\n",
      "Epoch 35/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3300 - val_loss: 0.3723\n",
      "Epoch 36/100\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "7740/7740 [==============================] - 0s 42us/sample - loss: 0.3287 - val_loss: 0.3593\n",
      "Epoch 37/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3278 - val_loss: 0.3276\n",
      "Epoch 38/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3272 - val_loss: 0.4318\n",
      "Epoch 39/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3264 - val_loss: 0.3511\n",
      "Epoch 40/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3252 - val_loss: 0.3221\n",
      "Epoch 41/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3240 - val_loss: 0.3791\n",
      "Epoch 42/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3236 - val_loss: 0.3305\n",
      "Epoch 43/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3225 - val_loss: 0.3361\n",
      "Epoch 44/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3210 - val_loss: 0.3781\n",
      "Epoch 45/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3205 - val_loss: 0.3291\n",
      "Epoch 46/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3197 - val_loss: 0.3419\n",
      "Epoch 47/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3189 - val_loss: 0.3624\n",
      "Epoch 48/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3179 - val_loss: 0.3237\n",
      "Epoch 49/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3168 - val_loss: 0.3165\n",
      "Epoch 50/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3160 - val_loss: 0.3233\n",
      "Epoch 51/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3148 - val_loss: 0.4150\n",
      "Epoch 52/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3152 - val_loss: 0.3100\n",
      "Epoch 53/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3138 - val_loss: 0.4137\n",
      "Epoch 54/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3140 - val_loss: 0.3866\n",
      "Epoch 55/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3125 - val_loss: 0.3258\n",
      "Epoch 56/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3112 - val_loss: 0.3279\n",
      "Epoch 57/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3109 - val_loss: 0.4006\n",
      "Epoch 58/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3111 - val_loss: 0.3077\n",
      "Epoch 59/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3094 - val_loss: 0.3585\n",
      "Epoch 60/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3097 - val_loss: 0.3600\n",
      "Epoch 61/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3089 - val_loss: 0.3933\n",
      "Epoch 62/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3080 - val_loss: 0.3050\n",
      "Epoch 63/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3066 - val_loss: 0.3039\n",
      "Epoch 64/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3057 - val_loss: 0.3046\n",
      "Epoch 65/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3053 - val_loss: 0.3040\n",
      "Epoch 66/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3047 - val_loss: 0.3039\n",
      "Epoch 67/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3036 - val_loss: 0.4107\n",
      "Epoch 68/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3036 - val_loss: 0.3624\n",
      "Epoch 69/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3031 - val_loss: 0.4021\n",
      "Epoch 70/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3026 - val_loss: 0.3481\n",
      "Epoch 71/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3021 - val_loss: 0.5111\n",
      "Epoch 72/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3034 - val_loss: 0.3270\n",
      "Epoch 73/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.2998 - val_loss: 0.4186\n",
      "3870/3870 [==============================] - 0s 15us/sample - loss: 0.3326\n",
      "7740/7740 [==============================] - 0s 17us/sample - loss: 0.2985\n",
      "[CV]  learning_rate=0.0017196854145436866, n_hidden=2, n_neurons=77, total=  22.5s\n",
      "[CV] learning_rate=0.0017196854145436866, n_hidden=2, n_neurons=77 ...\n",
      "Train on 7740 samples, validate on 3870 samples\n",
      "Epoch 1/100\n",
      "7740/7740 [==============================] - 0s 56us/sample - loss: 1.6301 - val_loss: 4.2039\n",
      "Epoch 2/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.6970 - val_loss: 0.6768\n",
      "Epoch 3/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.6090 - val_loss: 0.8804\n",
      "Epoch 4/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.5529 - val_loss: 1.6374\n",
      "Epoch 5/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.5127 - val_loss: 2.1626\n",
      "Epoch 6/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.4821 - val_loss: 2.8482\n",
      "Epoch 7/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.4595 - val_loss: 2.3233\n",
      "Epoch 8/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.4420 - val_loss: 2.1937\n",
      "Epoch 9/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.4277 - val_loss: 2.1441\n",
      "Epoch 10/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.4166 - val_loss: 1.2412\n",
      "Epoch 11/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.4067 - val_loss: 1.0166\n",
      "Epoch 12/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3987 - val_loss: 0.7848\n",
      "3870/3870 [==============================] - 0s 15us/sample - loss: 0.4227\n",
      "7740/7740 [==============================] - 0s 17us/sample - loss: 0.3932\n",
      "[CV]  learning_rate=0.0017196854145436866, n_hidden=2, n_neurons=77, total=   4.1s\n",
      "[CV] learning_rate=0.0017196854145436866, n_hidden=2, n_neurons=77 ...\n",
      "Train on 7740 samples, validate on 3870 samples\n",
      "Epoch 1/100\n",
      "7740/7740 [==============================] - 0s 56us/sample - loss: 1.5410 - val_loss: 1.7610\n",
      "Epoch 2/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.6859 - val_loss: 0.6030\n",
      "Epoch 3/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.6079 - val_loss: 0.5475\n",
      "Epoch 4/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.5578 - val_loss: 0.5058\n",
      "Epoch 5/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.5189 - val_loss: 0.4733\n",
      "Epoch 6/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.4871 - val_loss: 0.4425\n",
      "Epoch 7/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.4617 - val_loss: 0.4225\n",
      "Epoch 8/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.4422 - val_loss: 0.4064\n",
      "Epoch 9/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.4270 - val_loss: 0.4141\n",
      "Epoch 10/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.4156 - val_loss: 0.4001\n",
      "Epoch 11/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.4056 - val_loss: 0.4111\n",
      "Epoch 12/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3978 - val_loss: 0.3833\n",
      "Epoch 13/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3928 - val_loss: 0.4107\n",
      "Epoch 14/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3881 - val_loss: 0.3695\n",
      "Epoch 15/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3824 - val_loss: 0.4339\n",
      "Epoch 16/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3784 - val_loss: 0.4015\n",
      "Epoch 17/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3751 - val_loss: 0.3944\n",
      "Epoch 18/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3715 - val_loss: 0.3888\n",
      "Epoch 19/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3681 - val_loss: 0.4058\n",
      "Epoch 20/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3654 - val_loss: 0.3471\n",
      "Epoch 21/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3638 - val_loss: 0.3469\n",
      "Epoch 22/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3600 - val_loss: 0.3725\n",
      "Epoch 23/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3581 - val_loss: 0.4448\n",
      "Epoch 24/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3571 - val_loss: 0.3868\n",
      "Epoch 25/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3549 - val_loss: 0.3871\n",
      "Epoch 26/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3532 - val_loss: 0.3699\n",
      "Epoch 27/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3510 - val_loss: 0.3456\n",
      "Epoch 28/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3492 - val_loss: 0.4108\n",
      "Epoch 29/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3481 - val_loss: 0.3753\n",
      "Epoch 30/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3461 - val_loss: 0.3960\n",
      "Epoch 31/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3445 - val_loss: 0.3945\n",
      "Epoch 32/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3434 - val_loss: 0.3682\n",
      "Epoch 33/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3419 - val_loss: 0.3298\n",
      "Epoch 34/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3411 - val_loss: 0.3609\n",
      "Epoch 35/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3398 - val_loss: 0.3399\n",
      "Epoch 36/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3385 - val_loss: 0.3297\n",
      "Epoch 37/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3368 - val_loss: 0.3665\n",
      "Epoch 38/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3364 - val_loss: 0.3196\n",
      "Epoch 39/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3353 - val_loss: 0.4043\n",
      "Epoch 40/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3344 - val_loss: 0.3602\n",
      "Epoch 41/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3329 - val_loss: 0.3659\n",
      "Epoch 42/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3321 - val_loss: 0.3848\n",
      "Epoch 43/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3310 - val_loss: 0.3145\n",
      "Epoch 44/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3296 - val_loss: 0.3836\n",
      "Epoch 45/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3294 - val_loss: 0.3203\n",
      "Epoch 46/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3272 - val_loss: 0.3455\n",
      "Epoch 47/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3271 - val_loss: 0.3557\n",
      "Epoch 48/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3262 - val_loss: 0.3176\n",
      "Epoch 49/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3252 - val_loss: 0.3626\n",
      "Epoch 50/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3246 - val_loss: 0.3258\n",
      "Epoch 51/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3235 - val_loss: 0.3191\n",
      "Epoch 52/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3231 - val_loss: 0.3160\n",
      "Epoch 53/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3216 - val_loss: 0.3293\n",
      "3870/3870 [==============================] - 0s 15us/sample - loss: 0.3243\n",
      "7740/7740 [==============================] - 0s 16us/sample - loss: 0.3189\n",
      "[CV]  learning_rate=0.0017196854145436866, n_hidden=2, n_neurons=77, total=  16.3s\n",
      "[CV] learning_rate=0.003284284607688981, n_hidden=2, n_neurons=75 ....\n",
      "Train on 7740 samples, validate on 3870 samples\n",
      "Epoch 1/100\n",
      "7740/7740 [==============================] - 0s 56us/sample - loss: 1.4842 - val_loss: 2.1657\n",
      "Epoch 2/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.5991 - val_loss: 1.9354\n",
      "Epoch 3/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.5146 - val_loss: 0.6279\n",
      "Epoch 4/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.4586 - val_loss: 0.5010\n",
      "Epoch 5/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.4262 - val_loss: 0.3873\n",
      "Epoch 6/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.4064 - val_loss: 0.3757\n",
      "Epoch 7/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3914 - val_loss: 0.3797\n",
      "Epoch 8/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3825 - val_loss: 0.3684\n",
      "Epoch 9/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3742 - val_loss: 0.3503\n",
      "Epoch 10/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3679 - val_loss: 0.3745\n",
      "Epoch 11/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3622 - val_loss: 0.3999\n",
      "Epoch 12/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3582 - val_loss: 0.3889\n",
      "Epoch 13/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3541 - val_loss: 0.3470\n",
      "Epoch 14/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3500 - val_loss: 0.4274\n",
      "Epoch 15/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3486 - val_loss: 0.3568\n",
      "Epoch 16/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3443 - val_loss: 0.4425\n",
      "Epoch 17/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3429 - val_loss: 0.3352\n",
      "Epoch 18/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3390 - val_loss: 0.3316\n",
      "Epoch 19/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3365 - val_loss: 0.4038\n",
      "Epoch 20/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3357 - val_loss: 0.3435\n",
      "Epoch 21/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3330 - val_loss: 0.3226\n",
      "Epoch 22/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3316 - val_loss: 0.3796\n",
      "Epoch 23/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3293 - val_loss: 0.3240\n",
      "Epoch 24/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3274 - val_loss: 0.3713\n",
      "Epoch 25/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3268 - val_loss: 0.3210\n",
      "Epoch 26/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3225 - val_loss: 0.3229\n",
      "Epoch 27/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3219 - val_loss: 0.3197\n",
      "Epoch 28/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3205 - val_loss: 0.3354\n",
      "Epoch 29/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3195 - val_loss: 0.3137\n",
      "Epoch 30/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3180 - val_loss: 0.3713\n",
      "Epoch 31/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3170 - val_loss: 0.3105\n",
      "Epoch 32/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3153 - val_loss: 0.3809\n",
      "Epoch 33/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3137 - val_loss: 0.3351\n",
      "Epoch 34/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3127 - val_loss: 0.3326\n",
      "Epoch 35/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3109 - val_loss: 0.3667\n",
      "Epoch 36/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3089 - val_loss: 0.3167\n",
      "Epoch 37/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3073 - val_loss: 0.3544\n",
      "Epoch 38/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3082 - val_loss: 0.3108\n",
      "Epoch 39/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3065 - val_loss: 0.3324\n",
      "Epoch 40/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3063 - val_loss: 0.3016\n",
      "Epoch 41/100\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3038 - val_loss: 0.3123\n",
      "Epoch 42/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3025 - val_loss: 0.3372\n",
      "Epoch 43/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3027 - val_loss: 0.3443\n",
      "Epoch 44/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3021 - val_loss: 0.3409\n",
      "Epoch 45/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.3002 - val_loss: 0.3001\n",
      "Epoch 46/100\n",
      "7740/7740 [==============================] - 0s 43us/sample - loss: 0.2992 - val_loss: 0.3370\n",
      "Epoch 47/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2981 - val_loss: 0.3304\n",
      "Epoch 48/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2979 - val_loss: 0.3872\n",
      "Epoch 49/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2984 - val_loss: 0.3408\n",
      "Epoch 50/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2955 - val_loss: 0.3122\n",
      "Epoch 51/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2942 - val_loss: 0.3237\n",
      "Epoch 52/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2932 - val_loss: 0.3119\n",
      "Epoch 53/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2924 - val_loss: 0.3206\n",
      "Epoch 54/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2910 - val_loss: 0.4156\n",
      "Epoch 55/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2920 - val_loss: 0.3537\n",
      "3870/3870 [==============================] - 0s 15us/sample - loss: 0.3294\n",
      "7740/7740 [==============================] - 0s 17us/sample - loss: 0.2875\n",
      "[CV]  learning_rate=0.003284284607688981, n_hidden=2, n_neurons=75, total=  17.3s\n",
      "[CV] learning_rate=0.003284284607688981, n_hidden=2, n_neurons=75 ....\n",
      "Train on 7740 samples, validate on 3870 samples\n",
      "Epoch 1/100\n",
      "7740/7740 [==============================] - 0s 56us/sample - loss: 1.0847 - val_loss: 5.8122\n",
      "Epoch 2/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.6135 - val_loss: 0.7386\n",
      "Epoch 3/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.5218 - val_loss: 0.6025\n",
      "Epoch 4/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.4632 - val_loss: 0.7112\n",
      "Epoch 5/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.4265 - val_loss: 0.5877\n",
      "Epoch 6/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.4039 - val_loss: 0.4176\n",
      "Epoch 7/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3898 - val_loss: 0.3684\n",
      "Epoch 8/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3788 - val_loss: 0.4108\n",
      "Epoch 9/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3707 - val_loss: 0.4843\n",
      "Epoch 10/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3641 - val_loss: 0.6541\n",
      "Epoch 11/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3576 - val_loss: 0.7952\n",
      "Epoch 12/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3545 - val_loss: 0.8378\n",
      "Epoch 13/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3505 - val_loss: 0.8978\n",
      "Epoch 14/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3467 - val_loss: 0.8281\n",
      "Epoch 15/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3450 - val_loss: 0.9674\n",
      "Epoch 16/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3422 - val_loss: 0.9459\n",
      "Epoch 17/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3400 - val_loss: 0.8946\n",
      "3870/3870 [==============================] - 0s 15us/sample - loss: 0.3573\n",
      "7740/7740 [==============================] - 0s 16us/sample - loss: 0.3355\n",
      "[CV]  learning_rate=0.003284284607688981, n_hidden=2, n_neurons=75, total=   5.7s\n",
      "[CV] learning_rate=0.003284284607688981, n_hidden=2, n_neurons=75 ....\n",
      "Train on 7740 samples, validate on 3870 samples\n",
      "Epoch 1/100\n",
      "7740/7740 [==============================] - 0s 56us/sample - loss: 1.3023 - val_loss: 2.6927\n",
      "Epoch 2/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.5547 - val_loss: 0.5447\n",
      "Epoch 3/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.4631 - val_loss: 0.4596\n",
      "Epoch 4/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.4296 - val_loss: 0.4099\n",
      "Epoch 5/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.4112 - val_loss: 0.4182\n",
      "Epoch 6/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3992 - val_loss: 0.4050\n",
      "Epoch 7/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3907 - val_loss: 0.3833\n",
      "Epoch 8/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3824 - val_loss: 0.4306\n",
      "Epoch 9/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3773 - val_loss: 0.3806\n",
      "Epoch 10/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3720 - val_loss: 0.3851\n",
      "Epoch 11/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3677 - val_loss: 0.3504\n",
      "Epoch 12/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3632 - val_loss: 0.3865\n",
      "Epoch 13/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3596 - val_loss: 0.4215\n",
      "Epoch 14/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3577 - val_loss: 0.3906\n",
      "Epoch 15/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3538 - val_loss: 0.4588\n",
      "Epoch 16/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3525 - val_loss: 0.3350\n",
      "Epoch 17/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3488 - val_loss: 0.4020\n",
      "Epoch 18/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.3459 - val_loss: 0.4451\n",
      "Epoch 19/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3445 - val_loss: 0.4082\n",
      "Epoch 20/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3429 - val_loss: 0.3690\n",
      "Epoch 21/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3413 - val_loss: 0.3326\n",
      "Epoch 22/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3389 - val_loss: 0.3475\n",
      "Epoch 23/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3376 - val_loss: 0.3381\n",
      "Epoch 24/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3352 - val_loss: 0.3333\n",
      "Epoch 25/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3340 - val_loss: 0.4147\n",
      "Epoch 26/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3325 - val_loss: 0.4207\n",
      "Epoch 27/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3319 - val_loss: 0.3328\n",
      "Epoch 28/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3302 - val_loss: 0.3656\n",
      "Epoch 29/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3288 - val_loss: 0.3180\n",
      "Epoch 30/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3266 - val_loss: 0.3994\n",
      "Epoch 31/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3269 - val_loss: 0.3344\n",
      "Epoch 32/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3252 - val_loss: 0.3281\n",
      "Epoch 33/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3226 - val_loss: 0.3339\n",
      "Epoch 34/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3233 - val_loss: 0.3283\n",
      "Epoch 35/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3208 - val_loss: 0.3834\n",
      "Epoch 36/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3209 - val_loss: 0.3260\n",
      "Epoch 37/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3187 - val_loss: 0.3149\n",
      "Epoch 38/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3175 - val_loss: 0.3136\n",
      "Epoch 39/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3159 - val_loss: 0.4119\n",
      "Epoch 40/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3158 - val_loss: 0.3252\n",
      "Epoch 41/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3144 - val_loss: 0.3174\n",
      "Epoch 42/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3129 - val_loss: 0.3911\n",
      "Epoch 43/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3131 - val_loss: 0.3167\n",
      "Epoch 44/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3117 - val_loss: 0.4159\n",
      "Epoch 45/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3113 - val_loss: 0.3414\n",
      "Epoch 46/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3078 - val_loss: 0.3597\n",
      "Epoch 47/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3089 - val_loss: 0.3062\n",
      "Epoch 48/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3074 - val_loss: 0.3133\n",
      "Epoch 49/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3052 - val_loss: 0.3587\n",
      "Epoch 50/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3060 - val_loss: 0.3166\n",
      "Epoch 51/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3038 - val_loss: 0.3144\n",
      "Epoch 52/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3038 - val_loss: 0.3803\n",
      "Epoch 53/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3024 - val_loss: 0.3023\n",
      "Epoch 54/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3012 - val_loss: 0.3206\n",
      "Epoch 55/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3006 - val_loss: 0.3453\n",
      "Epoch 56/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3008 - val_loss: 0.3081\n",
      "Epoch 57/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.2989 - val_loss: 0.3795\n",
      "Epoch 58/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.2982 - val_loss: 0.3208\n",
      "Epoch 59/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.2973 - val_loss: 0.3284\n",
      "Epoch 60/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.2957 - val_loss: 0.3058\n",
      "Epoch 61/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.2966 - val_loss: 0.3155\n",
      "Epoch 62/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.2947 - val_loss: 0.3156\n",
      "Epoch 63/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.2948 - val_loss: 0.3417\n",
      "3870/3870 [==============================] - 0s 15us/sample - loss: 0.3149\n",
      "7740/7740 [==============================] - 0s 16us/sample - loss: 0.2935\n",
      "[CV]  learning_rate=0.003284284607688981, n_hidden=2, n_neurons=75, total=  19.2s\n",
      "[CV] learning_rate=0.004038022680351922, n_hidden=2, n_neurons=69 ....\n",
      "Train on 7740 samples, validate on 3870 samples\n",
      "Epoch 1/100\n",
      "7740/7740 [==============================] - 0s 56us/sample - loss: 1.1144 - val_loss: 1.6181\n",
      "Epoch 2/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.5747 - val_loss: 1.2753\n",
      "Epoch 3/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.4971 - val_loss: 0.5712\n",
      "Epoch 4/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.4514 - val_loss: 0.5843\n",
      "Epoch 5/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.4202 - val_loss: 0.3850\n",
      "Epoch 6/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.4007 - val_loss: 0.3774\n",
      "Epoch 7/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3869 - val_loss: 0.4252\n",
      "Epoch 8/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3780 - val_loss: 0.3768\n",
      "Epoch 9/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3698 - val_loss: 0.3744\n",
      "Epoch 10/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3627 - val_loss: 0.3604\n",
      "Epoch 11/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3577 - val_loss: 0.4952\n",
      "Epoch 12/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3563 - val_loss: 0.4150\n",
      "Epoch 13/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3503 - val_loss: 0.3910\n",
      "Epoch 14/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3469 - val_loss: 0.3373\n",
      "Epoch 15/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3428 - val_loss: 0.3744\n",
      "Epoch 16/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3414 - val_loss: 0.3325\n",
      "Epoch 17/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3371 - val_loss: 0.4917\n",
      "Epoch 18/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3410 - val_loss: 0.6252\n",
      "Epoch 19/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3394 - val_loss: 1.0122\n",
      "Epoch 20/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3407 - val_loss: 0.9887\n",
      "Epoch 21/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3384 - val_loss: 0.7173\n",
      "Epoch 22/100\n",
      "7740/7740 [==============================] - 0s 43us/sample - loss: 0.3325 - val_loss: 0.3328\n",
      "Epoch 23/100\n",
      "7740/7740 [==============================] - 0s 45us/sample - loss: 0.3257 - val_loss: 0.3620\n",
      "Epoch 24/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3255 - val_loss: 0.3204\n",
      "Epoch 25/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3224 - val_loss: 0.3355\n",
      "Epoch 26/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3205 - val_loss: 0.3574\n",
      "Epoch 27/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3197 - val_loss: 0.3340\n",
      "Epoch 28/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3172 - val_loss: 0.3134\n",
      "Epoch 29/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3155 - val_loss: 0.3252\n",
      "Epoch 30/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3151 - val_loss: 0.3228\n",
      "Epoch 31/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3131 - val_loss: 0.3122\n",
      "Epoch 32/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3109 - val_loss: 0.3072\n",
      "Epoch 33/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3095 - val_loss: 0.4399\n",
      "Epoch 34/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3111 - val_loss: 0.5012\n",
      "Epoch 35/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3116 - val_loss: 0.4788\n",
      "Epoch 36/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3081 - val_loss: 0.3058\n",
      "Epoch 37/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3061 - val_loss: 0.3707\n",
      "Epoch 38/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3038 - val_loss: 0.3291\n",
      "Epoch 39/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3021 - val_loss: 0.3020\n",
      "Epoch 40/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3014 - val_loss: 0.3682\n",
      "Epoch 41/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3014 - val_loss: 0.3114\n",
      "Epoch 42/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2999 - val_loss: 0.3104\n",
      "Epoch 43/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2986 - val_loss: 0.3037\n",
      "Epoch 44/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2966 - val_loss: 0.3074\n",
      "Epoch 45/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2953 - val_loss: 0.3298\n",
      "Epoch 46/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2956 - val_loss: 0.4312\n",
      "Epoch 47/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2953 - val_loss: 0.4607\n",
      "Epoch 48/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2959 - val_loss: 0.3917\n",
      "Epoch 49/100\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2955 - val_loss: 0.3009\n",
      "Epoch 50/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2912 - val_loss: 0.3113\n",
      "Epoch 51/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2909 - val_loss: 0.3125\n",
      "Epoch 52/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2903 - val_loss: 0.3306\n",
      "Epoch 53/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2884 - val_loss: 0.2966\n",
      "Epoch 54/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2876 - val_loss: 0.3536\n",
      "Epoch 55/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2877 - val_loss: 0.4683\n",
      "Epoch 56/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2902 - val_loss: 0.4054\n",
      "Epoch 57/100\n",
      "7740/7740 [==============================] - 0s 42us/sample - loss: 0.2866 - val_loss: 0.3190\n",
      "Epoch 58/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.2854 - val_loss: 0.3752\n",
      "Epoch 59/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2839 - val_loss: 0.2895\n",
      "Epoch 60/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2828 - val_loss: 0.3034\n",
      "Epoch 61/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2829 - val_loss: 0.3833\n",
      "Epoch 62/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2839 - val_loss: 0.2852\n",
      "Epoch 63/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2806 - val_loss: 0.2916\n",
      "Epoch 64/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2787 - val_loss: 0.2897\n",
      "Epoch 65/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2794 - val_loss: 0.2838\n",
      "Epoch 66/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2785 - val_loss: 0.3772\n",
      "Epoch 67/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2787 - val_loss: 0.3277\n",
      "Epoch 68/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2771 - val_loss: 0.5036\n",
      "Epoch 69/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2791 - val_loss: 0.3969\n",
      "Epoch 70/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2765 - val_loss: 0.6491\n",
      "Epoch 71/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2857 - val_loss: 0.6069\n",
      "Epoch 72/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2822 - val_loss: 0.8506\n",
      "Epoch 73/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2840 - val_loss: 0.4954\n",
      "Epoch 74/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2785 - val_loss: 0.4551\n",
      "Epoch 75/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2754 - val_loss: 0.2963\n",
      "3870/3870 [==============================] - 0s 15us/sample - loss: 0.3199\n",
      "7740/7740 [==============================] - 0s 16us/sample - loss: 0.2753\n",
      "[CV]  learning_rate=0.004038022680351922, n_hidden=2, n_neurons=69, total=  23.2s\n",
      "[CV] learning_rate=0.004038022680351922, n_hidden=2, n_neurons=69 ....\n",
      "Train on 7740 samples, validate on 3870 samples\n",
      "Epoch 1/100\n",
      "7740/7740 [==============================] - 0s 55us/sample - loss: 1.1371 - val_loss: 1.3452\n",
      "Epoch 2/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.5859 - val_loss: 0.5920\n",
      "Epoch 3/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.5003 - val_loss: 0.4443\n",
      "Epoch 4/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.4476 - val_loss: 0.4072\n",
      "Epoch 5/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.4162 - val_loss: 0.3839\n",
      "Epoch 6/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3974 - val_loss: 0.3671\n",
      "Epoch 7/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3837 - val_loss: 0.3693\n",
      "Epoch 8/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3740 - val_loss: 0.3850\n",
      "Epoch 9/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3688 - val_loss: 0.4103\n",
      "Epoch 10/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3634 - val_loss: 0.4479\n",
      "Epoch 11/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3587 - val_loss: 0.5044\n",
      "Epoch 12/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3554 - val_loss: 0.5236\n",
      "Epoch 13/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3508 - val_loss: 0.5970\n",
      "Epoch 14/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3474 - val_loss: 0.6589\n",
      "Epoch 15/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3454 - val_loss: 0.6735\n",
      "Epoch 16/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3421 - val_loss: 0.7284\n",
      "3870/3870 [==============================] - 0s 15us/sample - loss: 0.3575\n",
      "7740/7740 [==============================] - 0s 16us/sample - loss: 0.3373\n",
      "[CV]  learning_rate=0.004038022680351922, n_hidden=2, n_neurons=69, total=   5.2s\n",
      "[CV] learning_rate=0.004038022680351922, n_hidden=2, n_neurons=69 ....\n",
      "Train on 7740 samples, validate on 3870 samples\n",
      "Epoch 1/100\n",
      "7740/7740 [==============================] - 0s 56us/sample - loss: 1.2970 - val_loss: 7.5366\n",
      "Epoch 2/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.6151 - val_loss: 2.4204\n",
      "Epoch 3/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.5010 - val_loss: 0.4373\n",
      "Epoch 4/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.4434 - val_loss: 0.4774\n",
      "Epoch 5/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.4185 - val_loss: 0.4510\n",
      "Epoch 6/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.4012 - val_loss: 0.4292\n",
      "Epoch 7/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3888 - val_loss: 0.3930\n",
      "Epoch 8/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3802 - val_loss: 0.4631\n",
      "Epoch 9/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3740 - val_loss: 0.3844\n",
      "Epoch 10/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3680 - val_loss: 0.4246\n",
      "Epoch 11/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3648 - val_loss: 0.4520\n",
      "Epoch 12/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3609 - val_loss: 0.4205\n",
      "Epoch 13/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3570 - val_loss: 0.3746\n",
      "Epoch 14/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3529 - val_loss: 0.3670\n",
      "Epoch 15/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3511 - val_loss: 0.3522\n",
      "Epoch 16/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3471 - val_loss: 0.4065\n",
      "Epoch 17/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3453 - val_loss: 0.3991\n",
      "Epoch 18/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3426 - val_loss: 0.3844\n",
      "Epoch 19/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3402 - val_loss: 0.3834\n",
      "Epoch 20/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3375 - val_loss: 0.4403\n",
      "Epoch 21/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3362 - val_loss: 0.3458\n",
      "Epoch 22/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3335 - val_loss: 0.3672\n",
      "Epoch 23/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3317 - val_loss: 0.3860\n",
      "Epoch 24/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3304 - val_loss: 0.3250\n",
      "Epoch 25/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3278 - val_loss: 0.3844\n",
      "Epoch 26/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3265 - val_loss: 0.4097\n",
      "Epoch 27/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3250 - val_loss: 0.3139\n",
      "Epoch 28/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3238 - val_loss: 0.3452\n",
      "Epoch 29/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3215 - val_loss: 0.3300\n",
      "Epoch 30/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3198 - val_loss: 0.3281\n",
      "Epoch 31/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3173 - val_loss: 0.3994\n",
      "Epoch 32/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3172 - val_loss: 0.3340\n",
      "Epoch 33/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3153 - val_loss: 0.3991\n",
      "Epoch 34/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.3152 - val_loss: 0.3153\n",
      "Epoch 35/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3128 - val_loss: 0.3425\n",
      "Epoch 36/100\n",
      "7740/7740 [==============================] - 0s 48us/sample - loss: 0.3115 - val_loss: 0.3047\n",
      "Epoch 37/100\n",
      "7740/7740 [==============================] - 0s 44us/sample - loss: 0.3098 - val_loss: 0.3154\n",
      "Epoch 38/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3085 - val_loss: 0.3139\n",
      "Epoch 39/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3074 - val_loss: 0.3774\n",
      "Epoch 40/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3071 - val_loss: 0.3249\n",
      "Epoch 41/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3071 - val_loss: 0.3420\n",
      "Epoch 42/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3026 - val_loss: 0.3788\n",
      "Epoch 43/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3027 - val_loss: 0.3213\n",
      "Epoch 44/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3014 - val_loss: 0.3695\n",
      "Epoch 45/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3001 - val_loss: 0.2987\n",
      "Epoch 46/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2987 - val_loss: 0.3171\n",
      "Epoch 47/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2981 - val_loss: 0.3631\n",
      "Epoch 48/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2975 - val_loss: 0.2914\n",
      "Epoch 49/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2959 - val_loss: 0.3695\n",
      "Epoch 50/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.2942 - val_loss: 0.2942\n",
      "Epoch 51/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2925 - val_loss: 0.3925\n",
      "Epoch 52/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2939 - val_loss: 0.2917\n",
      "Epoch 53/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2916 - val_loss: 0.3588\n",
      "Epoch 54/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.2899 - val_loss: 0.3232\n",
      "Epoch 55/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2914 - val_loss: 0.2941\n",
      "Epoch 56/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2878 - val_loss: 0.2984\n",
      "Epoch 57/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2871 - val_loss: 0.3576\n",
      "Epoch 58/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.2862 - val_loss: 0.3397\n",
      "3870/3870 [==============================] - 0s 15us/sample - loss: 0.3052\n",
      "7740/7740 [==============================] - 0s 17us/sample - loss: 0.2829\n",
      "[CV]  learning_rate=0.004038022680351922, n_hidden=2, n_neurons=69, total=  18.3s\n",
      "[CV] learning_rate=0.0006477779307312751, n_hidden=1, n_neurons=59 ...\n",
      "Train on 7740 samples, validate on 3870 samples\n",
      "Epoch 1/100\n",
      "7740/7740 [==============================] - 0s 51us/sample - loss: 2.7840 - val_loss: 3.1844\n",
      "Epoch 2/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 1.2596 - val_loss: 0.9812\n",
      "Epoch 3/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.8758 - val_loss: 0.7441\n",
      "Epoch 4/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.7312 - val_loss: 0.6454\n",
      "Epoch 5/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.6688 - val_loss: 0.6013\n",
      "Epoch 6/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.6341 - val_loss: 0.5979\n",
      "Epoch 7/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.6103 - val_loss: 0.5533\n",
      "Epoch 8/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.5899 - val_loss: 0.5331\n",
      "Epoch 9/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.5719 - val_loss: 0.5214\n",
      "Epoch 10/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.5560 - val_loss: 0.5139\n",
      "Epoch 11/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.5418 - val_loss: 0.4965\n",
      "Epoch 12/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.5286 - val_loss: 0.4943\n",
      "Epoch 13/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.5171 - val_loss: 0.4726\n",
      "Epoch 14/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.5061 - val_loss: 0.4608\n",
      "Epoch 15/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.4962 - val_loss: 0.4600\n",
      "Epoch 16/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.4876 - val_loss: 0.4459\n",
      "Epoch 17/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4794 - val_loss: 0.4467\n",
      "Epoch 18/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4722 - val_loss: 0.4322\n",
      "Epoch 19/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4656 - val_loss: 0.4305\n",
      "Epoch 20/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.4597 - val_loss: 0.4219\n",
      "Epoch 21/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.4540 - val_loss: 0.4187\n",
      "Epoch 22/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4492 - val_loss: 0.4126\n",
      "Epoch 23/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4442 - val_loss: 0.4112\n",
      "Epoch 24/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4402 - val_loss: 0.4057\n",
      "Epoch 25/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4363 - val_loss: 0.4048\n",
      "Epoch 26/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.4327 - val_loss: 0.4011\n",
      "Epoch 27/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4291 - val_loss: 0.3973\n",
      "Epoch 28/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4259 - val_loss: 0.3977\n",
      "Epoch 29/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.4230 - val_loss: 0.3935\n",
      "Epoch 30/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4201 - val_loss: 0.3995\n",
      "Epoch 31/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.4176 - val_loss: 0.3938\n",
      "Epoch 32/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4149 - val_loss: 0.3924\n",
      "Epoch 33/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.4125 - val_loss: 0.3842\n",
      "Epoch 34/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.4102 - val_loss: 0.3922\n",
      "Epoch 35/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4080 - val_loss: 0.3810\n",
      "Epoch 36/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4059 - val_loss: 0.3883\n",
      "Epoch 37/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4039 - val_loss: 0.3848\n",
      "Epoch 38/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4018 - val_loss: 0.3784\n",
      "Epoch 39/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4001 - val_loss: 0.3841\n",
      "Epoch 40/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3984 - val_loss: 0.3793\n",
      "Epoch 41/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3966 - val_loss: 0.3837\n",
      "Epoch 42/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3951 - val_loss: 0.3736\n",
      "Epoch 43/100\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3934 - val_loss: 0.3825\n",
      "Epoch 44/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3921 - val_loss: 0.3693\n",
      "Epoch 45/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3905 - val_loss: 0.3694\n",
      "Epoch 46/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3891 - val_loss: 0.3673\n",
      "Epoch 47/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3875 - val_loss: 0.3753\n",
      "Epoch 48/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3862 - val_loss: 0.3868\n",
      "Epoch 49/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3850 - val_loss: 0.3735\n",
      "Epoch 50/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3837 - val_loss: 0.3745\n",
      "Epoch 51/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3825 - val_loss: 0.3742\n",
      "Epoch 52/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3814 - val_loss: 0.3672\n",
      "Epoch 53/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3800 - val_loss: 0.3624\n",
      "Epoch 54/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3787 - val_loss: 0.3769\n",
      "Epoch 55/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3778 - val_loss: 0.3609\n",
      "Epoch 56/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3766 - val_loss: 0.3719\n",
      "Epoch 57/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3757 - val_loss: 0.3732\n",
      "Epoch 58/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3748 - val_loss: 0.3588\n",
      "Epoch 59/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3736 - val_loss: 0.3681\n",
      "Epoch 60/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3726 - val_loss: 0.3557\n",
      "Epoch 61/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3716 - val_loss: 0.3600\n",
      "Epoch 62/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3706 - val_loss: 0.3524\n",
      "Epoch 63/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3699 - val_loss: 0.3550\n",
      "Epoch 64/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3689 - val_loss: 0.3609\n",
      "Epoch 65/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3681 - val_loss: 0.3504\n",
      "Epoch 66/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3670 - val_loss: 0.3500\n",
      "Epoch 67/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3662 - val_loss: 0.3657\n",
      "Epoch 68/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3655 - val_loss: 0.3561\n",
      "Epoch 69/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3647 - val_loss: 0.3559\n",
      "Epoch 70/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3638 - val_loss: 0.3566\n",
      "Epoch 71/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3631 - val_loss: 0.3482\n",
      "Epoch 72/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3623 - val_loss: 0.3550\n",
      "Epoch 73/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3617 - val_loss: 0.3494\n",
      "Epoch 74/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3609 - val_loss: 0.3559\n",
      "Epoch 75/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3602 - val_loss: 0.3543\n",
      "Epoch 76/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3595 - val_loss: 0.3439\n",
      "Epoch 77/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3588 - val_loss: 0.3552\n",
      "Epoch 78/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3582 - val_loss: 0.3423\n",
      "Epoch 79/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3576 - val_loss: 0.3480\n",
      "Epoch 80/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3569 - val_loss: 0.3429\n",
      "Epoch 81/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3562 - val_loss: 0.3474\n",
      "Epoch 82/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3555 - val_loss: 0.3482\n",
      "Epoch 83/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3551 - val_loss: 0.3444\n",
      "Epoch 84/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3545 - val_loss: 0.3417\n",
      "Epoch 85/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3538 - val_loss: 0.3400\n",
      "Epoch 86/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3533 - val_loss: 0.3438\n",
      "Epoch 87/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3526 - val_loss: 0.3545\n",
      "Epoch 88/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3523 - val_loss: 0.3430\n",
      "Epoch 89/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3515 - val_loss: 0.3515\n",
      "Epoch 90/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3511 - val_loss: 0.3432\n",
      "Epoch 91/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3505 - val_loss: 0.3404\n",
      "Epoch 92/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3499 - val_loss: 0.3428\n",
      "Epoch 93/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3496 - val_loss: 0.3470\n",
      "Epoch 94/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3490 - val_loss: 0.3530\n",
      "Epoch 95/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3485 - val_loss: 0.3673\n",
      "3870/3870 [==============================] - 0s 13us/sample - loss: 0.3662\n",
      "7740/7740 [==============================] - 0s 15us/sample - loss: 0.3478\n",
      "[CV]  learning_rate=0.0006477779307312751, n_hidden=1, n_neurons=59, total=  27.3s\n",
      "[CV] learning_rate=0.0006477779307312751, n_hidden=1, n_neurons=59 ...\n",
      "Train on 7740 samples, validate on 3870 samples\n",
      "Epoch 1/100\n",
      "7740/7740 [==============================] - 0s 51us/sample - loss: 2.9243 - val_loss: 18.8442\n",
      "Epoch 2/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 1.2074 - val_loss: 16.0418\n",
      "Epoch 3/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.8573 - val_loss: 11.0685\n",
      "Epoch 4/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.7260 - val_loss: 7.3959\n",
      "Epoch 5/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.6690 - val_loss: 5.0236\n",
      "Epoch 6/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.6350 - val_loss: 3.4955\n",
      "Epoch 7/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.6096 - val_loss: 2.3247\n",
      "Epoch 8/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.5878 - val_loss: 1.5633\n",
      "Epoch 9/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.5687 - val_loss: 1.0776\n",
      "Epoch 10/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.5517 - val_loss: 0.7538\n",
      "Epoch 11/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.5365 - val_loss: 0.5741\n",
      "Epoch 12/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.5228 - val_loss: 0.5018\n",
      "Epoch 13/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.5106 - val_loss: 0.4949\n",
      "Epoch 14/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4997 - val_loss: 0.5297\n",
      "Epoch 15/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4899 - val_loss: 0.5825\n",
      "Epoch 16/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4812 - val_loss: 0.6560\n",
      "Epoch 17/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.4736 - val_loss: 0.7104\n",
      "Epoch 18/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.4666 - val_loss: 0.7667\n",
      "Epoch 19/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4604 - val_loss: 0.7889\n",
      "Epoch 20/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4546 - val_loss: 0.8336\n",
      "Epoch 21/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4495 - val_loss: 0.8446\n",
      "Epoch 22/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4448 - val_loss: 0.8488\n",
      "Epoch 23/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4403 - val_loss: 0.8735\n",
      "3870/3870 [==============================] - 0s 14us/sample - loss: 0.4598\n",
      "7740/7740 [==============================] - 0s 15us/sample - loss: 0.4382\n",
      "[CV]  learning_rate=0.0006477779307312751, n_hidden=1, n_neurons=59, total=   6.9s\n",
      "[CV] learning_rate=0.0006477779307312751, n_hidden=1, n_neurons=59 ...\n",
      "Train on 7740 samples, validate on 3870 samples\n",
      "Epoch 1/100\n",
      "7740/7740 [==============================] - 0s 51us/sample - loss: 2.6866 - val_loss: 8.0515\n",
      "Epoch 2/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 1.0829 - val_loss: 1.5668\n",
      "Epoch 3/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.7425 - val_loss: 0.6838\n",
      "Epoch 4/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.6664 - val_loss: 0.6391\n",
      "Epoch 5/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.6360 - val_loss: 0.6036\n",
      "Epoch 6/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.6139 - val_loss: 0.5849\n",
      "Epoch 7/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.5957 - val_loss: 0.5632\n",
      "Epoch 8/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.5793 - val_loss: 0.5505\n",
      "Epoch 9/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.5648 - val_loss: 0.5358\n",
      "Epoch 10/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.5513 - val_loss: 0.5225\n",
      "Epoch 11/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.5389 - val_loss: 0.5139\n",
      "Epoch 12/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.5273 - val_loss: 0.4968\n",
      "Epoch 13/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.5164 - val_loss: 0.4871\n",
      "Epoch 14/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.5063 - val_loss: 0.4761\n",
      "Epoch 15/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.4972 - val_loss: 0.4666\n",
      "Epoch 16/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.4882 - val_loss: 0.4606\n",
      "Epoch 17/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4801 - val_loss: 0.4513\n",
      "Epoch 18/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4728 - val_loss: 0.4440\n",
      "Epoch 19/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4654 - val_loss: 0.4390\n",
      "Epoch 20/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4594 - val_loss: 0.4303\n",
      "Epoch 21/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.4531 - val_loss: 0.4275\n",
      "Epoch 22/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4476 - val_loss: 0.4231\n",
      "Epoch 23/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.4425 - val_loss: 0.4187\n",
      "Epoch 24/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4376 - val_loss: 0.4136\n",
      "Epoch 25/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.4333 - val_loss: 0.4054\n",
      "Epoch 26/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4293 - val_loss: 0.4118\n",
      "Epoch 27/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.4256 - val_loss: 0.4052\n",
      "Epoch 28/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4221 - val_loss: 0.3981\n",
      "Epoch 29/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4190 - val_loss: 0.3979\n",
      "Epoch 30/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4159 - val_loss: 0.3940\n",
      "Epoch 31/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.4132 - val_loss: 0.3969\n",
      "Epoch 32/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.4106 - val_loss: 0.3897\n",
      "Epoch 33/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.4080 - val_loss: 0.3888\n",
      "Epoch 34/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4058 - val_loss: 0.3832\n",
      "Epoch 35/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4036 - val_loss: 0.3949\n",
      "Epoch 36/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.4017 - val_loss: 0.3838\n",
      "Epoch 37/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3996 - val_loss: 0.3866\n",
      "Epoch 38/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3978 - val_loss: 0.3872\n",
      "Epoch 39/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3961 - val_loss: 0.3955\n",
      "Epoch 40/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3944 - val_loss: 0.3855\n",
      "Epoch 41/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3929 - val_loss: 0.3826\n",
      "Epoch 42/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3914 - val_loss: 0.3714\n",
      "Epoch 43/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3896 - val_loss: 0.3914\n",
      "Epoch 44/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3885 - val_loss: 0.3794\n",
      "Epoch 45/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3870 - val_loss: 0.3651\n",
      "Epoch 46/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3854 - val_loss: 0.3952\n",
      "Epoch 47/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3843 - val_loss: 0.3603\n",
      "Epoch 48/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3831 - val_loss: 0.3862\n",
      "Epoch 49/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3819 - val_loss: 0.3959\n",
      "Epoch 50/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3807 - val_loss: 0.3726\n",
      "Epoch 51/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3797 - val_loss: 0.3636\n",
      "Epoch 52/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3783 - val_loss: 0.3939\n",
      "Epoch 53/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3776 - val_loss: 0.3632\n",
      "Epoch 54/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3760 - val_loss: 0.3598\n",
      "Epoch 55/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3753 - val_loss: 0.3850\n",
      "Epoch 56/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3745 - val_loss: 0.3665\n",
      "Epoch 57/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3738 - val_loss: 0.3574\n",
      "Epoch 58/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3724 - val_loss: 0.3859\n",
      "Epoch 59/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3720 - val_loss: 0.3780\n",
      "Epoch 60/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3710 - val_loss: 0.3745\n",
      "Epoch 61/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3703 - val_loss: 0.3562\n",
      "Epoch 62/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3694 - val_loss: 0.3705\n",
      "Epoch 63/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3685 - val_loss: 0.3532\n",
      "Epoch 64/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3675 - val_loss: 0.3531\n",
      "Epoch 65/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3670 - val_loss: 0.3522\n",
      "Epoch 66/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3660 - val_loss: 0.3824\n",
      "Epoch 67/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3654 - val_loss: 0.3472\n",
      "Epoch 68/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3649 - val_loss: 0.3528\n",
      "Epoch 69/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3639 - val_loss: 0.3488\n",
      "Epoch 70/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3634 - val_loss: 0.3521\n",
      "Epoch 71/100\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3628 - val_loss: 0.3633\n",
      "Epoch 72/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3621 - val_loss: 0.3724\n",
      "Epoch 73/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3615 - val_loss: 0.3504\n",
      "Epoch 74/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3606 - val_loss: 0.3757\n",
      "Epoch 75/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3601 - val_loss: 0.3428\n",
      "Epoch 76/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3597 - val_loss: 0.3600\n",
      "Epoch 77/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3590 - val_loss: 0.3468\n",
      "Epoch 78/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3584 - val_loss: 0.3391\n",
      "Epoch 79/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3576 - val_loss: 0.3450\n",
      "Epoch 80/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3571 - val_loss: 0.3658\n",
      "Epoch 81/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3566 - val_loss: 0.3612\n",
      "Epoch 82/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3561 - val_loss: 0.3525\n",
      "Epoch 83/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3554 - val_loss: 0.3548\n",
      "Epoch 84/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3550 - val_loss: 0.3449\n",
      "Epoch 85/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3542 - val_loss: 0.3533\n",
      "Epoch 86/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3542 - val_loss: 0.3527\n",
      "Epoch 87/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3534 - val_loss: 0.3655\n",
      "Epoch 88/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3530 - val_loss: 0.3569\n",
      "3870/3870 [==============================] - 0s 14us/sample - loss: 0.3494\n",
      "7740/7740 [==============================] - 0s 14us/sample - loss: 0.3521\n",
      "[CV]  learning_rate=0.0006477779307312751, n_hidden=1, n_neurons=59, total=  24.8s\n",
      "[CV] learning_rate=0.0032123503761513073, n_hidden=0, n_neurons=87 ...\n",
      "Train on 7740 samples, validate on 3870 samples\n",
      "Epoch 1/100\n",
      "7740/7740 [==============================] - 0s 46us/sample - loss: 1.3665 - val_loss: 30.0074\n",
      "Epoch 2/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 1.0455 - val_loss: 22.4981\n",
      "Epoch 3/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.8411 - val_loss: 0.5801\n",
      "Epoch 4/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.5291 - val_loss: 0.5369\n",
      "Epoch 5/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.4956 - val_loss: 0.5124\n",
      "Epoch 6/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4722 - val_loss: 0.4677\n",
      "Epoch 7/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4550 - val_loss: 0.4384\n",
      "Epoch 8/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4421 - val_loss: 0.4157\n",
      "Epoch 9/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4312 - val_loss: 0.3990\n",
      "Epoch 10/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.4227 - val_loss: 0.3904\n",
      "Epoch 11/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4157 - val_loss: 0.3844\n",
      "Epoch 12/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4095 - val_loss: 0.3846\n",
      "Epoch 13/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4053 - val_loss: 0.3783\n",
      "Epoch 14/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4016 - val_loss: 0.3811\n",
      "Epoch 15/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3976 - val_loss: 0.3855\n",
      "Epoch 16/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3946 - val_loss: 0.3793\n",
      "Epoch 17/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3917 - val_loss: 0.3790\n",
      "Epoch 18/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3892 - val_loss: 0.3795\n",
      "Epoch 19/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3868 - val_loss: 0.3751\n",
      "Epoch 20/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3844 - val_loss: 0.3905\n",
      "Epoch 21/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3830 - val_loss: 0.3758\n",
      "Epoch 22/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3806 - val_loss: 0.3810\n",
      "Epoch 23/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3790 - val_loss: 0.3810\n",
      "Epoch 24/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3773 - val_loss: 0.3796\n",
      "Epoch 25/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3763 - val_loss: 0.3838\n",
      "Epoch 26/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3739 - val_loss: 0.3877\n",
      "Epoch 27/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3731 - val_loss: 0.3776\n",
      "Epoch 28/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3716 - val_loss: 0.3778\n",
      "Epoch 29/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3702 - val_loss: 0.3812\n",
      "3870/3870 [==============================] - 0s 13us/sample - loss: 0.3857\n",
      "7740/7740 [==============================] - 0s 13us/sample - loss: 0.3677\n",
      "[CV]  learning_rate=0.0032123503761513073, n_hidden=0, n_neurons=87, total=   7.9s\n",
      "[CV] learning_rate=0.0032123503761513073, n_hidden=0, n_neurons=87 ...\n",
      "Train on 7740 samples, validate on 3870 samples\n",
      "Epoch 1/100\n",
      "7740/7740 [==============================] - 0s 46us/sample - loss: 1.3253 - val_loss: 1.7528\n",
      "Epoch 2/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.6961 - val_loss: 0.6533\n",
      "Epoch 3/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.6193 - val_loss: 1.3104\n",
      "Epoch 4/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.5629 - val_loss: 2.2476\n",
      "Epoch 5/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.5220 - val_loss: 2.6183\n",
      "Epoch 6/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4918 - val_loss: 2.7712\n",
      "Epoch 7/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.4692 - val_loss: 2.4734\n",
      "Epoch 8/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4537 - val_loss: 2.0569\n",
      "Epoch 9/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4415 - val_loss: 1.6704\n",
      "Epoch 10/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4329 - val_loss: 1.3330\n",
      "Epoch 11/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4249 - val_loss: 0.9791\n",
      "Epoch 12/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4199 - val_loss: 0.8599\n",
      "3870/3870 [==============================] - 0s 13us/sample - loss: 0.4479\n",
      "7740/7740 [==============================] - 0s 14us/sample - loss: 0.4163\n",
      "[CV]  learning_rate=0.0032123503761513073, n_hidden=0, n_neurons=87, total=   3.5s\n",
      "[CV] learning_rate=0.0032123503761513073, n_hidden=0, n_neurons=87 ...\n",
      "Train on 7740 samples, validate on 3870 samples\n",
      "Epoch 1/100\n",
      "7740/7740 [==============================] - 0s 47us/sample - loss: 1.5445 - val_loss: 15.2564\n",
      "Epoch 2/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.9030 - val_loss: 5.7446\n",
      "Epoch 3/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.6692 - val_loss: 0.6443\n",
      "Epoch 4/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.5637 - val_loss: 0.5236\n",
      "Epoch 5/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.5269 - val_loss: 0.4840\n",
      "Epoch 6/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4986 - val_loss: 0.4984\n",
      "Epoch 7/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4779 - val_loss: 0.4490\n",
      "Epoch 8/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4620 - val_loss: 0.4321\n",
      "Epoch 9/100\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4504 - val_loss: 0.4292\n",
      "Epoch 10/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4414 - val_loss: 0.4454\n",
      "Epoch 11/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4341 - val_loss: 0.4018\n",
      "Epoch 12/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.4281 - val_loss: 0.4089\n",
      "Epoch 13/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4226 - val_loss: 0.4120\n",
      "Epoch 14/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4179 - val_loss: 0.4455\n",
      "Epoch 15/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4138 - val_loss: 0.4541\n",
      "Epoch 16/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4108 - val_loss: 0.4184\n",
      "Epoch 17/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4075 - val_loss: 0.4369\n",
      "Epoch 18/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4044 - val_loss: 0.3751\n",
      "Epoch 19/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4026 - val_loss: 0.3771\n",
      "Epoch 20/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3994 - val_loss: 0.4666\n",
      "Epoch 21/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3988 - val_loss: 0.4131\n",
      "Epoch 22/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3952 - val_loss: 0.3811\n",
      "Epoch 23/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3930 - val_loss: 0.3736\n",
      "Epoch 24/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3915 - val_loss: 0.3913\n",
      "Epoch 25/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3897 - val_loss: 0.3709\n",
      "Epoch 26/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3880 - val_loss: 0.3942\n",
      "Epoch 27/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3863 - val_loss: 0.3592\n",
      "Epoch 28/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3848 - val_loss: 0.3652\n",
      "Epoch 29/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3830 - val_loss: 0.3596\n",
      "Epoch 30/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3815 - val_loss: 0.4866\n",
      "Epoch 31/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3811 - val_loss: 0.3524\n",
      "Epoch 32/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3783 - val_loss: 0.5189\n",
      "Epoch 33/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3787 - val_loss: 0.3859\n",
      "Epoch 34/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3761 - val_loss: 0.3516\n",
      "Epoch 35/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3747 - val_loss: 0.5188\n",
      "Epoch 36/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3748 - val_loss: 0.3542\n",
      "Epoch 37/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3734 - val_loss: 0.5031\n",
      "Epoch 38/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3737 - val_loss: 0.3494\n",
      "Epoch 39/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3713 - val_loss: 0.3880\n",
      "Epoch 40/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3697 - val_loss: 0.3591\n",
      "Epoch 41/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3686 - val_loss: 0.4488\n",
      "Epoch 42/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3683 - val_loss: 0.3543\n",
      "Epoch 43/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3671 - val_loss: 0.3664\n",
      "Epoch 44/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3662 - val_loss: 0.3420\n",
      "Epoch 45/100\n",
      "7740/7740 [==============================] - 0s 32us/sample - loss: 0.3657 - val_loss: 0.3827\n",
      "Epoch 46/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3648 - val_loss: 0.3587\n",
      "Epoch 47/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3640 - val_loss: 0.3558\n",
      "Epoch 48/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3629 - val_loss: 0.5029\n",
      "Epoch 49/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3633 - val_loss: 0.3404\n",
      "Epoch 50/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3614 - val_loss: 0.5297\n",
      "Epoch 51/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3627 - val_loss: 0.3581\n",
      "Epoch 52/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3605 - val_loss: 0.4724\n",
      "Epoch 53/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3612 - val_loss: 0.3414\n",
      "Epoch 54/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3593 - val_loss: 0.3880\n",
      "Epoch 55/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3583 - val_loss: 0.3962\n",
      "Epoch 56/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3576 - val_loss: 0.4286\n",
      "Epoch 57/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3569 - val_loss: 0.3945\n",
      "Epoch 58/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3561 - val_loss: 0.3448\n",
      "Epoch 59/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3563 - val_loss: 0.4321\n",
      "3870/3870 [==============================] - 0s 13us/sample - loss: 0.3558\n",
      "7740/7740 [==============================] - 0s 14us/sample - loss: 0.3563\n",
      "[CV]  learning_rate=0.0032123503761513073, n_hidden=0, n_neurons=87, total=  15.7s\n",
      "[CV] learning_rate=0.004576909498448105, n_hidden=2, n_neurons=87 ....\n",
      "Train on 7740 samples, validate on 3870 samples\n",
      "Epoch 1/100\n",
      "7740/7740 [==============================] - 0s 58us/sample - loss: 1.0841 - val_loss: 1.6728\n",
      "Epoch 2/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.5127 - val_loss: 3.2044\n",
      "Epoch 3/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.4594 - val_loss: 0.4278\n",
      "Epoch 4/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.4003 - val_loss: 0.4229\n",
      "Epoch 5/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.3793 - val_loss: 0.4236\n",
      "Epoch 6/100\n",
      "7740/7740 [==============================] - 0s 42us/sample - loss: 0.3653 - val_loss: 0.4169\n",
      "Epoch 7/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.3580 - val_loss: 0.4311\n",
      "Epoch 8/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.3524 - val_loss: 0.3587\n",
      "Epoch 9/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.3437 - val_loss: 0.4140\n",
      "Epoch 10/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.3405 - val_loss: 0.3898\n",
      "Epoch 11/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.3361 - val_loss: 0.3298\n",
      "Epoch 12/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.3320 - val_loss: 0.3523\n",
      "Epoch 13/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.3285 - val_loss: 0.3785\n",
      "Epoch 14/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3275 - val_loss: 0.3206\n",
      "Epoch 15/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.3239 - val_loss: 0.3414\n",
      "Epoch 16/100\n",
      "7740/7740 [==============================] - 0s 42us/sample - loss: 0.3224 - val_loss: 0.3518\n",
      "Epoch 17/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.3189 - val_loss: 0.3833\n",
      "Epoch 18/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.3168 - val_loss: 0.3239\n",
      "Epoch 19/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.3165 - val_loss: 0.4082\n",
      "Epoch 20/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.3133 - val_loss: 0.3160\n",
      "Epoch 21/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.3114 - val_loss: 0.3317\n",
      "Epoch 22/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.3098 - val_loss: 0.3171\n",
      "Epoch 23/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.3073 - val_loss: 0.3239\n",
      "Epoch 24/100\n",
      "7740/7740 [==============================] - 0s 44us/sample - loss: 0.3062 - val_loss: 0.3073\n",
      "Epoch 25/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.3044 - val_loss: 0.3243\n",
      "Epoch 26/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.3034 - val_loss: 0.3801\n",
      "Epoch 27/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.3025 - val_loss: 0.3579\n",
      "Epoch 28/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.3026 - val_loss: 0.3355\n",
      "Epoch 29/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.3003 - val_loss: 0.3455\n",
      "Epoch 30/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.2983 - val_loss: 0.3128\n",
      "Epoch 31/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.2955 - val_loss: 0.3165\n",
      "Epoch 32/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.2940 - val_loss: 0.3149\n",
      "Epoch 33/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.2909 - val_loss: 0.3912\n",
      "Epoch 34/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.2930 - val_loss: 0.3209\n",
      "3870/3870 [==============================] - 0s 16us/sample - loss: 0.3271\n",
      "7740/7740 [==============================] - 0s 17us/sample - loss: 0.2891\n",
      "[CV]  learning_rate=0.004576909498448105, n_hidden=2, n_neurons=87, total=  11.3s\n",
      "[CV] learning_rate=0.004576909498448105, n_hidden=2, n_neurons=87 ....\n",
      "Train on 7740 samples, validate on 3870 samples\n",
      "Epoch 1/100\n",
      "7740/7740 [==============================] - 0s 58us/sample - loss: 1.0489 - val_loss: 1.0151\n",
      "Epoch 2/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.5291 - val_loss: 0.5506\n",
      "Epoch 3/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.4577 - val_loss: 0.4543\n",
      "Epoch 4/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.4166 - val_loss: 0.4211\n",
      "Epoch 5/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.3932 - val_loss: 0.4991\n",
      "Epoch 6/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.3774 - val_loss: 0.6786\n",
      "Epoch 7/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.3659 - val_loss: 0.9418\n",
      "Epoch 8/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.3593 - val_loss: 0.8760\n",
      "Epoch 9/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.3531 - val_loss: 0.9404\n",
      "Epoch 10/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.3499 - val_loss: 1.0715\n",
      "Epoch 11/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.3454 - val_loss: 0.9127\n",
      "Epoch 12/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.3412 - val_loss: 1.0588\n",
      "Epoch 13/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.3385 - val_loss: 0.9105\n",
      "Epoch 14/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.3345 - val_loss: 0.9548\n",
      "3870/3870 [==============================] - 0s 16us/sample - loss: 0.3628\n",
      "7740/7740 [==============================] - 0s 17us/sample - loss: 0.3311\n",
      "[CV]  learning_rate=0.004576909498448105, n_hidden=2, n_neurons=87, total=   4.9s\n",
      "[CV] learning_rate=0.004576909498448105, n_hidden=2, n_neurons=87 ....\n",
      "Train on 7740 samples, validate on 3870 samples\n",
      "Epoch 1/100\n",
      "7740/7740 [==============================] - 0s 58us/sample - loss: 0.9595 - val_loss: 10.8814\n",
      "Epoch 2/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.6306 - val_loss: 2.7405\n",
      "Epoch 3/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.4783 - val_loss: 0.4214\n",
      "Epoch 4/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.4204 - val_loss: 0.3964\n",
      "Epoch 5/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3979 - val_loss: 0.4372\n",
      "Epoch 6/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3852 - val_loss: 0.4389\n",
      "Epoch 7/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3759 - val_loss: 0.4287\n",
      "Epoch 8/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3691 - val_loss: 0.4075\n",
      "Epoch 9/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3640 - val_loss: 0.4152\n",
      "Epoch 10/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3600 - val_loss: 0.4284\n",
      "Epoch 11/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3548 - val_loss: 0.3699\n",
      "Epoch 12/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3526 - val_loss: 0.4052\n",
      "Epoch 13/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3497 - val_loss: 0.3468\n",
      "Epoch 14/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3462 - val_loss: 0.3865\n",
      "Epoch 15/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3427 - val_loss: 0.3764\n",
      "Epoch 16/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3404 - val_loss: 0.3697\n",
      "Epoch 17/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3379 - val_loss: 0.3259\n",
      "Epoch 18/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3358 - val_loss: 0.3488\n",
      "Epoch 19/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3335 - val_loss: 0.3473\n",
      "Epoch 20/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3308 - val_loss: 0.3886\n",
      "Epoch 21/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3314 - val_loss: 0.3202\n",
      "Epoch 22/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3279 - val_loss: 0.3620\n",
      "Epoch 23/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3263 - val_loss: 0.3380\n",
      "Epoch 24/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3246 - val_loss: 0.4124\n",
      "Epoch 25/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3234 - val_loss: 0.3171\n",
      "Epoch 26/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3224 - val_loss: 0.3410\n",
      "Epoch 27/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3195 - val_loss: 0.3358\n",
      "Epoch 28/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3169 - val_loss: 0.3893\n",
      "Epoch 29/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3175 - val_loss: 0.3208\n",
      "Epoch 30/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3168 - val_loss: 0.3512\n",
      "Epoch 31/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3147 - val_loss: 0.3236\n",
      "Epoch 32/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3111 - val_loss: 0.3870\n",
      "Epoch 33/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3105 - val_loss: 0.4080\n",
      "Epoch 34/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.3108 - val_loss: 0.3167\n",
      "Epoch 35/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3102 - val_loss: 0.3398\n",
      "Epoch 36/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.3071 - val_loss: 0.3612\n",
      "Epoch 37/100\n",
      "7740/7740 [==============================] - 0s 42us/sample - loss: 0.3062 - val_loss: 0.3460\n",
      "Epoch 38/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3046 - val_loss: 0.4104\n",
      "Epoch 39/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3079 - val_loss: 0.2997\n",
      "Epoch 40/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3056 - val_loss: 0.3803\n",
      "Epoch 41/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3041 - val_loss: 0.3085\n",
      "Epoch 42/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3026 - val_loss: 0.3959\n",
      "Epoch 43/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.3006 - val_loss: 0.3498\n",
      "Epoch 44/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.2981 - val_loss: 0.3024\n",
      "Epoch 45/100\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.2986 - val_loss: 0.3475\n",
      "Epoch 46/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.2949 - val_loss: 0.3191\n",
      "Epoch 47/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.2946 - val_loss: 0.3201\n",
      "Epoch 48/100\n",
      "7740/7740 [==============================] - 0s 40us/sample - loss: 0.2937 - val_loss: 0.3001\n",
      "Epoch 49/100\n",
      "7740/7740 [==============================] - 0s 41us/sample - loss: 0.2929 - val_loss: 0.3431\n",
      "3870/3870 [==============================] - 0s 15us/sample - loss: 0.3356\n",
      "7740/7740 [==============================] - 0s 16us/sample - loss: 0.3133\n",
      "[CV]  learning_rate=0.004576909498448105, n_hidden=2, n_neurons=87, total=  15.6s\n",
      "[CV] learning_rate=0.0011714615165291644, n_hidden=0, n_neurons=56 ...\n",
      "Train on 7740 samples, validate on 3870 samples\n",
      "Epoch 1/100\n",
      "7740/7740 [==============================] - 0s 46us/sample - loss: 2.4510 - val_loss: 5.1025\n",
      "Epoch 2/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.9274 - val_loss: 0.8434\n",
      "Epoch 3/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.7575 - val_loss: 0.7224\n",
      "Epoch 4/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.7072 - val_loss: 0.6691\n",
      "Epoch 5/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.6692 - val_loss: 0.6309\n",
      "Epoch 6/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.6383 - val_loss: 0.6009\n",
      "Epoch 7/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.6099 - val_loss: 0.5716\n",
      "Epoch 8/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.5841 - val_loss: 0.5599\n",
      "Epoch 9/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.5598 - val_loss: 0.5321\n",
      "Epoch 10/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.5414 - val_loss: 0.5070\n",
      "Epoch 11/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.5225 - val_loss: 0.5337\n",
      "Epoch 12/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.5066 - val_loss: 0.4755\n",
      "Epoch 13/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.4927 - val_loss: 0.4675\n",
      "Epoch 14/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.4805 - val_loss: 0.4664\n",
      "Epoch 15/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.4693 - val_loss: 0.4675\n",
      "Epoch 16/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.4607 - val_loss: 0.4314\n",
      "Epoch 17/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.4532 - val_loss: 0.4594\n",
      "Epoch 18/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.4460 - val_loss: 0.4296\n",
      "Epoch 19/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.4400 - val_loss: 0.4329\n",
      "Epoch 20/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.4350 - val_loss: 0.4289\n",
      "Epoch 21/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.4304 - val_loss: 0.4344\n",
      "Epoch 22/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.4264 - val_loss: 0.4139\n",
      "Epoch 23/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.4227 - val_loss: 0.4219\n",
      "Epoch 24/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.4191 - val_loss: 0.4107\n",
      "Epoch 25/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.4165 - val_loss: 0.4236\n",
      "Epoch 26/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.4137 - val_loss: 0.4210\n",
      "Epoch 27/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.4110 - val_loss: 0.4112\n",
      "Epoch 28/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.4091 - val_loss: 0.4074\n",
      "Epoch 29/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.4068 - val_loss: 0.4051\n",
      "Epoch 30/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.4047 - val_loss: 0.3898\n",
      "Epoch 31/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.4025 - val_loss: 0.4200\n",
      "Epoch 32/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.4012 - val_loss: 0.3944\n",
      "Epoch 33/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3991 - val_loss: 0.3930\n",
      "Epoch 34/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3976 - val_loss: 0.4333\n",
      "Epoch 35/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3957 - val_loss: 0.3851\n",
      "Epoch 36/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3948 - val_loss: 0.3853\n",
      "Epoch 37/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3930 - val_loss: 0.3914\n",
      "Epoch 38/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3915 - val_loss: 0.4464\n",
      "Epoch 39/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3907 - val_loss: 0.3995\n",
      "Epoch 40/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3896 - val_loss: 0.4028\n",
      "Epoch 41/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3882 - val_loss: 0.3802\n",
      "Epoch 42/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3859 - val_loss: 0.3964\n",
      "Epoch 43/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3854 - val_loss: 0.4256\n",
      "Epoch 44/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3844 - val_loss: 0.3978\n",
      "Epoch 45/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3836 - val_loss: 0.3725\n",
      "Epoch 46/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3819 - val_loss: 0.4094\n",
      "Epoch 47/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3822 - val_loss: 0.3933\n",
      "Epoch 48/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3808 - val_loss: 0.3953\n",
      "Epoch 49/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3797 - val_loss: 0.4138\n",
      "Epoch 50/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3788 - val_loss: 0.3889\n",
      "Epoch 51/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3782 - val_loss: 0.3666\n",
      "Epoch 52/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3773 - val_loss: 0.4196\n",
      "Epoch 53/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3768 - val_loss: 0.3661\n",
      "Epoch 54/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3759 - val_loss: 0.3684\n",
      "Epoch 55/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3753 - val_loss: 0.3827\n",
      "Epoch 56/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3739 - val_loss: 0.3700\n",
      "Epoch 57/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3738 - val_loss: 0.4040\n",
      "Epoch 58/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3734 - val_loss: 0.4116\n",
      "Epoch 59/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3726 - val_loss: 0.3979\n",
      "Epoch 60/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3716 - val_loss: 0.3631\n",
      "Epoch 61/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3714 - val_loss: 0.4177\n",
      "Epoch 62/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3709 - val_loss: 0.3661\n",
      "Epoch 63/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3700 - val_loss: 0.4091\n",
      "Epoch 64/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3696 - val_loss: 0.3683\n",
      "Epoch 65/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3692 - val_loss: 0.3736\n",
      "Epoch 66/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3687 - val_loss: 0.3813\n",
      "Epoch 67/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3679 - val_loss: 0.3978\n",
      "Epoch 68/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3676 - val_loss: 0.3629\n",
      "Epoch 69/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3667 - val_loss: 0.3627\n",
      "Epoch 70/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3669 - val_loss: 0.3584\n",
      "Epoch 71/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3658 - val_loss: 0.3676\n",
      "Epoch 72/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3658 - val_loss: 0.3675\n",
      "Epoch 73/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3650 - val_loss: 0.4163\n",
      "Epoch 74/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3647 - val_loss: 0.3869\n",
      "Epoch 75/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3644 - val_loss: 0.3578\n",
      "Epoch 76/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3634 - val_loss: 0.4146\n",
      "Epoch 77/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3635 - val_loss: 0.3891\n",
      "Epoch 78/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3629 - val_loss: 0.3620\n",
      "Epoch 79/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3623 - val_loss: 0.3643\n",
      "Epoch 80/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3620 - val_loss: 0.3540\n",
      "Epoch 81/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3616 - val_loss: 0.4140\n",
      "Epoch 82/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3612 - val_loss: 0.3953\n",
      "Epoch 83/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3608 - val_loss: 0.4081\n",
      "Epoch 84/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3603 - val_loss: 0.3990\n",
      "Epoch 85/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3602 - val_loss: 0.3479\n",
      "Epoch 86/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3590 - val_loss: 0.4146\n",
      "Epoch 87/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3596 - val_loss: 0.3958\n",
      "Epoch 88/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3588 - val_loss: 0.3643\n",
      "Epoch 89/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3581 - val_loss: 0.3519\n",
      "Epoch 90/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3577 - val_loss: 0.4248\n",
      "Epoch 91/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3577 - val_loss: 0.3980\n",
      "Epoch 92/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3572 - val_loss: 0.3431\n",
      "Epoch 93/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3568 - val_loss: 0.4402\n",
      "Epoch 94/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3570 - val_loss: 0.3865\n",
      "Epoch 95/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3560 - val_loss: 0.3469\n",
      "Epoch 96/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3558 - val_loss: 0.3514\n",
      "Epoch 97/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3556 - val_loss: 0.4285\n",
      "Epoch 98/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3553 - val_loss: 0.3403\n",
      "Epoch 99/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3549 - val_loss: 0.3584\n",
      "Epoch 100/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3539 - val_loss: 0.3579\n",
      "3870/3870 [==============================] - 0s 12us/sample - loss: 0.3695\n",
      "7740/7740 [==============================] - 0s 14us/sample - loss: 0.3534\n",
      "[CV]  learning_rate=0.0011714615165291644, n_hidden=0, n_neurons=56, total=  26.8s\n",
      "[CV] learning_rate=0.0011714615165291644, n_hidden=0, n_neurons=56 ...\n",
      "Train on 7740 samples, validate on 3870 samples\n",
      "Epoch 1/100\n",
      "7740/7740 [==============================] - 0s 46us/sample - loss: 2.2833 - val_loss: 9.3171\n",
      "Epoch 2/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.8871 - val_loss: 6.2420\n",
      "Epoch 3/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.7421 - val_loss: 3.4939\n",
      "Epoch 4/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.6852 - val_loss: 1.9102\n",
      "Epoch 5/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.6445 - val_loss: 1.0602\n",
      "Epoch 6/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.6116 - val_loss: 0.6847\n",
      "Epoch 7/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.5845 - val_loss: 0.5604\n",
      "Epoch 8/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.5618 - val_loss: 0.5700\n",
      "Epoch 9/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.5430 - val_loss: 0.6534\n",
      "Epoch 10/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.5268 - val_loss: 0.7650\n",
      "Epoch 11/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.5133 - val_loss: 0.8208\n",
      "Epoch 12/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.5014 - val_loss: 0.8494\n",
      "Epoch 13/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4916 - val_loss: 0.8949\n",
      "Epoch 14/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4826 - val_loss: 0.9046\n",
      "Epoch 15/100\n",
      "7740/7740 [==============================] - 0s 32us/sample - loss: 0.4747 - val_loss: 0.8815\n",
      "Epoch 16/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4677 - val_loss: 0.8587\n",
      "Epoch 17/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4616 - val_loss: 0.7927\n",
      "3870/3870 [==============================] - 0s 12us/sample - loss: 0.4803\n",
      "7740/7740 [==============================] - 0s 14us/sample - loss: 0.4579\n",
      "[CV]  learning_rate=0.0011714615165291644, n_hidden=0, n_neurons=56, total=   4.7s\n",
      "[CV] learning_rate=0.0011714615165291644, n_hidden=0, n_neurons=56 ...\n",
      "Train on 7740 samples, validate on 3870 samples\n",
      "Epoch 1/100\n",
      "7740/7740 [==============================] - 0s 47us/sample - loss: 2.0724 - val_loss: 1.4705\n",
      "Epoch 2/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.7958 - val_loss: 0.7538\n",
      "Epoch 3/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.6896 - val_loss: 0.6508\n",
      "Epoch 4/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.6477 - val_loss: 0.6159\n",
      "Epoch 5/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.6167 - val_loss: 0.5844\n",
      "Epoch 6/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.5906 - val_loss: 0.5521\n",
      "Epoch 7/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.5675 - val_loss: 0.5322\n",
      "Epoch 8/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.5478 - val_loss: 0.5150\n",
      "Epoch 9/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.5305 - val_loss: 0.5061\n",
      "Epoch 10/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.5161 - val_loss: 0.4909\n",
      "Epoch 11/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.5025 - val_loss: 0.4726\n",
      "Epoch 12/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4914 - val_loss: 0.4612\n",
      "Epoch 13/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4815 - val_loss: 0.4534\n",
      "Epoch 14/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4728 - val_loss: 0.4438\n",
      "Epoch 15/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4650 - val_loss: 0.4357\n",
      "Epoch 16/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4585 - val_loss: 0.4313\n",
      "Epoch 17/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4522 - val_loss: 0.4274\n",
      "Epoch 18/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4470 - val_loss: 0.4222\n",
      "Epoch 19/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4419 - val_loss: 0.4129\n",
      "Epoch 20/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4377 - val_loss: 0.4086\n",
      "Epoch 21/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4337 - val_loss: 0.4057\n",
      "Epoch 22/100\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.4300 - val_loss: 0.4034\n",
      "Epoch 23/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4266 - val_loss: 0.3982\n",
      "Epoch 24/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4237 - val_loss: 0.3950\n",
      "Epoch 25/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4206 - val_loss: 0.4003\n",
      "Epoch 26/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.4182 - val_loss: 0.3956\n",
      "Epoch 27/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4158 - val_loss: 0.3966\n",
      "Epoch 28/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4134 - val_loss: 0.3945\n",
      "Epoch 29/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4114 - val_loss: 0.3836\n",
      "Epoch 30/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4094 - val_loss: 0.3941\n",
      "Epoch 31/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4074 - val_loss: 0.3814\n",
      "Epoch 32/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.4053 - val_loss: 0.3792\n",
      "Epoch 33/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.4041 - val_loss: 0.3928\n",
      "Epoch 34/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.4027 - val_loss: 0.3802\n",
      "Epoch 35/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.4010 - val_loss: 0.3835\n",
      "Epoch 36/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3997 - val_loss: 0.3741\n",
      "Epoch 37/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3981 - val_loss: 0.3885\n",
      "Epoch 38/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3974 - val_loss: 0.3860\n",
      "Epoch 39/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3962 - val_loss: 0.3697\n",
      "Epoch 40/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3949 - val_loss: 0.3734\n",
      "Epoch 41/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3940 - val_loss: 0.3706\n",
      "Epoch 42/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3928 - val_loss: 0.3909\n",
      "Epoch 43/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3920 - val_loss: 0.3772\n",
      "Epoch 44/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3908 - val_loss: 0.3750\n",
      "Epoch 45/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3900 - val_loss: 0.3648\n",
      "Epoch 46/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3891 - val_loss: 0.3807\n",
      "Epoch 47/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3887 - val_loss: 0.3625\n",
      "Epoch 48/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3876 - val_loss: 0.3746\n",
      "Epoch 49/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3867 - val_loss: 0.3754\n",
      "Epoch 50/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3864 - val_loss: 0.3645\n",
      "Epoch 51/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3851 - val_loss: 0.3951\n",
      "Epoch 52/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3845 - val_loss: 0.3832\n",
      "Epoch 53/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3841 - val_loss: 0.3613\n",
      "Epoch 54/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3829 - val_loss: 0.3953\n",
      "Epoch 55/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3825 - val_loss: 0.3757\n",
      "Epoch 56/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3818 - val_loss: 0.3647\n",
      "Epoch 57/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3810 - val_loss: 0.3959\n",
      "Epoch 58/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3807 - val_loss: 0.3563\n",
      "Epoch 59/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3802 - val_loss: 0.3882\n",
      "Epoch 60/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3795 - val_loss: 0.3784\n",
      "Epoch 61/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3786 - val_loss: 0.3908\n",
      "Epoch 62/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3785 - val_loss: 0.3719\n",
      "Epoch 63/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3778 - val_loss: 0.3788\n",
      "Epoch 64/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3774 - val_loss: 0.3566\n",
      "Epoch 65/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3766 - val_loss: 0.3630\n",
      "Epoch 66/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3764 - val_loss: 0.3537\n",
      "Epoch 67/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3756 - val_loss: 0.3850\n",
      "Epoch 68/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3755 - val_loss: 0.3769\n",
      "Epoch 69/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3749 - val_loss: 0.3656\n",
      "Epoch 70/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3744 - val_loss: 0.3584\n",
      "Epoch 71/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3739 - val_loss: 0.3706\n",
      "Epoch 72/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3735 - val_loss: 0.3533\n",
      "Epoch 73/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3731 - val_loss: 0.3556\n",
      "Epoch 74/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3724 - val_loss: 0.3960\n",
      "Epoch 75/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3724 - val_loss: 0.3755\n",
      "Epoch 76/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3718 - val_loss: 0.3667\n",
      "Epoch 77/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3715 - val_loss: 0.3575\n",
      "Epoch 78/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3708 - val_loss: 0.3851\n",
      "Epoch 79/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3707 - val_loss: 0.3694\n",
      "Epoch 80/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3702 - val_loss: 0.3824\n",
      "Epoch 81/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3700 - val_loss: 0.3673\n",
      "Epoch 82/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3696 - val_loss: 0.3491\n",
      "Epoch 83/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3691 - val_loss: 0.3479\n",
      "Epoch 84/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3683 - val_loss: 0.3513\n",
      "Epoch 85/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3683 - val_loss: 0.3882\n",
      "Epoch 86/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3679 - val_loss: 0.3776\n",
      "Epoch 87/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3677 - val_loss: 0.3607\n",
      "Epoch 88/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3670 - val_loss: 0.3911\n",
      "Epoch 89/100\n",
      "7740/7740 [==============================] - 0s 33us/sample - loss: 0.3670 - val_loss: 0.3468\n",
      "Epoch 90/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3667 - val_loss: 0.3470\n",
      "Epoch 91/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3661 - val_loss: 0.3466\n",
      "Epoch 92/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3658 - val_loss: 0.3778\n",
      "Epoch 93/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3654 - val_loss: 0.3447\n",
      "Epoch 94/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.3649 - val_loss: 0.3542\n",
      "Epoch 95/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3647 - val_loss: 0.3634\n",
      "Epoch 96/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3646 - val_loss: 0.3474\n",
      "Epoch 97/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3640 - val_loss: 0.3834\n",
      "Epoch 98/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3640 - val_loss: 0.3535\n",
      "Epoch 99/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3636 - val_loss: 0.3502\n",
      "Epoch 100/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3630 - val_loss: 0.3472\n",
      "3870/3870 [==============================] - 0s 13us/sample - loss: 0.3650\n",
      "7740/7740 [==============================] - 0s 14us/sample - loss: 0.3623\n",
      "[CV]  learning_rate=0.0011714615165291644, n_hidden=0, n_neurons=56, total=  26.5s\n",
      "[CV] learning_rate=0.0012578066689117673, n_hidden=1, n_neurons=60 ...\n",
      "Train on 7740 samples, validate on 3870 samples\n",
      "Epoch 1/100\n",
      "7740/7740 [==============================] - 0s 52us/sample - loss: 2.0678 - val_loss: 1.4867\n",
      "Epoch 2/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.7582 - val_loss: 0.7377\n",
      "Epoch 3/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.6402 - val_loss: 0.5802\n",
      "Epoch 4/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.5891 - val_loss: 0.5485\n",
      "Epoch 5/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.5528 - val_loss: 0.5201\n",
      "Epoch 6/100\n",
      "7740/7740 [==============================] - 0s 39us/sample - loss: 0.5260 - val_loss: 0.4890\n",
      "Epoch 7/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.5025 - val_loss: 0.4603\n",
      "Epoch 8/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.4839 - val_loss: 0.4542\n",
      "Epoch 9/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.4686 - val_loss: 0.4417\n",
      "Epoch 10/100\n",
      "7740/7740 [==============================] - 0s 38us/sample - loss: 0.4565 - val_loss: 0.4411\n",
      "Epoch 11/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.4466 - val_loss: 0.4201\n",
      "Epoch 12/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.4381 - val_loss: 0.4113\n",
      "Epoch 13/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4314 - val_loss: 0.4170\n",
      "Epoch 14/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4250 - val_loss: 0.4002\n",
      "Epoch 15/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.4195 - val_loss: 0.4316\n",
      "Epoch 16/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4149 - val_loss: 0.3887\n",
      "Epoch 17/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.4103 - val_loss: 0.3882\n",
      "Epoch 18/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.4060 - val_loss: 0.4235\n",
      "Epoch 19/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.4030 - val_loss: 0.4081\n",
      "Epoch 20/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3990 - val_loss: 0.4274\n",
      "Epoch 21/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3962 - val_loss: 0.3974\n",
      "Epoch 22/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3929 - val_loss: 0.3794\n",
      "Epoch 23/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3899 - val_loss: 0.4050\n",
      "Epoch 24/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3879 - val_loss: 0.3742\n",
      "Epoch 25/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3849 - val_loss: 0.3693\n",
      "Epoch 26/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3824 - val_loss: 0.3617\n",
      "Epoch 27/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3800 - val_loss: 0.3715\n",
      "Epoch 28/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3780 - val_loss: 0.3569\n",
      "Epoch 29/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3758 - val_loss: 0.3695\n",
      "Epoch 30/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3735 - val_loss: 0.3967\n",
      "Epoch 31/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3723 - val_loss: 0.3551\n",
      "Epoch 32/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3700 - val_loss: 0.3600\n",
      "Epoch 33/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3680 - val_loss: 0.3780\n",
      "Epoch 34/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3665 - val_loss: 0.3552\n",
      "Epoch 35/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3650 - val_loss: 0.3626\n",
      "Epoch 36/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3631 - val_loss: 0.3454\n",
      "Epoch 37/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3617 - val_loss: 0.3514\n",
      "Epoch 38/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3603 - val_loss: 0.3436\n",
      "Epoch 39/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3589 - val_loss: 0.3621\n",
      "Epoch 40/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3575 - val_loss: 0.3398\n",
      "Epoch 41/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3559 - val_loss: 0.3806\n",
      "Epoch 42/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3551 - val_loss: 0.3377\n",
      "Epoch 43/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3535 - val_loss: 0.3823\n",
      "Epoch 44/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3525 - val_loss: 0.3376\n",
      "Epoch 45/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3513 - val_loss: 0.3599\n",
      "Epoch 46/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3497 - val_loss: 0.3378\n",
      "Epoch 47/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3489 - val_loss: 0.3848\n",
      "Epoch 48/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3482 - val_loss: 0.3528\n",
      "Epoch 49/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3468 - val_loss: 0.3474\n",
      "Epoch 50/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3459 - val_loss: 0.3603\n",
      "Epoch 51/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3451 - val_loss: 0.3538\n",
      "Epoch 52/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3440 - val_loss: 0.3315\n",
      "Epoch 53/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3433 - val_loss: 0.3428\n",
      "Epoch 54/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3425 - val_loss: 0.3466\n",
      "Epoch 55/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3414 - val_loss: 0.3303\n",
      "Epoch 56/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3408 - val_loss: 0.3706\n",
      "Epoch 57/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3401 - val_loss: 0.3334\n",
      "Epoch 58/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3386 - val_loss: 0.3698\n",
      "Epoch 59/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3383 - val_loss: 0.3272\n",
      "Epoch 60/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3373 - val_loss: 0.4134\n",
      "Epoch 61/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3370 - val_loss: 0.3250\n",
      "Epoch 62/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3361 - val_loss: 0.4060\n",
      "Epoch 63/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3357 - val_loss: 0.3332\n",
      "Epoch 64/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3347 - val_loss: 0.3328\n",
      "Epoch 65/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3341 - val_loss: 0.3264\n",
      "Epoch 66/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3335 - val_loss: 0.3295\n",
      "Epoch 67/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3329 - val_loss: 0.3599\n",
      "Epoch 68/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3321 - val_loss: 0.3459\n",
      "Epoch 69/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3313 - val_loss: 0.3226\n",
      "Epoch 70/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3308 - val_loss: 0.3513\n",
      "Epoch 71/100\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3302 - val_loss: 0.3653\n",
      "Epoch 72/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3300 - val_loss: 0.3384\n",
      "Epoch 73/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3289 - val_loss: 0.3897\n",
      "Epoch 74/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3286 - val_loss: 0.3191\n",
      "Epoch 75/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3282 - val_loss: 0.3525\n",
      "Epoch 76/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3274 - val_loss: 0.3197\n",
      "Epoch 77/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3268 - val_loss: 0.4065\n",
      "Epoch 78/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3270 - val_loss: 0.3257\n",
      "Epoch 79/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3256 - val_loss: 0.3260\n",
      "Epoch 80/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3253 - val_loss: 0.3684\n",
      "Epoch 81/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3251 - val_loss: 0.3202\n",
      "Epoch 82/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3245 - val_loss: 0.4489\n",
      "Epoch 83/100\n",
      "7740/7740 [==============================] - 0s 35us/sample - loss: 0.3246 - val_loss: 0.3344\n",
      "Epoch 84/100\n",
      "7740/7740 [==============================] - 0s 34us/sample - loss: 0.3239 - val_loss: 0.4869\n",
      "3870/3870 [==============================] - 0s 13us/sample - loss: 0.3442\n",
      "7740/7740 [==============================] - 0s 15us/sample - loss: 0.3227\n",
      "[CV]  learning_rate=0.0012578066689117673, n_hidden=1, n_neurons=60, total=  23.7s\n",
      "[CV] learning_rate=0.0012578066689117673, n_hidden=1, n_neurons=60 ...\n",
      "Train on 7740 samples, validate on 3870 samples\n",
      "Epoch 1/100\n",
      "7740/7740 [==============================] - 0s 50us/sample - loss: 1.7817 - val_loss: 9.1129\n",
      "Epoch 2/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.7881 - val_loss: 4.1277\n",
      "Epoch 3/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.6598 - val_loss: 1.6439\n",
      "Epoch 4/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.6022 - val_loss: 0.8057\n",
      "Epoch 5/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.5604 - val_loss: 0.5135\n",
      "Epoch 6/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.5278 - val_loss: 0.5437\n",
      "Epoch 7/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.5023 - val_loss: 0.7848\n",
      "Epoch 8/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4825 - val_loss: 0.9492\n",
      "Epoch 9/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4660 - val_loss: 1.0810\n",
      "Epoch 10/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4530 - val_loss: 1.1596\n",
      "Epoch 11/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4423 - val_loss: 1.1965\n",
      "Epoch 12/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4338 - val_loss: 1.2383\n",
      "Epoch 13/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.4256 - val_loss: 1.1621\n",
      "Epoch 14/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4193 - val_loss: 1.0986\n",
      "Epoch 15/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4133 - val_loss: 1.0196\n",
      "3870/3870 [==============================] - 0s 14us/sample - loss: 0.4393\n",
      "7740/7740 [==============================] - 0s 15us/sample - loss: 0.4098\n",
      "[CV]  learning_rate=0.0012578066689117673, n_hidden=1, n_neurons=60, total=   4.6s\n",
      "[CV] learning_rate=0.0012578066689117673, n_hidden=1, n_neurons=60 ...\n",
      "Train on 7740 samples, validate on 3870 samples\n",
      "Epoch 1/100\n",
      "7740/7740 [==============================] - 0s 51us/sample - loss: 2.1548 - val_loss: 2.2374\n",
      "Epoch 2/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.8591 - val_loss: 0.7902\n",
      "Epoch 3/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.7066 - val_loss: 0.6506\n",
      "Epoch 4/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.6480 - val_loss: 0.6075\n",
      "Epoch 5/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.6086 - val_loss: 0.5687\n",
      "Epoch 6/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.5752 - val_loss: 0.5391\n",
      "Epoch 7/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.5474 - val_loss: 0.5183\n",
      "Epoch 8/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.5239 - val_loss: 0.4882\n",
      "Epoch 9/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.5039 - val_loss: 0.4673\n",
      "Epoch 10/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4862 - val_loss: 0.4519\n",
      "Epoch 11/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4721 - val_loss: 0.4395\n",
      "Epoch 12/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4603 - val_loss: 0.4273\n",
      "Epoch 13/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4498 - val_loss: 0.4201\n",
      "Epoch 14/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4415 - val_loss: 0.4119\n",
      "Epoch 15/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4346 - val_loss: 0.4132\n",
      "Epoch 16/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4276 - val_loss: 0.3996\n",
      "Epoch 17/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4230 - val_loss: 0.4010\n",
      "Epoch 18/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4180 - val_loss: 0.3988\n",
      "Epoch 19/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4138 - val_loss: 0.3940\n",
      "Epoch 20/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4100 - val_loss: 0.3974\n",
      "Epoch 21/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4067 - val_loss: 0.4028\n",
      "Epoch 22/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4044 - val_loss: 0.4049\n",
      "Epoch 23/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.4006 - val_loss: 0.3939\n",
      "Epoch 24/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3971 - val_loss: 0.3815\n",
      "Epoch 25/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3951 - val_loss: 0.3838\n",
      "Epoch 26/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3922 - val_loss: 0.3768\n",
      "Epoch 27/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3902 - val_loss: 0.3716\n",
      "Epoch 28/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3872 - val_loss: 0.3778\n",
      "Epoch 29/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3859 - val_loss: 0.3884\n",
      "Epoch 30/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3841 - val_loss: 0.3782\n",
      "Epoch 31/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3822 - val_loss: 0.3698\n",
      "Epoch 32/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3807 - val_loss: 0.3677\n",
      "Epoch 33/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3786 - val_loss: 0.3782\n",
      "Epoch 34/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3770 - val_loss: 0.3646\n",
      "Epoch 35/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3757 - val_loss: 0.3699\n",
      "Epoch 36/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3742 - val_loss: 0.3663\n",
      "Epoch 37/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3728 - val_loss: 0.3685\n",
      "Epoch 38/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3710 - val_loss: 0.3635\n",
      "Epoch 39/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3699 - val_loss: 0.3762\n",
      "Epoch 40/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3693 - val_loss: 0.3657\n",
      "Epoch 41/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3679 - val_loss: 0.3567\n",
      "Epoch 42/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3662 - val_loss: 0.3659\n",
      "Epoch 43/100\n",
      "7740/7740 [==============================] - 0s 37us/sample - loss: 0.3652 - val_loss: 0.3720\n",
      "Epoch 44/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3645 - val_loss: 0.3576\n",
      "Epoch 45/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3633 - val_loss: 0.3676\n",
      "Epoch 46/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3623 - val_loss: 0.3505\n",
      "Epoch 47/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3613 - val_loss: 0.3488\n",
      "Epoch 48/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3604 - val_loss: 0.3592\n",
      "Epoch 49/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3598 - val_loss: 0.3445\n",
      "Epoch 50/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3590 - val_loss: 0.3681\n",
      "Epoch 51/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3581 - val_loss: 0.3576\n",
      "Epoch 52/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3569 - val_loss: 0.3442\n",
      "Epoch 53/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3563 - val_loss: 0.3443\n",
      "Epoch 54/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3558 - val_loss: 0.3566\n",
      "Epoch 55/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3552 - val_loss: 0.3392\n",
      "Epoch 56/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3546 - val_loss: 0.3462\n",
      "Epoch 57/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3537 - val_loss: 0.3421\n",
      "Epoch 58/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3527 - val_loss: 0.3472\n",
      "Epoch 59/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3526 - val_loss: 0.3521\n",
      "Epoch 60/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3519 - val_loss: 0.3422\n",
      "Epoch 61/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3509 - val_loss: 0.3555\n",
      "Epoch 62/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3506 - val_loss: 0.3409\n",
      "Epoch 63/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3500 - val_loss: 0.3394\n",
      "Epoch 64/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3493 - val_loss: 0.3418\n",
      "Epoch 65/100\n",
      "7740/7740 [==============================] - 0s 36us/sample - loss: 0.3486 - val_loss: 0.3506\n",
      "3870/3870 [==============================] - 0s 14us/sample - loss: 0.3470\n",
      "7740/7740 [==============================] - 0s 15us/sample - loss: 0.3468\n",
      "[CV]  learning_rate=0.0012578066689117673, n_hidden=1, n_neurons=60, total=  18.6s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  30 out of  30 | elapsed:  7.3min finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 11610 samples, validate on 3870 samples\n",
      "Epoch 1/100\n",
      "11610/11610 [==============================] - 1s 46us/sample - loss: 0.8992 - val_loss: 2.7901\n",
      "Epoch 2/100\n",
      "11610/11610 [==============================] - 0s 35us/sample - loss: 0.4828 - val_loss: 0.4578\n",
      "Epoch 3/100\n",
      "11610/11610 [==============================] - 0s 35us/sample - loss: 0.4082 - val_loss: 0.3782\n",
      "Epoch 4/100\n",
      "11610/11610 [==============================] - 0s 35us/sample - loss: 0.3842 - val_loss: 0.4179\n",
      "Epoch 5/100\n",
      "11610/11610 [==============================] - 0s 35us/sample - loss: 0.3707 - val_loss: 0.3564\n",
      "Epoch 6/100\n",
      "11610/11610 [==============================] - 0s 35us/sample - loss: 0.3624 - val_loss: 0.3539\n",
      "Epoch 7/100\n",
      "11610/11610 [==============================] - 0s 35us/sample - loss: 0.3541 - val_loss: 0.3745\n",
      "Epoch 8/100\n",
      "11610/11610 [==============================] - 0s 35us/sample - loss: 0.3493 - val_loss: 0.4061\n",
      "Epoch 9/100\n",
      "11610/11610 [==============================] - 0s 35us/sample - loss: 0.3448 - val_loss: 0.3380\n",
      "Epoch 10/100\n",
      "11610/11610 [==============================] - 0s 35us/sample - loss: 0.3405 - val_loss: 0.3352\n",
      "Epoch 11/100\n",
      "11610/11610 [==============================] - 0s 34us/sample - loss: 0.3361 - val_loss: 0.4455\n",
      "Epoch 12/100\n",
      "11610/11610 [==============================] - 0s 35us/sample - loss: 0.3349 - val_loss: 0.6610\n",
      "Epoch 13/100\n",
      "11610/11610 [==============================] - 0s 35us/sample - loss: 0.3335 - val_loss: 0.4351\n",
      "Epoch 14/100\n",
      "11610/11610 [==============================] - 0s 35us/sample - loss: 0.3284 - val_loss: 0.3318\n",
      "Epoch 15/100\n",
      "11610/11610 [==============================] - 0s 35us/sample - loss: 0.3254 - val_loss: 0.3144\n",
      "Epoch 16/100\n",
      "11610/11610 [==============================] - 0s 34us/sample - loss: 0.3223 - val_loss: 0.4413\n",
      "Epoch 17/100\n",
      "11610/11610 [==============================] - 0s 35us/sample - loss: 0.3226 - val_loss: 0.3295\n",
      "Epoch 18/100\n",
      "11610/11610 [==============================] - 0s 34us/sample - loss: 0.3182 - val_loss: 0.4551\n",
      "Epoch 19/100\n",
      "11610/11610 [==============================] - 0s 34us/sample - loss: 0.3182 - val_loss: 0.7770\n",
      "Epoch 20/100\n",
      "11610/11610 [==============================] - 0s 35us/sample - loss: 0.3176 - val_loss: 0.8286\n",
      "Epoch 21/100\n",
      "11610/11610 [==============================] - 0s 34us/sample - loss: 0.3168 - val_loss: 0.4178\n",
      "Epoch 22/100\n",
      "11610/11610 [==============================] - 0s 35us/sample - loss: 0.3118 - val_loss: 0.3867\n",
      "Epoch 23/100\n",
      "11610/11610 [==============================] - 0s 34us/sample - loss: 0.3082 - val_loss: 0.3094\n",
      "Epoch 24/100\n",
      "11610/11610 [==============================] - 0s 34us/sample - loss: 0.3063 - val_loss: 0.3018\n",
      "Epoch 25/100\n",
      "11610/11610 [==============================] - 0s 35us/sample - loss: 0.3046 - val_loss: 0.3086\n",
      "Epoch 26/100\n",
      "11610/11610 [==============================] - 0s 35us/sample - loss: 0.3035 - val_loss: 0.2955\n",
      "Epoch 27/100\n",
      "11610/11610 [==============================] - 0s 35us/sample - loss: 0.3016 - val_loss: 0.2874\n",
      "Epoch 28/100\n",
      "11610/11610 [==============================] - 0s 35us/sample - loss: 0.2998 - val_loss: 0.3067\n",
      "Epoch 29/100\n",
      "11610/11610 [==============================] - 0s 35us/sample - loss: 0.2987 - val_loss: 0.2855\n",
      "Epoch 30/100\n",
      "11610/11610 [==============================] - 0s 35us/sample - loss: 0.2985 - val_loss: 0.4065\n",
      "Epoch 31/100\n",
      "11610/11610 [==============================] - 0s 35us/sample - loss: 0.2964 - val_loss: 0.4888\n",
      "Epoch 32/100\n",
      "11610/11610 [==============================] - 0s 35us/sample - loss: 0.2971 - val_loss: 0.6767\n",
      "Epoch 33/100\n",
      "11610/11610 [==============================] - 0s 34us/sample - loss: 0.2964 - val_loss: 0.4657\n",
      "Epoch 34/100\n",
      "11610/11610 [==============================] - 0s 34us/sample - loss: 0.2943 - val_loss: 0.3589\n",
      "Epoch 35/100\n",
      "11610/11610 [==============================] - 0s 35us/sample - loss: 0.2922 - val_loss: 0.3978\n",
      "Epoch 36/100\n",
      "11610/11610 [==============================] - 0s 35us/sample - loss: 0.2909 - val_loss: 0.6603\n",
      "Epoch 37/100\n",
      "11610/11610 [==============================] - 0s 36us/sample - loss: 0.2923 - val_loss: 0.4158\n",
      "Epoch 38/100\n",
      "11610/11610 [==============================] - 0s 35us/sample - loss: 0.2899 - val_loss: 0.3226\n",
      "Epoch 39/100\n",
      "11610/11610 [==============================] - 0s 35us/sample - loss: 0.2873 - val_loss: 0.2782\n",
      "Epoch 40/100\n",
      "11610/11610 [==============================] - 0s 36us/sample - loss: 0.2866 - val_loss: 0.3285\n",
      "Epoch 41/100\n",
      "11610/11610 [==============================] - 0s 35us/sample - loss: 0.2850 - val_loss: 0.4074\n",
      "Epoch 42/100\n",
      "11610/11610 [==============================] - 0s 35us/sample - loss: 0.2858 - val_loss: 0.3352\n",
      "Epoch 43/100\n",
      "11610/11610 [==============================] - 0s 35us/sample - loss: 0.2827 - val_loss: 0.2797\n",
      "Epoch 44/100\n",
      "11610/11610 [==============================] - 0s 34us/sample - loss: 0.2824 - val_loss: 0.7323\n",
      "Epoch 45/100\n",
      "11610/11610 [==============================] - 0s 35us/sample - loss: 0.2838 - val_loss: 0.7834\n",
      "Epoch 46/100\n",
      "11610/11610 [==============================] - 0s 34us/sample - loss: 0.2888 - val_loss: 0.6376\n",
      "Epoch 47/100\n",
      "11610/11610 [==============================] - 0s 35us/sample - loss: 0.2844 - val_loss: 0.5744\n",
      "Epoch 48/100\n",
      "11610/11610 [==============================] - 0s 35us/sample - loss: 0.2819 - val_loss: 0.3071\n",
      "Epoch 49/100\n",
      "11610/11610 [==============================] - 0s 35us/sample - loss: 0.2790 - val_loss: 0.3484\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "RandomizedSearchCV(cv=3, error_score='raise-deprecating',\n",
       "          estimator=<tensorflow.python.keras.wrappers.scikit_learn.KerasRegressor object at 0x11213ed68>,\n",
       "          fit_params=None, iid='warn', n_iter=10, n_jobs=None,\n",
       "          param_distributions={'n_hidden': [0, 1, 2, 3], 'n_neurons': array([ 1,  2, ..., 98, 99]), 'learning_rate': <scipy.stats._distn_infrastructure.rv_frozen object at 0x1a3bcea4a8>},\n",
       "          pre_dispatch='2*n_jobs', random_state=None, refit=True,\n",
       "          return_train_score='warn', scoring=None, verbose=2)"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rnd_search_cv.fit(X_train_scaled, y_train, epochs=100,\n",
    "                  validation_data=(X_valid_scaled, y_valid),\n",
    "                  callbacks=[keras.callbacks.EarlyStopping(patience=10)])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'learning_rate': 0.004038022680351922, 'n_hidden': 2, 'n_neurons': 69}"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rnd_search_cv.best_params_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "-0.3275334420977329"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rnd_search_cv.best_score_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.wrappers.scikit_learn.KerasRegressor at 0x1a3d960588>"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rnd_search_cv.best_estimator_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Evaluate the best model found on the test set. You can either use the best estimator's `score()` method, or get its underlying Keras model *via* its `model` attribute, and call this model's `evaluate()` method. Note that the estimator returns the negative mean square error (it's a score, not a loss, so higher is better)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5160/5160 [==============================] - 0s 19us/sample - loss: 0.2968\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "-0.2968322378604911"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rnd_search_cv.score(X_test_scaled, y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5160/5160 [==============================] - 0s 15us/sample - loss: 0.2968\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0.2968322378604911"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model = rnd_search_cv.best_estimator_.model\n",
    "model.evaluate(X_test_scaled, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, save the best Keras model found. **Tip**: it is available via the best estimator's `model` attribute, and just need to call its `save()` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.save(\"my_fine_tuned_housing_model.h5\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}