FinalAssignment-ML-with-Python.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "<a href=\"https://www.bigdatauniversity.com\"><img src=\"https://ibm.box.com/shared/static/cw2c7r3o20w9zn8gkecaeyjhgw3xdgbj.png\" width=\"400\" align=\"center\"></a>\n",
    "\n",
    "<h1 align=\"center\"><font size=\"5\">Classification with Python</font></h1>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "In this notebook we try to practice all the classification algorithms that we learned in this course.\n",
    "\n",
    "We load a dataset using Pandas library, and apply the following algorithms, and find the best one for this specific dataset by accuracy evaluation methods.\n",
    "\n",
    "Lets first load required libraries:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "import itertools\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from matplotlib.ticker import NullFormatter\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.ticker as ticker\n",
    "from sklearn import preprocessing\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### About dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "This dataset is about past loans. The __Loan_train.csv__ data set includes details of 346 customers whose loan are already paid off or defaulted. It includes following fields:\n",
    "\n",
    "| Field          | Description                                                                           |\n",
    "|----------------|---------------------------------------------------------------------------------------|\n",
    "| Loan_status    | Whether a loan is paid off on in collection                                           |\n",
    "| Principal      | Basic principal loan amount at the                                                    |\n",
    "| Terms          | Origination terms which can be weekly (7 days), biweekly, and monthly payoff schedule |\n",
    "| Effective_date | When the loan got originated and took effects                                         |\n",
    "| Due_date       | Since it’s one-time payoff schedule, each loan has one single due date                |\n",
    "| Age            | Age of applicant                                                                      |\n",
    "| Education      | Education of applicant                                                                |\n",
    "| Gender         | The gender of applicant                                                               |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Lets download the dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2020-03-24 14:20:16--  https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_train.csv\n",
      "Resolving s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)... 67.228.254.196\n",
      "Connecting to s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)|67.228.254.196|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 23101 (23K) [text/csv]\n",
      "Saving to: ‘loan_train.csv’\n",
      "\n",
      "100%[======================================>] 23,101      --.-K/s   in 0.07s   \n",
      "\n",
      "2020-03-24 14:20:16 (305 KB/s) - ‘loan_train.csv’ saved [23101/23101]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!wget -O loan_train.csv https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_train.csv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Load Data From CSV File  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/8/2016</td>\n",
       "      <td>10/7/2016</td>\n",
       "      <td>45</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/8/2016</td>\n",
       "      <td>10/7/2016</td>\n",
       "      <td>33</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>9/8/2016</td>\n",
       "      <td>9/22/2016</td>\n",
       "      <td>27</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/9/2016</td>\n",
       "      <td>10/8/2016</td>\n",
       "      <td>28</td>\n",
       "      <td>college</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6</td>\n",
       "      <td>6</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/9/2016</td>\n",
       "      <td>10/8/2016</td>\n",
       "      <td>29</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           0             0     PAIDOFF       1000     30       9/8/2016   \n",
       "1           2             2     PAIDOFF       1000     30       9/8/2016   \n",
       "2           3             3     PAIDOFF       1000     15       9/8/2016   \n",
       "3           4             4     PAIDOFF       1000     30       9/9/2016   \n",
       "4           6             6     PAIDOFF       1000     30       9/9/2016   \n",
       "\n",
       "    due_date  age             education  Gender  \n",
       "0  10/7/2016   45  High School or Below    male  \n",
       "1  10/7/2016   33              Bechalor  female  \n",
       "2  9/22/2016   27               college    male  \n",
       "3  10/8/2016   28               college  female  \n",
       "4  10/8/2016   29               college    male  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv('loan_train.csv')\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(346, 10)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Convert to date time object "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>45</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>33</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-09-22</td>\n",
       "      <td>27</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>28</td>\n",
       "      <td>college</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6</td>\n",
       "      <td>6</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>29</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           0             0     PAIDOFF       1000     30     2016-09-08   \n",
       "1           2             2     PAIDOFF       1000     30     2016-09-08   \n",
       "2           3             3     PAIDOFF       1000     15     2016-09-08   \n",
       "3           4             4     PAIDOFF       1000     30     2016-09-09   \n",
       "4           6             6     PAIDOFF       1000     30     2016-09-09   \n",
       "\n",
       "    due_date  age             education  Gender  \n",
       "0 2016-10-07   45  High School or Below    male  \n",
       "1 2016-10-07   33              Bechalor  female  \n",
       "2 2016-09-22   27               college    male  \n",
       "3 2016-10-08   28               college  female  \n",
       "4 2016-10-08   29               college    male  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['due_date'] = pd.to_datetime(df['due_date'])\n",
    "df['effective_date'] = pd.to_datetime(df['effective_date'])\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "# Data visualization and pre-processing\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Let’s see how many of each class is in our data set "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "PAIDOFF       260\n",
       "COLLECTION     86\n",
       "Name: loan_status, dtype: int64"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['loan_status'].value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "260 people have paid off the loan on time while 86 have gone into collection \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lets plot some columns to underestand data better:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Solving environment: done\n",
      "\n",
      "## Package Plan ##\n",
      "\n",
      "  environment location: /opt/conda/envs/Python36\n",
      "\n",
      "  added / updated specs: \n",
      "    - seaborn\n",
      "\n",
      "\n",
      "The following packages will be downloaded:\n",
      "\n",
      "    package                    |            build\n",
      "    ---------------------------|-----------------\n",
      "    ca-certificates-2020.1.1   |                0         132 KB  anaconda\n",
      "    seaborn-0.10.0             |             py_0         161 KB  anaconda\n",
      "    certifi-2019.11.28         |           py36_1         157 KB  anaconda\n",
      "    openssl-1.1.1              |       h7b6447c_0         5.0 MB  anaconda\n",
      "    ------------------------------------------------------------\n",
      "                                           Total:         5.5 MB\n",
      "\n",
      "The following packages will be UPDATED:\n",
      "\n",
      "    ca-certificates: 2019.11.27-0       --> 2020.1.1-0        anaconda\n",
      "    certifi:         2019.11.28-py36_0  --> 2019.11.28-py36_1 anaconda\n",
      "    openssl:         1.1.1d-h7b6447c_3  --> 1.1.1-h7b6447c_0  anaconda\n",
      "    seaborn:         0.9.0-pyh91ea838_1 --> 0.10.0-py_0       anaconda\n",
      "\n",
      "\n",
      "Downloading and Extracting Packages\n",
      "ca-certificates-2020 | 132 KB    | ##################################### | 100% \n",
      "seaborn-0.10.0       | 161 KB    | ##################################### | 100% \n",
      "certifi-2019.11.28   | 157 KB    | ##################################### | 100% \n",
      "openssl-1.1.1        | 5.0 MB    | ##################################### | 100% \n",
      "Preparing transaction: done\n",
      "Verifying transaction: done\n",
      "Executing transaction: done\n"
     ]
    }
   ],
   "source": [
    "# notice: installing seaborn might takes a few minutes\n",
    "!conda install -c anaconda seaborn -y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAG4xJREFUeJzt3XucFOWd7/HPV5wVFaIioyKIMyKKqGTAWY3XJbCyqPF2jAbjUdx4DtFoXDbxeMt5aTa+1nghMclRibhyyCaKGrKgSxINUTmKiRfAEcELITrqKCAQN8YgBPB3/qiaSYM9zKV7pmu6v+/Xq15T9VTVU7+umWd+XU9XP6WIwMzMLGt2KHUAZmZm+ThBmZlZJjlBmZlZJjlBmZlZJjlBmZlZJjlBmZlZJjlBdRFJe0u6T9LrkhZJ+q2kM4tU92hJc4tRV3eQNF9SfanjsNIop7YgqVrSs5JekHR8Fx7nw66quydxguoCkgTMAZ6MiAMi4ghgAjCoRPHsWIrjmpVhWxgLvBoRIyPiqWLEZK1zguoaY4C/RMQPmwsi4s2I+D8AknpJulXS85KWSPpyWj46vdqYJelVSfemDRxJ49OyBcB/a65X0q6Spqd1vSDp9LT8Qkk/lfSfwK8KeTGSZkiaKumJ9F3w36XHfEXSjJztpkpaKGmZpH9ppa5x6TvoxWl8fQqJzTKvbNqCpDrgFuBkSQ2Sdm7t71lSo6Qb03ULJY2S9Kik30u6ON2mj6TH0n1fao43z3H/V875yduuylZEeCryBFwO3Lad9ZOA/53O7wQsBGqB0cAfSd5d7gD8FjgO6A28DQwFBDwIzE33vxH47+n87sByYFfgQqAJ6NdKDE8BDXmmv8+z7Qzg/vTYpwMfAIenMS4C6tLt+qU/ewHzgRHp8nygHugPPAnsmpZfBVxX6t+Xp66byrAtXAjcns63+vcMNAKXpPO3AUuAvkA18F5aviPwqZy6VgBKlz9Mf44DpqWvdQdgLnBCqX+v3TW566cbSLqDpHH9JSL+luSPboSkz6eb7EbS4P4CPBcRTel+DUAN8CHwRkT8Li3/CUnDJq3rNElXpMu9gcHp/LyI+EO+mCKio/3n/xkRIeklYHVEvJTGsiyNsQE4R9IkkoY3ABhO0jCbfSYtezp9M/w3JP94rEKUSVto1tbf88Ppz5eAPhHxJ+BPkjZI2h34M3CjpBOAj4GBwN7Aqpw6xqXTC+lyH5Lz82QnY+5RnKC6xjLgrOaFiLhUUn+Sd4eQvBv6akQ8mruTpNHAxpyiLfz1d9TaoIkCzoqI17ap6yiSBpB/J+kpknd027oiIn6dp7w5ro+3ifFjYEdJtcAVwN9GxPtp11/vPLHOi4hzW4vLyk45toXc423v73m7bQY4j+SK6oiI2CSpkfxt5tsRcdd24ihb/gyqazwO9JZ0SU7ZLjnzjwKXSKoCkHSQpF23U9+rQK2kIelyboN4FPhqTv/8yPYEGBHHR0Rdnml7DXJ7PkXyT+CPkvYGTsqzzTPAsZIOTGPdRdJBnTye9Qzl3BYK/XvejaS7b5OkzwL759nmUeBLOZ9tDZS0VweO0aM5QXWBSDqPzwD+TtIbkp4DfkTSRw3wb8DLwGJJS4G72M7VbERsIOnG+Hn6wfCbOatvAKqAJWldNxT79bRHRLxI0g2xDJgOPJ1nmzUkffgzJS0haeDDujFM62bl3BaK8Pd8L1AvaSHJ1dSreY7xK+A+4Ldp9/os8l/tlaXmD+TMzMwyxVdQZmaWSU5QZmaWSU5QZmaWSU5QZmaWSZlIUOPHjw+S7zZ48lQuU9G4fXgqs6ndMpGg1q5dW+oQzDLL7cMqVSYSlJmZ2bacoMzMLJOcoMzMLJM8WKyZlZVNmzbR1NTEhg0bSh1KRevduzeDBg2iqqqq03U4QZlZWWlqaqJv377U1NSQjhtr3SwiWLduHU1NTdTW1na6HnfxmVlZ2bBhA3vuuaeTUwlJYs899yz4KtYJyirG/gMGIKko0/4DBpT65dh2ODmVXjF+B+7is4rx1qpVNO07qCh1DXq3qSj1mFnrfAVlZmWtmFfO7b167tWrF3V1dRx22GGcffbZrF+/vmXd7NmzkcSrr/718U+NjY0cdthhAMyfP5/ddtuNkSNHcvDBB3PCCScwd+7creqfNm0aw4YNY9iwYRx55JEsWLCgZd3o0aM5+OCDqauro66ujlmzZm0VU/PU2NhYyGntFr6CMrOyVswrZ2jf1fPOO+9MQ0MDAOeddx4//OEP+drXvgbAzJkzOe6447j//vv55je/mXf/448/viUpNTQ0cMYZZ7DzzjszduxY5s6dy1133cWCBQvo378/ixcv5owzzuC5555jn332AeDee++lvr6+1Zh6ijavoCRNl/Re+oTK5rJvSnpHUkM6nZyz7hpJKyS9JukfuipwM7Oe4Pjjj2fFihUAfPjhhzz99NPcc8893H///e3av66ujuuuu47bb78dgJtvvplbb72V/v37AzBq1CgmTpzIHXfc0TUvoITa08U3Axifp/y2iKhLp18ASBoOTAAOTfe5U1KvYgVrZtaTbN68mV/+8pccfvjhAMyZM4fx48dz0EEH0a9fPxYvXtyuekaNGtXSJbhs2TKOOOKIrdbX19ezbNmyluXzzjuvpStv3bp1AHz00UctZWeeeWYxXl6Xa7OLLyKelFTTzvpOB+6PiI3AG5JWAEcCv+10hGZmPUxzMoDkCuqiiy4Cku69yZMnAzBhwgRmzpzJqFGj2qwvYvuDgEfEVnfNlUsXXyGfQV0m6QJgIfD1iHgfGAg8k7NNU1r2CZImAZMABg8eXEAYZuXH7aNny5cM1q1bx+OPP87SpUuRxJYtW5DELbfc0mZ9L7zwAocccggAw4cPZ9GiRYwZM6Zl/eLFixk+fHhxX0QGdPYuvqnAEKAOWAl8Jy3Pd+N73tQfEdMioj4i6qurqzsZhll5cvsoP7NmzeKCCy7gzTffpLGxkbfffpva2tqt7sDLZ8mSJdxwww1ceumlAFx55ZVcddVVLV13DQ0NzJgxg6985Std/hq6W6euoCJidfO8pLuB5nsgm4D9cjYdBLzb6ejMzAo0eJ99ivq9tcHpnXIdNXPmTK6++uqtys466yzuu+8+rrrqqq3Kn3rqKUaOHMn69evZa6+9+MEPfsDYsWMBOO2003jnnXc45phjkETfvn35yU9+woAy/PK42urbBEg/g5obEYelywMiYmU6/8/AURExQdKhwH0knzvtCzwGDI2ILdurv76+PhYuXFjI6zBrk6SiflG3jbZTtKEM3D465pVXXmnpDrPSauV30e620eYVlKSZwGigv6Qm4HpgtKQ6ku67RuDLABGxTNKDwMvAZuDStpKTmZlZPu25i+/cPMX3bGf7fwX+tZCgzMzMPNSRmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZW1fQcNLurjNvYd1L6RPVatWsWECRMYMmQIw4cP5+STT2b58uUsW7aMMWPGcNBBBzF06FBuuOGGlq8szJgxg8suu+wTddXU1LB27dqtymbMmEF1dfVWj9B4+eWXAVi+fDknn3wyBx54IIcccgjnnHMODzzwQMt2ffr0aXkkxwUXXMD8+fP53Oc+11L3nDlzGDFiBMOGDePwww9nzpw5LesuvPBCBg4cyMaNGwFYu3YtNTU1HfqdtJcft2FmZW3lO29z1HWPFK2+Z7+Vb+zsrUUEZ555JhMnTmwZtbyhoYHVq1dz4YUXMnXqVMaNG8f69es566yzuPPOO1tGiuiIL3zhCy2jnDfbsGEDp5xyCt/97nc59dRTAXjiiSeorq5uGX5p9OjRTJkypWW8vvnz57fs/+KLL3LFFVcwb948amtreeONNzjxxBM54IADGDFiBJA8W2r69OlccsklHY65I3wFZWZWZE888QRVVVVcfPHFLWV1dXUsX76cY489lnHjxgGwyy67cPvtt3PTTTcV7dj33XcfRx99dEtyAvjsZz/b8kDEtkyZMoVrr72W2tpaAGpra7nmmmu49dZbW7aZPHkyt912G5s3by5a3Pk4QZmZFdnSpUs/8UgMyP+ojCFDhvDhhx/ywQcfdPg4ud12dXV1fPTRR60eu73a8ziPwYMHc9xxx/HjH/+408dpD3fxmZl1k20fi5GrtfLtydfFV6h8MeYru/baaznttNM45ZRTinr8XL6CMjMrskMPPZRFixblLd92XMXXX3+dPn360Ldv3y49dkf23zbGfI/zOPDAA6mrq+PBBx/s9LHa4gRlZlZkY8aMYePGjdx9990tZc8//zxDhw5lwYIF/PrXvwaSBxtefvnlXHnllUU79he/+EV+85vf8POf/7yl7JFHHuGll15q1/5XXHEF3/72t2lsbASgsbGRG2+8ka9//euf2PYb3/gGU6ZMKUrc+biLz8zK2oCB+7XrzruO1NcWScyePZvJkydz00030bt3b2pqavje977HQw89xFe/+lUuvfRStmzZwvnnn7/VreUzZszY6rbuZ55JngE7YsQIdtghuaY455xzGDFiBA888MBWz5O68847OeaYY5g7dy6TJ09m8uTJVFVVMWLECL7//e+36/XV1dVx8803c+qpp7Jp0yaqqqq45ZZbWp4QnOvQQw9l1KhR7X50fUe163EbXc2PE7Du4MdtVAY/biM7Cn3cRptdfJKmS3pP0tKcslslvSppiaTZknZPy2skfSSpIZ1+2N5AzMzMcrXnM6gZwLbXx/OAwyJiBLAcuCZn3e8joi6dLsbMzKwT2kxQEfEk8Idtyn4VEc3f0HqG5NHuZmaZkIWPLipdMX4HxbiL70vAL3OWayW9IOn/STq+tZ0kTZK0UNLCNWvWFCEMs/Lh9tF5vXv3Zt26dU5SJRQRrFu3jt69exdUT0F38Un6Bsmj3e9Ni1YCgyNinaQjgDmSDo2IT3xFOiKmAdMg+RC4kDjMyo3bR+cNGjSIpqYmnNhLq3fv3gwaVFjnWqcTlKSJwOeAsZG+VYmIjcDGdH6RpN8DBwG+BcnMukVVVVXLOHLWs3Wqi0/SeOAq4LSIWJ9TXi2pVzp/ADAUeL0YgZqZWWVp8wpK0kxgNNBfUhNwPcldezsB89LxmZ5J79g7AfiWpM3AFuDiiPhD3orNzMy2o80EFRHn5im+p5Vtfwb8rNCgzMzMPBafmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllUrsSlKTpkt6TtDSnrJ+keZJ+l/7cIy2XpB9IWiFpiaRRXRW8mZmVr/ZeQc0Axm9TdjXwWEQMBR5LlwFOInmS7lBgEjC18DDNzKzStCtBRcSTwLZPxj0d+FE6/yPgjJzyf4/EM8DukgYUI1gzM6schXwGtXdErARIf+6Vlg8E3s7Zrikt24qkSZIWSlq4Zs2aAsIwKz9uH2Zdc5OE8pTFJwoipkVEfUTUV1dXd0EYZj2X24dZYQlqdXPXXfrzvbS8CdgvZ7tBwLsFHMfMzCpQIQnqYWBiOj8ReCin/IL0br7PAH9s7go0MzNrrx3bs5GkmcBooL+kJuB64CbgQUkXAW8BZ6eb/wI4GVgBrAf+scgxm5lZBWhXgoqIc1tZNTbPtgFcWkhQZmZmHknCzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyqV2jmecj6WDggZyiA4DrgN2B/wk0P6f62oj4RacjNDOzitTpBBURrwF1AJJ6Ae8As0me/3RbREwpSoRmZlaRitXFNxb4fUS8WaT6zMyswhUrQU0AZuYsXyZpiaTpkvbIt4OkSZIWSlq4Zs2afJuYVSy3D7MiJChJfwOcBvw0LZoKDCHp/lsJfCfffhExLSLqI6K+urq60DDMyorbh1lxrqBOAhZHxGqAiFgdEVsi4mPgbuDIIhzDzMwqTDES1LnkdO9JGpCz7kxgaRGOYWZmFabTd/EBSNoFOBH4ck7xLZLqgAAat1lnZmbWLgUlqIhYD+y5Tdn5BUVkZmaGR5IwM7OMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMKug2c7OeRL2qGPRuU9HqMrOu5QRlFSO2bOKo6x4pSl3Pfmt8Ueoxs9a5i8/MzDLJCcrMzDLJCcrMzDLJCcrMzDLJCcrMzDLJCcrMzDKp4NvMJTUCfwK2AJsjol5SP+ABoIbkmVDnRMT7hR7LzMwqR7GuoD4bEXURUZ8uXw08FhFDgcfSZasw+w8YgKSCp/0HDGj7YGZWdrrqi7qnA6PT+R8B84GruuhYllFvrVpF076DCq6nWKM/mFnPUowrqAB+JWmRpElp2d4RsRIg/bnXtjtJmiRpoaSFa9asKUIYZuXD7cOsOAnq2IgYBZwEXCrphPbsFBHTIqI+Iuqrq6uLEIZZ+XD7MCtCgoqId9Of7wGzgSOB1ZIGAKQ/3yv0OGZmVlkKSlCSdpXUt3keGAcsBR4GJqabTQQeKuQ4ZmZWeQq9SWJvYLak5rrui4hHJD0PPCjpIuAt4OwCj2NmZhWmoAQVEa8Dn85Tvg4YW0jdZmZW2TyShJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZfIJ2F31RF0zM+tBsvgEbF9BmZlZJnU6QUnaT9ITkl6RtEzSP6Xl35T0jqSGdDq5eOGamVmlKKSLbzPw9YhYnD60cJGkeem62yJiSuHhmZlZpep0goqIlcDKdP5Pkl4BBhYrMDMzq2xF+QxKUg0wEng2LbpM0hJJ0yXt0co+kyQtlLRwzZo1xQjDrGy4fZgVIUFJ6gP8DJgcER8AU4EhQB3JFdZ38u0XEdMioj4i6qurqwsNw6ysuH2YFZigJFWRJKd7I+I/ACJidURsiYiPgbuBIwsP08zMKk0hd/EJuAd4JSK+m1Oe+y2tM4GlnQ/PzMwqVSF38R0LnA+8JKkhLbsWOFdSHRBAI/DlgiI0M7OKVMhdfAsA5Vn1i86HY2ZmlvBIEmZmlkkei8+6jHpVFWVcLvWqKkI0ZtbTOEFZl4ktmzjqukcKrufZb40vQjRm1tO4i8/MzDLJCcrMzDLJCcrMzDLJCcrMzDLJCcrMrJtl8fHqWeS7+MzMulkWH6+eRb6CMjOzTHKCMjOzTHIXn5mZZXLkFycoMzPL5Mgv7uIzM7NM6rIEJWm8pNckrZB0daH1+bZMM7PK0iVdfJJ6AXcAJwJNwPOSHo6Ilztbp2/LNDOrLF31GdSRwIqIeB1A0v3A6UCnE1TW7D9gAG+tWlVwPYP32Yc3V64sQkTlTcr3bEzLIreNthXrhoQdelWVddtQRBS/UunzwPiI+B/p8vnAURFxWc42k4BJ6eLBwGtFD6T9+gNrS3j8Qjj20mgr9rUR0elPizPUPsr5d5Rl5Rx7u9tGV11B5UvpW2XCiJgGTOui43eIpIURUV/qODrDsZdGV8eelfbh31FpOPZEV90k0QTsl7M8CHi3i45lZmZlqKsS1PPAUEm1kv4GmAA83EXHMjOzMtQlXXwRsVnSZcCjQC9gekQs64pjFUnJu1IK4NhLoyfH3hE9+XU69tIoWuxdcpOEmZlZoTyShJmZZZITlJmZZVLFJChJvSS9IGluulwr6VlJv5P0QHozB5J2SpdXpOtrShz37pJmSXpV0iuSjpbUT9K8NPZ5kvZIt5WkH6SxL5E0qsSx/7OkZZKWSpopqXdWz7uk6ZLek7Q0p6zD51nSxHT730ma2J2vobPcNkoSu9tGO1RMggL+CXglZ/lm4LaIGAq8D1yUll8EvB8RBwK3pduV0veBRyJiGPBpktdwNfBYGvtj6TLAScDQdJoETO3+cBOSBgKXA/URcRjJzTITyO55nwFs++XBDp1nSf2A64GjSEZTub654Wac20Y3ctvoQNuIiLKfSL6H9RgwBphL8kXitcCO6fqjgUfT+UeBo9P5HdPtVKK4PwW8se3xSUYVGJDODwBeS+fvAs7Nt10JYh8IvA30S8/jXOAfsnzegRpgaWfPM3AucFdO+VbbZXFy23DbaGfMJWkblXIF9T3gSuDjdHlP4L8iYnO63ETyRwN//eMhXf/HdPtSOABYA/zftAvm3yTtCuwdESvTGFcCe6Xbt8Seyn1d3Soi3gGmAG8BK0nO4yJ6xnlv1tHznJnz3wFuG93MbWOr8u0q+wQl6XPAexGxKLc4z6bRjnXdbUdgFDA1IkYCf+avl9L5ZCb29PL9dKAW2BfYleTyf1tZPO9taS3WnvQa3DbcNrpCUdtG2Sco4FjgNEmNwP0kXRnfA3aX1PxF5dyhmFqGaUrX7wb8oTsDztEENEXEs+nyLJJGuVrSAID053s522dliKm/B96IiDURsQn4D+AYesZ5b9bR85yl898ebhul4bbRzvNf9gkqIq6JiEERUUPyQeTjEXEe8ATw+XSzicBD6fzD6TLp+scj7TTtbhGxCnhb0sFp0ViSR5bkxrht7Bekd9J8Bvhj82V4CbwFfEbSLpLEX2PP/HnP0dHz/CgwTtIe6bvkcWlZJrltuG0UoHvaRik+JCzVBIwG5qbzBwDPASuAnwI7peW90+UV6foDShxzHbAQWALMAfYg6X9+DPhd+rNfuq1IHhT5e+AlkruEShn7vwCvAkuBHwM7ZfW8AzNJPg/YRPJu76LOnGfgS+lrWAH8Y6n/5jvw+t02ujd2t412HNtDHZmZWSaVfRefmZn1TE5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QGSZpi6SGdMTjn0rapZXtfiFp907Uv6+kWQXE1yipf2f3N+sst43K4NvMM0zShxHRJ52/F1gUEd/NWS+S3+HHrdXRxfE1knzPYW0pjm+Vy22jMvgKqud4CjhQUo2SZ9/cCSwG9mt+t5az7m4lz5r5laSdASQdKOnXkl6UtFjSkHT7pen6CyU9JOkRSa9Jur75wJLmSFqU1jmpJK/erHVuG2XKCaoHSMffOonkm9kABwP/HhEjI+LNbTYfCtwREYcC/wWclZbfm5Z/mmTcr3zDvBwJnEfyDf2zJdWn5V+KiCOAeuBySaUeSdkMcNsod05Q2bazpAaS4VzeAu5Jy9+MiGda2eeNiGhI5xcBNZL6AgMjYjZARGyIiPV59p0XEesi4iOSASyPS8svl/Qi8AzJgI9DC35lZoVx26gAO7a9iZXQRxFRl1uQdK3z5+3sszFnfguwM/mHus9n2w8kQ9JoktGXj46I9ZLmk4wNZlZKbhsVwFdQFSAiPgCaJJ0BIGmnVu56OlFSv7Rv/gzgaZKh/d9PG+Aw4DPdFrhZF3PbyDYnqMpxPkl3xBLgN8A+ebZZQDKycgPws4hYCDwC7JjudwNJV4ZZOXHbyCjfZm5AcqcSyW2xl5U6FrMscdsoHV9BmZlZJvkKyszMMslXUGZmlklOUGZmlklOUGZmlklOUGZmlklOUGZmlkn/H+LDZoiBEQ8dAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x216 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import seaborn as sns\n",
    "\n",
    "bins = np.linspace(df.Principal.min(), df.Principal.max(), 10)\n",
    "g = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\n",
    "g.map(plt.hist, 'Principal', bins=bins, ec=\"k\")\n",
    "\n",
    "g.axes[-1].legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAGfZJREFUeJzt3XuQVOW57/HvTxgdFbygo4yMwKgoopIBZ3tDDYJy2N49XuKOR7GOJx4Naqjo8ZZTVrLdZbyVmhwvkUQLK1HUmA26SUWDCidi4gVwRBBv0UFHQS7RKAchgs/5o9fMHqBhembWTK/u+X2qVnWvt1e/61lMvzy93vX2uxQRmJmZZc02xQ7AzMwsHycoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCeolEjaU9Ijkt6XNE/SXySdkVLdoyXNSKOu7iBptqT6YsdhxVdO7UJSlaSXJb0m6Zgu3M/qrqq71DhBpUCSgOnAnyJin4g4FDgXqClSPL2LsV+z1sqwXYwF3oqIERHxQhox2dY5QaVjDPCPiPhFc0FELImI/wMgqZek2yS9KmmBpP+ZlI9OzjaekPSWpIeTRo2k8UnZHOC/NtcraUdJDyZ1vSbptKT8Qkm/lfQfwB87czCSpki6T9Ks5Jvvt5N9LpY0pdV290maK2mRpJ9soa5xybfm+Ul8fToTm5WUsmkXkuqAW4ETJTVI2n5Ln21JjZJuSl6bK2mkpGck/VXSJck2fSQ9l7z3jeZ48+z3f7X698nbxspaRHjp5AJcAdy5ldcvBv538nw7YC5QC4wG/k7uG+U2wF+Ao4FK4CNgCCDgcWBG8v6bgP+WPN8FeAfYEbgQaAL6bSGGF4CGPMvxebadAjya7Ps04AvgkCTGeUBdsl2/5LEXMBsYnqzPBuqB3YE/ATsm5dcANxT77+Wle5YybBcXAncnz7f42QYagUuT53cCC4C+QBWwPCnvDezUqq73ACXrq5PHccDk5Fi3AWYAxxb779qdi7uCuoCke8g1qH9ExD+R+6ANl3RWssnO5BrZP4BXIqIpeV8DMBhYDXwQEe8m5b8h15hJ6jpV0lXJeiUwMHk+MyL+li+miGhvn/l/RERIegP4NCLeSGJZlMTYAJwj6WJyja0aGEauMTY7Iil7MfkCvC25/2ysByqTdtGsrc/2U8njG0CfiPgS+FLSWkm7AP8PuEnSscA3wABgT2BZqzrGJctryXofcv8+f+pgzCXHCSodi4Azm1ciYqKk3cl9I4TcN6DLI+KZ1m+SNBpY16poA//5N9nSJIkCzoyItzep63ByH/r8b5JeIPctblNXRcSzecqb4/pmkxi/AXpLqgWuAv4pIj5Luv4q88Q6MyL+ZUtxWVkrx3bRen9b+2xvtf0A55E7ozo0Ir6W1Ej+9vPTiLh/K3GUNV+DSsfzQKWkS1uV7dDq+TPApZIqACTtL2nHrdT3FlArad9kvXUjeAa4vFWf/IhCAoyIYyKiLs+ytUa4NTuRa/h/l7Qn8M95tnkJGCVpvyTWHSTt38H9Wekp53bR2c/2zuS6+76WdBwwKM82zwD/vdW1rQGS9mjHPkqeE1QKItdhfDrwbUkfSHoFeIhcvzTAr4A3gfmSFgL3s5Wz14hYS67r4vfJxeAlrV6+EagAFiR13Zj28RQiIl4n1/WwCHgQeDHPNivI9dtPlbSAXKMe2o1hWhGVc7tI4bP9MFAvaS65s6m38uzjj8AjwF+SrvYnyH+2V7aaL8qZmZllis+gzMwsk5ygzMwsk5ygzMwsk5ygzMwsk7o1QY0fPz7I/Y7Bi5dyXTrN7cRLD1gK0q0JauXKld25O7OS5HZiluMuPjMzyyQnKDMzyyQnKDMzyyRPFmtmZefrr7+mqamJtWvXFjuUHq2yspKamhoqKio69H4nKDMrO01NTfTt25fBgweTzB9r3SwiWLVqFU1NTdTW1naoDnfxmVnZWbt2LbvttpuTUxFJYrfdduvUWawTVDcaVF2NpFSWQdXVxT4cs0xzciq+zv4N3MXXjT5ctoymvWpSqavmk6ZU6jEzyyqfQZlZ2Uuz96LQHoxevXpRV1fHwQcfzNlnn82aNWtaXps2bRqSeOut/7wNVGNjIwcffDAAs2fPZuedd2bEiBEccMABHHvsscyYMWOj+idPnszQoUMZOnQohx12GHPmzGl5bfTo0RxwwAHU1dVRV1fHE088sVFMzUtjY2Nn/lm7nM+gzKzspdl7AYX1YGy//fY0NDQAcN555/GLX/yCH/7whwBMnTqVo48+mkcffZQf//jHed9/zDHHtCSlhoYGTj/9dLbffnvGjh3LjBkzuP/++5kzZw6777478+fP5/TTT+eVV16hf//+ADz88MPU19dvMaZS4DMoM7Mudswxx/Dee+8BsHr1al588UUeeOABHn300YLeX1dXxw033MDdd98NwC233MJtt93G7rvvDsDIkSOZMGEC99xzT9ccQJE4QZmZdaH169fzhz/8gUMOOQSA6dOnM378ePbff3/69evH/PnzC6pn5MiRLV2CixYt4tBDD93o9fr6ehYtWtSyft5557V05a1atQqAr776qqXsjDPOSOPwupS7+MzMukBzMoDcGdRFF10E5Lr3Jk2aBMC5557L1KlTGTlyZJv1RWx9EvCI2GjUXDl08RWUoCQ1Al8CG4D1EVEvqR/wGDAYaATOiYjPuiZMM7PSki8ZrFq1iueff56FCxciiQ0bNiCJW2+9tc36XnvtNQ488EAAhg0bxrx58xgzZkzL6/Pnz2fYsGHpHkSRtaeL77iIqIuI5pR8LfBcRAwBnkvWzcxsC5544gkuuOAClixZQmNjIx999BG1tbUbjcDLZ8GCBdx4441MnDgRgKuvvpprrrmmpeuuoaGBKVOm8P3vf7/Lj6E7daaL7zRgdPL8IWA2cE0n4zEzS93A/v1T/e3gwGSkXHtNnTqVa6/d+Lv8mWeeySOPPMI112z83+cLL7zAiBEjWLNmDXvssQc///nPGTt2LACnnnoqH3/8MUcddRSS6Nu3L7/5zW+oLrMf8Kutfk0ASR8An5G7E+L9ETFZ0ucRsUurbT6LiF3zvPdi4GKAgQMHHrpkyZLUgi81klL9oW4hfzvrdh366bzbSboWL17c0h1mxbWFv0VB7aTQLr5RETES+GdgoqRjCw0uIiZHRH1E1FdVVRX6NrMexe3EbHMFJaiI+CR5XA5MAw4DPpVUDZA8Lu+qIM3MrOdpM0FJ2lFS3+bnwDhgIfAUMCHZbALwZFcFaWZmPU8hgyT2BKYl4+t7A49ExNOSXgUel3QR8CFwdteFaWZmPU2bCSoi3ge+lad8FTC2K4IyMzPzVEdmZpZJTlBmVvb2qhmY6u029qoZWNB+ly1bxrnnnsu+++7LsGHDOPHEE3nnnXdYtGgRY8aMYf/992fIkCHceOONLT8bmTJlCpdddtlmdQ0ePJiVK1duVDZlyhSqqqo2uoXGm2++CcA777zDiSeeyH777ceBBx7IOeecw2OPPdayXZ8+fVpuyXHBBRcwe/ZsTj755Ja6p0+fzvDhwxk6dCiHHHII06dPb3ntwgsvZMCAAaxbtw6AlStXMnjw4Hb9TQrhufgKMKi6mg+XLSt2GGbWQUs//ojDb3g6tfpe/tfxbW4TEZxxxhlMmDChZdbyhoYGPv30Uy688ELuu+8+xo0bx5o1azjzzDO59957W2aKaI/vfOc7LbOcN1u7di0nnXQSd9xxB6eccgoAs2bNoqqqqmX6pdGjR3P77be3zNc3e/bslve//vrrXHXVVcycOZPa2lo++OADTjjhBPbZZx+GDx8O5O4t9eCDD3LppZe2O+ZCOUEVIK17yfguuGY9x6xZs6ioqOCSSy5pKaurq+OBBx5g1KhRjBs3DoAddtiBu+++m9GjR3coQeXzyCOPcOSRR7YkJ4Djjjuu4PfffvvtXH/99dTW1gJQW1vLddddx2233cavf/1rACZNmsSdd97J9773vVRizsddfGZmXWDhwoWb3RID8t8qY99992X16tV88cUX7d5P6267uro6vvrqqy3uu1CF3M5j4MCBHH300S0Jqyv4DMrMrBtteluM1rZUvjX5uvg6K1+M+cquv/56Tj31VE466aRU99/MZ1BmZl3goIMOYt68eXnL586du1HZ+++/T58+fejbt2+X7rs97980xny389hvv/2oq6vj8ccf7/C+tsYJysysC4wZM4Z169bxy1/+sqXs1VdfZciQIcyZM4dnn30WyN3Y8IorruDqq69Obd/f/e53+fOf/8zvf//7lrKnn36aN954o6D3X3XVVfz0pz+lsbERgMbGRm666SauvPLKzbb90Y9+xO23355K3JtyF5+Zlb3qAXsXNPKuPfW1RRLTpk1j0qRJ3HzzzVRWVjJ48GDuuusunnzySS6//HImTpzIhg0bOP/88zcaWj5lypSNhnW/9NJLAAwfPpxttsmdV5xzzjkMHz6cxx57bKP7Sd17770cddRRzJgxg0mTJjFp0iQqKioYPnw4P/vZzwo6vrq6Om655RZOOeUUvv76ayoqKrj11ltb7hDc2kEHHcTIkSMLvnV9exR0u4201NfXx6anjaUgrdtk1HzS5NttlL8O3W6jtVJtJ1ni221kR3fcbsPMzKxbOUGZmVkmOUGZWVlyF3jxdfZv4ARlZmWnsrKSVatWOUkVUUSwatUqKisrO1yHR/GZWdmpqamhqamJFStWFDuUHq2yspKamo4PDHOCKlHb0bFfneczsH9/lixdmkpdZllQUVHRMo+clS4nqBK1DlIdsm5mljUFX4OS1EvSa5JmJOu1kl6W9K6kxyRt23VhmplZT9OeQRI/ABa3Wr8FuDMihgCfARelGZiZmfVsBSUoSTXAScCvknUBY4Ankk0eAk7vigDNzKxnKvQM6i7gauCbZH034POIWJ+sNwED8r1R0sWS5kqa6xE1Zvm5nZhtrs0EJelkYHlEtJ67Pd/wsbw/OIiIyRFRHxH1VVVVHQzTrLy5nZhtrpBRfKOAUyWdCFQCO5E7o9pFUu/kLKoG+KTrwjQzs56mzTOoiLguImoiYjBwLvB8RJwHzALOSjabADzZZVGamVmP05mpjq4BfijpPXLXpB5IJyQzM7N2/lA3ImYDs5Pn7wOHpR+SmZmZJ4s1M7OMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMajNBSaqU9Iqk1yUtkvSTpLxW0suS3pX0mKRtuz5cMzPrKQo5g1oHjImIbwF1wHhJRwC3AHdGxBDgM+CirgvTzMx6mjYTVOSsTlYrkiWAMcATSflDwOldEqGZmfVIBV2DktRLUgOwHJgJ/BX4PCLWJ5s0AQO28N6LJc2VNHfFihVpxGxWdtxOzDZXUIKKiA0RUQfUAIcBB+bbbAvvnRwR9RFRX1VV1fFIzcqY24nZ5to1ii8iPgdmA0cAu0jqnbxUA3ySbmhmZtaTFTKKr0rSLsnz7YHjgcXALOCsZLMJwJNdFaSZmfU8vdvehGrgIUm9yCW0xyNihqQ3gUcl/RvwGvBAF8ZpZmY9TJsJKiIWACPylL9P7nqUmZlZ6jyThJmZZZITlJmZZZITlJmZZZITlJmZZVLZJqhB1dVISmUxM7PuV8gw85L04bJlNO1Vk0pdNZ80pVKPmZkVrmzPoMzMrLQ5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSa1maAk7S1plqTFkhZJ+kFS3k/STEnvJo+7dn24ZmbWUxRyBrUeuDIiDgSOACZKGgZcCzwXEUOA55J1MzOzVLSZoCJiaUTMT55/CSwGBgCnAQ8lmz0EnN5VQZqZWc/TrmtQkgYDI4CXgT0jYinkkhiwxxbec7GkuZLmrlixonPRmpUptxOzzRWcoCT1AX4HTIqILwp9X0RMjoj6iKivqqrqSIxmZc/txGxzBSUoSRXkktPDEfHvSfGnkqqT16uB5V0TopmZ9USFjOIT8ACwOCLuaPXSU8CE5PkE4Mn0w7PusB20edv7QpZB1dXFPhQzKyOF3PJ9FHA+8IakhqTseuBm4HFJFwEfAmd3TYjW1dYBTXvVdLqemk+aOh+MmVmizQQVEXMAbeHlsemGk03qVZHKf77qvW1q/4mrV0Uq9ZiZZVUhZ1A9Xmz4msNveLrT9bz8r+NTqae5LjOzcuapjszMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJPKdiaJtKYnMjOz4ijbBJXW9ETgaYXMzIrBXXxmZpZJTlBmZpZJTlBmZpZJZXsNqtylOQjE95ayrBlUXc2Hy5Z1up7tt+nFV99sSCEiGNi/P0uWLk2lLiuME1SJ8iAQK2cfLluW2l2e06inuS7rXm128Ul6UNJySQtblfWTNFPSu8njrl0bppmZ9TSFXIOaAmz6Ffta4LmIGAI8l6xbD7cdICmVZVB1dbEPx8yKrM0uvoj4k6TBmxSfBoxOnj8EzAauSTEuK0HrwN0pZpaajo7i2zMilgIkj3tsaUNJF0uaK2nuihUrOrg7s/JWDu1kUHV1amfQZtANgyQiYjIwGaC+vj66en9mpagc2klaAxvAZ9CW09EzqE8lVQMkj8vTC8nMzKzjCeopYELyfALwZDrhmJmZ5RQyzHwq8BfgAElNki4CbgZOkPQucEKybmZmlppCRvH9yxZeGptyLGZmZi0yNRefRwGZmVmzTE115FFAZmbWLFMJyoojrYlnPemsmaXJCcpSm3jWk86aWZoydQ3KzMysmROUmZllkhOUmZllkhOUmZllkhOUZZLvLdU9/NtDyzKP4rNM8r2luod/e2hZ5gRlqUnr91TNdZlZz+YEZalJ6/dU4N9UmZmvQZmZWUb5DMoyKc3uwm16VaRyEX9g//4sWbo0hYjKU6pdvL239fRbBRhUXc2Hy5alUlcWP99OUJZJaXcXpjEQwIMAti7tv5mn32pbuQ9ycRefmZllUqbOoNLsIjAzs9KWqQTlUWBmZtasUwlK0njgZ0Av4FcRcXMqUZmlqBzvd5XmxXErTFqDbQC26V3BN+u/TqWuctbhBCWpF3APcALQBLwq6amIeDOt4MzSUI73u0rr4ri71Av3jQfudLvODJI4DHgvIt6PiH8AjwKnpROWmZn1dIqIjr1ROgsYHxH/I1k/Hzg8Ii7bZLuLgYuT1QOAtzsebovdgZUp1JMFPpZs6uixrIyIdp9qdVE7Af9NsqqnH0tB7aQz16DydcZulu0iYjIwuRP72XzH0tyIqE+zzmLxsWRTdx9LV7QT8N8kq3wshelMF18TsHer9Rrgk86FY2ZmltOZBPUqMERSraRtgXOBp9IJy8zMeroOd/FFxHpJlwHPkBtm/mBELEotsq1LvSukiHws2VQux1IuxwE+lqzqsmPp8CAJMzOzruS5+MzMLJOcoMzMLJMyn6Ak7S1plqTFkhZJ+kFS3k/STEnvJo+7FjvWtkiqlPSKpNeTY/lJUl4r6eXkWB5LBp1knqRekl6TNCNZL8njAJDUKOkNSQ2S5iZlJfMZczvJtnJpK93dTjKfoID1wJURcSBwBDBR0jDgWuC5iBgCPJesZ906YExEfAuoA8ZLOgK4BbgzOZbPgIuKGGN7/ABY3Gq9VI+j2XERUdfqNx2l9BlzO8m2cmor3ddOIqKkFuBJcvP/vQ1UJ2XVwNvFjq2dx7EDMB84nNyvsHsn5UcCzxQ7vgLir0k+jGOAGeR+uF1yx9HqeBqB3TcpK9nPmNtJdpZyaivd3U5K4QyqhaTBwAjgZWDPiFgKkDzuUbzICpec6jcAy4GZwF+BzyNifbJJEzCgWPG1w13A1cA3yfpulOZxNAvgj5LmJdMOQel+xgbjdpIl5dRWurWdZOp+UFsjqQ/wO2BSRHyR1rT33S0iNgB1knYBpgEH5tuse6NqH0knA8sjYp6k0c3FeTbN9HFsYlREfCJpD2CmpLeKHVBHuJ1kSxm2lW5tJyWRoCRVkGt0D0fEvyfFn0qqjoilkqrJfdMqGRHxuaTZ5K4X7CKpd/KNqhSmjBoFnCrpRKAS2Inct8RSO44WEfFJ8rhc0jRys/WX1GfM7SSTyqqtdHc7yXwXn3JfAR8AFkfEHa1eegqYkDyfQK7PPdMkVSXfCJG0PXA8uQuns4Czks0yfywRcV1E1ETEYHJTXD0fEedRYsfRTNKOkvo2PwfGAQspoc+Y20k2lVNbKUo7KfZFtwIuyh1N7vR3AdCQLCeS68d9Dng3eexX7FgLOJbhwGvJsSwEbkjK9wFeAd4DfgtsV+xY23FMo4EZpXwcSdyvJ8si4EdJecl8xtxOsr+UelspRjvxVEdmZpZJme/iMzOznskJyszMMskJyszMMskJyszMMskJyszMMskJyszMMskJyszMMskJqsRJmp5M3LioefJGSRdJekfSbEm/lHR3Ul4l6XeSXk2WUcWN3qx7uJ2UJv9Qt8RJ6hcRf0umhHkV+C/Ai8BI4EvgeeD1iLhM0iPAvRExR9JAclP855uE06ysuJ2UppKYLNa26gpJZyTP9wbOB/5vRPwNQNJvgf2T148HhrWa4XonSX0j4svuDNisCNxOSpATVAlLpu8/HjgyItYksz6/Tf5bE0CuS/fIiPiqeyI0Kz63k9Lla1ClbWfgs6TRDSV3S4IdgG9L2lVSb+DMVtv/EbiseUVSXbdGa1YcbiclygmqtD0N9Ja0ALgReAn4GLiJ3N1UnwXeBP6ebH8FUC9pgaQ3gUu6P2Szbud2UqI8SKIMSeoTEauTb4bTgAcjYlqx4zLLEreT7PMZVHn6saQGcvfS+QCYXuR4zLLI7STjfAZlZmaZ5DMoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLpP8PlTlGZbaTvVAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x216 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "bins = np.linspace(df.age.min(), df.age.max(), 10)\n",
    "g = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\n",
    "g.map(plt.hist, 'age', bins=bins, ec=\"k\")\n",
    "\n",
    "g.axes[-1].legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "# Pre-processing:  Feature selection/extraction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Lets look at the day of the week people get the loan "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAGepJREFUeJzt3XmcVPW55/HPV2gvIriC2tIBWkQQldtgR+OCQUh4EdzwuoTEKGTMdTQuYQyDSzImN84YF8YlcSVq8EbEhUTMJTcaVIjgztKCiCFebbEVFJgYYxQFfeaPOt1poKGr6VPU6erv+/WqV1edOud3ntNdTz91fnXq91NEYGZmljU7FDsAMzOzprhAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlApUTS3pLuk/S6pAWSnpV0ckptD5U0M422tgdJcyRVFzsOK75SygtJ3SU9L2mRpCEF3M+HhWq7rXGBSoEkATOApyJiv4g4FBgDVBQpno7F2K9ZYyWYF8OBVyNiUETMTSMm2zoXqHQMAz6NiNvrF0TEmxHxcwBJHSRdJ+lFSYsl/fdk+dDkbGO6pFclTU2SGkkjk2XzgH+pb1fSzpLuTtpaJOmkZPk4SQ9J+g/gD605GElTJN0maXbyzvfLyT6XSZrSaL3bJM2XtFTSv22hrRHJu+aFSXxdWhObtSklkxeSqoBrgVGSaiTttKXXtqRaSVclz82XNFjSY5L+S9K5yTpdJD2RbLukPt4m9vs/G/1+msyxkhYRvrXyBlwE3LCV588Bfpjc/ydgPlAJDAX+Su4d5Q7As8DRQCfgLaAvIOBBYGay/VXAt5L7uwHLgZ2BcUAdsMcWYpgL1DRx+0oT604B7k/2fRLwAXBIEuMCoCpZb4/kZwdgDjAweTwHqAa6AU8BOyfLLwGuKPbfy7ftcyvBvBgH3Jzc3+JrG6gFzkvu3wAsBroC3YH3kuUdgV0atfUaoOTxh8nPEcDk5Fh3AGYCxxT777o9b+4KKgBJt5BLqE8j4ovkXmgDJZ2arLIruST7FHghIuqS7WqA3sCHwBsR8edk+b3kkpmkrRMlTUgedwJ6JvdnRcT/ayqmiGhpn/l/RERIWgK8GxFLkliWJjHWAKdLOodcspUDA8glY70vJcueTt4A70jun421QyWSF/Wae23/Nvm5BOgSEX8D/iZpnaTdgL8DV0k6Bvgc6AHsDaxq1MaI5LYoedyF3O/nqW2Muc1xgUrHUuCU+gcRcb6kbuTeEULuHdCFEfFY440kDQU+abToM/7xN9nSIIkCTomIP23S1uHkXvRNbyTNJfcublMTIuLxJpbXx/X5JjF+DnSUVAlMAL4YEX9Juv46NRHrrIj4xpbispJWinnReH9be21vNX+AM8idUR0aEesl1dJ0/vw0Iu7YShwlzZ9BpeNJoJOk8xot69zo/mPAeZLKACQdIGnnrbT3KlApqU/yuHESPAZc2KhPflA+AUbEkIioauK2tSTcml3IJf5fJe0NfK2JdZ4DjpK0fxJrZ0kHbOP+rO0p5bxo7Wt7V3LdfeslHQv0amKdx4D/1uizrR6S9mrBPto8F6gURK7DeDTwZUlvSHoBuIdcvzTAncArwEJJLwN3sJWz14hYR67r4nfJh8FvNnr6SqAMWJy0dWXax5OPiHiJXNfDUuBu4Okm1llNrt9+mqTF5JK6/3YM04qolPMihdf2VKBa0nxyZ1OvNrGPPwD3Ac8mXe3Tafpsr2TVfyhnZmaWKT6DMjOzTHKBMjOzTHKBMjOzTHKBMjOzTNquBWrkyJFB7nsMvvlWqrdWc5741g5uedmuBWrNmjXbc3dmbZLzxCzHXXxmZpZJLlBmZpZJLlBmZpZJHizWzErO+vXrqaurY926dcUOpV3r1KkTFRUVlJWVbdP2LlBmVnLq6uro2rUrvXv3Jhk/1raziGDt2rXU1dVRWVm5TW24i8/MSs66devYc889XZyKSBJ77rlnq85iXaCs5PUqL0dSq2+9ysuLfSjWAi5Oxdfav4G7+KzkrVi1irp9K1rdTsU7dSlEY2b58hmUmZW8tM6iW3I23aFDB6qqqjj44IM57bTT+Oijjxqee/jhh5HEq6/+Yxqo2tpaDj74YADmzJnDrrvuyqBBg+jXrx/HHHMMM2fO3Kj9yZMn079/f/r3789hhx3GvHnzGp4bOnQo/fr1o6qqiqqqKqZPn75RTPW32tra1vxaCy6vMyhJ/wP4DrkhKpYA3wbKgfuBPYCFwJkR8WmB4jQz22ZpnUXXy+dseqeddqKmpgaAM844g9tvv52LL74YgGnTpnH00Udz//338+Mf/7jJ7YcMGdJQlGpqahg9ejQ77bQTw4cPZ+bMmdxxxx3MmzePbt26sXDhQkaPHs0LL7zAPvvsA8DUqVOprq7eYkxtQbNnUJJ6ABcB1RFxMNABGANcA9wQEX2BvwBnFzJQM7O2asiQIbz22msAfPjhhzz99NPcdddd3H///XltX1VVxRVXXMHNN98MwDXXXMN1111Ht27dABg8eDBjx47llltuKcwBFEm+XXwdgZ0kdQQ6AyuBYeSmIIbcNM6j0w/PzKxt27BhA7///e855JBDAJgxYwYjR47kgAMOYI899mDhwoV5tTN48OCGLsGlS5dy6KGHbvR8dXU1S5cubXh8xhlnNHTlrV27FoCPP/64YdnJJ5+cxuEVVLNdfBHxtqRJwArgY+APwALg/YjYkKxWB/RoantJ5wDnAPTs2TONmM1KjvOk9NQXA8idQZ19dq6Tadq0aYwfPx6AMWPGMG3aNAYPHtxsexFbHwQ8Ija6aq4UuviaLVCSdgdOAiqB94GHgK81sWqTv72ImAxMBqiurs57mHWz9sR5UnqaKgZr167lySef5OWXX0YSn332GZK49tprm21v0aJFHHjggQAMGDCABQsWMGzYsIbnFy5cyIABA9I9iCLLp4vvK8AbEbE6ItYDvwGOBHZLuvwAKoB3ChSjmVlJmD59OmeddRZvvvkmtbW1vPXWW1RWVm50BV5TFi9ezJVXXsn5558PwMSJE7nkkksauu5qamqYMmUK3/3udwt+DNtTPlfxrQC+JKkzuS6+4cB8YDZwKrkr+cYCjxQqSDOz1ui5zz6pfo+tZ3KlXEtNmzaNSy+9dKNlp5xyCvfddx+XXHLJRsvnzp3LoEGD+Oijj9hrr7342c9+xvDhwwE48cQTefvttznyyCORRNeuXbn33nspL7Evk6u5fk0ASf8GfB3YACwid8l5D/5xmfki4FsR8cnW2qmuro758+e3NmazFpGU2hd188iXVg9f4DxpvWXLljV0h1lxbeFvkVee5PU9qIj4EfCjTRa/DhyWz/ZmZmYt5ZEkzMwsk1ygzMwsk1ygzMwsk1ygzMwsk1ygzMwsk1ygzKzk7VvRM9XpNvatyG84qlWrVjFmzBj69OnDgAEDGDVqFMuXL2fp0qUMGzaMAw44gL59+3LllVc2fIVhypQpXHDBBZu11bt3b9asWbPRsilTptC9e/eNptB45ZVXAFi+fDmjRo1i//3358ADD+T000/ngQceaFivS5cuDVNynHXWWcyZM4fjjz++oe0ZM2YwcOBA+vfvzyGHHMKMGTManhs3bhw9evTgk09y3yxas2YNvXv3btHfJB+esNDMSt7Kt9/i8CseTa29538ystl1IoKTTz6ZsWPHNoxaXlNTw7vvvsu4ceO47bbbGDFiBB999BGnnHIKt956a8NIES3x9a9/vWGU83rr1q3juOOO4/rrr+eEE04AYPbs2XTv3r1h+KWhQ4cyadKkhvH65syZ07D9Sy+9xIQJE5g1axaVlZW88cYbfPWrX2W//fZj4MCBQG5uqbvvvpvzzjuvxTHny2dQZmYFMHv2bMrKyjj33HMbllVVVbF8+XKOOuooRowYAUDnzp25+eabufrqq1Pb93333ccRRxzRUJwAjj322IYJEZszadIkLr/8ciorKwGorKzksssu47rrrmtYZ/z48dxwww1s2LBhS820mguUmVkBvPzyy5tNiQFNT5XRp08fPvzwQz744IMW76dxt11VVRUff/zxFvedr3ym8+jZsydHH300v/rVr7Z5P81xF5+Z2Xa06bQYjW1p+dY01cXXWk3F2NSyyy+/nBNPPJHjjjsu1f3X8xmUmVkBHHTQQSxYsKDJ5ZuOtfj666/TpUsXunbtWtB9t2T7TWNsajqP/fffn6qqKh588MFt3tfWuECZmRXAsGHD+OSTT/jFL37RsOzFF1+kb9++zJs3j8cffxzITWx40UUXMXHixNT2/c1vfpNnnnmG3/3udw3LHn30UZYsWZLX9hMmTOCnP/0ptbW1ANTW1nLVVVfx/e9/f7N1f/CDHzBp0qRU4t6Uu/jMrOSV9/hCXlfetaS95kji4YcfZvz48Vx99dV06tSJ3r17c+ONN/LII49w4YUXcv755/PZZ59x5plnbnRp+ZQpUza6rPu5554DYODAgeywQ+684vTTT2fgwIE88MADG80ndeutt3LkkUcyc+ZMxo8fz/jx4ykrK2PgwIHcdNNNeR1fVVUV11xzDSeccALr16+nrKyMa6+9tmGG4MYOOuggBg8enPfU9S2R13QbafE0AlYMnm6j/fF0G9nRmuk23MVnZmaZlKkC1au8PLVvevcqsZklzczam0x9BrVi1apUumKAVKd3NrO2Z2uXc9v20dqPkDJ1BmVmloZOnTqxdu3aVv+DtG0XEaxdu5ZOnTptcxuZOoMyM0tDRUUFdXV1rF69utihtGudOnWiomLbe8VcoMys5JSVlTWMI2dtl7v4zMwsk1ygzMwsk1ygzMwsk1ygzMwsk1ygzMwsk/IqUJJ2kzRd0quSlkk6QtIekmZJ+nPyc/dCB2tmZu1HvmdQNwGPRkR/4J+BZcClwBMR0Rd4InlsZmaWimYLlKRdgGOAuwAi4tOIeB84CbgnWe0eYHShgjQzs/YnnzOo/YDVwC8lLZJ0p6Sdgb0jYiVA8nOvpjaWdI6k+ZLm+1vdZk1znphtLp8C1REYDNwWEYOAv9OC7ryImBwR1RFR3b17920M06y0OU/MNpdPgaoD6iLi+eTxdHIF611J5QDJz/cKE6KZmbVHzRaoiFgFvCWpX7JoOPAK8FtgbLJsLPBIQSI0M7N2Kd/BYi8EpkraEXgd+Da54vagpLOBFcBphQnRrHXUoSyV+cHUoSyFaMwsX3kVqIioAaqbeGp4uuGYpS8+W8/hVzza6nae/8nIFKIxs3x5JAkzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8ukvAuUpA6SFkmamTyulPS8pD9LekDSjoUL08zM2puWnEF9D1jW6PE1wA0R0Rf4C3B2moGZmVn7lleBklQBHAfcmTwWMAyYnqxyDzC6EAGamVn7lO8Z1I3ARODz5PGewPsRsSF5XAf0aGpDSedImi9p/urVq1sVrFmpcp6Yba7ZAiXpeOC9iFjQeHETq0ZT20fE5Iiojojq7t27b2OYZqXNeWK2uY55rHMUcKKkUUAnYBdyZ1S7SeqYnEVVAO8ULkwzM2tvmj2DiojLIqIiInoDY4AnI+IMYDZwarLaWOCRgkVpZmbtTmu+B3UJcLGk18h9JnVXOiGZmZnl18XXICLmAHOS+68Dh6UfkpmZmUeSMDOzjHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKB2o56lZcjKZVbr/LyYh+OmVlBtWg+KGudFatWUbdvRSptVbxTl0o7ZmZZ5TMoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLpGYLlKQvSJotaZmkpZK+lyzfQ9IsSX9Ofu5e+HDNzKy9yOcMagPw/Yg4EPgScL6kAcClwBMR0Rd4InlsZmaWimYLVESsjIiFyf2/AcuAHsBJwD3JavcAowsVpJmZtT8t+gxKUm9gEPA8sHdErIRcEQP22sI250iaL2n+6tWrWxetWYlynphtLu8CJakL8GtgfER8kO92ETE5Iqojorp79+7bEqNZyXOemG0urwIlqYxccZoaEb9JFr8rqTx5vhx4rzAhmplZe5TPVXwC7gKWRcT1jZ76LTA2uT8WeCT98MzMrL3KZ8LCo4AzgSWSapJllwNXAw9KOhtYAZxWmBDNzKw9arZARcQ8QFt4eni64ZiZWTH0Ki9nxapVqbTVc599eHPlyla34ynfzcyMFatWUbdvRSptVbxTl0o7HurIMqlXeTmSUrmVorR+P73Ky4t9KGZb5DMoy6QsvpvLkrR+P6X4u7HS4TMoMzPLpJI9g/onSK17J60P/Cx/6lDmd/dm7VzJFqhPwF1EbVh8tp7Dr3g0lbae/8nIVNoxs+3LXXxmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJJTvUkZmZ5S/N8S/VoSyVdlygzMwsk+NfuovPrB2rH/Xfkx9aFvkMyqwd86j/lmU+gzIzs0xygbLU7FvRM7XuIjMzd/FZala+/VbmPmQ1s7YrUwUqi5c5mtn216u8nBWrVrW6nZ777MObK1emEJEVQ6YKVBYvc8yq+quv0uAktqxZsWpVKhdv+MKNtq1VBUrSSOAmoANwZ0RcnUpU1ixffWVmpW6bL5KQ1AG4BfgaMAD4hqQBaQVmZtZaWf2eV6/y8lRi6tyhY0lfmNSaM6jDgNci4nUASfcDJwGvpBGYmVlrZbWnIc0uzCweX1oUEdu2oXQqMDIivpM8PhM4PCIu2GS9c4Bzkof9gD9tpdluwJptCqht8PG1bfkc35qIaPEHoC3Mk3xjact8fG1bc8eXV5605gyqqXPCzapdREwGJufVoDQ/IqpbEVOm+fjatkIeX0vypNCxZIGPr21L6/ha80XdOuALjR5XAO+0LhwzM7Oc1hSoF4G+kiol7QiMAX6bTlhmZtbebXMXX0RskHQB8Bi5y8zvjoilrYwn7y6ONsrH17Zl6fiyFEsh+PjatlSOb5svkjAzMyskDxZrZmaZ5AJlZmaZlJkCJWmkpD9Jek3SpcWOJ02SviBptqRlkpZK+l6xY0qbpA6SFkmaWexYCkHSbpKmS3o1+TseUaQ4nCdtXCnnStp5konPoJJhk5YDXyV3+fqLwDcioiRGpZBUDpRHxEJJXYEFwOhSOT4ASRcD1cAuEXF8seNJm6R7gLkRcWdy1WrniHh/O8fgPCkBpZwraedJVs6gGoZNiohPgfphk0pCRKyMiIXJ/b8By4AexY0qPZIqgOOAO4sdSyFI2gU4BrgLICI+3d7FKeE8aeNKOVcKkSdZKVA9gLcaPa6jxF6Y9ST1BgYBzxc3klTdCEwEPi92IAWyH7Aa+GXSNXOnpJ2LEIfzpO0r5VxJPU+yUqDyGjaprZPUBfg1MD4iPih2PGmQdDzwXkQsKHYsBdQRGAzcFhGDgL8Dxfj8x3nShrWDXEk9T7JSoEp+2CRJZeSSbmpE/KbY8aToKOBESbXkupyGSbq3uCGlrg6oi4j6d/PTySViMeJwnrRdpZ4rqedJVgpUSQ+bpNxkK3cByyLi+mLHk6aIuCwiKiKiN7m/25MR8a0ih5WqiFgFvCWpX7JoOMWZVsZ50oaVeq4UIk8yMeV7gYZNypKjgDOBJZJqkmWXR8R/FjEma5kLgalJYXgd+Pb2DsB5Ym1AqnmSicvMzczMNpWVLj4zM7ONuECZmVkmuUCZmVkmuUCZmVkmuUCZmVkmuUBlgKQfS5qQYnv9JdUkw430SavdRu3PkVSddrtmW+M8aX9coErTaOCRiBgUEf9V7GDMMsp5knEuUEUi6QfJvD6PA/2SZf8q6UVJL0n6taTOkrpKeiMZAgZJu0iqlVQmqUrSc5IWS3pY0u6SRgHjge8kc+tMlHRRsu0Nkp5M7g+vH2ZF0ghJz0paKOmhZCw0JB0q6Y+SFkh6LJkOofEx7CDpHkn/e7v94qxdcZ60by5QRSDpUHJDnQwC/gX4YvLUbyLiixHxz+SmGjg7mXZgDrkh+km2+3VErAf+HbgkIgYCS4AfJd+6vx24ISKOBZ4ChiTbVgNdkiQ+GpgrqRvwQ+ArETEYmA9cnKzzc+DUiDgUuBv4P40OoyMwFVgeET9M8ddjBjhPLCNDHbVDQ4CHI+IjAEn146kdnLzL2g3oQm5IG8jNHTMRmEFu6JB/lbQrsFtE/DFZ5x7goSb2tQA4VLkJ4D4BFpJLwCHARcCXgAHA07mh0NgReJbcu9WDgVnJ8g7Aykbt3gE8GBGNk9EsTc6Tds4FqniaGmNqCrkZRF+SNA4YChART0vqLenLQIeIeDlJvOZ3ErFeudGTvw08AywGjgX6kHv32QeYFRHfaLydpEOApRGxpSmbnwGOlfR/I2JdPrGYbQPnSTvmLr7ieAo4WdJOyTu2E5LlXYGVSbfBGZts8+/ANOCXABHxV+Avkuq7Jc4E/kjTngImJD/nAucCNZEbiPE54ChJ+wMk/fkHAH8Cuks6IlleJumgRm3eBfwn8JAkv9GxQnCetHMuUEWQTGv9AFBDbu6buclT/4vcDKKzgFc32WwqsDu55Ks3FrhO0mKgCvjJFnY5FygHno2Id4F19fuMiNXAOGBa0s5zQP9kSvFTgWskvZTEeuQmx3E9ua6QX0nya8lS5Twxj2beRkg6FTgpIs4sdixmWeU8KS0+5WwDJP0c+BowqtixmGWV86T0+AzKzMwyyf2hZmaWSS5QZmaWSS5QZmaWSS5QZmaWSS5QZmaWSf8feZ3K8s9z83MAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x216 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "df['dayofweek'] = df['effective_date'].dt.dayofweek\n",
    "bins = np.linspace(df.dayofweek.min(), df.dayofweek.max(), 10)\n",
    "g = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\n",
    "g.map(plt.hist, 'dayofweek', bins=bins, ec=\"k\")\n",
    "g.axes[-1].legend()\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "We see that people who get the loan at the end of the week dont pay it off, so lets use Feature binarization to set a threshold values less then day 4 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "      <th>dayofweek</th>\n",
       "      <th>weekend</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>45</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>male</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>33</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>female</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-09-22</td>\n",
       "      <td>27</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>28</td>\n",
       "      <td>college</td>\n",
       "      <td>female</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6</td>\n",
       "      <td>6</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>29</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           0             0     PAIDOFF       1000     30     2016-09-08   \n",
       "1           2             2     PAIDOFF       1000     30     2016-09-08   \n",
       "2           3             3     PAIDOFF       1000     15     2016-09-08   \n",
       "3           4             4     PAIDOFF       1000     30     2016-09-09   \n",
       "4           6             6     PAIDOFF       1000     30     2016-09-09   \n",
       "\n",
       "    due_date  age             education  Gender  dayofweek  weekend  \n",
       "0 2016-10-07   45  High School or Below    male          3        0  \n",
       "1 2016-10-07   33              Bechalor  female          3        0  \n",
       "2 2016-09-22   27               college    male          3        0  \n",
       "3 2016-10-08   28               college  female          4        1  \n",
       "4 2016-10-08   29               college    male          4        1  "
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['weekend'] = df['dayofweek'].apply(lambda x: 1 if (x>3)  else 0)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "## Convert Categorical features to numerical values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Lets look at gender:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Gender  loan_status\n",
       "female  PAIDOFF        0.865385\n",
       "        COLLECTION     0.134615\n",
       "male    PAIDOFF        0.731293\n",
       "        COLLECTION     0.268707\n",
       "Name: loan_status, dtype: float64"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.groupby(['Gender'])['loan_status'].value_counts(normalize=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "86 % of female pay there loans while only 73 % of males pay there loan\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Lets convert male to 0 and female to 1:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "      <th>dayofweek</th>\n",
       "      <th>weekend</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>45</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>33</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-09-22</td>\n",
       "      <td>27</td>\n",
       "      <td>college</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>28</td>\n",
       "      <td>college</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6</td>\n",
       "      <td>6</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>29</td>\n",
       "      <td>college</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           0             0     PAIDOFF       1000     30     2016-09-08   \n",
       "1           2             2     PAIDOFF       1000     30     2016-09-08   \n",
       "2           3             3     PAIDOFF       1000     15     2016-09-08   \n",
       "3           4             4     PAIDOFF       1000     30     2016-09-09   \n",
       "4           6             6     PAIDOFF       1000     30     2016-09-09   \n",
       "\n",
       "    due_date  age             education  Gender  dayofweek  weekend  \n",
       "0 2016-10-07   45  High School or Below       0          3        0  \n",
       "1 2016-10-07   33              Bechalor       1          3        0  \n",
       "2 2016-09-22   27               college       0          3        0  \n",
       "3 2016-10-08   28               college       1          4        1  \n",
       "4 2016-10-08   29               college       0          4        1  "
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['Gender'].replace(to_replace=['male','female'], value=[0,1],inplace=True)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "## One Hot Encoding  \n",
    "#### How about education?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "education             loan_status\n",
       "Bechalor              PAIDOFF        0.750000\n",
       "                      COLLECTION     0.250000\n",
       "High School or Below  PAIDOFF        0.741722\n",
       "                      COLLECTION     0.258278\n",
       "Master or Above       COLLECTION     0.500000\n",
       "                      PAIDOFF        0.500000\n",
       "college               PAIDOFF        0.765101\n",
       "                      COLLECTION     0.234899\n",
       "Name: loan_status, dtype: float64"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.groupby(['education'])['loan_status'].value_counts(normalize=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "#### Feature befor One Hot Encoding"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>age</th>\n",
       "      <th>Gender</th>\n",
       "      <th>education</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>45</td>\n",
       "      <td>0</td>\n",
       "      <td>High School or Below</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>33</td>\n",
       "      <td>1</td>\n",
       "      <td>Bechalor</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>27</td>\n",
       "      <td>0</td>\n",
       "      <td>college</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>28</td>\n",
       "      <td>1</td>\n",
       "      <td>college</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>college</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Principal  terms  age  Gender             education\n",
       "0       1000     30   45       0  High School or Below\n",
       "1       1000     30   33       1              Bechalor\n",
       "2       1000     15   27       0               college\n",
       "3       1000     30   28       1               college\n",
       "4       1000     30   29       0               college"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[['Principal','terms','age','Gender','education']].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "#### Use one hot encoding technique to conver categorical varables to binary variables and append them to the feature Data Frame "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>age</th>\n",
       "      <th>Gender</th>\n",
       "      <th>weekend</th>\n",
       "      <th>Bechalor</th>\n",
       "      <th>High School or Below</th>\n",
       "      <th>college</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>45</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>33</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>27</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>28</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Principal  terms  age  Gender  weekend  Bechalor  High School or Below  \\\n",
       "0       1000     30   45       0        0         0                     1   \n",
       "1       1000     30   33       1        0         1                     0   \n",
       "2       1000     15   27       0        0         0                     0   \n",
       "3       1000     30   28       1        1         0                     0   \n",
       "4       1000     30   29       0        1         0                     0   \n",
       "\n",
       "   college  \n",
       "0        0  \n",
       "1        0  \n",
       "2        1  \n",
       "3        1  \n",
       "4        1  "
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Feature = df[['Principal','terms','age','Gender','weekend']]\n",
    "Feature = pd.concat([Feature,pd.get_dummies(df['education'])], axis=1)\n",
    "Feature.drop(['Master or Above'], axis = 1,inplace=True)\n",
    "Feature.head()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Feature selection"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Lets defind feature sets, X:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>age</th>\n",
       "      <th>Gender</th>\n",
       "      <th>weekend</th>\n",
       "      <th>Bechalor</th>\n",
       "      <th>High School or Below</th>\n",
       "      <th>college</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>45</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>33</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>27</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>28</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Principal  terms  age  Gender  weekend  Bechalor  High School or Below  \\\n",
       "0       1000     30   45       0        0         0                     1   \n",
       "1       1000     30   33       1        0         1                     0   \n",
       "2       1000     15   27       0        0         0                     0   \n",
       "3       1000     30   28       1        1         0                     0   \n",
       "4       1000     30   29       0        1         0                     0   \n",
       "\n",
       "   college  \n",
       "0        0  \n",
       "1        0  \n",
       "2        1  \n",
       "3        1  \n",
       "4        1  "
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = Feature\n",
    "X[0:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "What are our lables?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF'],\n",
       "      dtype=object)"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = df['loan_status'].values\n",
    "y[0:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "## Normalize Data "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Data Standardization give data zero mean and unit variance (technically should be done after train test split )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/envs/Python36/lib/python3.6/site-packages/sklearn/preprocessing/data.py:645: DataConversionWarning: Data with input dtype uint8, int64 were all converted to float64 by StandardScaler.\n",
      "  return self.partial_fit(X, y)\n",
      "/opt/conda/envs/Python36/lib/python3.6/site-packages/ipykernel/__main__.py:1: DataConversionWarning: Data with input dtype uint8, int64 were all converted to float64 by StandardScaler.\n",
      "  if __name__ == '__main__':\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([[ 0.51578458,  0.92071769,  2.33152555, -0.42056004, -1.20577805,\n",
       "        -0.38170062,  1.13639374, -0.86968108],\n",
       "       [ 0.51578458,  0.92071769,  0.34170148,  2.37778177, -1.20577805,\n",
       "         2.61985426, -0.87997669, -0.86968108],\n",
       "       [ 0.51578458, -0.95911111, -0.65321055, -0.42056004, -1.20577805,\n",
       "        -0.38170062, -0.87997669,  1.14984679],\n",
       "       [ 0.51578458,  0.92071769, -0.48739188,  2.37778177,  0.82934003,\n",
       "        -0.38170062, -0.87997669,  1.14984679],\n",
       "       [ 0.51578458,  0.92071769, -0.3215732 , -0.42056004,  0.82934003,\n",
       "        -0.38170062, -0.87997669,  1.14984679]])"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X= preprocessing.StandardScaler().fit(X).transform(X)\n",
    "X[0:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "# Classification "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Now, it is your turn, use the training set to build an accurate model. Then use the test set to report the accuracy of the model\n",
    "You should use the following algorithm:\n",
    "- K Nearest Neighbor(KNN)\n",
    "- Decision Tree\n",
    "- Support Vector Machine\n",
    "- Logistic Regression\n",
    "\n",
    "\n",
    "\n",
    "__ Notice:__ \n",
    "- You can go above and change the pre-processing, feature selection, feature-extraction, and so on, to make a better model.\n",
    "- You should use either scikit-learn, Scipy or Numpy libraries for developing the classification algorithms.\n",
    "- You should include the code of the algorithm in the following cells."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# K Nearest Neighbor(KNN)\n",
    "Notice: You should find the best k to build the model with the best accuracy.  \n",
    "**warning:** You should not use the __loan_test.csv__ for finding the best k, however, you can split your train_loan.csv into train and test to find the best __k__."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train set: (276, 8) (276,)\n",
      "Test set: (70, 8) (70,)\n"
     ]
    }
   ],
   "source": [
    "# split train_loan\n",
    "from sklearn.model_selection import train_test_split\n",
    "X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.2, random_state=4 )\n",
    "print ('Train set:', X_train.shape,  y_train.shape)\n",
    "print ('Test set:', X_test.shape,  y_test.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.67142857 0.65714286 0.71428571 0.68571429 0.75714286 0.71428571\n",
      " 0.78571429 0.75714286 0.75714286]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xd4k2X3wPHvSdJBBxsVBQEVFAcWrDhQhmwVEAegOHC/Ku6Be+sPFffmdYADEFAQVPYSeJEpogwHSyoiu7tJk9y/P5LWUlqatkmeJD2f6+pFmz7Jc1qanNz3fZ5zizEGpZRSKtLYrA5AKaWUKosmKKWUUhFJE5RSSqmIpAlKKaVURNIEpZRSKiJpglJKKRWRNEEppZSKSJqglFJKRSRNUEoppSKSw+oAgqVhw4amefPmVoehlFKqAitXrtxtjGlU0XExk6CaN2/OihUrrA5DKaVUBURkayDH6RSfUkqpiKQJSimlVETSBKWUUioixcwalFJKVVdhYSEZGRkUFBRYHUpMSExMpEmTJsTFxVXp/pqglFLKLyMjg9TUVJo3b46IWB1OVDPGsGfPHjIyMmjRokWVHkOn+JRSyq+goIAGDRpocgoCEaFBgwbVGo1qglJKqRI0OQVPdX+XmqCUUkpFJE1QSqmY5/Z4cbm9VocRsEmTJiEibNiwwepQLKUJSikV8zLzC9mX5yIzrxCP11gdToXGjh3LOeecw7hx40J6Ho/HE9LHry6t4lNKxbQcpxu3PykVuD04czwkJThIjrcfco3krul3sXrH6qDGknZEGq/1eu3Q8ebksHjxYubNm0ffvn158skni7/34osv8umnn2Kz2ejduzfDhw/njz/+4D//+Q+7du3CbrczYcIEtm3bxogRI/jmm28AGDp0KOnp6QwZMoTmzZtz3XXXMXPmTIYOHUp2djYjR47E5XJx3HHH8emnn5KUlMQ///zDf/7zHzZt2gTAu+++y7Rp02jYsCF33nknAI888giHH344d9xxR1B/T0U0QSmlYlahx0uu033AbQbIdbrJd3lITXSQGGe3JrhyTJ48mV69etGqVSvq16/PqlWraNeuHdOmTWPy5MksXbqUpKQk9u7dC8DgwYN58MEH6d+/PwUFBXi9XrZt23bIcyQmJrJo0SIA9uzZw4033gjAo48+yocffsjtt9/OHXfcQadOnZg0aRIej4ecnByOPPJILr74Yu688068Xi/jxo1j2bJlIftdaIJSSsUkYwyZ+YXlft/r/36ey0NKgoN4x4ErHhWNdEJl7Nix3HXXXQAMGjSIsWPH0q5dO2bPns21115LUlISAPXr1yc7O5u//vqL/v37A77EE4iBAwcWf/7LL7/w6KOPsn//fnJycujZsycAc+fO5ZNPPgHAbrdTp04d6tSpQ4MGDfjxxx/5559/aNu2LQ0aNAjaz16aJiilVEzKdroDWm8q9HjZl+ci0WHHWLw8tWfPHubOncsvv/yCiODxeBARXnzxRYwxB01JmnICdjgceL3/FoWUvhYpOTm5+PMhQ4YwefJkTj31VEaNGsX8+fMPGeMNN9zAqFGj2LFjB9ddd10lf8LK0SIJpVTMcbo95LsqVwBQ4PbgNQaP15T7wh9qEydO5Oqrr2br1q1s2bKFbdu20aJFCxYtWkSPHj346KOPyMvLA2Dv3r3Url2bJk2aMHnyZACcTid5eXk0a9aMdevW4XQ6yczMZM6cOeWeMzs7m8aNG1NYWMjnn39efHvXrl159913AV8xRVZWFgD9+/dn+vTpLF++vHi0FSqaoJRSMcXrNWTluys+sLz7G4Pba/BaUO03duzY4um6IpdccgljxoyhV69e9O3bl/T0dNLS0hgxYgQAn376KW+88QZt2rTh7LPPZseOHTRt2pQBAwbQpk0bBg8eTNu2bcs95zPPPMMZZ5xB9+7dOeGEE4pvf/3115k3bx6nnHIKp512GmvXrgUgPj6eLl26MGDAAOz20K7fiVXvFIItPT3d6IaFSqnMvEIK3FUrn961bRPHl3iRFsBmE2zaXaKY1+ulXbt2TJgwgZYtW1Z4/Pr162nduvUBt4nISmNMekX31RGUUipmFBR6qpycymIAj9fg9notm/aLJOvWreO4446ja9euASWn6tIiCaVUTPB4DVkF5VftVYcx4DYGm4BNam6/vhNPPLH4uqhw0BGUUiomZOUXhrwKz8r1qZpIE5RSKurludy4POHrtecxBrfHi1en/UJKE5RSKqq5PV5yCqpetVdVuj4VeroGpZSKalkFbqxMD7o+FTo6glJKRa0cp5vCEE7t/ZNVEPDH35n5/LU/n7/355d7TLAsWbKkuH9eed577z1OOeUU0tLSOOecc1i3bl2lz7NlyxbGjBlT7vc7d+5MKC/v0QSllIpKhR4vec7wT+1VpGh9qrrTfvPnz2fIkCFlfm/69On06tXrkPe/4oor+Pnnn1m9ejUPPPAA99xzT6VjqChBhZomKKVU1ClqBBupKz8GcHsNnhCtT82ZM4du3bod8pjatWsXf56bm1s89fjKK68U99D7+eefOfnkk8nLy2PBggWkpaWRlpZG27Ztyc7O5sEHH2ThwoWkpaXx6quvkp+fz6BBg2jTpg0DBw4kPz8/6D9bSboGpZSKOoE2grWa1/hK020Cdltw1qZ2795NXFwcderUqfDYt99+m1deeQWXy8XcuXMBuOuuu+jcuTOTJk3iueee4/333ycpKYkRI0bw9ttv06FDB3JyckhMTGT48OEH7Cv1yiuvkJSUxJo1a1izZg3t2rULys9UHh1BKaWiSlUawVrNawyFAZaln3HGGaSlpXHDDTcwZcqU4lHNjBkzAJg5cyY9evQI6Ly33XYbGzdu5IUXXuDZZ58FwGazMWrUKK666io6depEhw4dAOjQoQP33HMPb7zxBvv378fhOHj88v3333PllVcC0KZNG9q0aRNQHFWlCUopFTWq2wjWah5vxetTS5cuZfXq1XzwwQf07duX1atXs3r16uLO4dOmTStef7r22mtJS0vj/PPPP+R5Bw0aVNzxHOD3338nJSWF7du3F9/24IMP8sEHH5Cfn8+ZZ57Jhg0bynyscFYpaoJSSkWNbKc76i+Orc76lDGGNWvWkJaWBsDHH3/M6tWr+e677w469vfffy/+/Ntvvy3unZeZmcmdd97J999/z549e5g4cSIAGzdu5JRTTmHYsGGkp6ezYcMGUlNTyc7OLn6cjh07Fm/J8csvv7BmzZpKxV9ZugallIoKBYUeCgrDO7V3eO3AdqitiqqsT61cuZK2bdsGNIp56623mD17NnFxcdSrV4/Ro0cDcPfdd3PrrbfSqlUrPvzwQ7p06ULHjh157bXXmDdvHna7nRNPPJHevXtjs9lwOByceuqpDBkyhFtuuYVrr72WNm3akJaWRvv27av1O6iIbrehlIp4Xq9hd64z5L32Sm+3EU72ALb1ePbZZznuuOMYNGhQmKKqvupst6EjKKUilNdryHG5SU1w1PjuBFkFoW8EazWP1+CpoHB+2EMPA4T04uSK2ESCVpFY4bnCchalVKVlF7jJd3nYn1dYo7tn57s8ON3WvSCr0sL3t6gJSqkIVHLjPZfHy948V1Rc9xNsHq8hO0R7PJUnVpY9IkF1f5eaoJSKMN4yNt7zeA17cp2WTu1YIdzdIhzxCezds1eTVBAYY9izZw+JiVUvNNE1KKUiTHnrLcbAvlwXtWvFkRhnD39gYZYb4kawZand4HD27/mH3bt3hfW80USECos5iiQmJtKkSZMqnyukCUpEegGvA3bgA2PM8FLfPxoYDdT1H/OgMeY7//ceAq4HPMAdxpgZoYxVqUhQ0XqLwTeq8BpDUnzsvr8s9HjJtaARrN3hoN7hR4X9vNGkVryd2olxYTlXyKb4RMQOvA30Bk4ELheRE0sd9igw3hjTFhgEvOO/74n+r08CegHv+B9PqZhVmfWW7AJ32NdmwsUY49u+3epAlOVCuQbVHvjDGLPJGOMCxgH9Sh1jgKKWu3WAor4b/YBxxhinMWYz8If/8ZSKWZV9Uc5zecjMK4y59ZIcpxt3DSwIUQcLZYI6CthW4usM/20lPQlcKSIZwHfA7ZW4LyJyk4isEJEVu3bpnLGKXnkuN64qrLcUuD3si6EydJfbS16UNYJVoRPKBFXWKlrpZ9HlwChjTBPgfOBTEbEFeF+MMSONMenGmPRGjRpVO2ClrOD2eMkpqPp6S2GMlKEX7fGkVJFQJqgMoGmJr5vw7xRekeuB8QDGmCVAItAwwPsqFROyCtzVXm+JhTL0rILobwSrgiuUCWo50FJEWohIPL6ihymljvkT6AogIq3xJahd/uMGiUiCiLQAWgLLQhirUpbICWIpdVEZergbqgaDFY1gVeQLWZ2qMcYtIkOBGfhKyD8yxqwVkaeBFcaYKcC9wH9F5G58U3hDjG/Fd62IjAfWAW7gNmOM/vWqmFLo8ZIX5FLqaCxDL+vCZKVAu5krZQljDHtzXSGtVkuKt5MaputVqmN/nkt77UWRYFwHFWg3c211pJQFwlFKHQ1l6NoIVh2KJiilwiycpdSRXIZuRSNYFV00QSkVRsaEf70lUsvQw90IVkUfTVBKhVG2021JoigqQ3dFyHSaFY1gVfTRBKVUmDjdHvIt7JJgjK8gwepybrdFjWBV9NEEpVQYeL2GrHzrX5SLytDzXNbEUtQtItxjyCxnJnmFeWE+q6ouTVBKhUF2hHVJyC5wW3LtkRWNYHNc2XT97AzO+OhEJm34IqKrGtWBNEEpFWIlt2+PJPkuD/vzXGF7wbaqEezzix8nI+tPGiY14pZpQ7h8Ul+27N8U9jhU5WmCUiqEIr1LgtPtDUsZuhXViwDLty/h49Xvc0PbW5k9+Aee7fwyK/5eSudPTuO1pS/g8rjCHpMKnCYopUKovO3bI0lRGbo7hFV1WQXhr150up3cM+tWjqrdlAfPfhK7zc4NbW9l4TU/0u2Y3gz/35N0/ewMlmQsCmtcKnCaoJQKkWjqkuDxGvbmuUJShm5VI9jXl73I73s38FLXt0iOTym+vXHKUXxw4Rg+7fcV+e48+k/ozl0zb2Zv/p6wx6gOTROUUiHg8RqynZE7tVeWUJShWzXFuX73Wt5c/hKXtr6CLs27l3lM92N6s+DqVdyWfg8T1n3OOaPT+GLtZ1pEEUE0QSkVAln5kT+1V5aiMvRgXadkxRSnx+vh3lm3kJpQh6c6vXDIY5Pjknns3OeYfeUPHFP3WO6ceSOXTOzF73t/DVO06lA0QSkVZFXdvj2S5DirX4ZeUGjNFOdHq99l1Y7lPNd5BA1qNQzoPq0bnsyUgXN5qetbrN21hq6ftefF/z1DgbsgxNGqQ9EEpVQQVXf79khSnTJ0j0VTe39mbuX5xU/QtUUvLjp+QKXuaxMbV7W5nkXXrObClv15ZenznPfp6Sz8c16IolUV0QSlVBAFY/v2SFLVMnQrpjiNMQybczs2sfHCea8jIlV6nEbJh/NO71F8cfE3GAyXfXk+Q6ddx668nUGOWFVEE5RSQRKrDVALPV725AZehm7VFOfE9WOZt3UWj5zzDE1qH13tx+vUrCtzr1rO3Wc8yNe/TeTcUWl89vNHeE3s/R9HKk1QSgVBYYw3QPWawMrQrZri3JW3k8cX3M/pjc9kyKk3Be1xazlqMezsJ5hz5TJOaHgS982+jYvGd2P97rVBO4cqnyYoparJGOOb0rI6kBCrqAzdqkawAI/Pv5/cwhxe7v4uNgn+y1qrBicw6bKZvNZjJH/s/Y3un5/Jc4se0wa0IaYJSqlqsqIBqlUOVYae6/JY8nuYtWkak34dz13tH6RVgxNCdh4RYdBJV7FwyGouOeFy3lw+gs6fnMaczTNCds6aThOUUtVgVQNUq5UuQ3e5rZnizHFlM2zO7ZzQ4CSGnn5vWM7ZoFZDXu85kq8um0mCI4HBky/ixm8GsyNne1jOX5NoglKqiqxqgBopisrQrWyI+9yix/g7ZzuvdH+HeHt8WM99dpNzmT14KcPOfoKZm77l3NFt+Wj1e3i8Ne8NS6hoglJBl+N0R8zW4qFk1fbtkcTp9rI7x2nJ72HZX/9j1E8juaHtrbRr3D7s5wdIcCRw9xkPMv/qlbQ9Ip2H593NBeM68cvOnyyJJ9ZoglJBledyk+t0sy/PRWZeYcy+gFu9fXskseJ/2Ol2cu/sfzuVW61F3WP54uJveKf3x2Rkb6PnmA48sWAYua4cq0OLapqgVNAUFHrILlFiXOD2sCfHSa7THVMNOH1Ve7FbUh4NXlv2Ar/v/fWgTuVWEhEuPmEQi65ZzeCTr+X9VW/Q8ZO2TN841erQopYmKBUUhR4vWfkHr0MYfFN+e3KD2yXbSlkRtn17TbN+9y8Vdiq3Ut3EerzY7U2mDpxLanwdhkwZwJApl/FX9jarQ4s6mqBUtXm9hv15h77+xeP1XSOzP8Qb44WaVXsbKR+P18M9s26hTkI9nu70otXhHNLpR57FrMFLePScZ1mwdQ7njm7L+6vewO3V0XegNEGpajHGsD+/MOARhdPtZW+ui+yCwqib9ov07dtrgg9Xv8OPO1bwbOeXqF+rgdXhVCjOHsfQ0+9lwdU/claTc3liwTB6jTmHH3essDq0qKAJSlVLVn7l+88ZIM/lYXdOdE37RcP27bFsa+YW/m/xk3Rr0bvSncqtdnSdZnzW7yv+e8Hn7M7byfljO/LQ3LvIcmZaHVpE0wSlqizH6abAXfUE4/W3xtmb64r4JqtW7W2kfA7oVN616p3KrSQi9Gl1MQuvWc11af9h1E8jOXd0GlN++zLqZhPCRROUqpKCQk/QOgcUenzTflkFld/WIRys2ttI/WvC+jHM3zqbR855hqNSm1odTrWkJtTmuS6v8N3l33NY8hHc9O2VDJ7cn62ZW6wOLeJIrGTu9PR0s2KFzuuGg8vt9W1kF4LHFoHUhDhqxdtD8OhVsy/XZcn2ETtytvO/jO/pf/zAqBwxBMuuvJ10HN2WlvWPZ/KA2SFpBmsVt9fNR6vf5YX/PY3XeLjqlOupm1jf6rAOKc4uDG4zkNaNWlf5MURkpTEmvaLjHFU+g6qRPF7D/vzQJCfwdczOKigkz+UmNTGOeIe1L0ZW7W3kNV5u+vZKlm1fwvd/zmNEt7dx2Grm0/Wx+feRW5jDiG7vxFRyAnDYHNzU7nYubNmfx+bfx8gf37I6pIC0O7JNtRJUoGrmX7yqEmMM+/JcYSkUcHt950qMs5Oa4MBmC/8Iwsrt20f9NJJl25fQpXkPxq39hGxnJu/0Hk2CI8GSeKwyc9N3TP51Ag+c9XhIO5Vb7cjUJnzYZ1xU9PGrFW+nbq3w/B3G1tsRFVKZ+eFvXVRQ6GG3Rd0orNq+/a/sbTy36DE6Hd2VMRdN5plOL/HtH19z9deX1KjWOdnOLB6cc0dYO5VbzW6zR8VHuKacQ5qgRKSXiPwqIn+IyINlfP9VEVnt//hNRPaX+J6nxPemhDJOVbGsgkLLqthKdqNwVqNqsDKs2r7dV612J17j5cVubyEi3NhuKK/3+C+Lts1nwFcXsL9gX9jjssJzix+3rFO5igwhm+ITETvwNtAdyACWi8gUY8y6omOMMXeXOP52oG2Jh8g3xqSFKj4VuDyXOyIao3r8HSsSHB5SE+Owh2jaz23h9u1f/zaB2Zun8VSnF2hWp3nx7QNPupLaCbW5+bur6D+hB+P6T+HwlMaWxBgOS/9azKif3uemtkMt61SurBfKEVR74A9jzCZjjAsYB/Q7xPGXA2NDGI+qAqf7wAawkcDp9rInx0lOCKb9rNy2fG/+Hh6Zdx9ph5/GDWm3HfT93sf15fOLJrM1czP9xneL2bLkAncB982+jaa1mzGsw5NWh6MsFMoEdRRQsjtihv+2g4hIM6AFMLfEzYkiskJEfhCRi0IXpiqP2+Mls4wGsJHA4JuGC3Y3Cqu2LQd4fMEDZDr38Ur3d7Hbyi6zP/foLky8dBqZzv30++I8NuxeV+Zx0ez1ok7l3d4iOS7Z6nCUhUKZoMqafynvmT8ImGiMKflKc7S/Tv4K4DUROfagE4jc5E9iK3bt2lX9iFUxr9ewLy/yW/sUdaPYl1v9JrRWbVsOMG/LLCauH8Ptp9/HiY1OOeSx7Y44na8um4nB0H9C95jq67Zu18+8uXwEl7UeTOdm3awOR1kslAkqAyh5yXcTYHs5xw6i1PSeMWa7/99NwHwOXJ8qOmakMSbdGJPeqFGjYMSsqHwD2Ejg8njZU40mtFZu357ryuH+OUM5rl4r7mw/LKD7tG54ElMGzKV2Qh0undibRdsWhDjK0CvZqfypTi9YHY6KAKFMUMuBliLSQkTi8SWhg6rxROR4oB6wpMRt9UQkwf95Q6ADEHtzGREqq8CaCrZgyHN52JXjrHRRh5Xbtw//35NkZP3Jy93fIdGRGPD9mtVtwZQBc2hauxmDJ/WL+o3xPlj9Nqv/WclzXUZERadyFXohS1DGGDcwFJgBrAfGG2PWisjTItK3xKGXA+PMgW97WwMrROQnYB4wvGT1nwqdXKc7qjqMl6WoG0WgTWit3L591d/L+ODHdxhy6s2ccVSHSt//8JTGTBowk5MateH6qZczYd2YEEQZelsztzB88VN0b3E+/VpdZnU4KkJoLz5VrKDQE7FFEdVxqG4Uxhh257gsmc50eVz0+Pxsspz7WXD1KlITalf5sXJdOQyZMoCF2+bxbOeXuaHtrUGMNLSMMQz6qg8rdyxjwdUro74ZbKyrFW+ndmJctR4j0F582klCAeVv2R4LCgo97M51kuc6uADCyu3b31w+gg171jK86xvVSk4AyfEpfHbRJM4/ri+Pzr+Xl394Pmq2cBi/7nMW/DknJjqVq+DSBKWKL4CNjpezqjEGsgvc7MlxFnejsHL79l/3rOe1pcO56PjL6HHM+UF5zARHAiMv+JyBJ17FS0ue4YkFD+A1kb2WuCv3H55Y8ADtjzyLa9rcaHU4KsJos9gazhjD/jxrpris4PYn40SHF6fHmuTkNV7um3UrKfGpPNN5RFAf22Fz8GqP96iTUIeRP75FpjOTl7u/E7Gd0B+dfx957lxe7v5uzHUqV9UXmX+1Kmwy8wstuzDVStXZCbi6Rv00kuV//8AbPT+gUdJhQX98m9h4qtOL1Emsx0tLniHblcm7vT+JuE7oMzd+y9e/TWTY2U/Qsv7xVoejIlCFb1lEZKiI1AtHMCq8si1sAFtTZWT9yXOLHqNzs25c1vqKkJ1HRLj3zId5tvMIvvtjCld9fXFEdULPcmYybO4dtG54Mrel32N1OCpCBTKmPgJfo9fx/u7kNXdrzxiS7/KQFwENYGsSYwzD5vo7lXd9KyxbFtzQ9jbe6PkBi7ct4LIvz2dfwd6QnzMQzy16nH9yd/CydipXh1BhgjLGPAq0BD4EhgC/i8jzZbUeUtHB1wA2Niv2ItnkX8czZ/N0HurwJEfXaRa28w44cTAf9hnLL7t+ov/4HvyT83fYzl2WHzIWMXrNSG5sexvtjjg9LOcUyu69piJbQKuS/otod/g/3Pg6P0wUkRdDGJsKgaIGsDVv1clae/J388j8+2h7RDrXp4X/GqVex/ZhzEVfsy1rK33Hd2Xr/s1hjwEO7FT+wNlPhO28KYkOGqQkEG/XQoxoEsga1B0ishJ4EVgMnGKMuQU4DbgkxPGpIPJ6fT32akjBXkR5Yv4DZDn3H7JTeaidc3RnJlz6HVnOTPqOt6YT+mtLh/PHvt/C2qncYROS4h3YbUK95HhSEx06mooSgbydaAhcbIzpaYyZYIwpBDDGeIELQxqdCpqifY6s6jdXk83dMpOJG8Zy++n307rhyZbG0u6I05l02SwEof+E7qzasTxs516362feWvEyA068MqydylNLdT1IitfRVLQI5H/oO6B4ZVVEUkXkDABjzPpQBaaCK6vAjStKG8BGs1xXDvfPHkrL+sdzV4CdykPthIYn8vVAXyf0yyaez6I/54f8nB6vh7tn/Yc6CfV4suPwkJ+vSGKcnXjHwS9zOpqKDoEkqHeBkvWpuf7bVJSIhQaw0er/Fj/B9uwMXu72TkRdh9SsTvPiTuhXTO7HtD8O2mggqP7741v89M8qnuvyctg6lYtAasKhL/UsGk3F6WgqIgXyvyIlO437p/b0At8oUVDoIceiTfhqupV/L+XD1e8y5NSbaH/U2VaHc5CiTuinHJbGDd9cwfh1n4fkPFv3b+aF/z1Fj2MuoF+rS0NyjrKklNMguDS7Taivo6mIFEiC2uQvlIjzf9wJbAp1YKr6YrkBbKRzeVzcM+tWGqccycMdnrY6nHLVS6zP+Iu/pUPTTtwx4wY++PHtoD6+MYb75wzFbnMw/LzXwnLtF/xbGFEZSfEO6ifH62gqggTyP/Ef4GzgL3y75J4B3BTKoKwSLd2fA1ETGsBGsjeWvcSve9bxQtc3q92pPNSS41P4tN9XXHBcPx6dfx8jljwXtOfCF+s+4/s/5/LoOc9yZGqToDxmIEoXRgTKYbfpaCqCVPgWwxizE99uuDHNGMOubCfxDhsJDjsJDltA0wORqKY1gI00v+5Zz+vLXqD/8QPofkxvq8MJSIIjgfcv+Ix7Z93KiB+eJdO5j6c6vVitBq67cv/hyQXDOOPIs7m6zQ1BjPbQyiuMqIykeAfxdltU7y4dCypMUCKSCFwPnAQU70dtjLkuhHFZwgBOt7e4P12c3eZPWLaoGvZn5btrZAPYSODxerh31i2kxKfydOeXrA6nUoo6oddNrMv7q94k05nJK93frXIn9Efm30ueO5cR3d8JW6fyQAojAlU0msp1usl1unU2wgKB/E9+CmwAegJPA4PxbeEe8wo9Xgo9XnKdYBMhIc5GvN2XsCK1JWF2QaGlnbprulFrRrLi76W82fPDkHQqDzWb2Hiy4wvUSajHi0ueJtuZybvnf0KiI7HiO5cwY+M3TPntSx48+8mwdioPtDCiMpITHCQ4dDRlhUDe1hxnjHkMyDXGjAYuAE4JbViRx2sM+S7flui7sp3sz3OR53JH1IWv2gDWWhlZf/L8osfp3Kwbl7a+3OpwqkxEuOfMh3i288uTBAbbAAAgAElEQVRM2ziVKyf3J8eVHfD9s5yZDJvj61R+a/rdIYz0QFUpjAj4sf2jqZQEXZsKp0ASVFEZ2H4RORmoAzQPWURRoGgqMLvAze4cJ3tynOQ43bgs3LrC5fZqA1gLGWN4YM4dGEzYOpWH2g1tb+XNnh+yJGNhpTqhP7voMXbm/RP2TuVVLYyojOQErfQLp0B+yyP9+0E9CkwB1gEvhDSqKOP2GnKdbvbludiZXUBmfiEFhZ6wVQV6vIb9+S6dI7fQpF+/YO6WGTx0dng7lYfaZSdewUd9xrFu189cNL47O3K2H/L4JRmL+GTNf8PaqRyCUxgRKB1NhY8c6kVURGzApcaY8eELqWrS09PNihUrqnx/Yww7s51BjMjX3j/ObiMhzlcZaA9BVaDXa9ib54qoqcaaZk/+bs4d3ZbmdVowdeA8y5rBhtKibQu45utLaVCrIRMu+Y5mdVscdEyBu4Cun7Wn0FPIvKtXhK0ZrAg0TE6wpOq2aHeAmlSUVCveTu1qjlZFZKUxJr2i4w75lsPfNWJotSKpwQzg8vw7Fbg7x0l2QWFQpwK1Aaz1Hp9/P9nOTF62sFN5qJ3TtBMTL51GtiuLvuPPY/3utQcd8+rS/2Pjvt/D2qkcQlMYESiH3UaDlASSdTQVEoGMiWeJyH0i0lRE6hd9hDyyGOTxGvJcnn+nAvOqNxWYVVCoDWAtNmfzDL7cMM7fqfwkq8MJqbZHpDN5wGxfJ/Tx3Vn197Li763dtYa3V7zCgBOvpFOzrmGLKZSFEZWRkuCgXnI8jii9djJSHXKKD0BEytrZzBhjjglNSFUTiVN8gSqaCiy65soRwAJsnstNdoH22LNSjiubTp+cRlJcMrMH/xBRzWBDaWvmFgZ+eQE78/5hdN8JnNXkXC4Y14m/sjNYeM2P1EsM3/vXSCxYyHG6yYvh66bCOcUXSCeJgyebVVAVTQW6PF5ynL7mlQkOW3FXi9IKCj2anCJAUafyKQPn1JjkBL5O6F8PmM2gSX0ZPPkieh3bh5/+WcX7538a1uSUGGePuOQEvtFUgsNGVg1bmwqFQDpJXF3W7caYT4IfjoJ/pwLzXB5ECkmw24svEvYaE9ENYDfu+53rp15O92N6c/cZD5EUl2R1SCGxYvsPfLT6Pa5Nu5nTjzzL6nDC7vCUxnx12QyunNyfKb99SY9jLqBvq/BtsB3MjhGhEFfUhcLlienRVKgFMsX3ZokvE4GuwCpjTPj65gcgmqf4KkMgYv/Ys51ZnD+uExlZW8l353N07eYM7/o65zXvYXVoQeV0O+n++VnkFmaz4OpVpMSnWh2SZXILcxn900guO3FwWDtn1E6Mo1Z8dBSkFO0qECujqUib4ru91APXwdf+SFkgUv/EvcbL7TOuZ9O+3/nikm8RhGFzbueKSf3o2+oSnun0EoenNLY6zKB4c/kIftu7nk/7fVWjkxNAclxyWLtFgG90Ei3JCQ4cTeXq3myVUpUJ3DygZbADUdHt1aXDmb7xG57sOJxzmnaiQ9OOzLlyGQ+c9TgzNn7DOaPT+Gj1e3i80d2KacPudVHXqTzWpCZG7tReeUSEFH8XilBcDxmrKkxQIjJVRKb4P74BfgW+Dn1oKlrM2PgNLy15hstaD+aGtrcV357gSOCeMx9i3lUraHtEOg/Pu5sLv+jMLzt/sjDaqvN4Pdw7+xZS42vzTOcRVodTI9WKj8zCiEDF2W00SI4nKYpGgFYK5K1IyWeiG9hqjMkIUTwqyvy2ZwO3Tb+OUw9vx4vd3iyzB90x9Y7ji4u/YdKvX/D4gmH0HNOBG9rexgNnPUZyfIoFUVfNxz+9z8q/l/FWr49omNTI6nBqHBFIiYBrnqpLREhNjCMxzq4X2lcgkLcifwJLjTELjDGLgT0i0jykUamokOXM5NqpA0h01OKjPuOo5ahV7rEiwsUnDGLRNau54uQhvL/qDTp+0pbpG6eGMeKq25a1lecXP06XZt255ISY378zIqUmxEXtJqJl0dFUxQJJUBOAku0KPP7bVA3mNV5unXYtWzM388EFn3NUatOA7lc3sR4vdXuLqQPnkhpfhyFTBnDtlAH8lb0txBFXXVGncqDcUaIKrWgrjAhU0WiqXpKuTZUlkPGywxjjKvrCGOMSkfD10FcR6aUlzzB78zT+77zXOLPJOZW+/+lHnsWswUt4b9XrvPLD83Qc3Y4Hzn6M69NurfIOrqHy1YZxzNsyk2c7j6Bp7fB2Kk9w2Ip3eK7JorEwojLiHb7RlNPtJUybIFRZOBNpINdBzQLeNMZM8X/dD7jDGBO+hlsBqCnXQUWCb36fxA3fXMEVJw/h5W7vVHtEsTVzCw/Pu5s5m6dzymFpvNj1TdoeUeElEmGxO28XHUe3pXndY8LeqbzoepOsgkLya/BGlMG47kZFlqB0M/f7D/CwiPwpIn8Cw4CbAwyil4j8KiJ/iMiDZXz/VRFZ7f/4TUT2l/jeNSLyu//jmkDOp0Jv/e613DHjRk5r3J7/6/JaUKa7mtVpzmf9vuK/F3zOztwdnD+2Iw/Pu5ssZ2YQIq6exxfcT7Yri1e6vxfW5FSyICCaLkoNtlgpjFBVU+EIqvhAkRT/8QHt/SwiduA3oDuQASwHLjfGrCvn+NuBtsaY6/zd0lcA6fiuTV0JnGaM2Vfe+XQEFXr7C/bRa8w55BXmMXPwYo5IOTLo58hyZjL8f0/y8er3OSz5cJ7t/DIXtuxvybrP7M3TuXJyf+498xHuP+vRsJ67rKRUE0dSNTk5x7KgjaBE5HkRqWuMyTHGZItIPRF5NoAY2gN/GGM2+dewxgH9DnH85cBY/+c9gVnGmL3+pDQL6BXAOVWIeLwebvnuGv7K3sZHfcaGJDkB1E6ow/NdXuW7y7/nsOQjuPHbwVz59cX8mbk1JOcrT44rm2Fzbqdl/RO44/T7w3ru8goCatqLdawWRqjABTLF19sYUzz15k8Y5wdwv6OAkqVZGf7bDiIizYAWwNzK3leFx/8tfoJ5W2fxf+e9RvqRZ4b8fG2PSGfa5Qt5utOLLMlYSKdP2vLm8hEUesLTKNfXqfwvXu3+btg7ldc+REFATUpSsV4YoSoWSIKyi0jxM1REagGBPGPLmpMpbz5xEDDRGFM0fxHQfUXkJhFZISIrdu3aFUBIqiom/zqBt1a8zNVtbuTKU64L23kdNgc3tbudhdf8SOdm3Xhu0WN0//wslm9fEtLzluxUHo5kXFJSvL3C/cBqJ8bF/LUz0d4xQgVHIH8BnwFzROR6EbkO33RbIFttZAAlL45pAmwv59hB/Du9F/B9jTEjjTHpxpj0Ro30yv5QWLtrDXfPvJn2R57Fsxa19zkqtSkf9x3P6L4TyHZl0ueL87h/9lD2F5S7JFllTreTu2fdwpGpR/Fwh6eD/viHYvP3awtEagwnKS2MUEUqTFDGmBeBZ4HWwEnAM8aYFwJ47OVASxFp4b9uahAwpfRBInI8UA8o+bZ4BtDDv95VD+jhv02F0d78PQyZMoC6ifX44MIxxNutvfyt57EX8v3VP3JzuzsY88sozhmdxlcbxhFooU8g3lj+Er/v3cCLXd8Ke6fy1ERHpYpBYjVJxVrHCFV1AY2hjTHTjTH3GWPuBXJE5O0A7uMGhuJLLOuB8caYtSLytIj0LXHo5cA4U+JVxhizF3gGX5JbDjztv02Fidvr5uZvr2Jn7g4+6vMFhyUfYXVIACTHp/BUpxeYccVimtY+mlunXcvAry5k8/6N1X7sDbvX8cayF7n4hIF0bdEzCNEGLt5uIzGu8skm1pKUFkaokgIqMxeRNHyJZCCwGfjKGPPmoe8VXlpmHlxPLBjG+6ve4LUeIxl00lVWh1Mmj9fDJz9/wPOLHsflcXJn+2Hcln5PlYoaPF4Pfb7owpb9m/j+mh/D2gxWgPrJ8RWuPR1KdkEheTFQgl4/OV7XnmqAapeZi0grEXlcRNYDb+FbFxJjTJdIS04quCauH8v7q97g+rRbIjY5Adhtdq499WYWXbOansdeyItLnqbb52fwv4yFlX6sj396j1U7lvNM55fC3qk8KcFRreQEvpFUcgRvgR4ILYxQpR3qr2EDvu3d+xhjzvEnpeh/i6YO6ad/VnHfrFs5q8m5PNkxkKVG6x2e0piRF3zGmP5f43Q7uXhCD+6ccRN78ncHdH9fp/InOK95Ty4Oc6dymwjJQZrSSklwRG2S0sIIVZZDJahLgB3APBH5r4h0pezybxUjduXt5LqpA2mQ1Ij/XvA5cfbo6n92XvMezL96JXecfj9fbhjLuaPSGLv2k0MWURhjeGD27QC80PX1sHesqGxhREWiNUlpYYQqS7kJyhgzyRgzEDgBmA/cDRwuIu+KSI8wxafCpNBTyE3fXsmevN183OeLqN2QLykuiYfPeZpZg3/guPqtuHvmzfSf0IPf9mwo8/gvN4xj3tZZPNzhaUs6lVelMKIi0ZaktDBClSeQMvNcY8znxpgL8V2PtBo4qPGrim5Pfj+MJRkLebn7O7Q5vK3V4VRb64YnMXnAbF7u9g4bdq+l62fteeF/T5Hvzi8+ZnfeLh6ffz+nNW7PtacG1P84aATfulGoRFOS0o4RqjyVWpH098Z73xhzXqgCUuE3du0nfLj6XW5udweXtL7c6nCCxiY2Bp9yLQuHrKZfq0t5delwunySzoKtc4B/O5W/3O3dsHYqB0hOcIR8X51oSFJaGKEORf8yarhVfy9j2JzbObdpFx479zmrwwmJRkmH8Vbvj5hwyXfYxMbAry7ksonn89WGL7ij/QOc0PDEsMZjt0nYrl1KSXAE3J0i3LQwQlVEE1QNtjN3B9dNHcThyY15/4JPI24n22A79+guzL1qOfee+QhLty+mVf3WYe9UDsEvjKhIcoQmqdqJWhihDi3y/mpVWLg8Lm745goynZl8M2ge9Ws1sDqksEh0JHL/WY8y+OQhJDpqhb1TeaLDToIj/AUBRVN9OU532M9dlrgqds5QNYsmqBrq0fn3sWz7Et4//1NOatTG6nDC7sjUJmE/pwApFhYERFKS0sIIFQid4quBPl3zIZ+s+S9D0++l3/GXWh1OjRGOwohAYrB6uk8LI1Sg9K+khlm+fQkPz7ubLs2681CHp6wOp8Zw2CRiKuqSExyWjWBsIqRGyO9BRT5NUDXIjpztXD/1Co5Kbcq7548Oe2l1TRbKa56qIinemiQV7gIRFd30rUwN4XQ7uW7qIHIKsxl/ybfUTaxndUg1RmKcnXhH5L0XTPKXeGcXhGdNSgsjVGVpgqoBjDE8OPdOVu1YzocXjg37dT81mQgRPaUVziRVWwsjVCVF3ts6FXSj1oxk7NrR3H3Gg1zQ8iKrw6lRUhIcEX+tTzim+5Li7dXeUkTVPPoXE+OWZCzisfn30b3F+dx/1mNWh1OjOGxSPEKJdKFMUjYRyysHVXTSBBXD/srexo3fDqZ5nWN4u/dH2ET/u8Opdq3IKoyoSFK8g9ohKObQwghVVfq2Jkblu/O5dsogCtz5fHzZTGon1LE6pBolWq/1Kdr2IqugMCiPF6+FEaoaNEH5Tfl1Cm5PImce1SHqe9IVbcK3ZucqPuk7kZb1j7c6pBol2pugBitJ+bYUid7fg7Ke/vX4PT7/cdb8s4Z6ifXp2qIXvY/tQ+fm3UmOS7Y6tEr74Me3mbD+c+4/6zF6HHuB1eHUOLGwO2wwklQtLYxQ1aQJym/xtYuZsPZbpv0xldmbpzFx/RgS7Ymce/R59D6uD92POZ9GSYdZHWaFFm1bwJPfP0jvY/tw9xm6r2S4xdLusNVJUloYoYJBjDFWxxAU6enpZsWKFVW+vzGGndlOANxeN0v/+h/TN05h+sZv2Ja1FUE4/cgz6XVsH3od24dj6h0XrNCD5s/MrfQa04GGSYfx3eULSIlPtTqkGqdBcnzMjRoKCj1k5lcuSdWpFadrT6pcIrLSGJNe4XGaoHxKJqjSt6/b/TPTN37D9I1T+XnnagBa1W9Nr2MvpNdxfUg7/DTLK+TyCvPo+8V5/Jm1hemXL4rIBBrrkuLtEdfSKFgqk6Ti7TbqJceHOCIVzTRBVVJ5Caq0bVlbmbHxW6ZvnMqSjIV4jIcjkhvT49gL6X1sHzo07US8PbxPTmMMt04bwuRfJ/DZRZPo2qJnWM+vfFNaDVPiY7qcOpAkJUD9GBxFquDSBFVJgSaokvYV7GXO5hlM+2MK87bOIq8wl9T42pzXvAe9j+vDec17hqW8+50Vr/L0wod5pMPT3N4+/DvEqpozpVVRkorlUaQKHk1QlVSVBFVSgbuAhX/OY/rGqczY9C2783YSZ4ujQ9NO9Dq2Dz2PvYDGKUdV+fHLs2DrHC6f1JcLjruIkRd8FtPv4CNVTZvSKi9J1YRRpAoOTVCVVN0EVZLH62HVjmXF61Yb9/0OQNrhpxUXWRzfoHW1n8hb92+m55gONE49im8GziM5PiUY4atKqKlTWmUlqZoyilTVpwmqkoKZoEr7fe+vTP9jCtM2TmXVjuUAtKh7LD3961bpjc+s9N5Mua4cLvyiC39n/8WMKxbTrG6LUISuKhAJO9RapWSSqmmjSFU9mqAqKZQJqqQdOduZuek7pm2cyuJt83F5XDSo1Ygex5xP72P7cG6z86jlqFVhrDd+O5jv/viasf2n0KlZ15DHrQ6mU1q+JJWVX1gjR5Gq6jRBVVK4ElRJ2c4s5m6ZyYxN3zB783SynJnUciTRpXk3eh7Th+7H9KZ+rQYH3e+NZS/x/OLHeaLj/3HLaXeFNeZAJDhsFHoM3hj52yqPTmn5eLwGe5R3zlDhpQmqkqxIUCW5PC6WZCxk+sapTN84lb9ztmMXO2cc1YFex15Iz2P70KxOc2Zvns5Vky+m/wkDeLvXxxH37j3ObqN+cjxeryHb6aag0GN1SCGR4LBRN0mntJSqCk1QlWR1girJGMOanT8yfeNUpv0xlQ171gJwYsNTyMj+k6NrN2fKwLkkxSVZHOmBRKBBcsIB76adbg9Z+e6YGk0J0CAlQUcNSlWRJqhKiqQEVdqW/Zv8I6tv2JGznQmXfkfT2s2sDusgdZPiSHAcPOVljG80le+KjdFUSoKD5BpaGKFUMGiCqqRITlDRIJAXbZfbS1ZBIR5v9P7N2W1Cg+SaXRihVHUFmqC07EZVW4LDFtCIIt5ho0FyPElR3O1bd4dVKnxCmqBEpJeI/Coif4hImXs/iMgAEVknImtFZEyJ2z0istr/MSWUcaqqs4lUaptwESE1MY76yfFRt4aT6LCXOYWplAqNkE2ki4gdeBvoDmQAy0VkijFmXYljWgIPAR2MMftEpOSGS/nGmLRQxaeqT/CtO1Vlc744u280levykOd0E+mTfgKk6O6wSoVVKEdQ7YE/jDGbjDEuYBzQr9QxNwJvG2P2ARhjdoYwHhVkKYkO4qpxcab4N7WrlxyPI8JHU8kJjqgb8SkV7UKZoI4CtpX4OsN/W0mtgFYislhEfhCRXiW+lygiK/y3XxTCOFUVJDrsJMUHZ0QRZ7fRICWBlAQHkZgCHDbRqj2lLBDKZ11ZrzWlZ3IcQEugM9AEWCgiJxtj9gNHG2O2i8gxwFwR+dkYs/GAE4jcBNwEcPTRRwc7flUOu02oXSv4fzrJCQ4SHDayCtwUerxBf/yq0u0jlLJGKEdQGUDTEl83AbaXcczXxphCY8xm4Fd8CQtjzHb/v5uA+UDb0icwxow0xqQbY9IbNWoU/J9AHUSAurXiQlbJ5vB3okhNjIzRVGKcnXiHFrsqZYVQPvOWAy1FpIWIxAODgNLVeJOBLgAi0hDflN8mEaknIgklbu8ArENZrnatuLA0BU2Kd9AgJYF4CxuQikCqTu0pZZmQPfuMMW4RGQrMAOzAR8aYtSLyNLDCGDPF/70eIrIO8AD3G2P2iMjZwPsi4sWXRIeXrP5T1qgVbw9rc1S7TaiXHE++y0N2QWHYK/1SEhxVqlBUSgWHdpLw004ShxZnt1EvKXRTexXxeA3ZBYU43eFZm3LYhAYpCWE5l1I1jXaSUEEj4ttawsoOCnabUDcp3h9H6M9Xu5YWRihlNZ1gVxWqUysuYq4BSoyzE2+3kV3gpsAdmuazteLt1bq+SykVHPosVIfkK/2OrPY+NptQJykuJKMpEUgJ0vVdSqnq0WeiKle83UZKBFexFY+mgrgxYmpC1Vo3KaWCT0dQqkw2EepEwTqMzeaLs25SHLZqDqfi7DZqRXGndaVijSYodRDBt+4UTSOJBIedhinx1UowtbUZrFIRRROUOkhKoiMquyeIf+uPukmVL+pIireH5QJkpVTg9BmpDhDMJrBWSXDYaZAc+GjK5u+qrpSKLJqgVLFQNYG1QtFoql5SxRsj6i65SkUmTVAKCH0TWKtUtM18vN0W1vZNSqnAaYJSQPiawFqh5DbzJTdGFHyjJ6VUZIrNVyRVKYlx4W0Ca5WijRGT/RsjJiU4YjYpKxUL9O1jDeewSY0rr05JcJDosEVM+yalVNlq1iuTOoAI1E2Kj7l1p0DoyEmpyKfP0hqsdmLkNIFVSqnSNEHVUElh3nxQKaUqSxNUDRRvt5GaGPl99pRSNZsmqBomWprAKqWUJqgaJtqawCqlai5NUDVISkJ0NoFVStVM+mpVQyQ4bCRrQ1SlVBTRBFUD2PyNU5VSKppogopxAr7dZnXdSSkVZTRBxbjUxDjitGuCUioK6StXDEuMs1drC3SllLKSJig/ESGWJsFqYhNYpVRs0VewEhqmJODyeCko9OByezFWB1RFgu96p5rYBFYpFTs0QZVgswmJNl+POmMMTrcXZ6EXp9sTVckqljcfVErVHJqgyiEixRv5GeP4N1l5PJgIzlbaBFYpFSs0QQWgdLLyTQP6R1YRlKzitAmsUiqGaIKqJBEhwWEnwWEH4nC6PcWjK6+F2UoEbQKrlIopmqCqqThZJYLL7RtVFViQrOrU0s0HlVKxRRNUEMU7bMQ7bKQmQqHHi9Ptqwj0eEObrJITHP4RnVJKxQ5NUCESZ7cRZ7eRkuDA7fFS4PbiLPTgDnKySnD4zqGUUrFGX9nCwGG3kVIiWTndvo9Cj7daj6tNYJVSsUwTVJg57DYcdhvJCeDxmuI1q8omK20Cq5SKdSG9mlNEeonIryLyh4g8WM4xA0RknYisFZExJW6/RkR+939cE8o4rWK3CUnxDuonx9MoJYHURAfxAV5gm5Lo0CawSqmYFrIRlIjYgbeB7kAGsFxEphhj1pU4piXwENDBGLNPRA7z314feAJIBwyw0n/ffaGK12o2f7JKigev19/Fwl12y6VEh52keB38KqViWyjfgrcH/jDGbDLGuIBxQL9Sx9wIvF2UeIwxO/239wRmGWP2+r83C+gVwlgjis0m1Iq3UzcpnkapCdSpFUeCw4bgG3XVrqXJSSkV+0L5SncUsK3E1xnAGaWOaQUgIosBO/CkMWZ6Ofc9KnShRq4Du1gYjEGbwCqlaoRQJqiyXkVLz1Y5gJZAZ6AJsFBETg7wvojITcBNAEcffXR1Yo0KIoLmJqVUTRHKKb4MoGmJr5sA28s45mtjTKExZjPwK76EFch9McaMNMakG2PSGzVqFNTglVJKWSuUCWo50FJEWohIPDAImFLqmMlAFwARaYhvym8TMAPoISL1RKQe0MN/m1JKqRoiZFN8xhi3iAzFl1jswEfGmLUi8jSwwhgzhX8T0TrAA9xvjNkDICLP4EtyAE8bY/aGKlallFKRR0wk7RdRDenp6WbFihVWh6GUUqoCIrLSGJNe0XF6padSSqmIpAlKKaVURNIEpZRSKiJpglJKKRWRNEEppZSKSDFTxSciu4Ct1XyYhsDuIIQTStEQI0RHnBpj8ERDnBpj8FQ3zmbGmAq7K8RMggoGEVkRSOmjlaIhRoiOODXG4ImGODXG4AlXnDrFp5RSKiJpglJKKRWRNEEdaKTVAQQgGmKE6IhTYwyeaIhTYwyesMSpa1BKKaUiko6glFJKRSRNUEoppSKSJihARD4SkZ0i8ovVsZRHRJqKyDwRWS8ia0XkTqtjKk1EEkVkmYj85I/xKatjKo+I2EXkRxH5xupYyiMiW0TkZxFZLSIR2apfROqKyEQR2eD/2zzL6phKE5Hj/b/Doo8sEbnL6rhKE5G7/c+bX0RkrIgkWh1TaSJypz++teH4HeoaFCAiHYEc4BNjzMlWx1MWEWkMNDbGrBKRVGAlcJExZp3FoRUTEQGSjTE5IhIHLALuNMb8YHFoBxGRe4B0oLYx5kKr4ymLiGwB0o0xEXvhpoiMBhYaYz7wb0yaZIzZb3Vc5RERO/AXcIYxproX9geNiByF7/lyojEmX0TGA98ZY0ZZG9m/RORkYBzQHnAB04FbjDG/h+qcOoICjDHfAxG9IaIx5m9jzCr/59nAeuAoa6M6kPHJ8X8Z5/+IuHdAItIEuAD4wOpYopmI1AY6Ah8CGGNckZyc/LoCGyMpOZXgAGqJiANIArZbHE9prYEfjDF5xhg3sADoH8oTaoKKQiLSHGgLLLU2koP5p85WAzuBWcaYiIsReA14APBaHUgFDDBTRFaKyE1WB1OGY4BdwMf+6dIPRCTZ6qAqMAgYa3UQpRlj/gJGAH8CfwOZxpiZ1kZ1kF+AjiLSQESSgPOBpqE8oSaoKCMiKcCXwF3GmCyr4ynNGOMxxqQBTYD2/mmBiCEiFwI7jTErrY4lAB2MMe2A3sBt/qnoSOIA2gHvGmPaArnAg9aGVD7/FGRfYILVsZQmIvWAfkAL4EggWUSutDaqAxlj1gMvALPwTe/9BLhDeU5NUFHEv67zJfC5MeYrq+M5FP9Uz3ygl8WhlNYB6Otf3xkHnCcin1kbUtmMMdv9/+4EJuGb+48kGUBGiVHyRHwJK1L1BlYZY/6xOpAydAM2G2N2GWMKga+Asy2O6SDGmA+NMe2MMR3xLYuEbP0JNEFFDX8BwofAemPMK9k9ZJYAAAVfSURBVFbHUxYRaSQidf2f18L3pNtgbVQHMsY8ZIxpYoxpjm+6Z64xJqLeqQKISLK/GAb/tFkPfFMsEcMYswPYJiLH+2/qCkRM0U4ZLicCp/f8/gTOFJEk/3O9K7515ogiIof5/z0auJgQ/z4doXzwaCEiY4HOQEMRyQCeMMZ8aG1UB+kAXAX87F/jAXjYGPOdhTGV1hgY7a+UsgHjjTERW8Yd4Q4HJvleq3AAY4wx060NqUy3A5/7p882AddaHE+Z/Gsm3YGbrY6lLMaYpSIyEViFb9rsRyKz7dGXItIAKARuM8bsC+XJtMxcKaVURNIpPqWUUhFJE5RSSqmIpAlKKaVURNIEpZRSKiJpglJKKRWRNEGpmCEiRkReLvH1fSLyZJAee5SIXBqMx6rgPJf5u4LPK3V7c//Pd3uJ294SkSEVPF644p4oIsf4P98iIg39n58mIptFpK2IXBjJHe5V5NEEpWKJE7i46MUxUvivCwvU9cCtxpguZXxvJ3Cn/5qjkPM3LQ3kuJMAuzFmU6nb2+DrLjHQGPMj8C2+Lh5JQQ9WxSRNUCqWuPFd3Hh36W+UHkmISI7/384iskBExovIbyIyXEQGi29fq59F5NgSD9NNRBb6j7vQf3+7iLwkIstFZI2I3FziceeJyBjg5zLiudz/+L+IyAv+2x4HzgHeE5GXyvj5dgFzgGvKeLwb/TH8JCJflkoCZcWdKCIf+2P4UUS6+G8fIiITRGQqvka1jUXke/Hto/SLiJxbRlyDga9L3dYamAxcZYxZBr5u9/jaX0Xk9iYq8miCUrHmbWCwiNSpxH1OBe4ETsHXraOVMaY9vu04bi9xXHOgE76tOt4T34Zy1+PrPH06cDpwo4i08B/fHnjEGHNiyZOJyJH4mm6eB6QBp4vIRcaYp4EVwGBjzP3lxDocuLeMUdlXxpjTjTGn4muRc30Fcd8GYIw5BV8LoNHy7wZ5ZwHXGGPOA64AZvgbAJ8KrOZgHfDtT1bS18BQY8yiUrevAMpKckodRBOUiin+Du+fAHdU4m7L/fttOYGNQNE2Bz/je3EvMt4Y4/Vv0LYJOAFfj7yr/e2nlgINgJb+45cZYzaXcb7Tgfn+xqBu4HN8+yoF8vNtBpbhSxwlnewfJf2Mb0RzUgVxnwN86n/MDcBWoJX/+FnGmKL90ZYD1/rX8k7x70VWWmN8o7uSZgM3lJFId+Lr1q1UhTRBqVj0Gr4RRMm9idz4/979zThLruM4S3zuLfG1lwP7VZbuC2YAAW43xqT5P1qU2Mcnt5z4JNAfpBzPA8M48Pk7Ct+I5RTgKaDkduHlxV2e4rj9m3l2xLcL7acicnUZx+eXOh/AUP+/75S6PdF/vFIV0gSlYo7/3f94Dpzm2gKc5v+8H77dfivrMhGx+deljgF+BWYAt4hvKxREpJVUvGnfUqCTiDT0jzAux7c7aUD8I551HLiWkwr87Y9jcABxf190nIi0Ao72334AEWmGb/+s/+Lrpl/WdhrrgeNK3eb1/1zHi8jTJW5vRYR1ZVeRSxOUilUvAyWr+f6LLyksA86g/NHNofyKL5FMA/5jjCnAt061DlglIr8A71PBLgHGmL+Bh4B5+DZ9W2WMKV1kUJHn8G0KWeQxfIlvFgdvcVJW3O8Adv+U4BfAEP8UZ2mdgdUi8iNwCfB6Gcd86z/uAP7H64evcu82/81d/McrVSHtZq6Uqhbx7f01D98OwJ5DHHc4vm1DuoYtOBXVNEEppapNRHri20zzz0McczpQaIwpqxJQqYNoglJKKRWRdA1KKaVURNIEpZRSKiJpglJKKRWRNEEppZSKSJqglFJKRaT/BzLGJ24T/lLbAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The best accuracy was with 0.7857142857142857 with k= 7\n"
     ]
    }
   ],
   "source": [
    "# import library\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn import metrics\n",
    "\n",
    "# try with 10 different values of k to find the best one\n",
    "Ks = 10\n",
    "mean_acc = np.zeros((Ks-1))\n",
    "std_acc = np.zeros((Ks-1))\n",
    "ConfustionMx = [];\n",
    "for n in range(1,Ks):\n",
    "    #Train Model and Predict  \n",
    "    neigh = KNeighborsClassifier(n_neighbors = n).fit(X_train,y_train)\n",
    "    yhat=neigh.predict(X_test)\n",
    "    mean_acc[n-1] = metrics.accuracy_score(y_test, yhat)\n",
    "    std_acc[n-1]=np.std(yhat==y_test)/np.sqrt(yhat.shape[0])\n",
    "\n",
    "# accuracy\n",
    "print(mean_acc)\n",
    "\n",
    "# Plot model accuracy for Different number of Neighbors\n",
    "plt.plot(range(1,Ks),mean_acc,'g')\n",
    "plt.fill_between(range(1,Ks),mean_acc - 1 * std_acc,mean_acc + 1 * std_acc, alpha=0.10)\n",
    "plt.legend(('Accuracy ', '+/- 3xstd'))\n",
    "plt.ylabel('Accuracy ')\n",
    "plt.xlabel('Number of Nabors (K)')\n",
    "plt.tight_layout()\n",
    "plt.show()\n",
    "\n",
    "# result\n",
    "print( \"The best accuracy was with\", mean_acc.max(), \"with k=\", mean_acc.argmax()+1 )\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "KNeighborsClassifier(algorithm='auto', leaf_size=30, metric='minkowski',\n",
       "           metric_params=None, n_jobs=None, n_neighbors=7, p=2,\n",
       "           weights='uniform')"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# train model with k=7\n",
    "neigh = KNeighborsClassifier(n_neighbors = 7).fit(X_train,y_train)\n",
    "neigh"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Decision Tree"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import library\n",
    "from sklearn.tree import DecisionTreeClassifier\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create an instance of the DecisionTreeClassifier called loanTree\n",
    "loanTree = DecisionTreeClassifier(criterion=\"entropy\", max_depth = 4)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DecisionTreeClassifier(class_weight=None, criterion='entropy', max_depth=4,\n",
       "            max_features=None, max_leaf_nodes=None,\n",
       "            min_impurity_decrease=0.0, min_impurity_split=None,\n",
       "            min_samples_leaf=1, min_samples_split=2,\n",
       "            min_weight_fraction_leaf=0.0, presort=False, random_state=None,\n",
       "            splitter='best')"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# training\n",
    "loanTree.fit(X_train,y_train)\n",
    "loanTree"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Support Vector Machine"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import library\n",
    "from sklearn import svm\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/envs/Python36/lib/python3.6/site-packages/sklearn/svm/base.py:196: FutureWarning: The default value of gamma will change from 'auto' to 'scale' in version 0.22 to account better for unscaled features. Set gamma explicitly to 'auto' or 'scale' to avoid this warning.\n",
      "  \"avoid this warning.\", FutureWarning)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0,\n",
       "  decision_function_shape='ovr', degree=3, gamma='auto_deprecated',\n",
       "  kernel='rbf', max_iter=-1, probability=False, random_state=None,\n",
       "  shrinking=True, tol=0.001, verbose=False)"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# training\n",
    "clf = svm.SVC()\n",
    "clf.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Logistic Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import library\n",
    "from sklearn.linear_model import LogisticRegression\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/envs/Python36/lib/python3.6/site-packages/sklearn/linear_model/logistic.py:433: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.\n",
      "  FutureWarning)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "LogisticRegression(C=0.01, class_weight=None, dual=False, fit_intercept=True,\n",
       "          intercept_scaling=1, max_iter=100, multi_class='warn',\n",
       "          n_jobs=None, penalty='l2', random_state=None, solver='warn',\n",
       "          tol=0.0001, verbose=0, warm_start=False)"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# training\n",
    "LR = LogisticRegression(C=0.01).fit(X_train,y_train)\n",
    "LR"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Model Evaluation using Test set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import jaccard_similarity_score\n",
    "from sklearn.metrics import f1_score\n",
    "from sklearn.metrics import log_loss"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, download and load the test set:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2020-03-24 14:36:22--  https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_test.csv\n",
      "Resolving s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)... 67.228.254.196\n",
      "Connecting to s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)|67.228.254.196|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 3642 (3.6K) [text/csv]\n",
      "Saving to: ‘loan_test.csv’\n",
      "\n",
      "100%[======================================>] 3,642       --.-K/s   in 0s      \n",
      "\n",
      "2020-03-24 14:36:22 (292 MB/s) - ‘loan_test.csv’ saved [3642/3642]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!wget -O loan_test.csv https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_test.csv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Load Test set for evaluation "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/8/2016</td>\n",
       "      <td>10/7/2016</td>\n",
       "      <td>50</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>300</td>\n",
       "      <td>7</td>\n",
       "      <td>9/9/2016</td>\n",
       "      <td>9/15/2016</td>\n",
       "      <td>35</td>\n",
       "      <td>Master or Above</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>21</td>\n",
       "      <td>21</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/10/2016</td>\n",
       "      <td>10/9/2016</td>\n",
       "      <td>43</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>24</td>\n",
       "      <td>24</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/10/2016</td>\n",
       "      <td>10/9/2016</td>\n",
       "      <td>26</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>35</td>\n",
       "      <td>35</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>800</td>\n",
       "      <td>15</td>\n",
       "      <td>9/11/2016</td>\n",
       "      <td>9/25/2016</td>\n",
       "      <td>29</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           1             1     PAIDOFF       1000     30       9/8/2016   \n",
       "1           5             5     PAIDOFF        300      7       9/9/2016   \n",
       "2          21            21     PAIDOFF       1000     30      9/10/2016   \n",
       "3          24            24     PAIDOFF       1000     30      9/10/2016   \n",
       "4          35            35     PAIDOFF        800     15      9/11/2016   \n",
       "\n",
       "    due_date  age             education  Gender  \n",
       "0  10/7/2016   50              Bechalor  female  \n",
       "1  9/15/2016   35       Master or Above    male  \n",
       "2  10/9/2016   43  High School or Below  female  \n",
       "3  10/9/2016   26               college    male  \n",
       "4  9/25/2016   29              Bechalor    male  "
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_df = pd.read_csv('loan_test.csv')\n",
    "test_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/envs/Python36/lib/python3.6/site-packages/sklearn/preprocessing/data.py:645: DataConversionWarning: Data with input dtype uint8, int64 were all converted to float64 by StandardScaler.\n",
      "  return self.partial_fit(X, y)\n",
      "/opt/conda/envs/Python36/lib/python3.6/site-packages/ipykernel/__main__.py:11: DataConversionWarning: Data with input dtype uint8, int64 were all converted to float64 by StandardScaler.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([[ 0.49362588,  0.92844966,  3.05981865,  1.97714211, -4.12310563,\n",
       "         2.39791576, -0.79772404, -0.86135677],\n",
       "       [-3.56269116, -1.70427745,  0.53336288, -0.50578054, -4.12310563,\n",
       "        -0.41702883, -0.79772404, -0.86135677],\n",
       "       [ 0.49362588,  0.92844966,  1.88080596,  1.97714211, -4.12310563,\n",
       "        -0.41702883,  1.25356634, -0.86135677],\n",
       "       [ 0.49362588,  0.92844966, -0.98251057, -0.50578054,  0.24253563,\n",
       "        -0.41702883, -0.79772404,  1.16095912],\n",
       "       [-0.66532184, -0.78854628, -0.47721942, -0.50578054,  0.24253563,\n",
       "         2.39791576, -0.79772404, -0.86135677]])"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Pre-processing loan_test\n",
    "test_df['due_date'] = pd.to_datetime(test_df['due_date'])\n",
    "test_df['effective_date'] = pd.to_datetime(test_df['effective_date'])\n",
    "test_df['dayofweek'] = df['effective_date'].dt.dayofweek\n",
    "test_df['weekend'] = test_df['dayofweek'].apply(lambda x: 1 if (x>3)  else 0)\n",
    "test_df['Gender'].replace(to_replace=['male','female'], value=[0,1],inplace=True)\n",
    "Feature_test = test_df[['Principal','terms','age','Gender','weekend']]\n",
    "Feature_test = pd.concat([Feature_test,pd.get_dummies(test_df['education'])], axis=1)\n",
    "Feature_test.drop(['Master or Above'], axis = 1,inplace=True)\n",
    "test_X = Feature_test\n",
    "test_X = preprocessing.StandardScaler().fit(test_X).transform(test_X)\n",
    "test_X[0:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF'],\n",
       "      dtype=object)"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Pre-processing loan_test (cont)\n",
    "test_y = test_df['loan_status'].values\n",
    "test_y[0:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## KNN"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "KNN Jaccard index:  0.6851851851851852\n",
      "KNN F1-score:  0.6453810131971051\n"
     ]
    }
   ],
   "source": [
    "# predicted y\n",
    "yhat_knn = neigh.predict(test_X)\n",
    "\n",
    "# jaccard\n",
    "jaccard_knn = jaccard_similarity_score(test_y, yhat_knn)\n",
    "print(\"KNN Jaccard index: \", jaccard_knn)\n",
    "\n",
    "# f1_score\n",
    "f1_score_knn = f1_score(test_y, yhat_knn, average='weighted')\n",
    "print(\"KNN F1-score: \", f1_score_knn)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Decision tree"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "DT Jaccard index:  0.5185185185185185\n",
      "DT F1-score:  0.5385802469135802\n"
     ]
    }
   ],
   "source": [
    "# predicted y\n",
    "yhat_dt = loanTree.predict(test_X)\n",
    "\n",
    "# jaccard\n",
    "jaccard_dt = jaccard_similarity_score(test_y, yhat_dt)\n",
    "print(\"DT Jaccard index: \", jaccard_dt)\n",
    "\n",
    "# f1_score\n",
    "f1_score_dt = f1_score(test_y, yhat_dt, average='weighted')\n",
    "print(\"DT F1-score: \", f1_score_dt)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Supported vector machine (SVM)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SVM Jaccard index:  0.8148148148148148\n",
      "SVM F1-score:  0.7861952861952862\n"
     ]
    }
   ],
   "source": [
    "# predicted y\n",
    "yhat_svm = clf.predict(test_X)\n",
    "\n",
    "# jaccard\n",
    "jaccard_svm = jaccard_similarity_score(test_y, yhat_svm)\n",
    "print(\"SVM Jaccard index: \", jaccard_svm)\n",
    "\n",
    "# f1_score\n",
    "f1_score_svm = f1_score(test_y, yhat_svm, average='weighted')\n",
    "print(\"SVM F1-score: \", f1_score_svm)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Logistics Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LR Jaccard index:  0.7407407407407407\n",
      "LR F1-score:  0.6304176516942475\n",
      "LR log loss:  0.6037871272191607\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/envs/Python36/lib/python3.6/site-packages/sklearn/metrics/classification.py:1143: UndefinedMetricWarning: F-score is ill-defined and being set to 0.0 in labels with no predicted samples.\n",
      "  'precision', 'predicted', average, warn_for)\n"
     ]
    }
   ],
   "source": [
    "# predicted y\n",
    "yhat_lg = LR.predict(test_X)\n",
    "yhat_lg_prob = LR.predict_proba(test_X)\n",
    "\n",
    "# jaccard\n",
    "jaccard_lg = jaccard_similarity_score(test_y, yhat_lg)\n",
    "print(\"LR Jaccard index: \", jaccard_lg)\n",
    "\n",
    "# f1_score\n",
    "f1_score_lg = f1_score(test_y, yhat_lg, average='weighted')\n",
    "print(\"LR F1-score: \", f1_score_lg)\n",
    "\n",
    "# logloss\n",
    "logloss_lg = log_loss(test_y, yhat_lg_prob)\n",
    "print(\"LR log loss: \", logloss_lg)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Report\n",
    "You should be able to report the accuracy of the built model using different evaluation metrics:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "| Algorithm          | Jaccard | F1-score | LogLoss |\n",
    "|--------------------|---------|----------|---------|\n",
    "| KNN                | ?       | ?        | NA      |\n",
    "| Decision Tree      | ?       | ?        | NA      |\n",
    "| SVM                | ?       | ?        | NA      |\n",
    "| LogisticRegression | ?       | ?        | ?       |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "<h2>Want to learn more?</h2>\n",
    "\n",
    "IBM SPSS Modeler is a comprehensive analytics platform that has many machine learning algorithms. It has been designed to bring predictive intelligence to decisions made by individuals, by groups, by systems – by your enterprise as a whole. A free trial is available through this course, available here: <a href=\"http://cocl.us/ML0101EN-SPSSModeler\">SPSS Modeler</a>\n",
    "\n",
    "Also, you can use Watson Studio to run these notebooks faster with bigger datasets. Watson Studio is IBM's leading cloud solution for data scientists, built by data scientists. With Jupyter notebooks, RStudio, Apache Spark and popular libraries pre-packaged in the cloud, Watson Studio enables data scientists to collaborate on their projects without having to install anything. Join the fast-growing community of Watson Studio users today with a free account at <a href=\"https://cocl.us/ML0101EN_DSX\">Watson Studio</a>\n",
    "\n",
    "<h3>Thanks for completing this lesson!</h3>\n",
    "\n",
    "<h4>Author:  <a href=\"https://ca.linkedin.com/in/saeedaghabozorgi\">Saeed Aghabozorgi</a></h4>\n",
    "<p><a href=\"https://ca.linkedin.com/in/saeedaghabozorgi\">Saeed Aghabozorgi</a>, PhD is a Data Scientist in IBM with a track record of developing enterprise level applications that substantially increases clients’ ability to turn data into actionable knowledge. He is a researcher in data mining field and expert in developing advanced analytic methods like machine learning and statistical modelling on large datasets.</p>\n",
    "\n",
    "<hr>\n",
    "\n",
    "<p>Copyright &copy; 2018 <a href=\"https://cocl.us/DX0108EN_CC\">Cognitive Class</a>. This notebook and its source code are released under the terms of the <a href=\"https://bigdatauniversity.com/mit-license/\">MIT License</a>.</p>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.6",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}