ml2lab.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Bach Chorales Harmony\n",
    "\n",
    "Учитывая музыкальный поток, задача анализа гармонии музыки состоит в том, чтобы связать метку аккорда с каждой точкой времени. Такие ярлыки показывают основную гармонию, указывая фундаментальную ноту (корень) и режим, используя имена аккордов, такие как C minor.\n",
    "\n",
    "Набор данных состоит из 60 хоралов (5665 событий) J.S. Бах (1675-1750). Каждое событие каждого хорала помечено, используя 1 из 101 аккордовых меток.\n",
    "Количество атрибутов: 17 (имя последовательности, номер события, наличие метки (x12), аккорд).\n",
    "\n",
    "Информация о высотных классах была извлечена из загруженных источников MIDI от (JSB Chorales) [http://www.jsbchorales.net]. Информация о метре были рассчитаны через программу Meter, которая является частью Melisma музыкальный анализатор (Melisma) [http://www.link.cs.cmu.edu/music-analysis/]. Рукописные ярлыки вручную аннотируются экспертом-человеком.\n",
    "\n",
    "## Описание данных\n",
    "\n",
    "   * ***Choral ID***: соответствует именам файлов из (Bach Central) [http://www.bachcentral.com];\n",
    "   * ***Event number***:  индекс (начиная с 1) события внутри хорала;\n",
    "   * ***Pitch classes(C..B) (x12)***: YES / NO в зависимости от того, присутствует ли данный высотный класс [https://ru.wikipedia.org/wiki/Высотный_класс];\n",
    "   * ***Pitch class Bass***: высотный класс басовой ноты;\n",
    "   * ***Meter***: целые числа от 1 до 5, меньшие числа обозначают менее акцентированные события, более высокие числа обозначают более акцентированные события;\n",
    "   * ***Chord label***: аккорд, резонирующий во время данного события."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Подключаем используемые в работе библиотеки"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy  as np\n",
    "import matplotlib.pyplot  as plt\n",
    "import pandas as pd\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Считываем данные из датасета"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = pd.read_csv('source/jsbach_chorals_harmony.data', sep = ',', engine = 'python')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Контроль ввода"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Choral ID</th>\n",
       "      <th>Event number</th>\n",
       "      <th>Pitch class C</th>\n",
       "      <th>Pitch class C#</th>\n",
       "      <th>Pitch class D</th>\n",
       "      <th>Pitch class D#</th>\n",
       "      <th>Pitch class E</th>\n",
       "      <th>Pitch class F</th>\n",
       "      <th>Pitch class F#</th>\n",
       "      <th>Pitch class G</th>\n",
       "      <th>Pitch class G#</th>\n",
       "      <th>Pitch class A</th>\n",
       "      <th>Pitch class A#</th>\n",
       "      <th>Pitch class B</th>\n",
       "      <th>Pitch class Bass</th>\n",
       "      <th>Meter</th>\n",
       "      <th>Chord label</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>000106b_</td>\n",
       "      <td>1</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>F</td>\n",
       "      <td>3</td>\n",
       "      <td>F_M</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>000106b_</td>\n",
       "      <td>2</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>E</td>\n",
       "      <td>5</td>\n",
       "      <td>C_M</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>000106b_</td>\n",
       "      <td>3</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>E</td>\n",
       "      <td>2</td>\n",
       "      <td>C_M</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>000106b_</td>\n",
       "      <td>4</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>F</td>\n",
       "      <td>3</td>\n",
       "      <td>F_M</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>000106b_</td>\n",
       "      <td>5</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>F</td>\n",
       "      <td>2</td>\n",
       "      <td>F_M</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>000106b_</td>\n",
       "      <td>6</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>D</td>\n",
       "      <td>4</td>\n",
       "      <td>D_m</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>000106b_</td>\n",
       "      <td>7</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>D</td>\n",
       "      <td>2</td>\n",
       "      <td>D_m</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>000106b_</td>\n",
       "      <td>8</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>A</td>\n",
       "      <td>3</td>\n",
       "      <td>F_M</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>000106b_</td>\n",
       "      <td>9</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>A</td>\n",
       "      <td>2</td>\n",
       "      <td>F_M</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>000106b_</td>\n",
       "      <td>10</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>Bb</td>\n",
       "      <td>5</td>\n",
       "      <td>BbM</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>000106b_</td>\n",
       "      <td>11</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>Bb</td>\n",
       "      <td>1</td>\n",
       "      <td>BbM</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>000106b_</td>\n",
       "      <td>12</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>C</td>\n",
       "      <td>2</td>\n",
       "      <td>BbM</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>000106b_</td>\n",
       "      <td>13</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>C</td>\n",
       "      <td>1</td>\n",
       "      <td>BbM</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>000106b_</td>\n",
       "      <td>14</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>D</td>\n",
       "      <td>3</td>\n",
       "      <td>BbM</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>000106b_</td>\n",
       "      <td>15</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>E</td>\n",
       "      <td>2</td>\n",
       "      <td>BbM</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>000106b_</td>\n",
       "      <td>16</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>F</td>\n",
       "      <td>4</td>\n",
       "      <td>F_M</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>000106b_</td>\n",
       "      <td>17</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>F</td>\n",
       "      <td>3</td>\n",
       "      <td>F_M</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>000106b_</td>\n",
       "      <td>18</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>Bb</td>\n",
       "      <td>5</td>\n",
       "      <td>BbM</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>000106b_</td>\n",
       "      <td>19</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>Bb</td>\n",
       "      <td>2</td>\n",
       "      <td>BbM</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>000106b_</td>\n",
       "      <td>20</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>Bb</td>\n",
       "      <td>1</td>\n",
       "      <td>BbM</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>000106b_</td>\n",
       "      <td>21</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>A</td>\n",
       "      <td>3</td>\n",
       "      <td>C_M7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>000106b_</td>\n",
       "      <td>22</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>G</td>\n",
       "      <td>2</td>\n",
       "      <td>C_M7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>000106b_</td>\n",
       "      <td>23</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>G</td>\n",
       "      <td>1</td>\n",
       "      <td>C_M7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>000106b_</td>\n",
       "      <td>24</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>A</td>\n",
       "      <td>4</td>\n",
       "      <td>F_M</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>000106b_</td>\n",
       "      <td>25</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>B</td>\n",
       "      <td>2</td>\n",
       "      <td>F_M</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>000106b_</td>\n",
       "      <td>26</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>C</td>\n",
       "      <td>3</td>\n",
       "      <td>C_M</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>000106b_</td>\n",
       "      <td>27</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>C</td>\n",
       "      <td>2</td>\n",
       "      <td>C_M</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>000106b_</td>\n",
       "      <td>28</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>F</td>\n",
       "      <td>5</td>\n",
       "      <td>D_m7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>000106b_</td>\n",
       "      <td>29</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>F</td>\n",
       "      <td>2</td>\n",
       "      <td>D_m7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>000106b_</td>\n",
       "      <td>30</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>F</td>\n",
       "      <td>1</td>\n",
       "      <td>D_m7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5635</th>\n",
       "      <td>015505b_</td>\n",
       "      <td>80</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>F</td>\n",
       "      <td>5</td>\n",
       "      <td>F_M</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5636</th>\n",
       "      <td>015505b_</td>\n",
       "      <td>81</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>F</td>\n",
       "      <td>3</td>\n",
       "      <td>F_M</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5637</th>\n",
       "      <td>015505b_</td>\n",
       "      <td>82</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>F</td>\n",
       "      <td>4</td>\n",
       "      <td>F_M</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5638</th>\n",
       "      <td>015505b_</td>\n",
       "      <td>83</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>G</td>\n",
       "      <td>3</td>\n",
       "      <td>F_M</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5639</th>\n",
       "      <td>015505b_</td>\n",
       "      <td>84</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>A</td>\n",
       "      <td>5</td>\n",
       "      <td>F_M</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5640</th>\n",
       "      <td>015505b_</td>\n",
       "      <td>85</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>A</td>\n",
       "      <td>3</td>\n",
       "      <td>F_M</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5641</th>\n",
       "      <td>015505b_</td>\n",
       "      <td>86</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>Bb</td>\n",
       "      <td>4</td>\n",
       "      <td>BbM</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5642</th>\n",
       "      <td>015505b_</td>\n",
       "      <td>87</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>C</td>\n",
       "      <td>3</td>\n",
       "      <td>BbM</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5643</th>\n",
       "      <td>015505b_</td>\n",
       "      <td>88</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>D</td>\n",
       "      <td>5</td>\n",
       "      <td>D_m</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5644</th>\n",
       "      <td>015505b_</td>\n",
       "      <td>89</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>D</td>\n",
       "      <td>3</td>\n",
       "      <td>D_m</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5645</th>\n",
       "      <td>015505b_</td>\n",
       "      <td>90</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>E</td>\n",
       "      <td>4</td>\n",
       "      <td>C_M</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5646</th>\n",
       "      <td>015505b_</td>\n",
       "      <td>91</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>F#</td>\n",
       "      <td>3</td>\n",
       "      <td>C_M</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5647</th>\n",
       "      <td>015505b_</td>\n",
       "      <td>92</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>G</td>\n",
       "      <td>5</td>\n",
       "      <td>G_m</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5648</th>\n",
       "      <td>015505b_</td>\n",
       "      <td>93</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>G</td>\n",
       "      <td>3</td>\n",
       "      <td>G_m</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5649</th>\n",
       "      <td>015505b_</td>\n",
       "      <td>94</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>D</td>\n",
       "      <td>4</td>\n",
       "      <td>D_M</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5650</th>\n",
       "      <td>015505b_</td>\n",
       "      <td>95</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>A</td>\n",
       "      <td>5</td>\n",
       "      <td>F_M</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5651</th>\n",
       "      <td>015505b_</td>\n",
       "      <td>96</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>Bb</td>\n",
       "      <td>3</td>\n",
       "      <td>F_M</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5652</th>\n",
       "      <td>015505b_</td>\n",
       "      <td>97</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>C</td>\n",
       "      <td>4</td>\n",
       "      <td>C_M</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5653</th>\n",
       "      <td>015505b_</td>\n",
       "      <td>98</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>C</td>\n",
       "      <td>3</td>\n",
       "      <td>C_M</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5654</th>\n",
       "      <td>015505b_</td>\n",
       "      <td>99</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>D</td>\n",
       "      <td>5</td>\n",
       "      <td>BbM</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5655</th>\n",
       "      <td>015505b_</td>\n",
       "      <td>100</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>E</td>\n",
       "      <td>3</td>\n",
       "      <td>BbM</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5656</th>\n",
       "      <td>015505b_</td>\n",
       "      <td>101</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>F</td>\n",
       "      <td>4</td>\n",
       "      <td>F_M</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5657</th>\n",
       "      <td>015505b_</td>\n",
       "      <td>102</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>F</td>\n",
       "      <td>3</td>\n",
       "      <td>F_M</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5658</th>\n",
       "      <td>015505b_</td>\n",
       "      <td>103</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>E</td>\n",
       "      <td>5</td>\n",
       "      <td>C_M</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5659</th>\n",
       "      <td>015505b_</td>\n",
       "      <td>104</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>F</td>\n",
       "      <td>3</td>\n",
       "      <td>C_M</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5660</th>\n",
       "      <td>015505b_</td>\n",
       "      <td>105</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>G</td>\n",
       "      <td>4</td>\n",
       "      <td>G_m</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5661</th>\n",
       "      <td>015505b_</td>\n",
       "      <td>106</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>G</td>\n",
       "      <td>3</td>\n",
       "      <td>G_m</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5662</th>\n",
       "      <td>015505b_</td>\n",
       "      <td>107</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>C</td>\n",
       "      <td>5</td>\n",
       "      <td>C_M</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5663</th>\n",
       "      <td>015505b_</td>\n",
       "      <td>108</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>C</td>\n",
       "      <td>3</td>\n",
       "      <td>C_M</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5664</th>\n",
       "      <td>015505b_</td>\n",
       "      <td>109</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>F</td>\n",
       "      <td>4</td>\n",
       "      <td>F_M</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5665 rows × 17 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     Choral ID  Event number Pitch class C Pitch class C# Pitch class D  \\\n",
       "0     000106b_             1           YES             NO            NO   \n",
       "1     000106b_             2           YES             NO            NO   \n",
       "2     000106b_             3           YES             NO            NO   \n",
       "3     000106b_             4           YES             NO            NO   \n",
       "4     000106b_             5           YES             NO            NO   \n",
       "5     000106b_             6            NO             NO           YES   \n",
       "6     000106b_             7            NO             NO           YES   \n",
       "7     000106b_             8           YES             NO            NO   \n",
       "8     000106b_             9           YES             NO            NO   \n",
       "9     000106b_            10            NO             NO           YES   \n",
       "10    000106b_            11            NO             NO           YES   \n",
       "11    000106b_            12           YES             NO           YES   \n",
       "12    000106b_            13           YES             NO           YES   \n",
       "13    000106b_            14            NO             NO           YES   \n",
       "14    000106b_            15            NO             NO           YES   \n",
       "15    000106b_            16           YES             NO            NO   \n",
       "16    000106b_            17           YES             NO            NO   \n",
       "17    000106b_            18            NO             NO           YES   \n",
       "18    000106b_            19            NO             NO           YES   \n",
       "19    000106b_            20            NO             NO           YES   \n",
       "20    000106b_            21            NO             NO            NO   \n",
       "21    000106b_            22           YES             NO            NO   \n",
       "22    000106b_            23            NO             NO            NO   \n",
       "23    000106b_            24           YES             NO            NO   \n",
       "24    000106b_            25            NO             NO           YES   \n",
       "25    000106b_            26           YES             NO            NO   \n",
       "26    000106b_            27           YES             NO            NO   \n",
       "27    000106b_            28           YES             NO           YES   \n",
       "28    000106b_            29            NO             NO           YES   \n",
       "29    000106b_            30            NO             NO           YES   \n",
       "...        ...           ...           ...            ...           ...   \n",
       "5635  015505b_            80           YES             NO            NO   \n",
       "5636  015505b_            81           YES             NO            NO   \n",
       "5637  015505b_            82           YES             NO            NO   \n",
       "5638  015505b_            83           YES             NO            NO   \n",
       "5639  015505b_            84           YES             NO            NO   \n",
       "5640  015505b_            85           YES             NO            NO   \n",
       "5641  015505b_            86            NO             NO           YES   \n",
       "5642  015505b_            87           YES             NO           YES   \n",
       "5643  015505b_            88            NO             NO           YES   \n",
       "5644  015505b_            89            NO             NO           YES   \n",
       "5645  015505b_            90           YES             NO            NO   \n",
       "5646  015505b_            91           YES             NO            NO   \n",
       "5647  015505b_            92            NO             NO           YES   \n",
       "5648  015505b_            93           YES             NO            NO   \n",
       "5649  015505b_            94            NO             NO           YES   \n",
       "5650  015505b_            95           YES             NO            NO   \n",
       "5651  015505b_            96           YES             NO            NO   \n",
       "5652  015505b_            97           YES             NO            NO   \n",
       "5653  015505b_            98           YES             NO            NO   \n",
       "5654  015505b_            99            NO             NO           YES   \n",
       "5655  015505b_           100            NO             NO           YES   \n",
       "5656  015505b_           101           YES             NO            NO   \n",
       "5657  015505b_           102            NO             NO           YES   \n",
       "5658  015505b_           103           YES             NO            NO   \n",
       "5659  015505b_           104           YES             NO            NO   \n",
       "5660  015505b_           105            NO             NO           YES   \n",
       "5661  015505b_           106            NO             NO           YES   \n",
       "5662  015505b_           107           YES             NO            NO   \n",
       "5663  015505b_           108           YES             NO            NO   \n",
       "5664  015505b_           109            NO             NO            NO   \n",
       "\n",
       "     Pitch class D# Pitch class E Pitch class F Pitch class F# Pitch class G  \\\n",
       "0                NO            NO           YES             NO            NO   \n",
       "1                NO           YES            NO             NO           YES   \n",
       "2                NO           YES            NO             NO           YES   \n",
       "3                NO            NO           YES             NO            NO   \n",
       "4                NO            NO           YES             NO            NO   \n",
       "5                NO            NO           YES             NO            NO   \n",
       "6                NO            NO           YES             NO            NO   \n",
       "7                NO            NO           YES             NO            NO   \n",
       "8                NO            NO           YES             NO            NO   \n",
       "9                NO            NO           YES             NO            NO   \n",
       "10               NO            NO           YES             NO            NO   \n",
       "11               NO            NO           YES             NO            NO   \n",
       "12               NO            NO           YES             NO           YES   \n",
       "13               NO            NO           YES             NO            NO   \n",
       "14               NO           YES           YES             NO           YES   \n",
       "15               NO            NO           YES             NO            NO   \n",
       "16               NO            NO           YES             NO            NO   \n",
       "17               NO            NO           YES             NO            NO   \n",
       "18               NO            NO           YES             NO           YES   \n",
       "19               NO            NO           YES             NO            NO   \n",
       "20               NO           YES            NO             NO           YES   \n",
       "21               NO           YES            NO             NO           YES   \n",
       "22               NO           YES            NO             NO           YES   \n",
       "23               NO            NO           YES             NO            NO   \n",
       "24               NO            NO           YES             NO           YES   \n",
       "25               NO           YES            NO             NO           YES   \n",
       "26               NO           YES            NO             NO           YES   \n",
       "27               NO            NO           YES             NO            NO   \n",
       "28               NO           YES           YES             NO            NO   \n",
       "29               NO           YES           YES             NO            NO   \n",
       "...             ...           ...           ...            ...           ...   \n",
       "5635             NO            NO           YES             NO            NO   \n",
       "5636             NO            NO           YES             NO           YES   \n",
       "5637             NO            NO           YES             NO            NO   \n",
       "5638             NO            NO           YES             NO           YES   \n",
       "5639             NO            NO           YES             NO            NO   \n",
       "5640             NO           YES            NO             NO           YES   \n",
       "5641             NO            NO           YES             NO            NO   \n",
       "5642             NO            NO           YES             NO            NO   \n",
       "5643             NO            NO           YES             NO            NO   \n",
       "5644             NO            NO           YES             NO            NO   \n",
       "5645             NO           YES            NO             NO           YES   \n",
       "5646             NO           YES            NO            YES            NO   \n",
       "5647             NO            NO            NO             NO           YES   \n",
       "5648             NO           YES            NO             NO           YES   \n",
       "5649             NO            NO            NO            YES            NO   \n",
       "5650             NO            NO           YES             NO            NO   \n",
       "5651             NO            NO           YES             NO            NO   \n",
       "5652             NO           YES            NO             NO           YES   \n",
       "5653             NO           YES            NO             NO            NO   \n",
       "5654             NO            NO           YES             NO            NO   \n",
       "5655             NO           YES            NO             NO           YES   \n",
       "5656             NO            NO           YES             NO            NO   \n",
       "5657             NO            NO           YES             NO            NO   \n",
       "5658             NO           YES            NO             NO           YES   \n",
       "5659             NO           YES           YES             NO           YES   \n",
       "5660             NO            NO            NO             NO           YES   \n",
       "5661             NO            NO            NO             NO           YES   \n",
       "5662             NO           YES            NO             NO           YES   \n",
       "5663             NO           YES            NO             NO           YES   \n",
       "5664             NO            NO           YES             NO            NO   \n",
       "\n",
       "     Pitch class G# Pitch class A Pitch class A# Pitch class B  \\\n",
       "0                NO           YES             NO            NO   \n",
       "1                NO            NO             NO            NO   \n",
       "2                NO            NO             NO            NO   \n",
       "3                NO           YES             NO            NO   \n",
       "4                NO           YES             NO            NO   \n",
       "5                NO           YES             NO            NO   \n",
       "6                NO           YES             NO            NO   \n",
       "7                NO           YES             NO            NO   \n",
       "8                NO           YES             NO            NO   \n",
       "9                NO            NO            YES            NO   \n",
       "10               NO            NO            YES            NO   \n",
       "11               NO           YES            YES            NO   \n",
       "12               NO            NO            YES            NO   \n",
       "13               NO            NO            YES            NO   \n",
       "14               NO            NO            YES            NO   \n",
       "15               NO           YES             NO            NO   \n",
       "16               NO           YES             NO            NO   \n",
       "17               NO            NO            YES            NO   \n",
       "18               NO            NO            YES            NO   \n",
       "19               NO           YES            YES            NO   \n",
       "20               NO           YES            YES            NO   \n",
       "21               NO            NO            YES            NO   \n",
       "22               NO            NO            YES            NO   \n",
       "23               NO           YES             NO            NO   \n",
       "24               NO            NO             NO           YES   \n",
       "25               NO            NO             NO            NO   \n",
       "26               NO            NO             NO            NO   \n",
       "27               NO           YES             NO            NO   \n",
       "28               NO           YES             NO           YES   \n",
       "29               NO           YES             NO            NO   \n",
       "...             ...           ...            ...           ...   \n",
       "5635             NO           YES             NO            NO   \n",
       "5636             NO            NO            YES            NO   \n",
       "5637             NO           YES             NO            NO   \n",
       "5638             NO            NO            YES            NO   \n",
       "5639             NO           YES             NO            NO   \n",
       "5640             NO           YES             NO            NO   \n",
       "5641             NO            NO            YES            NO   \n",
       "5642             NO            NO             NO            NO   \n",
       "5643             NO           YES             NO            NO   \n",
       "5644             NO            NO            YES            NO   \n",
       "5645             NO            NO             NO            NO   \n",
       "5646             NO           YES             NO            NO   \n",
       "5647             NO            NO            YES            NO   \n",
       "5648             NO            NO            YES            NO   \n",
       "5649             NO           YES             NO            NO   \n",
       "5650             NO           YES             NO            NO   \n",
       "5651             NO           YES            YES            NO   \n",
       "5652             NO            NO             NO            NO   \n",
       "5653             NO           YES             NO            NO   \n",
       "5654             NO            NO            YES            NO   \n",
       "5655             NO            NO            YES            NO   \n",
       "5656             NO           YES             NO            NO   \n",
       "5657             NO           YES            YES            NO   \n",
       "5658             NO            NO             NO            NO   \n",
       "5659             NO            NO             NO            NO   \n",
       "5660             NO            NO            YES            NO   \n",
       "5661             NO           YES             NO            NO   \n",
       "5662             NO            NO             NO            NO   \n",
       "5663             NO            NO            YES            NO   \n",
       "5664             NO           YES             NO            NO   \n",
       "\n",
       "     Pitch class Bass  Meter Chord label  \n",
       "0                   F      3         F_M  \n",
       "1                   E      5         C_M  \n",
       "2                   E      2         C_M  \n",
       "3                   F      3         F_M  \n",
       "4                   F      2         F_M  \n",
       "5                   D      4         D_m  \n",
       "6                   D      2         D_m  \n",
       "7                   A      3         F_M  \n",
       "8                   A      2         F_M  \n",
       "9                  Bb      5         BbM  \n",
       "10                 Bb      1         BbM  \n",
       "11                  C      2         BbM  \n",
       "12                  C      1         BbM  \n",
       "13                  D      3         BbM  \n",
       "14                  E      2         BbM  \n",
       "15                  F      4         F_M  \n",
       "16                  F      3         F_M  \n",
       "17                 Bb      5         BbM  \n",
       "18                 Bb      2         BbM  \n",
       "19                 Bb      1         BbM  \n",
       "20                  A      3        C_M7  \n",
       "21                  G      2        C_M7  \n",
       "22                  G      1        C_M7  \n",
       "23                  A      4         F_M  \n",
       "24                  B      2         F_M  \n",
       "25                  C      3         C_M  \n",
       "26                  C      2         C_M  \n",
       "27                  F      5        D_m7  \n",
       "28                  F      2        D_m7  \n",
       "29                  F      1        D_m7  \n",
       "...               ...    ...         ...  \n",
       "5635                F      5         F_M  \n",
       "5636                F      3         F_M  \n",
       "5637                F      4         F_M  \n",
       "5638                G      3         F_M  \n",
       "5639                A      5         F_M  \n",
       "5640                A      3         F_M  \n",
       "5641               Bb      4         BbM  \n",
       "5642                C      3         BbM  \n",
       "5643                D      5         D_m  \n",
       "5644                D      3         D_m  \n",
       "5645                E      4         C_M  \n",
       "5646               F#      3         C_M  \n",
       "5647                G      5         G_m  \n",
       "5648                G      3         G_m  \n",
       "5649                D      4         D_M  \n",
       "5650                A      5         F_M  \n",
       "5651               Bb      3         F_M  \n",
       "5652                C      4         C_M  \n",
       "5653                C      3         C_M  \n",
       "5654                D      5         BbM  \n",
       "5655                E      3         BbM  \n",
       "5656                F      4         F_M  \n",
       "5657                F      3         F_M  \n",
       "5658                E      5         C_M  \n",
       "5659                F      3         C_M  \n",
       "5660                G      4         G_m  \n",
       "5661                G      3         G_m  \n",
       "5662                C      5         C_M  \n",
       "5663                C      3         C_M  \n",
       "5664                F      4         F_M  \n",
       "\n",
       "[5665 rows x 17 columns]"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Контроль размеров таблицы и наименования признаков(столбцов)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(5665, 17)\n",
      "Index(['Choral ID', 'Event number', 'Pitch class C', 'Pitch class C#',\n",
      "       'Pitch class D', 'Pitch class D#', 'Pitch class E', 'Pitch class F',\n",
      "       'Pitch class F#', 'Pitch class G', 'Pitch class G#', 'Pitch class A',\n",
      "       'Pitch class A#', 'Pitch class B', 'Pitch class Bass', 'Meter',\n",
      "       'Chord label'],\n",
      "      dtype='object')\n"
     ]
    }
   ],
   "source": [
    "print(data.shape)\n",
    "print(data.columns)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Основные сведения по данным, из таблицы видно, что пропусков в данных нет, уже хорошо, большинство признаков бинарные."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Choral ID</th>\n",
       "      <th>Pitch class C</th>\n",
       "      <th>Pitch class C#</th>\n",
       "      <th>Pitch class D</th>\n",
       "      <th>Pitch class D#</th>\n",
       "      <th>Pitch class E</th>\n",
       "      <th>Pitch class F</th>\n",
       "      <th>Pitch class F#</th>\n",
       "      <th>Pitch class G</th>\n",
       "      <th>Pitch class G#</th>\n",
       "      <th>Pitch class A</th>\n",
       "      <th>Pitch class A#</th>\n",
       "      <th>Pitch class B</th>\n",
       "      <th>Pitch class Bass</th>\n",
       "      <th>Chord label</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>5665</td>\n",
       "      <td>5665</td>\n",
       "      <td>5665</td>\n",
       "      <td>5665</td>\n",
       "      <td>5665</td>\n",
       "      <td>5665</td>\n",
       "      <td>5665</td>\n",
       "      <td>5665</td>\n",
       "      <td>5665</td>\n",
       "      <td>5665</td>\n",
       "      <td>5665</td>\n",
       "      <td>5665</td>\n",
       "      <td>5665</td>\n",
       "      <td>5665</td>\n",
       "      <td>5665</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>unique</th>\n",
       "      <td>62</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>16</td>\n",
       "      <td>102</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>top</th>\n",
       "      <td>002908ch</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>D</td>\n",
       "      <td>D_M</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>freq</th>\n",
       "      <td>206</td>\n",
       "      <td>3875</td>\n",
       "      <td>4711</td>\n",
       "      <td>3300</td>\n",
       "      <td>4956</td>\n",
       "      <td>3540</td>\n",
       "      <td>4381</td>\n",
       "      <td>4253</td>\n",
       "      <td>3523</td>\n",
       "      <td>5006</td>\n",
       "      <td>3290</td>\n",
       "      <td>4644</td>\n",
       "      <td>3874</td>\n",
       "      <td>689</td>\n",
       "      <td>503</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       Choral ID Pitch class C Pitch class C# Pitch class D Pitch class D#  \\\n",
       "count       5665          5665           5665          5665           5665   \n",
       "unique        62             2              2             2              2   \n",
       "top     002908ch            NO             NO            NO             NO   \n",
       "freq         206          3875           4711          3300           4956   \n",
       "\n",
       "       Pitch class E Pitch class F Pitch class F# Pitch class G  \\\n",
       "count           5665          5665           5665          5665   \n",
       "unique             2             2              2             2   \n",
       "top               NO            NO             NO            NO   \n",
       "freq            3540          4381           4253          3523   \n",
       "\n",
       "       Pitch class G# Pitch class A Pitch class A# Pitch class B  \\\n",
       "count            5665          5665           5665          5665   \n",
       "unique              2             2              2             2   \n",
       "top                NO            NO             NO            NO   \n",
       "freq             5006          3290           4644          3874   \n",
       "\n",
       "       Pitch class Bass Chord label  \n",
       "count              5665        5665  \n",
       "unique               16         102  \n",
       "top                   D         D_M  \n",
       "freq                689         503  "
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.describe(include = ['object'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Слияние двух таблиц выше, максимальная полнота сведений о данных"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Choral ID</th>\n",
       "      <th>Event number</th>\n",
       "      <th>Pitch class C</th>\n",
       "      <th>Pitch class C#</th>\n",
       "      <th>Pitch class D</th>\n",
       "      <th>Pitch class D#</th>\n",
       "      <th>Pitch class E</th>\n",
       "      <th>Pitch class F</th>\n",
       "      <th>Pitch class F#</th>\n",
       "      <th>Pitch class G</th>\n",
       "      <th>Pitch class G#</th>\n",
       "      <th>Pitch class A</th>\n",
       "      <th>Pitch class A#</th>\n",
       "      <th>Pitch class B</th>\n",
       "      <th>Pitch class Bass</th>\n",
       "      <th>Meter</th>\n",
       "      <th>Chord label</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>5665</td>\n",
       "      <td>5665.000000</td>\n",
       "      <td>5665</td>\n",
       "      <td>5665</td>\n",
       "      <td>5665</td>\n",
       "      <td>5665</td>\n",
       "      <td>5665</td>\n",
       "      <td>5665</td>\n",
       "      <td>5665</td>\n",
       "      <td>5665</td>\n",
       "      <td>5665</td>\n",
       "      <td>5665</td>\n",
       "      <td>5665</td>\n",
       "      <td>5665</td>\n",
       "      <td>5665</td>\n",
       "      <td>5665.000000</td>\n",
       "      <td>5665</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>unique</th>\n",
       "      <td>62</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>16</td>\n",
       "      <td>NaN</td>\n",
       "      <td>102</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>top</th>\n",
       "      <td>002908ch</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>D</td>\n",
       "      <td>NaN</td>\n",
       "      <td>D_M</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>freq</th>\n",
       "      <td>206</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3875</td>\n",
       "      <td>4711</td>\n",
       "      <td>3300</td>\n",
       "      <td>4956</td>\n",
       "      <td>3540</td>\n",
       "      <td>4381</td>\n",
       "      <td>4253</td>\n",
       "      <td>3523</td>\n",
       "      <td>5006</td>\n",
       "      <td>3290</td>\n",
       "      <td>4644</td>\n",
       "      <td>3874</td>\n",
       "      <td>689</td>\n",
       "      <td>NaN</td>\n",
       "      <td>503</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>NaN</td>\n",
       "      <td>53.374404</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3.134863</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>NaN</td>\n",
       "      <td>37.268208</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.109710</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>NaN</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>NaN</td>\n",
       "      <td>24.000000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>NaN</td>\n",
       "      <td>48.000000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>NaN</td>\n",
       "      <td>75.000000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>NaN</td>\n",
       "      <td>207.000000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       Choral ID  Event number Pitch class C Pitch class C# Pitch class D  \\\n",
       "count       5665   5665.000000          5665           5665          5665   \n",
       "unique        62           NaN             2              2             2   \n",
       "top     002908ch           NaN            NO             NO            NO   \n",
       "freq         206           NaN          3875           4711          3300   \n",
       "mean         NaN     53.374404           NaN            NaN           NaN   \n",
       "std          NaN     37.268208           NaN            NaN           NaN   \n",
       "min          NaN      1.000000           NaN            NaN           NaN   \n",
       "25%          NaN     24.000000           NaN            NaN           NaN   \n",
       "50%          NaN     48.000000           NaN            NaN           NaN   \n",
       "75%          NaN     75.000000           NaN            NaN           NaN   \n",
       "max          NaN    207.000000           NaN            NaN           NaN   \n",
       "\n",
       "       Pitch class D# Pitch class E Pitch class F Pitch class F#  \\\n",
       "count            5665          5665          5665           5665   \n",
       "unique              2             2             2              2   \n",
       "top                NO            NO            NO             NO   \n",
       "freq             4956          3540          4381           4253   \n",
       "mean              NaN           NaN           NaN            NaN   \n",
       "std               NaN           NaN           NaN            NaN   \n",
       "min               NaN           NaN           NaN            NaN   \n",
       "25%               NaN           NaN           NaN            NaN   \n",
       "50%               NaN           NaN           NaN            NaN   \n",
       "75%               NaN           NaN           NaN            NaN   \n",
       "max               NaN           NaN           NaN            NaN   \n",
       "\n",
       "       Pitch class G Pitch class G# Pitch class A Pitch class A#  \\\n",
       "count           5665           5665          5665           5665   \n",
       "unique             2              2             2              2   \n",
       "top               NO             NO            NO             NO   \n",
       "freq            3523           5006          3290           4644   \n",
       "mean             NaN            NaN           NaN            NaN   \n",
       "std              NaN            NaN           NaN            NaN   \n",
       "min              NaN            NaN           NaN            NaN   \n",
       "25%              NaN            NaN           NaN            NaN   \n",
       "50%              NaN            NaN           NaN            NaN   \n",
       "75%              NaN            NaN           NaN            NaN   \n",
       "max              NaN            NaN           NaN            NaN   \n",
       "\n",
       "       Pitch class B Pitch class Bass        Meter Chord label  \n",
       "count           5665             5665  5665.000000        5665  \n",
       "unique             2               16          NaN         102  \n",
       "top               NO                D          NaN         D_M  \n",
       "freq            3874              689          NaN         503  \n",
       "mean             NaN              NaN     3.134863         NaN  \n",
       "std              NaN              NaN     1.109710         NaN  \n",
       "min              NaN              NaN     1.000000         NaN  \n",
       "25%              NaN              NaN     2.000000         NaN  \n",
       "50%              NaN              NaN     3.000000         NaN  \n",
       "75%              NaN              NaN     4.000000         NaN  \n",
       "max              NaN              NaN     5.000000         NaN  "
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.describe(include = 'all')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Подготовка данных\n",
    "\n",
    "   Выбросим признак Choral ID, поскольку от названия произведения мало что зависит, также мало что зависит от порядкового номера события, значит Event Number выбрасываем туда-же. \n",
    "   Задача состоит в предсказании резонирующего в данный момент аккорда по встречающимся или не встречающимся высотным классам основного исполнения и баса. Большинство признаков бинарных, остаётся только придумать, что делать с категориальным признаком Pitch Class Bass, например векторизовать его."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = data.drop('Choral ID', axis=1)\n",
    "data = data.drop('Event number', axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Поскольку столбец Chord label содержит ответ, уберём из таблицы и его"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [],
   "source": [
    "y = data['Chord label']\n",
    "data = data.drop('Chord label', axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Pitch class C</th>\n",
       "      <th>Pitch class C#</th>\n",
       "      <th>Pitch class D</th>\n",
       "      <th>Pitch class D#</th>\n",
       "      <th>Pitch class E</th>\n",
       "      <th>Pitch class F</th>\n",
       "      <th>Pitch class F#</th>\n",
       "      <th>Pitch class G</th>\n",
       "      <th>Pitch class G#</th>\n",
       "      <th>Pitch class A</th>\n",
       "      <th>Pitch class A#</th>\n",
       "      <th>Pitch class B</th>\n",
       "      <th>Pitch class Bass</th>\n",
       "      <th>Meter</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>F</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>E</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>E</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>F</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>YES</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>F</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Pitch class C Pitch class C# Pitch class D Pitch class D# Pitch class E  \\\n",
       "0           YES             NO            NO             NO            NO   \n",
       "1           YES             NO            NO             NO           YES   \n",
       "2           YES             NO            NO             NO           YES   \n",
       "3           YES             NO            NO             NO            NO   \n",
       "4           YES             NO            NO             NO            NO   \n",
       "\n",
       "  Pitch class F Pitch class F# Pitch class G Pitch class G# Pitch class A  \\\n",
       "0           YES             NO            NO             NO           YES   \n",
       "1            NO             NO           YES             NO            NO   \n",
       "2            NO             NO           YES             NO            NO   \n",
       "3           YES             NO            NO             NO           YES   \n",
       "4           YES             NO            NO             NO           YES   \n",
       "\n",
       "  Pitch class A# Pitch class B Pitch class Bass  Meter  \n",
       "0             NO            NO                F      3  \n",
       "1             NO            NO                E      5  \n",
       "2             NO            NO                E      2  \n",
       "3             NO            NO                F      3  \n",
       "4             NO            NO                F      2  "
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['F', 'E', 'D', 'A', 'Bb', 'C', 'G', 'B', 'F#', 'C#', 'Eb', 'D#',\n",
       "       'G#', 'A#', 'Ab', 'Db'], dtype=object)"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data['Pitch class Bass'].unique()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Имеются повторяющиеся с точки зрения музыки, но разные по названию ноты (классы), приведём их к стандарту как в заголовке таблицы для основного звучания(C, C# = Db, D, D# = Eb, E, F, F#, G, G# = Ab, A, A# = Bb, B)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [],
   "source": [
    "d = {'Bb' : 'A#', 'Db' : 'C#', 'Eb': 'D#', 'Ab' : 'G#'}\n",
    "data = data.replace({'Pitch class Bass': d})\n",
    "#data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Разобьём таблицу на части по типу признака"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Pitch class C', 'Pitch class C#', 'Pitch class D', 'Pitch class D#', 'Pitch class E', 'Pitch class F', 'Pitch class F#', 'Pitch class G', 'Pitch class G#', 'Pitch class A', 'Pitch class A#', 'Pitch class B', 'Pitch class Bass']\n",
      "['Meter']\n"
     ]
    }
   ],
   "source": [
    "categorical_columns = [c for c in data.columns if data[c].dtype.name == 'object']\n",
    "numerical_columns   = [c for c in data.columns if data[c].dtype.name != 'object']\n",
    "print(categorical_columns)\n",
    "print(numerical_columns)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Pitch class C', 'Pitch class C#', 'Pitch class D', 'Pitch class D#', 'Pitch class E', 'Pitch class F', 'Pitch class F#', 'Pitch class G', 'Pitch class G#', 'Pitch class A', 'Pitch class A#', 'Pitch class B'] ['Pitch class Bass']\n"
     ]
    }
   ],
   "source": [
    "data_describe = data.describe(include = [object])\n",
    "binary_columns    = [c for c in categorical_columns if data_describe[c]['unique'] == 2]\n",
    "nonbinary_columns = [c for c in categorical_columns if data_describe[c]['unique'] > 2]\n",
    "print(binary_columns, nonbinary_columns)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Бинарный признак делаем с циферками"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [],
   "source": [
    "data.at[data['Pitch class C'] != 'YES', 'Pitch class C'] = 0\n",
    "data.at[data['Pitch class C'] == 'YES', 'Pitch class C'] = 1\n",
    "data.at[data['Pitch class C#'] != 'YES', 'Pitch class C#'] = 0\n",
    "data.at[data['Pitch class C#'] == 'YES', 'Pitch class C#'] = 1\n",
    "data.at[data['Pitch class D'] != 'YES', 'Pitch class D'] = 0\n",
    "data.at[data['Pitch class D'] == 'YES', 'Pitch class D'] = 1\n",
    "data.at[data['Pitch class D#'] != 'YES', 'Pitch class D#'] = 0\n",
    "data.at[data['Pitch class D#'] == 'YES', 'Pitch class D#'] = 1\n",
    "data.at[data['Pitch class E'] != 'YES', 'Pitch class E'] = 0\n",
    "data.at[data['Pitch class E'] == 'YES', 'Pitch class E'] = 1\n",
    "data.at[data['Pitch class F'] != 'YES', 'Pitch class F'] = 0\n",
    "data.at[data['Pitch class F'] == 'YES', 'Pitch class F'] = 1\n",
    "data.at[data['Pitch class F#'] != 'YES', 'Pitch class F#'] = 0\n",
    "data.at[data['Pitch class F#'] == 'YES', 'Pitch class F#'] = 1\n",
    "data.at[data['Pitch class G'] != 'YES', 'Pitch class G'] = 0\n",
    "data.at[data['Pitch class G'] == 'YES', 'Pitch class G'] = 1\n",
    "data.at[data['Pitch class G#'] != 'YES', 'Pitch class G#'] = 0\n",
    "data.at[data['Pitch class G#'] == 'YES', 'Pitch class G#'] = 1\n",
    "data.at[data['Pitch class A'] != 'YES', 'Pitch class A'] = 0\n",
    "data.at[data['Pitch class A'] == 'YES', 'Pitch class A'] = 1\n",
    "data.at[data['Pitch class A#'] != 'YES', 'Pitch class A#'] = 0\n",
    "data.at[data['Pitch class A#'] == 'YES', 'Pitch class A#'] = 1\n",
    "data.at[data['Pitch class B'] != 'YES', 'Pitch class B'] = 0\n",
    "data.at[data['Pitch class B'] == 'YES', 'Pitch class B'] = 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Векторизуем единственный оставшийся категориальный признак с помощью метода get_dummies из pandas:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Index(['Pitch class Bass_A', 'Pitch class Bass_A#', 'Pitch class Bass_B',\n",
      "       'Pitch class Bass_C', 'Pitch class Bass_C#', 'Pitch class Bass_D',\n",
      "       'Pitch class Bass_D#', 'Pitch class Bass_E', 'Pitch class Bass_F',\n",
      "       'Pitch class Bass_F#', 'Pitch class Bass_G', 'Pitch class Bass_G#'],\n",
      "      dtype='object')\n"
     ]
    }
   ],
   "source": [
    "data_nonbinary = pd.get_dummies(data[nonbinary_columns])\n",
    "print(data_nonbinary.columns)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Нормализация количественных признаков\n",
    "\n",
    "Многие алгоритмы машинного обучения чувствительны к масштабированию данных. К таким алгоритмам, например, относится метод ближайших соседей, машина опорных векторов и др.\n",
    "\n",
    "В этом случае количественные признаки полезно нормализовать. Это можно делать разными способами. Например, каждый количественный признак приведем к нулевому среднему и единичному среднеквадратичному отклонению:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Meter</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>5665.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>3.134863</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>1.109710</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>2.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>3.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>4.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>5.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             Meter\n",
       "count  5665.000000\n",
       "mean      3.134863\n",
       "std       1.109710\n",
       "min       1.000000\n",
       "25%       2.000000\n",
       "50%       3.000000\n",
       "75%       4.000000\n",
       "max       5.000000"
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_numerical = data[numerical_columns]\n",
    "data_numerical.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_numerical = (data_numerical - data_numerical.mean(axis = 0))/data_numerical.std(axis = 0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Meter</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>5.665000e+03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>-1.053605e-15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>1.000000e+00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>-1.923803e+00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>-1.022667e+00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>-1.215302e-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>7.796064e-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.680743e+00</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              Meter\n",
       "count  5.665000e+03\n",
       "mean  -1.053605e-15\n",
       "std    1.000000e+00\n",
       "min   -1.923803e+00\n",
       "25%   -1.022667e+00\n",
       "50%   -1.215302e-01\n",
       "75%    7.796064e-01\n",
       "max    1.680743e+00"
      ]
     },
     "execution_count": 82,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_numerical.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Соединяем всё в одну таблицу"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(5665, 25)\n"
     ]
    }
   ],
   "source": [
    "data = pd.concat((data_numerical, data_nonbinary, data[binary_columns]), axis = 1)\n",
    "print(data.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Meter</th>\n",
       "      <th>Pitch class Bass_A</th>\n",
       "      <th>Pitch class Bass_A#</th>\n",
       "      <th>Pitch class Bass_B</th>\n",
       "      <th>Pitch class Bass_C</th>\n",
       "      <th>Pitch class Bass_C#</th>\n",
       "      <th>Pitch class Bass_D</th>\n",
       "      <th>Pitch class Bass_D#</th>\n",
       "      <th>Pitch class Bass_E</th>\n",
       "      <th>Pitch class Bass_F</th>\n",
       "      <th>...</th>\n",
       "      <th>Pitch class D</th>\n",
       "      <th>Pitch class D#</th>\n",
       "      <th>Pitch class E</th>\n",
       "      <th>Pitch class F</th>\n",
       "      <th>Pitch class F#</th>\n",
       "      <th>Pitch class G</th>\n",
       "      <th>Pitch class G#</th>\n",
       "      <th>Pitch class A</th>\n",
       "      <th>Pitch class A#</th>\n",
       "      <th>Pitch class B</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>5.665000e+03</td>\n",
       "      <td>5665.000000</td>\n",
       "      <td>5665.000000</td>\n",
       "      <td>5665.000000</td>\n",
       "      <td>5665.000000</td>\n",
       "      <td>5665.000000</td>\n",
       "      <td>5665.000000</td>\n",
       "      <td>5665.000000</td>\n",
       "      <td>5665.000000</td>\n",
       "      <td>5665.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>5665.000000</td>\n",
       "      <td>5665.000000</td>\n",
       "      <td>5665.000000</td>\n",
       "      <td>5665.000000</td>\n",
       "      <td>5665.000000</td>\n",
       "      <td>5665.000000</td>\n",
       "      <td>5665.000000</td>\n",
       "      <td>5665.000000</td>\n",
       "      <td>5665.000000</td>\n",
       "      <td>5665.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>-1.053605e-15</td>\n",
       "      <td>0.121447</td>\n",
       "      <td>0.055075</td>\n",
       "      <td>0.079259</td>\n",
       "      <td>0.105914</td>\n",
       "      <td>0.038482</td>\n",
       "      <td>0.121624</td>\n",
       "      <td>0.044307</td>\n",
       "      <td>0.119682</td>\n",
       "      <td>0.081553</td>\n",
       "      <td>...</td>\n",
       "      <td>0.417476</td>\n",
       "      <td>0.125154</td>\n",
       "      <td>0.375110</td>\n",
       "      <td>0.226655</td>\n",
       "      <td>0.249250</td>\n",
       "      <td>0.378111</td>\n",
       "      <td>0.116328</td>\n",
       "      <td>0.419241</td>\n",
       "      <td>0.180229</td>\n",
       "      <td>0.316152</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>1.000000e+00</td>\n",
       "      <td>0.326675</td>\n",
       "      <td>0.228147</td>\n",
       "      <td>0.270166</td>\n",
       "      <td>0.307754</td>\n",
       "      <td>0.192374</td>\n",
       "      <td>0.326880</td>\n",
       "      <td>0.205795</td>\n",
       "      <td>0.324618</td>\n",
       "      <td>0.273707</td>\n",
       "      <td>...</td>\n",
       "      <td>0.493186</td>\n",
       "      <td>0.330923</td>\n",
       "      <td>0.484194</td>\n",
       "      <td>0.418704</td>\n",
       "      <td>0.432617</td>\n",
       "      <td>0.484958</td>\n",
       "      <td>0.320647</td>\n",
       "      <td>0.493478</td>\n",
       "      <td>0.384412</td>\n",
       "      <td>0.465014</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>-1.923803e+00</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>-1.022667e+00</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>-1.215302e-01</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>7.796064e-01</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.680743e+00</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>8 rows × 25 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "              Meter  Pitch class Bass_A  Pitch class Bass_A#  \\\n",
       "count  5.665000e+03         5665.000000          5665.000000   \n",
       "mean  -1.053605e-15            0.121447             0.055075   \n",
       "std    1.000000e+00            0.326675             0.228147   \n",
       "min   -1.923803e+00            0.000000             0.000000   \n",
       "25%   -1.022667e+00            0.000000             0.000000   \n",
       "50%   -1.215302e-01            0.000000             0.000000   \n",
       "75%    7.796064e-01            0.000000             0.000000   \n",
       "max    1.680743e+00            1.000000             1.000000   \n",
       "\n",
       "       Pitch class Bass_B  Pitch class Bass_C  Pitch class Bass_C#  \\\n",
       "count         5665.000000         5665.000000          5665.000000   \n",
       "mean             0.079259            0.105914             0.038482   \n",
       "std              0.270166            0.307754             0.192374   \n",
       "min              0.000000            0.000000             0.000000   \n",
       "25%              0.000000            0.000000             0.000000   \n",
       "50%              0.000000            0.000000             0.000000   \n",
       "75%              0.000000            0.000000             0.000000   \n",
       "max              1.000000            1.000000             1.000000   \n",
       "\n",
       "       Pitch class Bass_D  Pitch class Bass_D#  Pitch class Bass_E  \\\n",
       "count         5665.000000          5665.000000         5665.000000   \n",
       "mean             0.121624             0.044307            0.119682   \n",
       "std              0.326880             0.205795            0.324618   \n",
       "min              0.000000             0.000000            0.000000   \n",
       "25%              0.000000             0.000000            0.000000   \n",
       "50%              0.000000             0.000000            0.000000   \n",
       "75%              0.000000             0.000000            0.000000   \n",
       "max              1.000000             1.000000            1.000000   \n",
       "\n",
       "       Pitch class Bass_F      ...        Pitch class D  Pitch class D#  \\\n",
       "count         5665.000000      ...          5665.000000     5665.000000   \n",
       "mean             0.081553      ...             0.417476        0.125154   \n",
       "std              0.273707      ...             0.493186        0.330923   \n",
       "min              0.000000      ...             0.000000        0.000000   \n",
       "25%              0.000000      ...             0.000000        0.000000   \n",
       "50%              0.000000      ...             0.000000        0.000000   \n",
       "75%              0.000000      ...             1.000000        0.000000   \n",
       "max              1.000000      ...             1.000000        1.000000   \n",
       "\n",
       "       Pitch class E  Pitch class F  Pitch class F#  Pitch class G  \\\n",
       "count    5665.000000    5665.000000     5665.000000    5665.000000   \n",
       "mean        0.375110       0.226655        0.249250       0.378111   \n",
       "std         0.484194       0.418704        0.432617       0.484958   \n",
       "min         0.000000       0.000000        0.000000       0.000000   \n",
       "25%         0.000000       0.000000        0.000000       0.000000   \n",
       "50%         0.000000       0.000000        0.000000       0.000000   \n",
       "75%         1.000000       0.000000        0.000000       1.000000   \n",
       "max         1.000000       1.000000        1.000000       1.000000   \n",
       "\n",
       "       Pitch class G#  Pitch class A  Pitch class A#  Pitch class B  \n",
       "count     5665.000000    5665.000000     5665.000000    5665.000000  \n",
       "mean         0.116328       0.419241        0.180229       0.316152  \n",
       "std          0.320647       0.493478        0.384412       0.465014  \n",
       "min          0.000000       0.000000        0.000000       0.000000  \n",
       "25%          0.000000       0.000000        0.000000       0.000000  \n",
       "50%          0.000000       0.000000        0.000000       0.000000  \n",
       "75%          0.000000       1.000000        0.000000       1.000000  \n",
       "max          1.000000       1.000000        1.000000       1.000000  \n",
       "\n",
       "[8 rows x 25 columns]"
      ]
     },
     "execution_count": 84,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.describe()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Всё в порядке, 25 и ожидалось"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Х и У"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = data\n",
    "feature_names = X.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count     5665\n",
       "unique     102\n",
       "top        D_M\n",
       "freq       503\n",
       "Name: Chord label, dtype: object"
      ]
     },
     "execution_count": 86,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "pandas.core.indexes.base.Index"
      ]
     },
     "execution_count": 87,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(feature_names)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['Meter', 'Pitch class Bass_A', 'Pitch class Bass_A#',\n",
       "       'Pitch class Bass_B', 'Pitch class Bass_C', 'Pitch class Bass_C#',\n",
       "       'Pitch class Bass_D', 'Pitch class Bass_D#', 'Pitch class Bass_E',\n",
       "       'Pitch class Bass_F', 'Pitch class Bass_F#', 'Pitch class Bass_G',\n",
       "       'Pitch class Bass_G#', 'Pitch class C', 'Pitch class C#',\n",
       "       'Pitch class D', 'Pitch class D#', 'Pitch class E', 'Pitch class F',\n",
       "       'Pitch class F#', 'Pitch class G', 'Pitch class G#', 'Pitch class A',\n",
       "       'Pitch class A#', 'Pitch class B'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 88,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "feature_names"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(5665, 25)\n",
      "(5665,)\n"
     ]
    }
   ],
   "source": [
    "print(X.shape)\n",
    "print(y.shape)\n",
    "N, d = X.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "pandas.core.frame.DataFrame"
      ]
     },
     "execution_count": 90,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "pandas.core.series.Series"
      ]
     },
     "execution_count": 91,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {},
   "outputs": [],
   "source": [
    "#from sklearn.model_selection import train_test_split\n",
    "#X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.3, random_state = 42)\n",
    "\n",
    "#N_train, _ = X_train.shape \n",
    "#N_test,  _ = X_test.shape \n",
    "\n",
    "#print(N_train, N_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5665"
      ]
     },
     "execution_count": 93,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "3965+1700"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## MLP\n",
    "\n",
    "Разбиваем данные на тестовую и обучающую выборки"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.33, random_state = 7)\n",
    "N_test, _ = X_test.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Обучим классификатор"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.neural_network import MLPClassifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "MLPClassifier(activation='logistic', alpha=0.0001, batch_size='auto',\n",
       "       beta_1=0.9, beta_2=0.999, early_stopping=False, epsilon=1e-08,\n",
       "       hidden_layer_sizes=(100,), learning_rate='constant',\n",
       "       learning_rate_init=0.001, max_iter=200, momentum=0.9,\n",
       "       nesterovs_momentum=True, power_t=0.5, random_state=42, shuffle=True,\n",
       "       solver='lbfgs', tol=0.0001, validation_fraction=0.1, verbose=False,\n",
       "       warm_start=False)"
      ]
     },
     "execution_count": 96,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mlp_model = MLPClassifier(hidden_layer_sizes = (100,), solver = 'lbfgs', \n",
    "                          activation = 'logistic', random_state = 42)\n",
    "mlp_model.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Теперь можем классифицировать и проверить на правильность"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_train_pred = mlp_model.predict(X_train)\n",
    "y_test_pred = mlp_model.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.12042160737812911 0.26898395721925134\n"
     ]
    }
   ],
   "source": [
    "print(np.mean(y_train != y_train_pred), np.mean(y_test != y_test_pred))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.8795783926218709 0.7310160427807486\n"
     ]
    }
   ],
   "source": [
    "print(mlp_model.score(X_train, y_train), mlp_model.score(X_test, y_test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.2689839572192514\n"
     ]
    }
   ],
   "source": [
    "print(1 - mlp_model.score(X_test, y_test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.0001     0.00021544 0.00046416 0.001      0.00215443 0.00464159\n",
      " 0.01       0.02154435 0.04641589 0.1       ]\n"
     ]
    }
   ],
   "source": [
    "alpha_arr = np.logspace(-4, -1, 10)\n",
    "print(alpha_arr)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Подберем оптимальное значение параметра регуляризации alpha"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {},
   "outputs": [],
   "source": [
    "alpha_arr = np.logspace(-4, 2, 20)\n",
    "test_err = []\n",
    "train_err = []\n",
    "for alpha in alpha_arr:\n",
    "    mlp_model = MLPClassifier(alpha = alpha, hidden_layer_sizes = (100,), \n",
    "                              solver = 'lbfgs', activation = 'logistic', random_state = 42)\n",
    "    mlp_model.fit(X_train, y_train)\n",
    "\n",
    "    y_train_pred = mlp_model.predict(X_train)\n",
    "    y_test_pred = mlp_model.predict(X_test)\n",
    "    train_err.append(np.mean(y_train != y_train_pred))\n",
    "    test_err.append(np.mean(y_test != y_test_pred))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEaCAYAAAAG87ApAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl8lOW9///XJ2EJAURWlS3BXRQFidQFFSQiWhewVgU8rhXrUk/PaXuqtbXq+dJ6PL/aHqvWYvXYligipypWVAiCO2oQNxQEkSWissi+Ccnn98c9kSHMJDPJ3Jkl7+fjcT8y9zLXfK5MMp+5r+u+r8vcHRERkUTlpTsAERHJLkocIiKSFCUOERFJihKHiIgkRYlDRESSosQhIiJJUeIQyUBm9oiZ/b9UHyuSCkocktHMbKmZbTOzzVHLvemOS6Q5a5HuAEQScI67l9d3kJm1cPdd9W1LtgwR2ZPOOCRrmdnlZvaamf3ezL4GbouzLc/Mfmlmy8xslZn9zcw6RMooNjM3s6vMbDnwYozX+djMzo5ab2Fma8zsWDMrMLOJZrbWzNab2dtmtl+C8T9hZl+a2QYze9nMjoxz3BAzqzSzX0Red6mZja11WEcze9bMNpnZm2Z2UNTz/8fMVpjZRjOba2YnJxKfSDxKHJLtvgMsAboB4+NsuzyyDAUOBNoBtZu7TgWOAM6I8RqPAaOj1s8A1rj7O8BlQAegF9AZ+CGwLcHYnwMOicT5DlBWx7H7A12AHpHXnGBmh0XtHw3cDnQEFrP7dwHwNtAf6AQ8CjxhZgUJxiiyFyUOyQZPRb7N1yxXR+1b6e5/dPdd7r4tzraxwN3uvsTdNwM3AxebWXRT7W3uviWqjGiPAueaWWFkfUxkG8BOgoRxsLtXuftcd9+YSKXc/WF33+TuO4DbgGNqzoTi+JW773D3l4BngQuj9v3D3d+KNLOVESSKmteZ6O5rI7+P3wGtgcMQaSAlDskGI91936jlwah9K2IcX3tbd2BZ1Poygv696CalWOUA4O6LgY+BcyLJ41x2J46/Ay8Ak8xspZndZWYt66uQmeWb2Z1m9qmZbQSWRnZ1ifOUde6+pVYduketfxn1eCvBWVXNa/0k0ty2wczWE5whxXsdkXopcUi2izW8c+1tK4GiqPXewC7gq3rKiVbTXHUe8FEkmeDuO939dnfvC5wInA1cmkDcYyJllRJ8kBdHtluc4zuaWdtadVhZ34tE+jN+TnB20tHd9wU21PE6IvVS4pDm4DHg38ysj5m1A34DPJ7k1VOTgOHAtew+28DMhppZPzPLBzYSNF1VJVBee2AHsBYojMRUn9vNrFUkGZwNPJHg6+wCVgMtzOxWYJ8EnicSlxKHZINnat3H8WSSz3+YoEnpZeAzYDvwo2QKcPcvgDcIzioej9q1PzCFIGl8DLwETAQwswfM7IE4Rf6NoLnpc+AjYE49IXwJrCM4yygDfujuCxII/QWCTvhPIq+3nTqa5UQSYZrISSSzmdkQYKK790x3LCKgMw4REUlSqInDzEaY2UIzW2xmN8XY/3szezeyfBK54kNERDJYaE1Vkc7CT4DTgUqCm5BGu/tHcY7/ETDA3a8MJSAREUmJMM84BgGLIzddfUNwVcp5dRw/muDqFxERyWBhJo4e7Hn1RmVk217MrAjoQ4xxgkREJLOEOTpurBuM4rWLXQxMcfeY17+b2ThgHEDbtm0HHn744amJUESkmZg7d+4ad++airLCTByVBAO/1ehJ/DtdLwauj1eQu08AJgCUlJR4RUVFqmIUEWkWzGxZ/UclJsymqreBQyJ367YiSA5Tax8UGeGzI8HNVSIikuFCSxyR4RxuILhz9WNgsrvPN7M7zOzcqENHA5NcdyKKiGSFUGcAdPdpwLRa226ttX5bmDGIiEhqaepYEWkWdu7cSWVlJdu3b093KKEqKCigZ8+etGxZ7+j+DZZ1iWPuXCguhvHjYWztyTNFROKorKykffv2FBcXY5abo8q7O2vXrqWyspI+ffqE9jpZN1bVQOYye1kx5VeUUVbXRJsiIlG2b99O586dczZpAJgZnTt3Dv2sKusSB0Axy7h35zje/FdlDhFJXC4njRpNUcesTBwAbdnKv6+9Jd1hiIgkZP369dx///0Neu4f/vAHtm7dmuKIGi5rEwdAb5anOwQRyVFlZUF/al5e8LOxTeO5lDiyrnM82tbOvWmX7iBEJOeUlcG4cVDzWb1sWbAODb8o56abbuLTTz+lf//+nH766XTr1o3JkyezY8cORo0axe23386WLVu48MILqayspKqqil/96ld89dVXrFy5kqFDh9KlSxdmzZqVmko2QtYmjl2tCmn3P+PTHYaIZKEf/xjefTf+/jlzYMeOPbdt3QpXXQUPPhj7Of37wx/+EL/MO++8kw8//JB3332X6dOnM2XKFN566y3cnXPPPZeXX36Z1atX0717d5599lkANmzYQIcOHbj77ruZNWsWXbp0SbKm4cjapqqPL/2trscVkVDUThr1bU/W9OnTmT59OgMGDODYY49lwYIFLFq0iH79+lFeXs7Pf/5zXnnlFTp06JCaF0yxrDvj8L5HwkfzmfuO0S/dwYhIVqrrzACCPo1lMYYELCqC2bMb//ruzs0338w111yz1765c+cybdo0br75ZoYPH86tt94ao4T0yrozDmtTwBcdj6Do3afYvDnd0YhILho/HgoL99xWWBhsb6j27duzadMmAM444wwefvhhNkc+xD7//HNWrVrFypUrKSws5JJLLuGnP/0p77zzzl7PzQRZlzgAdp09ipOrX2La39emOxQRyUFjx8KECcEZhlnwc8KExrWOd+7cmZNOOomjjjqKGTNmMGbMGE444QT69evHBRdcwKZNm/jggw8YNGgQ/fv3Z/z48fzyl78EYNy4cZx55pkMHTo0RTVsnNDmHA9LSUmJv3Xfn8g7fhC/6/cIP3n/snSHJCJZ4OOPP+aII45IdxhNIlZdzWyuu5ekovysPOPIG1TC+vY9OeiDp1i1Kt3RiIg0L1mZODCj6uyRDOcFnizLnJtiRESag+xMHEDnq0ZSyDaW/vmFdIciItKsZG3i4JRT2NamI4cvfIrPPkt3MCIizUf2Jo6WLak68xzO4Rken7gz3dGIiDQb2Zs4gHaXjKQT6/jkLy+TZReHiYhkraxOHJxxBjtbtuHY5U/y/vvpDkZEJL6Gjo571llnsX79+hAiarjsThyFhVSffgYjeYpHy3TKISIplOJx1eMljqqqqjqfN23aNPbdd99GvXaqZXfiAFpfOJKefM78v1ZQXZ3uaEQkJ9SMq75sGbjvHle9Eckjelj14447jqFDhzJmzBj69QtG3Rs5ciQDBw7kyCOPZMKECd8+r7i4mDVr1rB06VKOOOIIrr76ao488kiGDx/Otm3bGl3Vhsi6QQ73cs45VOflc+KqJ3n11eM45ZR0ByQiGS8N46pHD6s+e/Zsvvvd7/Lhhx/Sp08fAB5++GE6derEtm3bOO644/je975H586d9yhj0aJFPPbYYzz44INceOGF/N///R+XXHJJQlVOpaw/46BTJ/zkUznfnmr0DF0iIkD446oDgwYN+jZpANxzzz0cc8wxHH/88axYsYJFixbt9Zw+ffrQv39/AAYOHMjSpUtTFk8ysv+MA8j/3kgOf+lG5k1ayDd/PIxWrdIdkYhktHSPqw60bdv228ezZ8+mvLycN954g8LCQoYMGcL27dv3ek7r1q2/fZyfn5+2pqrsP+MAGDkSgNM2PskLupFcRBorhHHV6xoafcOGDXTs2JHCwkIWLFjAnDlzGvw6TSE3EkevXlQPLOGCFmquEpEUCGFc9ehh1X/2s5/tsW/EiBHs2rWLo48+ml/96lccf/zxja1BqEIdVt3MRgD/A+QDf3H3O2MccyFwG+DAe+4+pq4yS0pKvKKiYu8d48fDL3/JwQWVzFvVg/btU1ABEckZGlY9C4ZVN7N84D7gTKAvMNrM+tY65hDgZuAkdz8S+HGDX3DUKACGb3+ap59ucCkiIlKPMJuqBgGL3X2Ju38DTALOq3XM1cB97r4OwN0bPrvGEUfghx7K6AI1V4mIhCnMxNEDWBG1XhnZFu1Q4FAze83M5kSatvZiZuPMrMLMKlavXh371cywUaM48ZtZvD19nSZ4EhEJSZiJw2Jsq92h0gI4BBgCjAb+YmZ73Vvv7hPcvcTdS7p27Rr/FUeOJL96FyOqn2Xy5AbHLSI5Ktumym6IpqhjmImjEugVtd4TWBnjmKfdfae7fwYsJEgkDTNoEBxwAJd1eIpHH21wKSKSgwoKCli7dm1OJw93Z+3atRQUFIT6OmHeAPg2cIiZ9QE+By4Gal8x9RTBmcYjZtaFoOlqSYNfMS8PRo7k1L/8lXlvbGPJkjYceGCDSxORHNKzZ08qKyuJ29ydIwoKCujZs2eorxFa4nD3XWZ2A/ACweW4D7v7fDO7A6hw96mRfcPN7COgCviZu69t1AuPHEmrP/2J05nBY4+dyy23NLIiIpITWrZsuccQH9Jwod7HEYa493HU+OYb6NaNZwvO52edHmb+/OD+HRGR5iwr7uNIm1at4OyzOW3zVD75eBfvvZfugEREckvuJQ6AkSNps2Utp+a9qk5yEZEUy83EMWIEtG7NjUVP8dhjaIInEZEUys3E0a4dDB9O6aYnqax0Xnkl3QGJiOSO3EwcACNH0nbNck4smKfmKhGRFMrdxHHOOZCXx08PfoonngguthIRkcbL3cTRtSucfDKlm55k3Tp4/vl0ByQikhtyN3EAjBxJ+2UfclzHxWquEhFJkZxPHAA3Hf4UU6dCnFkbRUQkCbmdOIqLYcAASjc/ybZt8NRT6Q5IRCT75XbigKC56sM3KOn5pZqrRERSIPcTx6hRmDu3HPU0M2agCZ5ERBop9xPHUUfBQQcxbPNTVFWhCZ5ERBop9xOHWdBc9eZMTjxyg+YjFxFppNxPHACjRsHOndx09DTmzIElDZ8qSkSk2WseieP442G//ThtU3BZ1bHHBpMFFhejMxARkSQ1j8SRnw/nnkur8mm0se1s2ADusGwZjBun5CEikozmkTgARo2i5fbNDPWZe2zeuhVNLysikoTmkzhOO42NtGcke98FuHx5GuIREclSzSdxtG7N7MLvch5Pk0fVHrt6905TTCIiWaj5JA6g05Uj6cZqTuT1b7cVFsL48WkMSkQkyzSrxDF4/JlUWz7T7LtUkcfyvGJeuKyMsWPTHZmISPZoVomDZ54hz5z2vok8nF7Vyxj8V11WJSKSjOaVOG65Baqr99ymy6pERJLSvBJHvMundFmViEjCmlfiiHf5lC6rEhFJWPNKHOPHB5dRRdmRr8uqRESSEWriMLMRZrbQzBab2U0x9l9uZqvN7N3I8oMw42HsWJgwAbp1A2BD6278W7sJ+BhdViUikqjQEoeZ5QP3AWcCfYHRZtY3xqGPu3v/yPKXsOL51tixwfC4rVqxbMil/GnDWD7+OPRXFRHJGWGecQwCFrv7Enf/BpgEnBfi6yWubVs48UQOqwzGrSovT3M8IiJZJMzE0QNYEbVeGdlW2/fM7H0zm2JmvWIVZGbjzKzCzCpWr16dmuiGDaP1/Hkc12eNEoeISBLCTBwWY5vXWn8GKHb3o4Fy4K+xCnL3Ce5e4u4lXbt2TU10paUAXH3Qi8yeDTt3pqZYEZFcF2biqASizyB6AiujD3D3te6+I7L6IDAwxHj2VFIC++zDMJvJpk3w9ttN9soiIlktzMTxNnCImfUxs1bAxcDU6APM7ICo1XOBpuumbtEChg6laFE5ZurnEBFJVGiJw913ATcALxAkhMnuPt/M7jCzcyOH3Whm883sPeBG4PKw4olp2DDyly7hnKM+U+IQEUlQizALd/dpwLRa226NenwzcHOYMdQp0s9xac+ZXDzjB2zeDO3apS0aEZGs0LzuHK/t8MOhe3cGbytn1y54+eV0ByQikvmad+Iwg2HD6PbhTNq0rlZzlYhIApp34gAoLcXWrOFf+n+gxCEikgAljmHDALioczkffABffpnmeEREMpwSR48ecPjhDFwfnG68+GKa4xERyXBKHAClpezz7svs1/EbNVeJiNRDiQOCfo6tWxl39BzKy8FrD4wiIiLfUuIAGDIE8vI4r205K1bAokXpDkhEJHMpcQB06ADHHceRXwTtVGquEhGJT4mjRmkprd9/i6N6b1TiEBGpgxJHjdJSrKqKcYe9xIsvQlVVugMSEclMShw1TjgB2rThjPxyNmyAuXPTHZCISGZS4qjRujWcfDIHLtV0siIidVHiiFZaSosF8yk98gslDhGROJQ4okWGH7myaCavvQZbt6Y5HhGRDKTEEa1/f+jUiVN2zuSbb+DVV9MdkIhI5lHiiJaXB8OGccBH5bRq6WquEhGJQYmjttJS8j6v5KIBnyhxiIjEoMRRW6SfY8x+M5k3D9asSXM8IiIZRomjtgMPhOJiBm3UMOsiIrEocdRmBqWldHxvFh33qVJzlYhILfUmDjPLN7N/a4pgMsawYdj69Vw14B0lDhGRWupNHO5eBZzXBLFkjtNOA+D8fcr57DNYsiTN8YiIZJBEm6peM7N7zexkMzu2Zgk1snTq1g2OOYajV2uYdRGR2lokeNyJkZ93RG1z4LTUhpNBhg2j8L77OLjHNsrL2zBuXLoDEhHJDAklDncfGnYgGae0FLv7bsYd+Rp3ziylujq4P1BEpLlL6KPQzDqY2d1mVhFZfmdmHcIOLq1OPhlatuSsVuV8/TW8+266AxIRyQyJfod+GNgEXBhZNgL/W9+TzGyEmS00s8VmdlMdx11gZm5mJQnGE7527eCEEzh0ufo5RESiJZo4DnL3X7v7kshyO3BgXU8ws3zgPuBMoC8w2sz6xjiuPXAj8GZyoTeBYcNo+cE7nHj410ocIiIRiSaObWY2uGbFzE4CttXznEHA4kii+QaYROzLev8TuAvYnmAsTae0FNz5wUGzeOUV2J55EYqINLlEE8cPgfvMbKmZLQXuBa6p5zk9gBVR65WRbd8yswFAL3f/Z10Fmdm4mv6V1atXJxhyChx3HLRvz2nV5WzfDq+/3nQvLSKSqRK5czwPOMzdjwGOBo529wHu/n59T42xzWuV+3vgJ/XF4O4T3L3E3Uu6du1a3+Gp07IlnHoqvT6ZSYsW6ucQEYHE7hyvBm6IPN7o7hsTLLsS6BW13hNYGbXeHjgKmB05izkemJpRHeQQDLP+6SLO679MiUNEhMSbqmaY2U/NrJeZdapZ6nnO28AhZtbHzFoBFwNTa3a6+wZ37+Luxe5eDMwBznX3ioZUJDSlpQD8S/eZVFTAunVpjkdEJM0STRxXAtcDLwNzI0udH/DuvovgTOUF4GNgsrvPN7M7zOzchofcxPr2hf3358RtM3GHWbPSHZCISHrVe+d4pC/iEnd/LdnC3X0aMK3WtlvjHDsk2fKbhBkMG0aXGTNo19YpLzfOPz/dQYmIpE+ifRz/XxPEkrlKS7FVq7hs4IfMmJHuYERE0ivRpqrpZvY9M4t1pVTui0wn+/1OM1m8GJYuTW84IiLplGji+HdgMrDDzDaa2SYzS/TqquzXqxcceijHfh1cVjVzZprjERFJo0QTRwfgcuD/ufs+wJHA6WEFlZFKS2k3dzY999upy3JFpFlLNHHcR3CfxejI+iaCu8ebj9JSbMsWxh3zJjNnQnV1ugMSEUmPRBPHd9z9eiLjSbn7OqBVaFFloiFDIC+Pcwpnsno1fPBBugMSEUmPRBPHzshotw5gZl2B5vWdu2NHGDiQIz7XMOsi0rwlmjjuAZ4EupnZeOBV4DehRZWpSktpPW8Oxx66WYlDRJqthBKHu5cB/wH8FvgCGOnuT4QZWEYaNgx27eIHh77Myy/Djh3pDkhEpOklNOc4gLsvABaEGEvmO+kkKCjg9Lxytm49izlz4NRT0x2UiEjTSrSpSgAKCmDwYDrNDdqphgyB4mIoK0trVCIiTUqJI0nzOpfS6fMP6MZXACxbBuPGKXmISPOhxJGkW18Khh85jRe/3bZ1K9xyS7oiEhFpWkocSXruywF8TUdK2fOyquXL0xSQiEgTU+JIUs+ifF7ktEji+HYmXPLy4KWX0heXiEhTUeJI0vjx8ErLYRSxnIP4FAj6zLt0gaFDgyarnTvTHKSISIiUOJI0dixcesFWABZxCCvyi5lxRRmLF8MVV8BvfgODB8Onn6Y5UBGRkChxJKusjIFPB5MYGtCzahmD/zqOdk+X8dBDMHkyfPIJ9O8Pf/87uNddnIhItlHiSNYttwSXUUWLuqzq+9+H996DAQPg0kuDM5QNG9IQp4hISJQ4khXv8qlly+DZZ+Gbb+jdG2bNgv/8z+AMpH9/eP31pg1TRCQsShzJ6t079nYzOPts2G8/uOIK8mc8zy9/vpNXXw12nXwy3H477NrVtOGKiKSaEkeyxo+HwsI9txUWwv/+b3DGcd558OSTcOaZsP/+HP+XH/DB76ZzycW7uO22YJiSZcvSEbiISGoocSRr7FiYMAGKioJTiaKiYP2yy+Css+CRR+Crr2Dq1GB98mTann8Gf31hfz4Zeg0d35nJsUfvYtIkePW6MipbFFNteVS2KObV6zRuiYhkPvMsu+ynpKTEKyoq0h1G4rZvh+efDzo7pk6FLVv4umU3Knb24xRepYDdY7NvoZB5105g8P1j0xiwiOQiM5vr7iUpKUuJowlt2wbPPUf1pMfhiSfIY+/ffWV+ET13LW362EQkp6Uycaipqim1aQPnn0/e5MfjHtK9SoNeiUhmCzVxmNkIM1toZovN7KYY+39oZh+Y2btm9qqZ9Q0znkyyMj/21Vmr6coHHzRxMCIiSQgtcZhZPnAfcCbQFxgdIzE86u793L0/cBdwd1jxZJql48azhT2vzqrG6Moqphx9B2MuqmLhwjQFJyJShzDPOAYBi919ibt/A0wCzos+wN03Rq22hRiN/jlq8P1jmXftBCrzi6jGqMwv4s0f/IWd37+E2/k1V085gyFHfMXll8OSJemOVkRktzATRw9gRdR6ZWTbHszsejP7lOCM48YQ48k4g+8fS89dS8nzanruWsoJD15J68f/Bg89xJBWr7GwsD9fPDqLww6Da66BFSvqL1NEJGxhJg6LsW2vMwp3v8/dDwJ+DvwyZkFm48yswswqVq9eneIwM4wZXHkl9tZb7NNrX56vKuXJY+/grw9XcfDBcOON8MUX6Q5SRJqzMBNHJdArar0nsLKO4ycBI2PtcPcJ7l7i7iVdu3ZNYYgZrF8/ePttbMwYzn7r16z7zhlcf8FX3H8/HHQQ/OxnUJNDy8qguDiYTKq4WPOfi0i4wkwcbwOHmFkfM2sFXAxMjT7AzA6JWv0usCjEeLJPu3bwt6Dpqs3c17j7xf4se2QWF1wAd98NBx4II0fC1VcHw5i4Bz/HjVPyEJHwhJY43H0XcAPwAvAxMNnd55vZHWZ2buSwG8xsvpm9C/w7cFlY8WStSNMVb70F++5Lj8tK+dvBd/Dhe1WcdRY8/XRwX2G0qFHeRURSTneOZ5PNm+Haa2HiRBg2DMrKyDtgv5iTRZlBdXXThyiS1crKgm9dy5cHI2GPHx+MT5cDdOd4cxXVdMVrr0H//lzUbRajKeMziqkij88oZjRldO+e7mAlFOrQCk9ZWdDOq3bf+rl7Vi0DBw50cff333c//HCvAt9BS/fgT90dfDOFfkneRP/Nb9x37Eh3oJIyEye6Fxbu8V57YWGwPZtNnOheVORuFvxsSH0aUsaOHe6LFrm/8IL7Aw+477PPnr/bmqVHD/fq6uRjaqxU/F6iABWeos/htCeCZBcljiibNrm3bRvzj/2rNkUO7ocf7j5zZroDlZQoKor9wda7d/JlpfhDqcFSkQzrKmPlSvdXX3X/+9/db7/d/fLL3U891b1XL/e8vNi/z1jLAQe4jxrl/l//5T57tvvmzaH9SuqtUwOlMnGojyPb5eUFf1YxrBh+Jf879xjK1/bn8O8fzR337Mv++8cpJ4fbdrOaO3zyCUyfHtzEE0/v3nDAAcHSvfvux9FL166Qn7+7SWbr1t3PLywM5pVJ5j1P9m9my5bgJqTo5dZbYePGvY8tKIDS0iDemiUvb8/1mm2PPRb0/yWie3fo0ye4JLFPnz0fDx4ce2rojh2DuXXmzIFPPw225efDUUfB8ccHy3e+A4cdFsST7O+mqiq4tj769/KTn8CGDXsf26lT0Fzds2ewdOoUdGjWJRJLybJlVLjXc3BilDiyXXFx7CkFCwqgffvdN3sAy6yIqqOOofi8/uQNOAaOOSb4h3nssdR8kIASUCqsWQMzZ8KMGUHCqBkyoEWL2HMP77NPMPNk9AfP11/vfVx+PnTrFpS/c+fe+/fdF37962AU54KCYIn3+Lnn4Kab9rykr3VruOKK4IO4doL44gvYtCm538OAAcGHalVVcKVHzePa619+Gb+Me+/dnRiKioI6xJNIQl29OrjCcc4cePPNYKlJfB06BAmksDD4/ezYPdcOrVvD6NHQq9fev5dVq4J6NERBwe4k0rMn9Oix5/pbbwU3fW3dSgkocUhEXX/sY8YE/1Tvvsuame/xwcT32O+r9ziMheQTueSqfXv45ps9/8hrdO8Ob7yx54dGixbxv+Gk6ptsLqoroe7YEfyep08PksXcucGZRocOwdVzw4fD6acHxyT6+92xI3jvY32AP/RQ+PUtLIx91lN7GTgw9rf8oiJYujSx14r35SmZMmok+8WnuhoWLAgSyJw5wfL++/GPNwvO/Oo6MzzgADjllNhjDPXoAVOmQGVlsHz++e7HNeuxvhRAShNH2vsskl3UxxFDgu3V1dXujz/ufuD+W/w43vK/n/qgb7v6hsTbeSFoF27b1r1z56DT8OCD3Y86yr2kxL1169jPKSpqyt9G6jW2PyBWe3VBgfvYse5nnbV7X4sW7oMHu99xh/sbb7jv3Jn6WNzj95X06uW+bl3QL7BkiftHH7nPnev+2mtBR9k//+k+ZUrwmvH+Pszc169PvDM57D6OdDCL/7uJ9Z7G0tA6VVW5f/ll8L49/fQezx+QYSVnAAAOvElEQVQI7urjkIbauDFokbjnHujcGZZUF9NubYxvbF26wH/9V9AcsX17sMR6XPPzuefiv+jvfx98cz7iiPrbZDNJomdRVVVBO/vGjXsvP/xh7KYjgEMPDc4mhg+HIUOCZqewpeLMMJ3f8sMqI1VS9btJRZ2iYtEZh6TEvHnuJ5zgPpqJvpk9v91sptBfuTbJb2zxvsm2aLH7cY8ewZUtjz7qvmpVKPX6VkO+nW/e7P7pp8E3/qefdu/UKXadWrUKLlnr3t29Xbvkztqiv4GmSxhnUblwaXAqZNLvJioWnXHojCNlqquDJtczvi7jN9xCb5aznN78gvG8XjQ2uS+PdX2TPemkoP1+xgwoL4d164L9Awbs/sZ90klBP0qqvoHWjqV1a7jqquBb/ldfBZ2SNUvNevTx9bnwwuAMofbSvv2e6yNGBG3PtTXk23kmyaRv+Zkmk343uqpKiSMM8a7obdCwJYn8w1RVBR3ANYnktdeCq4XatIGDDw46G6M7+Nq0gbvugtNOCxLOunWwfn3dj+fPr/tKlRYtgiuMapb99ou9ft55jf/Q10UDkgFSOeSIEofEbZLdf/8mmvtj82Z46aXgqqL77499yWld2rULrrXfd9/gZ8eOweiPsZgFl6N27JhYX0u67nkQSTElDiWOlIr12WgWLPfeG/TtNll/dh03NDJp0p7JYd99g6Vly72PzbTOW5E0S2XiaJGKQiS71XwGRn82/uIXwZf2664L7iG6//66751Kmd6943/gX3RR4uWMHx/7TGH8+ORjGjtWiUIkikbHFSD4XFy6NOjTWLo0+Mx95pngst1HHglGY2iSftzx44MP+GgN+cAfOzZoTioqCk6XiorUpyCSImqqknr9859wySXBiBWPPRZcABUqNQ2JpJzm45AmdfbZUFERjJAwYgT89rfxuyFSovbpj5KGSEZR4pCEHHxwMAzPxRcH/R/nnx97UFMRyX1KHJKwtm2DVqTf/z7o/xg0CD76KN1RiUhTU+KQpJjBj38cjPq9bl2QPKZMSXdUItKUlDikQU49Fd55B/r1g+9/H/7jP5K/b09EspMShzRYjx7BDd/XXQf//d9wxhnwwAPBvXd5ecHPsrJ0RykiqaYbAKVRWrWC++6D446DH/wAZs3afcXVsmXB/SCgC6NEconOOCQlLr88GGW39mW6W7cGt2SISO5Q4pCU+eqr2NtjzQwqItlLiUNSpnfv5LaLSHZS4pCUiTXMFMC11zZ9LCISnlATh5mNMLOFZrbYzG6Ksf/fzewjM3vfzGaaWVGY8Ui4ao8r2L17MAHeH/+o5iqRXBJa4jCzfOA+4EygLzDazPrWOmweUOLuRwNTgLvCikeaRvQwU59/Dq+8EszTNHw4rF6d7uhEJBXCPOMYBCx29yXu/g0wCTgv+gB3n+XuNRMmzAF6hhiPpMHRRwej6y5bBmedBZs2pTsiEWmsMBNHD2BF1HplZFs8VwHPhRiPpMngwTB5MsybFwyOuGNHuiMSkcYIM3HEmmw05mDcZnYJUAL8d5z948yswswqVqu9Iyudcw489BCUl8Oll0JVVbojEpGGCvPO8UqgV9R6T2Bl7YPMrBS4BTjV3WN+F3X3CcAECCZySn2o0hQuuwzWrIGf/hS6dAnmM2+yucxFJGXCTBxvA4eYWR/gc+BiYEz0AWY2APgzMMLdV4UYi2SIn/wEVq2Cu+6Cbt2CqWlFJLuEljjcfZeZ3QC8AOQDD7v7fDO7A6hw96kETVPtgCcs+Oq53N3PDSsmyQx33hmcedx2WzBMyXXXpTsiEUlGqIMcuvs0YFqtbbdGPS4N8/UlM5nBn/8cJI8bboDOneGii9IdlYgkSneOS1q0aAGTJsHJJ8O//AvMmJHuiEQkUUockjZt2sDUqdC3L4waBW+9le6IRCQRShySVh06wPPPw377BTcILliQ7ohEpD5KHJJ2++8P06cHzVfDh8OKFfU/R0TSR4lDMsJBBwVnHhs2wPHHQ69emn5WJFMpcUjG6N8ffvQjWLkSKiuD2QRrpp9V8hDJHEocklEmTtx7m6afFcksShySUeLN26H5PEQyhxKHZJR408y2agXvv9+0sYhIbEocklFiTT/bqhW0bAnHHgv/+q9BB7qIpI8Sh2SU2tPPFhXBww/v7iT/4x/hsMPgb38LOs9FpOmZZ9l/X0lJiVdUVKQ7DEmTuXPh+uvhzTeDCaLuvReOOSbdUYlkPjOb6+4lqShLZxySVQYOhNdfDyaFWrBgd/PV+vXpjkyk+VDikKyTlwdXXgkLF8I116j5SqSpKXFI1urUCe6/H95+G/r0CWYYPPlkeO+9dEcmktuUOCTrRTdfLVwYNF/deCM8+GAwZImGLhFJrVAnchJpKjXNVyNHwq9+FTRfRau5KguCK7dEpOF0xiE5pVMnuO++YMTd2rZuhV/8ouljEsk1ShySk776Kvb25cuDs5I//1nDmIg0lJqqJCf17h00T9XWrh3MmwdPPx2s9+0LI0bAmWcGHeutWzdtnCLZSGcckpNiDV1SWAgPPABLl8JHH8Hvfgc9egQ3EZ5+etDMdc45wZVan322+3llZY3vZE9FGZlWjmIJt5xUxxJcRpIi7p5Vy8CBA10kERMnuhcVuZsFPydOjH3c5s3uzzzjfv317gce6B7cDeJ+6KHuZ5zh3rr17m3gXlgYv6x4cRQWNq6MTCtHsWRjnQa6p+hzWEOOiERxh0WLgtkIn3su+BlLixZw8MHBz9pLy5Z7rpeXw7Zte5fRti2cf37isf3jH7BlS2aUo1jCLSecWEpwr7DEnx2fEodIHfLy4t+NfuGFsGtX7GXnzt2P582LX36fPonHEt18lu5yFEu45YQTS+oShzrHReoQr5O9qAgefzyxMoqL45exZEnisWRSOYol3HLCjqWx1DkuUod4nezjxzdtGZlWjmIJt5wwY0mJVHWWNNWiznFpaol2soddRqaVo1jCLSfVsWRN57iZjQD+B8gH/uLud9bafwrwB+Bo4GJ3n1JfmerjEBFJXlbMx2Fm+cB9wJlAX2C0mfWtddhy4HLg0bDiEBGR1Aqzc3wQsNjdlwCY2STgPOCjmgPcfWlkX3WIcYiISAqF2TneA1gRtV4Z2ZY0MxtnZhVmVrF69eqUBCciIg0TZuKIdb1wgzpU3H2Cu5e4e0nXrl0bGZaIiDRGmImjEugVtd4TWBni64mISBMIs4/jbeAQM+sDfA5cDIxpbKFz587dbGYLI6sdgA1RP6m1vfbjWPvr2pbIvoYcV5cuwBrqrkMu16+uutYVQybUr6ZutcvKhPql8r2rXV5j65cJ7x3Erl8u/e8d1sjn75aq63pjLcBZwCfAp8AtkW13AOdGHh9HcGayBVgLzE+gzIqoxxOif9beHmtfMtsS2deQ4xKpX111yOX61VXXumLIhPrF+tvMlPql8r1Ldf0y4b2LV79c/N9LxRLqkCPuPg2YVmvbrVGP3yZowmqoZ2r9rL091r5ktiWyryHHJVtWc6pfXXWtKwbVL/F4UiGV9cu09y66vFz832u0rBvk0MwqPEU3sWQi1S975XLdQPXLdqmsXzaOVTUh3QGETPXLXrlcN1D9sl3K6pd1ZxwiIpJe2XjGISIiaaTEISIiSVHiEBGRpGR94jCzkWb2oJk9bWbD0x1PqpnZgWb2kJnVO+R8tjGztmb218j7Nzbd8YQpl99HaBb/h0eY2QNmNsXMrk13PGGI/D/ONbOz6zs2IxOHmT1sZqvM7MNa20eY2UIzW2xmNwG4+1PufjXB8OwXpSHcpCVZvyXuflV6Ik1eMnUDzgemRN6/c5s82AQlWaeYMvl9TFH9Mvb/MEX1+9jdfwhcCGTUJbupqF/Ez4HJCb1oqu4kTOUCnAIcC3wYtS2f4A70A4FWwHtA36j9vwOOTXfsIdZvSrrjTnXdgJuB/pFjHk137CmqUz/gn7WWbpn8Pqa4fhn3f5iq+hF8uXkdGJPuOqW6fkApwbBQlwNn1/eaod453lDu/rKZFdfaHHN+DzP7GLgTeM7d32nSQBsomfoRNX9JNkiybpUEIwe8S4ae/UJydXL33wL1nupnklTUz8yMDP0/TNX75+5Tgalm9iwZNPlcit6/oUBbguSyzcymuXvceZIy9p81hnjze/yIIFteYGY/TEdgKRKzfmbW2cweAAaY2c3pCa3R4r13/wC+Z2Z/IsOGVEhAUvPNZOH7mOx8Otn2f5js+zfEzO4xsz9TaxilDJVU/dz9Fnf/MUFCfLCupAHhjo6bajHn93D3e4B7mjqYEMSr31ogG/4R6xKvbluAK5o6mBRJar6ZLHwfk61ftv0fJlu/2cDssIIJQYPmQ3L3RxIpPJvOOHJ9fo9crl8u1i0X6xRN9ctuodYvmxLHt/N7mFkrgo6cqWmOKZVyuX65WLdcrFM01S+7hVu/dF8REOcqgceAL4CdBJnzqsj2veb3yMYll+uXi3XLxTqpfqpfYxYNcigiIknJpqYqERHJAEocIiKSFCUOERFJihKHiIgkRYlDRESSosQhIiJJUeIQSYKZLTWzLo09RiSbKXGIiEhSlDhE4jCzpyIzos03s3G19hWb2QILZjB8PzIzXGHUIT8ys3fM7AMzOzzynEFm9rqZzYv8PKxJKySSIkocIvFd6e4DCWZ8u9HMOtfafxgwwd2PBjYC10XtW+PuxwJ/An4a2bYAOMXdBwC3Ar8JNXqRkChxiMR3o5m9B8whGGn0kFr7V7j7a5HHE4HBUfv+Efk5FyiOPO4APBGZ4vP3wJFhBC0SNiUOkRjMbAjBxEQnuPsxwDygoNZhtQd6i17fEflZxe55b/4TmOXuRwHnxChPJCsocYjE1gFY5+5bI30Ux8c4preZnRB5PBp4NYEyP488vjwlUYqkgRKHSGzPAy3M7H2CM4U5MY75GLgsckwngv6MutwF/NbMXgPyUxmsSFPSsOoiDWBmxcA/I81OIs2KzjhERCQpOuMQEZGk6IxDRESSosQhIiJJUeIQEZGkKHGIiEhSlDhERCQpShwiIpKU/x9S7+nxYSgSKgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.semilogx(alpha_arr, train_err, 'b-o', label = 'test')\n",
    "plt.semilogx(alpha_arr, test_err, 'r-o', label = 'train')\n",
    "plt.xlim([np.max(alpha_arr), np.min(alpha_arr)])\n",
    "plt.title('Error vs. alpha')\n",
    "plt.xlabel('alpha')\n",
    "plt.ylabel('error')\n",
    "plt.legend()\n",
    "pass"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Минимальное значение ошибки"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.12042160737812911 0.25240641711229944\n"
     ]
    }
   ],
   "source": [
    "min_train_err = np.min(train_err)\n",
    "min_test_err = np.min(test_err)\n",
    "print(min_train_err, min_test_err)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.61584821]\n"
     ]
    }
   ],
   "source": [
    "alpha_opt = alpha_arr[test_err == min_test_err]\n",
    "print(alpha_opt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {},
   "outputs": [],
   "source": [
    "alpha_opt = alpha_opt[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Повторим обучение"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.16469038208168643\n",
      "0.25240641711229944\n"
     ]
    }
   ],
   "source": [
    "mlp_model = MLPClassifier(alpha = alpha_opt, hidden_layer_sizes = (100,),\n",
    "                          solver = 'lbfgs', activation = 'logistic', random_state = 42)\n",
    "mlp_model.fit(X_train, y_train)\n",
    "\n",
    "y_train_pred = mlp_model.predict(X_train)\n",
    "y_test_pred = mlp_model.predict(X_test)\n",
    "print(np.mean(y_train != y_train_pred))\n",
    "print(np.mean(y_test != y_test_pred))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Матрица ошибок(рассогласования)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 1  0  0 ...  0  0  0]\n",
      " [ 0  0  0 ...  0  0  0]\n",
      " [ 0  0 94 ...  0  0  0]\n",
      " ...\n",
      " [ 0  0  0 ... 44  0  5]\n",
      " [ 0  0  0 ...  0  0  0]\n",
      " [ 0  0  0 ...  0  0  0]]\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import confusion_matrix\n",
    "print(confusion_matrix(y_test_pred, y_test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAATgAAAEKCAYAAACGzUnMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3X+0HGWd5/H3JwQkIL+DDpBkgBxW40oQvAuoExfB2WWVFfWgC4OKiDLuqoCo/HAZZREddDwoqyNuFDXMZFUG8cCMLogIa5wzBG8CBjW6GmAgiJC7IiAbxTt+94+qujSXTt+q7qruqurP65x70l3d/dTT1Z1vf596fpQiAjOzNpo36gqYmVXFAc7MWssBzsxaywHOzFrLAc7MWssBzsxaywHOzFprJAFO0rGSfirp55LOG0UdzKz9NOyBvpK2A/4P8KfAZuD7wEkR8eOhVsTMWm/+CPZ5OPDziLgLQNJXgOOBbQa4vfbaKxYvXsz8+aOortXB9PT0zG1/D8p3zz33MDU1pUHKkFQkW7ohIo4dZH95jOKbsh9wX8f9zcARvV6wePFivv3tb7Nw4cJKK2b1NTU1NXPb34PyTUxMDHuXQ/kQRxHguv1KPC3ySzodOB1gyZIl/lKPOX/+zSDlSwKHdWpsFJ0Mm4HFHfcXAb+Y/aSIWBkRExExsffeew+tcmbWv3nz5uX6G1p9hranJ30fOEjSAZJ2AE4ErhtBPcysZJJy/eUo5wuSHpL0w45tfyXpJ5I2SPq6pN3nKmfoTdSImJb0TuAGYDvgCxHxo6LlHHnkkQCsXr16ZtvSpUsBuO++5BTf4sVPJorZOZyiTZ1hnfspup9Rn5PasGEDAMuXLx/6vjObNm2auZ199t30+9mXse9xkjd45fQl4NPAlR3bbgTOT2PIR4HzgXN7FTKS7qiI+CbwzVHs28yqU1aAi4jvStp/1rZvddy9FThhrnIa299+6623bvOxzswt0++v97Cyo6L7GfVJ91Fmbpm8mVMVx8pZW3clZnBzeQvw1bme1NgAZ2b1UyDALZQ02XF/ZUSszLmP/wpMA6vneq4DnJmVQlKRHtKpiCg8+E7SKcBxwDGRY6yJA5yZlabKJqqkY0k6Ff5tRPy/PK/xaiJmVpoSh4l8Gfgn4DmSNks6jaRXdRfgRkl3SPrsXOU0NoO77bbbADj88MMLPTbq4RV5dBvmUjdNqGOV6vY9yj4PGO1nUmIv6kldNl9RtJzGBjgzq58h9qLm0tgA1y07y/NYHX5t59KErKgJdaxS3b5Hdfg8CnYyDEVjA5yZ1Y8zODNrLQe4kmzcuBGAZcuWPe2xbnMPTzopOWd5/vnnz2yrYjT+KOYoZvusy+j6bse/ivmgeU+sl7HvIp0q2XcTun8/+9XrfeTt9Kjic+jkAGdmrVTyZPtSDP2aDP2YmJiIycnJuZ9oZn2ZmJhgcnJyoOi0/fbbxx577JHruVu2bFnXz0yGopzBmVlp3ItqZq1VtybqWAe4a665BoDXvva1I66JWaLqToAq1fEc3FgHODMrlwNcjThzs7ppYubWyQHOzFrLnQwlqWI1iwsuuGDm9llnnQU8ddDmihUrutahUx3mBHZT9uoXTT5XVIbO45kZ12OR8Tk4M2s1Bzgzay0HuJJU0RS8+OKLn7ZtdrO06jpUpezm07g3x8b9/W+LA5yZtZYDXI11WwUiW/4cei+k2bYlvOu2JPcgxr1DZFi84KWZtZozODNrLQe4Guu2OGFns7RXMzTbNoqrG1XRBCu7OTfKRTnHoWna65TCME83OMCZWSt5oG/D5cnIRtHJ0IQMpS7LqbdVr+/AML8fDnBm1lruRa2xoufP3ve+983cPvfcc4FmZFNlqMPQi27nlvq94E0ThsWUcdGZqtUtg6tXuDWzxsrOweX5y1HWFyQ9JOmHHdv2lHSjpJ+l/855AQgHODMrTVkBDvgScOysbecBN0XEQcBN6f2e3ETtULSD4Iwzzpi5vXXr1rKrU2v9NoPKbNp2KyPvtn6eM2p16UjopawmakR8V9L+szYfDxyV3l4F3AKc26scBzgzK03FnQzPjogHACLiAUnPmusFYx3gBp0/2u11Bx988MztO++8s7+KFdRvVjSKRRvrkmlkqugs2bBhw8zt5cuXl1ZuGdavXw/AYYcdNrNtamqK6enpgcsuOA5uoaTOix2vjIiVA1dilrEOcGZWrgIBbqqPCz8/KGmfNHvbB3horhe4k8HMSlNiJ0M31wGnpLdPAa6d6wWVZXCSFgNXAn8E/IEkBb1M0p7AV4H9gXuA10fEw1XVo5cqZh0Mq1naqd/mVd2ai6NQxTGoW7O0U2fTNLNw4ULmzy8nFJTVySDpyyQdCgslbQY+CFwCXCXpNOBe4HVzlVNlE3UaeE9ErJe0C7BO0o3Am0m6ei+RdB5JV2/PnhAza4YSe1FP2sZDxxQpp7ImakQ8EBHr09uPARuB/Ui6elelT1sFvHqusqanp7ueEDez+sgWvMzzNyxD6WRIx7McCqwlZ1evpNOB0wEWLVo0jGqa2YDqNlWr8gAn6ZnA14CzIuLRvAcg7TJeCTAxMRFtOF+UddEDLFmyBMh/DqgOcz/N5lK3AFdprihpe5Lgtjoirkk3P5h28ZK3q9fMmqHiXtTCKgtwSt7FFcDGiLi046HCXb1mVn9lTrYvS5VN1JcAbwTulHRHuu399NHV2xadXfSdSzPl4aapNUHdmqiVBbiI+B6wrXdbqKvXzJrBC17WyOwT98NcNDAbZNxtbmBb5D2ea9asAWDFihWFyq1LVlu3+ozS2GRwZjZefNGZmpn9izuKX+A2Zm6ZvMczy9zyZnx1y5TqVp9RcoAzs9ZygDOz1nIng80pz9WT2tgsauN7Gic+B2dmreYAZ3Nq0gl2s04OcGbWWg5wZtZaDnDWl0GvANZUbe5UaZtswcs6cYAzs9I4g7O+ZJlbds3NOl/YpEyjmCds/XOAM7PWcoCzgWSZ27hlNKN4j+N2jAflgb5m1moOcGbWWu5FHZG2LSzZ2WTaddddAXj00UdntmXNq0ceeQSApUuXPq2MQZpgTRq2cttttwFw+OGHF3pdv83STZs2zdzudtyr0OuzHFZTu1FNVEl79nphRPyq/OqYWZM1JsAB64Cg+3UVAjiwkhpVpC2ZWzdZ5tbtl7qqea1NyNwyRTO3QQ0ra+tUl/nLZQY4Se8G3koSb+4ETo2I3xYpY5sBLiIOGKx6ZjZuygpwkvYDzgCeFxFbJV0FnAh8qUg5c54RVOINkv4ivb9E0nB/Es2s9rKpWnn+cpoPLJA0H9gJ+EXROuXpZPgM8AfgaOBDwGMkV6v/N0V3ZtXyWC0btbIyuIi4X9LHSa6dvBX4VkR8q2g5eULpERHxDuC36Y4fBnYouiMza78CV7ZfKGmy4+/0WeXsARwPHADsC+ws6Q1F65Mng/u9pO1ITvQhaW+SjM5qIruu6LJly2a2ZdnctddeC8Dxxx8//IqNUDaMBYbfIdK0GRBTU1NMT0+XUlaBDG4qIiZ6PP5y4O6I2JKWew3wYuBvi9QnTwb334GvA8+W9GHge8BHiuzEzMZDgQxuLvcCR0raSckLjgE2Fq3PnBlcRKyWtC7dAcCrI6Lwjqw6va4IP26ZW2aUw1iakLV1WrhwIfPnDz7mv8yBvhGxVtLVwHpgGrgdWFm0nLzvaicga6YuKLoTMxsPZU7ViogPAh8cpIw8w0Q+AKwC9gQWAl+UdMEgOzWzdiqxiVqKPBncScCh2QhiSZeQpI0XV1kxM2ueJk3VytwD7Eg6TAR4BrBpm882s7HUtMn2nyI55/Y74EeSbkzv/ylJT2rtld1d37YLoLz//e+fuf2Rj4xnx3i28keZ80dHOURl1BoT4IDJ9N91JMNEMrdUVhsza7TGrAcXEauGWREza74mZXAASDoI+EvgeSTn4gCIiNovl1R2U7ItTdNMZ7O0bc3vvPI2TYscnwULxnMkVR3PweXJJ78IXE4y2O5lwJXA31RZKTNrpiYOE1kQETdJUkT8M3ChpDUMOADPRifLRrZu3TqzLTsZ3qSlyIepSGY7bllwp7plcHkC3G8lzQN+JumdwP3As6qtlpk1URMD3FkkU7XOIFkP7mjglLw7SFcimQTuj4jjJB0AfIVkZsR64I0R8UTRig9L01aGyKPX+3Dmls8oLixTd9mCl3UyZ20i4vsR8ZuI2BwRp0bEayPi1gL7OJOnrgLwUeATEXEQ8DBwWrEqm1ldNeYcnKS/J10DrpuIeNVchUtaBLwS+DBwdrrsydHAn6VPWQVcSNKJYWYN16Qm6sdLKP+TwDnALun9vYBfR0S2ut5mYL8S9lOZtjRLrVxulnbXmAAXEf97kIIlHQc8FBHrJB2Vbe62q228/nTgdIAlS5YMUhUzG5LGBLgSvAR4laRXkAwQ3pUko9td0vw0i1vENq6UExErSRe4m5iY2GZT2apXdBBwmwcNt/m9DaqpA337EhHnR8SiiNif5HqG34mIk4GbgRPSp50CXFtVHcxsuEq+bODg9Rnanp50LkmHw89JzsldMYI6mFkFxqoXteO5t5CuQhIRdwEDXzi61zI3VSyB06sOw9hXplsTqd9mU94xft0ey76kP/jBDwDYd999Zx679957n/a6DRs2ALB8+fJC++6laBll7DN7Xd3GR3arT7fvRdUzVerWRM3Ti/pa4I948nJdJ5EsgmlmNqOO5+Dm7EWV9KGIeGnHQ38v6buV12wOvTKmYWVToxgq0C1bGDQb6UdEktxnWWxnWd3KzTK3MvbdbxllZlp1yNo65f1eVD1TpTEBrsPekg5Mm5akU632rrZaZtZEdZuqlSfAvRu4RdJd6f39gT+vrEbWKB7wap0al8FFxPXpopfPTTf9JCJ+V221zKxpGnUOLiNpJ+Bs4I8j4m2SDpL0nIj4h+qrZ2ZN0rgAR7Ki7zrgRen9zcDfAY0KcB6BXq41a9YAsGLFiqc9NntISBPUbdhHUXWpf90CXJ4zgksj4mPA7wEiYivd55Sa2Zgrc6CvpN0lXS3pJ5I2SnrR3K96qjwZ3BOSFpAO+pW0lORaqY3SxF/lOuuWuWWalLllmv79qEP9K1jw8jLg+og4QdIOJAvvFpInwF0IXA8slrSaZBL9qUV3ZGbtV1YTVdKuwEuBNwOkq34XXvk7Ty/qtyStA44kaZqeGRFTc7zMzMZQiefgDgS2AF+UdAhJP8CZEfF4kULy9KLeFBHHAN/osq2WypyvaeU4++yzAbj00kuB+pwUr8LGjU+u0L9s2TKg+vfbq/xs/inUaibDQkmTHfdXpkukZeYDhwHvioi1ki4DzgP+okh9ek2235GkzbtQ0h482bGwK7Dvtl5nZuOrQICbioiJHo9vBjZHxNr0/tUkAa6QXhncn5NcUWtfkvQwq/mjwF8X3dEwlTlfsxtng8VlmVuW3WSZTRt1e29Vf1fqcKW0Mgf6RsQvJd2Xjrn9KXAM8OOi5fSabH8ZcJmkd0XEpwaoq5mNiZJ7Ud8FrE57UO+ij87NPL2of5C0e0T8GiBtrp4UEZ8purNRKjNzcOZWXJb1Zsc/GwwMzRxWUtQwz4ONUpkDfSPiDqBXM3ZOecLt27Lglu70YeBtg+zUzNqpMSv6dpgnSZEuAJZeqX6HaqtlZk3TyMn2wA3AVZI+SzKb4e0kA38bpc0ntZtgdrO+s1la9TLaddDm99apiQHuXJIe1f9M0pP6LeDzVVbKzJqpcQteRsQfgMvTP7PSjUt2Mw4ak8FJuioiXi/pTrpcXSsi2t/1ZWa5Ne0c3Jnpv8cNoyJm1nyNCXAR8UD67z8Przpm1mSNCXCSHqP3hZ93raRGZtZYjQlwEbELgKSLgF8Cf0PSi3oysMtQamdmjVHBgpcDyzNM5N9HxBEd9y+XtBb4WEV1MrOGqlsGlyfc/oukkyVtJ2mepJOBf6m6YmbWPHWbqpUnwP0Z8HrgwfTvdek2M7OnqFuAyzPQ9x7g+OqrYmZN17gmqqR/JekmST9M7y+XdEH1VTOzJsmbvdWtifo54HyevC7qBuDEKitlZs00b968XH/DkqcXdaeIuG1W1J2uqD65bdq0CYClS5c+7bF+lxQvemGQzucXeZ0NJjvur3nNa2a2rVmzBui+sGmv78rsMiHfZ9htAcui37t+V1HprOvWrVv7KqMqdWui5glwU+nFnrP14E4AHqi0VmbWSE0McO8AVgLPlXQ/cDfJYF8zsxlNm2yPpHnARES8XNLOwLyIeGw4Vettt9122+Zj/TYTi77OzdHRyI7717/+9ZltWXOv28KmvZqms8vMq1uTsIwy8qjz965uAa7n2b50Lbh3prcfr0twM7N6amInw42S3gt8FXg82xgRv6qsVjnU+VdsHOQ5cV+1Rx55ZOZ2Vo/sal1tvlJX0Q6RYWlcEzX1lvTfd3RsC+DA8qtjZk3WuAAXEQf0W7ik3Umu3/B8kqD4FuCnJNng/sA9wOvTSxFag4wyc+tVhzZnbpk6ZW2z1S3A5ZnJsKOksyVdI+lrks6StGPO8i8Dro+I5wKHABuB84CbIuIg4Kb0vpm1QBNnMlwJ/GvgU8CngeeRrA3Xk6RdgZcCVwBExBPpBaSPB1alT1sFvLp4tc2sjuoW4PKcg3tORBzScf9mST/I8boDgS3AFyUdAqwjuc7DszuWQ39A0rOKVrofTbr25rBGxA+i39kiZZaf92T729/+dgAuvvjiXM8vWv7s54+iCZnN4IDRXQO47AUv04vMTwL3R0Rf14bJU5vbJR3ZsdMjgH/M8br5wGHA5RFxKEkPbO7mqKTTJU1KmtyyZUvel5nZCJWcwZ1Jclqrb3kyuCOAN0m6N72/BNiYXU6wx+UDNwObI2Jtev9qkgD3oKR90uxtH+Chbi+OiJUkMyiYmJgIyP+L2u2XtAmZW6bbe+uVHYzivVWdpeQpP28dPvvZz1ZWh0GeX6a99957ZPvuVFbzU9Ii4JXAh4Gz+y0nT4A7tp+CI+KXku6T9JyI+ClwDPDj9O8U4JL032v7Kd/M6qfE82ufBM5hwOu/5BkmMshlA98FrJa0A3AXcCpJs/gqSacB95KsEGxmDVew+blQ0mTH/ZVpqw1JxwEPRcQ6SUcNUqc8GVzfIuIOYKLLQ8f0U17eJkCdxwn1q43vaVi6LaHUbVuT1eX7UaCTYSoiusUGgJcAr5L0CmBHYFdJfxsRbyhcn6IvMDPbljI6GSLi/IhYFBH7kyyu+51+ghtUnMGNo86FEBcsWADU59d1trrOaSxbtywt2/aBD3wAgIsuumho9RnlcJKqF2mt20wGBzgzK0UVg3gj4hbgln5f7wA3gG4ZUNOHo4ybLHO78sorZ7a96U1vAp5cMQWePu91kOx3lMe96n07gzOz1nKAM7PWGuZilnk4wA3ATbz2yJqlAOvXrwfgsMMO2+bz/dk/XVMXvDQzy8UBrkUGmRtrgytzmEu21Dk8mbllmVznNuvNAc7MWssBzsxaywFuRKpoJo7z3Ng6KPO4druWQ2eztA5XEau7she8LMPYBDgzq54zuBFxFmWDcOaWjwOcmbWWA1xJuq3nNezhGGUMU6jrxXA6V0XpVrc6DH0pc5hO0c/yYx/72Mztc845J/d+2swDfc2s1dzJYGat5QyuJN0WMRx206CM/dWtaZqZq151aIaVOUyn6PvJmqXgpmknBzgzayWfgyuRfzVHq23Hv9+r2Hc+v23HpB8OcGbWWg5wJSn6K5kNe8guBNNPGXn0Wua6Tfo9/tm5vbodpzKWHc+2HXHEETPbPve5zwFPnQo27KFBw7y4kHtRzayVfA7OzFrNAW5EhtUcqENzq45mH/82H6e1a9f2fHzYQ4OG2enhAGdmreUAZ2at5QBnZq1UxwUv61UbM2u0rCd1rr8c5SyWdLOkjZJ+JOnMfurjDG6MdY6PyozzKPyyZEt5Qfc5092W+uqlSTMkSmyiTgPviYj1knYB1km6MSJ+XKQQBzgzK01ZAS4iHgAeSG8/JmkjsB/gAGf5NCEjaKK5MrO8mVumKZ9TVQN9Je0PHAr0Hn/ThQOcmZWmQCfDQkmTHfdXRsTK2U+S9Ezga8BZEfFo0fo4wJlZaQpkcFMRMTFHWduTBLfVEXFNP/VxgDOz0pTVRFVS0BXAxoi4tN9yPEzEzEqRd4hIziD4EuCNwNGS7kj/XlG0Ts7ghmiYy9ZYPqMYgtFrn2vWrAFgxYoVQ6tPmUrsRf0eMHBhDnBmVpqxmqol6d3AW4EA7gROBfYBvgLsCawH3hgRT1RZj7pw1lY/o/hMeu2zqZlbZmymaknaDzgDmIiI5wPbAScCHwU+EREHAQ8Dp1VVBzMbnpLPwZWi6nA7H1ggaT6wE8nI5KOBq9PHVwGvrrgOZjYkYxPgIuJ+4OPAvSSB7RFgHfDriJhOn7aZZPqFmW3D1NRU13nDdTQ2AU7SHsDxwAHAvsDOwH/o8tTYxutPlzQpaXLLli1VVdPMSlS3AFdlJ8PLgbsjYguApGuAFwO7S5qfZnGLgF90e3E6bWMlwMTERNcgOGoe9mHD0KTrrtatF7XKc3D3AkdK2ikdlXwMyUoANwMnpM85Bbi2wjqY2ZBkC17m+RuWyjK4iFgr6WqSoSDTwO0kGdk3gK9IujjddkVVdahanX9JrX1mZ3Kd2+qibhlcpePgIuKDwAdnbb4LOLzK/ZrZaIxVgDOz8eIAV0NeuruZ6txUq1Lne61Tx4OvbG9mrVa3qVqNDXBFf7l6Pb/or19W1iOPPDKzrYortefNUEbxK75p0yagfleo73YsNmzYAMDy5ct7Pq+JsvrvscceM9smJ5OFcrPPJnv/APvuu+9TXlc2Z3Bm1loOcGbWSj4HV6KiKXaZKXlWVtXNm7zlj6KZVYemad7TDZ1N017Pa7KHH3545vbsTrNu778qDnBm1loOcGZWqVFlp9lUrTpxgDOz0jiDa7Ds3MbWrVsBWLBgwcxjbTunY/3LhtBAvnOV3YYDFR3EvHHjxpnby5Yte8pjN99888ztgw8+OHeZ/XCAM7PWcoAzs9ZygGuwPGl9r+bJIHMn+x15X9WI/dlDEeYq/7777gNg8eLFpdZjUL2OT7+f12677VaoDmXMrpndLO30spe9bOb2eeedB8All1wys23Dhg0zp10G4XFwZtZqdetFVUQtVwN/iomJicjm15lZ+SYmJpicnBwo/XrhC18Yt956a67n7rDDDusiYmKQ/eVRr3BrZo1W5kVnJB0r6aeSfi7pvH7q4wBnZqUo88LPkrYD/prkSnzPA06S9LyidXKAM7PSlJjBHQ78PCLuiogngK+QXIa0EHcymFlpSuxk2A+4r+P+ZuCIooU0IsCtW7duStLjQDMu793dQlz/UWly3WE49f/jQQtYt27dDZLyjm/ZUVJnz+HK9FrImW5pXuEe0UYEuIjYW9LkMHpdquL6j06T6w7NqX9EHFticZuBzkGT27xIfC8+B2dmdfR94CBJB0jaATgRuK5oIY3I4MxsvETEtKR3AjcA2wFfiIgfFS2nSQFu5dxPqTXXf3SaXHdofv37EhHfBL45SBmNmMlgZtYPn4Mzs9ZqRIArY8rGsEhaLOlmSRsl/UjSmen2PSXdKOln6b97zFXWKEnaTtLtkv4hvX+ApLVp/b+anvitJUm7S7pa0k/Sz+FFTTr+kt6dfnd+KOnLknZs0vGvk9oHuLKmbAzRNPCeiFgGHAm8I63vecBNEXEQcFN6v87OBDZ23P8o8Im0/g8Dp42kVvlcBlwfEc8FDiF5H404/pL2A84AJiLi+SQn2E+kWce/Nmof4ChpysawRMQDEbE+vf0YyX+u/UjqvCp92irg1aOp4dwkLQJeCXw+vS/gaODq9Cm1rb+kXYGXAlcARMQTEfFrGnT8STr/FkiaD+wEPEBDjn/dNCHAdZuysd+I6lKIpP2BQ4G1wLMj4gFIgiDwrNHVbE6fBM4B/pDe3wv4dURMp/fr/BkcCGwBvpg2sT8vaWcacvwj4n7g48C9JIHtEWAdzTn+tdKEAFfKlI1hk/RM4GvAWRHx6Kjrk5ek44CHImJd5+YuT63rZzAfOAy4PCIOBR6nps3RbtJzg8cDBwD7AjuTnJ6Zra7Hv1aaEOBKmbIxTJK2JwluqyPimnTzg5L2SR/fB3hoVPWbw0uAV0m6h+R0wNEkGd3uaZMJ6v0ZbAY2R8Ta9P7VJAGvKcf/5cDdEbElIn4PXAO8mOYc/1ppQoArZcrGsKTnq64ANkbEpR0PXQeckt4+Bbh22HXLIyLOj4hFEbE/ybH+TkScDNwMnJA+rc71/yVwn6TnpJuOAX5MQ44/SdP0SEk7pd+lrP6NOP5104iBvpJeQZJFZFM2PjziKm2TpD8B1gB38uQ5rPeTnIe7ClhC8iV+XUT8aiSVzEnSUcB7I+I4SQeSZHR7ArcDb4iI342yftsi6QUkHSQ7AHcBp5L8mDfi+Ev6b8B/IumRvx14K8k5t0Yc/zppRIAzM+tHE5qoZmZ9cYAzs9ZygDOz1nKAM7PWcoAzs9ZygBtT6Yob/6XC8t8s6dNzPOdCSe8tWO5vBquZjRMHuPG1O9A1wKUruJg1ngPc+LoEWCrpDkl/JemodB27/wncKWl/ST/MnizpvZIuTG8vlXS9pHWS1kh6bq8dSfqP6Vpmt0v6tqRndzx8iKTvpOucva3jNe+T9H1JG9KBr2aFNemaDFau84DnR8QLYGbWwuHptrvTlVC2ZSXw9oj4maQjgM+QzFndlu8BR0ZESHoryUol70kfW06ybt7OwO2SvgE8HzgorY+A6yS9NCK+29c7tbHlAGedbouIu3s9IV0l5cXA3yVTJQF4xhzlLgK+mk5y3wHo3Me1EbEV2CrpZpKg9ifAvyOZkgTwTJKA5wBnhTjAWafHO25P89RTGDum/84jWZvsBQXK/RRwaURcl2aKF3Y8NnuuYJBkbX8ZEf+jwD7Mnsbn4MbXY8AuPR5/EHiWpL0kPQM4DiBd2+5uSa+DZPUUSYfMsa/dgPvT26fMeuz49JoDewFHkawecwPwljRbRNJ+kmq5QKXVmzO4MRUR/1fSP6YdCf8L+Masx38v6SKSVVDuBn7S8fDJwOWSLgC2J1mQ42SeAAAAYUlEQVTl4gc9dnchSZP2fuBWksUcM7el+14CfCgifgH8QtIy4J/SZvBvgDdQ3zXcrKa8moiZtZabqGbWWg5wZtZaDnBm1loOcGbWWg5wZtZaDnBm1loOcGbWWg5wZtZa/x/bGxpiRWmQAgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "M = confusion_matrix(y_test_pred, y_test)\n",
    "M = np.sqrt(M)\n",
    "plt.imshow(M, interpolation = 'nearest')\n",
    "plt.set_cmap('binary')\n",
    "plt.xlabel(\"true label\")\n",
    "plt.ylabel(\"predicted label\")\n",
    "plt.colorbar()\n",
    "pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "             precision    recall  f1-score   support\n",
      "\n",
      "        A#d       0.50      0.50      0.50         2\n",
      "       A#d7       0.00      0.00      0.00         1\n",
      "        A_M       0.82      0.78      0.80       120\n",
      "       A_M4       0.00      0.00      0.00         3\n",
      "       A_M6       0.00      0.00      0.00         1\n",
      "       A_M7       0.32      0.44      0.37        16\n",
      "        A_d       0.00      0.00      0.00         1\n",
      "        A_m       0.69      0.78      0.73        85\n",
      "       A_m7       0.00      0.00      0.00         7\n",
      "        AbM       0.94      0.84      0.89        19\n",
      "        Abm       0.00      0.00      0.00         1\n",
      "        B_M       0.67      0.82      0.74        50\n",
      "       B_M4       0.00      0.00      0.00         1\n",
      "       B_M7       0.47      0.39      0.42        18\n",
      "        B_d       0.20      0.50      0.29         4\n",
      "       B_d7       0.00      0.00      0.00         4\n",
      "        B_m       0.82      0.74      0.78        62\n",
      "       B_m7       0.40      0.25      0.31         8\n",
      "        BbM       0.85      0.89      0.87       118\n",
      "       BbM7       0.00      0.00      0.00         1\n",
      "        Bbd       0.00      0.00      0.00         1\n",
      "        Bbm       0.75      0.75      0.75         8\n",
      "       Bbm6       1.00      0.67      0.80         3\n",
      "        C#M       0.30      0.75      0.43         8\n",
      "       C#M7       0.00      0.00      0.00         1\n",
      "        C#d       0.50      0.25      0.33         4\n",
      "       C#d6       0.00      0.00      0.00         1\n",
      "       C#d7       0.67      0.40      0.50         5\n",
      "        C#m       0.83      0.71      0.77         7\n",
      "       C#m7       0.50      0.50      0.50         2\n",
      "        C_M       0.85      0.88      0.87       156\n",
      "       C_M4       1.00      0.50      0.67         2\n",
      "       C_M6       0.00      0.00      0.00         1\n",
      "       C_M7       0.52      0.79      0.62        19\n",
      "       C_d6       0.00      0.00      0.00         1\n",
      "       C_d7       0.00      0.00      0.00         1\n",
      "        C_m       0.86      0.86      0.86        49\n",
      "       C_m6       0.33      0.33      0.33         3\n",
      "       C_m7       0.40      1.00      0.57         2\n",
      "        D#d       0.00      0.00      0.00         3\n",
      "       D#d6       0.00      0.00      0.00         1\n",
      "       D#d7       0.00      0.00      0.00         0\n",
      "        D_M       0.81      0.84      0.82       184\n",
      "       D_M4       1.00      0.33      0.50         9\n",
      "       D_M6       0.00      0.00      0.00         1\n",
      "       D_M7       0.29      0.38      0.33        13\n",
      "       D_d7       0.00      0.00      0.00         1\n",
      "        D_m       0.81      0.77      0.79        62\n",
      "       D_m6       0.80      0.80      0.80         5\n",
      "       D_m7       0.42      0.50      0.45        10\n",
      "        DbM       1.00      0.18      0.31        11\n",
      "       DbM7       0.00      0.00      0.00         1\n",
      "        Dbd       0.00      0.00      0.00         1\n",
      "       Dbd7       0.00      0.00      0.00         1\n",
      "        Dbm       0.00      0.00      0.00         2\n",
      "       Dbm7       0.00      0.00      0.00         2\n",
      "        E_M       0.79      0.85      0.82       106\n",
      "       E_M4       0.50      1.00      0.67         2\n",
      "       E_M7       0.42      0.45      0.43        11\n",
      "        E_d       0.00      0.00      0.00         2\n",
      "        E_m       0.79      0.70      0.74        79\n",
      "       E_m6       0.00      0.00      0.00         2\n",
      "       E_m7       0.50      0.80      0.62         5\n",
      "        EbM       0.85      0.87      0.86        46\n",
      "        F#M       0.81      0.68      0.74        31\n",
      "       F#M4       1.00      1.00      1.00         6\n",
      "       F#M7       0.43      0.50      0.46        12\n",
      "        F#d       0.75      0.50      0.60         6\n",
      "       F#d7       0.00      0.00      0.00         1\n",
      "        F#m       0.69      0.72      0.70        46\n",
      "       F#m6       1.00      0.33      0.50         3\n",
      "       F#m7       0.20      0.25      0.22         4\n",
      "        F_M       0.83      0.77      0.80       123\n",
      "       F_M4       0.00      0.00      0.00         5\n",
      "       F_M6       0.00      0.00      0.00         3\n",
      "       F_M7       0.36      0.50      0.42         8\n",
      "        F_d       0.00      0.00      0.00         2\n",
      "       F_d7       0.00      0.00      0.00         1\n",
      "        F_m       0.75      0.90      0.82        10\n",
      "       F_m6       0.00      0.00      0.00         1\n",
      "       F_m7       1.00      0.50      0.67         2\n",
      "        G#d       0.50      0.33      0.40         3\n",
      "       G#d7       1.00      0.33      0.50         3\n",
      "        G#m       0.25      0.33      0.29         3\n",
      "        G_M       0.83      0.88      0.85       152\n",
      "       G_M4       1.00      0.25      0.40         4\n",
      "       G_M6       0.00      0.00      0.00         1\n",
      "       G_M7       0.42      0.50      0.45        20\n",
      "        G_m       0.71      0.79      0.75        56\n",
      "       G_m6       0.00      0.00      0.00         1\n",
      "       G_m7       0.00      0.00      0.00         7\n",
      "\n",
      "avg / total       0.74      0.75      0.74      1870\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Program Files (x86)\\Microsoft Visual Studio\\Shared\\Anaconda3_64\\lib\\site-packages\\sklearn\\metrics\\classification.py:1135: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples.\n",
      "  'precision', 'predicted', average, warn_for)\n",
      "C:\\Program Files (x86)\\Microsoft Visual Studio\\Shared\\Anaconda3_64\\lib\\site-packages\\sklearn\\metrics\\classification.py:1137: UndefinedMetricWarning: Recall and F-score are ill-defined and being set to 0.0 in labels with no true samples.\n",
      "  'recall', 'true', average, warn_for)\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import classification_report\n",
    "print(classification_report(y_test, y_test_pred))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Попробуем другие параметры"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "MLPClassifier(activation='logistic', alpha=0.0001, batch_size='auto',\n",
       "       beta_1=0.9, beta_2=0.999, early_stopping=False, epsilon=1e-08,\n",
       "       hidden_layer_sizes=(100, 2), learning_rate='constant',\n",
       "       learning_rate_init=0.001, max_iter=200, momentum=0.9,\n",
       "       nesterovs_momentum=True, power_t=0.5, random_state=42, shuffle=True,\n",
       "       solver='lbfgs', tol=0.0001, validation_fraction=0.1, verbose=False,\n",
       "       warm_start=False)"
      ]
     },
     "execution_count": 111,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.neural_network import MLPClassifier\n",
    "mlp_model = MLPClassifier(hidden_layer_sizes = (100,2), solver = 'lbfgs', \n",
    "                          activation = 'logistic', random_state = 42)\n",
    "mlp_model.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.43425559947299075 0.439572192513369\n"
     ]
    }
   ],
   "source": [
    "y_train_pred = mlp_model.predict(X_train)\n",
    "y_test_pred = mlp_model.predict(X_test)\n",
    "print(np.mean(y_train != y_train_pred), np.mean(y_test != y_test_pred))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.5657444005270093 0.560427807486631\n"
     ]
    }
   ],
   "source": [
    "print(mlp_model.score(X_train, y_train), mlp_model.score(X_test, y_test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.439572192513369\n"
     ]
    }
   ],
   "source": [
    "print(1 - mlp_model.score(X_test, y_test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {},
   "outputs": [],
   "source": [
    "alpha_arr = np.logspace(-4, 2, 20)\n",
    "test_err = []\n",
    "train_err = []\n",
    "for alpha in alpha_arr:\n",
    "    mlp_model = MLPClassifier(alpha = alpha, hidden_layer_sizes = (100,2), \n",
    "                              solver = 'lbfgs', activation = 'logistic', random_state = 42)\n",
    "    mlp_model.fit(X_train, y_train)\n",
    "\n",
    "    y_train_pred = mlp_model.predict(X_train)\n",
    "    y_test_pred = mlp_model.predict(X_test)\n",
    "    train_err.append(np.mean(y_train != y_train_pred))\n",
    "    test_err.append(np.mean(y_test != y_test_pred))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEaCAYAAAAG87ApAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl8VNX9//HXJwkBA7IIuEISbCkKKGERcavY1g0VtbYqRMVWpdbaurZqKf2qFbXfb637UlCrkig/pe5atVax7hoqioK4QFhEFJA9LEnm8/vjTiSECZkJM7kzk/fz8biPzNy5c+ZzMjP3M+ece881d0dERCReOWEHICIimUWJQ0REEqLEISIiCVHiEBGRhChxiIhIQpQ4REQkIUocImnIzO4zs2uSva1IMihxSFozs0ozW29ma+stt4Udl0hrlhd2ACJxOM7dX2xqIzPLc/eaptYlWoaIbEktDslYZnammb1uZjea2TfAlY2syzGzP5jZfDP72sweMLNO0TKKzczN7CwzWwC8FON1ZpvZsfXu55nZMjMbZGbtzKzMzJab2Uoze9fMdokz/kfMbImZrTKz/5hZv0a2G25mi8zs99HXrTSz0gabdTGzZ8xsjZm9bWbfqff8m81soZmtNrPpZnZIPPGJNEaJQzLd/sBcYGdgQiPrzowuhwF7Ah2Aht1dhwJ7A0fGeI2HgFH17h8JLHP3/wJjgE5AT6ArcC6wPs7Y/wn0jsb5X6B8G9vuCnQD9oi+5kQz61Pv8VHAVUAX4DM2/y8A3gVKgJ2AB4FHzKxdnDGKbEWJQzLB49Ff83XLOfUeW+zut7p7jbuvb2RdKfBXd5/r7muBK4BTzax+V+2V7r6uXhn1PQiMNLOC6P3R0XUA1QQJ47vuXuvu0919dTyVcvd73X2Nu28ErgQG1LWEGjHe3Te6+yvAM8DJ9R571N3fiXazlRMkirrXKXP35dH/xw1AW6APIs2kxCGZ4AR371xvmVTvsYUxtm+4bndgfr378wnG9+p3KcUqBwB3/wyYDRwXTR4j2Zw4JgPPA1PMbLGZ/a+ZtWmqQmaWa2bXm9nnZrYaqIw+1K2Rp6xw93UN6rB7vftL6t2uImhV1b3WJdHutlVmtpKghdTY64g0SYlDMl2s6Z0brlsMFNW7XwjUAF81UU59dd1VxwOzoskEd69296vcvS9wIHAscEYccY+OlvUjgh15cXS9NbJ9FzNr36AOi5t6keh4xmUErZMu7t4ZWLWN1xFpkhKHtAYPAReZWS8z6wBcC/y/BI+emgIcAfySza0NzOwwM9vHzHKB1QRdV7VxlLcjsBFYDhREY2rKVWaWH00GxwKPxPk6NcBSIM/M/gh0jON5Io1S4pBM8FSD8zgeS/D59xJ0Kf0HmAdsAH6dSAHu/iXwJkGr4v/Ve2hXYCpB0pgNvAKUAZjZXWZ2VyNFPkDQ3fQFMAt4q4kQlgArCFoZ5cC57v5xHKE/TzAI/0n09TawjW45kXiYLuQkkt7MbDhQ5u49wo5FBNTiEBGRBClxiIhIQtRVJSIiCVGLQ0REEqLEISIiCcm42XG7devmxcXFYYchIpJRpk+fvszduyejrIxLHMXFxVRUVIQdhohIRjGz+U1vFR91VYmISEKUOEREJCFKHCIikpCMG+MQEWmO6upqFi1axIYNG8IOJaXatWtHjx49aNOmydn9m8/dM2oZDO5FRe5lZd4sZWXB882aX04yykjHckSy2Ny5c33p0qUeiUTCDiVlIpGIL1261OfOnbvVY0CFJ2k/HHoiSHQZDEHYBQWJ7yDLyoLn1ZXRnHKSUUY6liOS5WbNmpXVSaNOJBLxWbNmbbU+mYkjc7uqqqrgootgxx3jf85FFwXP255yklFGS5QzbhyUlsZfjkgrYJb9169qiTpm3FxVQ8xcZ3HEwQwikbCjEEkbs2fPZu+99w7t9VeuXMmDDz7Ieeedl/Bzb7rpJsaOHUtBQUHTGxO7rmY23d2HJPziMWRuiwOo6rQrBS89E//2PziGglVLtl6fQDnJKKMlylm7U+Hmi06LSMLKy4OG+4IFUFgIEyZsXyN+5cqV3HHHHc1OHKeddlrciSPVMjZxrKOASyJ/YeSSQXE/58nIX7iBsbRnc9dOouUko4yWKOf3TOCWuEsRkfrKy2Hs2M29wPPnB/eh+cnj8ssv5/PPP6ekpITDDz+cnXfemYcffpiNGzdy4oknctVVV7Fu3TpOPvlkFi1aRG1tLePHj+err75i8eLFHHbYYXTr1o2XX345OZXcHskaLGmpZTD4PIp8FGVbjAfHu4yizOdR5LVYs8tJRhnJLmcN7T1S739j1oxRNZEsVn/A+IIL3A89tPGlbdvY37W2bRt/zgUXbPv1582b5/369XN39+eff97POeccj0QiXltb68ccc4y/8sorPnXqVD/77LO/fc7KlSvd3b2oqMiXLl3arLrWoTUPjk9nML0IRjl23RWeeCL+5x5/PDy0pJSH2PInQyLlJKOMVJSzC19zIxezP2/zNbtQVBh/GSKypY0bE1ufqBdeeIEXXniBgQMHArB27Vo+/fRTDjnkEC699FIuu+wyjj32WA455JDkvGCyJSsDxVqAo4A5wGfA5TEeLwL+DXwATAN6NF3mYB2NG6OcQ3nZHfwIntPRuCIxxPoV3piiotgtjqKi5r9+/RbHxRdf7HfddVfM7ZYvX+6TJ0/2gw46yK+66qpoPOnV4khl0sgFPgf2BPKB94G+DbZ5BBgTvf0DYHLT5Q4O/Vy5dDtvr6zMfY+Cb9zBr+98nZKGSAyJJI5UnB61bNkyLywsdPegq2ro0KG+Zs0ad3dftGiRf/XVV/7FF1/4+vXr3d39scce8+OPP97d3fv37x/zpL7GZHLiOAB4vt79K4ArGmzzUV0rAzBgdVPlDh48OO5/Xmty443ucyn2quNPCTsUkbSUSOJwT82EDKNGjfJ+/fr5pZde6jfddJP379/f+/fv78OGDfPPPvvMn3vuOd9nn318wIABPmTIEH/33Xfd3f2WW27xPn36+PDhw+N6nVQnjpSdx2FmPwGOcvezo/dPB/Z39/PrbfMg8La732xmPwb+AXRz9+UNyhoLjAUoLCwcPH9+0qaVzxqvvALfDD+Rw/eYRYdFc8IORyTthH0eR0tK9XkcqZwdN9bpiw2z1KXAoWb2HnAo8AVQs9WT3Ce6+xB3H9K9e1IuYJV1SkrgPQbSfvGnsHZt2OGISBZLZeJYBPSsd78HsLj+Bu6+2N1/7O4DgXHRdatSGFPW6tQJvty5BHOHDz4IOxwRyWKpTBzvAr3NrJeZ5QOnAk/W38DMuplZXQxXAPemMJ6slzOoJLgxY0a4gYhIVktZ4nD3GuB84HlgNvCwu39kZleb2cjoZsOBOWb2CbALMCFV8bQGPQ/syXJ2YtM7ShwikjopPQHQ3Z8Fnm2w7o/1bk8FpqYyhtZk4CBjBiUMffs98sMORkSyli4dm0UGDgwGyHf4bCbUbHWMgYhIUihxZJHddoO5HUvIq9kIH38cdjgiUk/d7LiJGjFiBCtXrkxBRM2nxJFFzKB2n2DuGw2Qi2yn8nIoLoacnOBvefl2FddY4qitrd3m85599lk6d+68Xa+dbEocWabbQX3YQFtqKt4LOxSRzFU3r/r8+cGMI3Xzqm9H8qg/rfp+++3HYYcdxujRo9lnn30AOOGEExg8eDD9+vVj4sSJ3z6vuLiYZcuWUVlZyd57780555xDv379OOKII1i/fv12V7U5Mm52XNm2fQflMZN96PPGDDqGHYxIurrwwm23yt96a+upcKuq4KyzYNKk2M8pKYGbbmq0yOuvv54PP/yQGTNmMG3aNI455hg+/PBDevXqBcC9997LTjvtxPr169lvv/046aST6Nq16xZlfPrppzz00ENMmjSJk08+mX/84x+cdtppcVU5mdTiyDJ1A+RtP3ov+KUkIolL9bzqwNChQ79NGgC33HILAwYMYNiwYSxcuJBPP/10q+f06tWLkpLgfK3BgwdTWVmZtHgSoRZHlvnud+HOtiW0rZoECxcG17wUkS1to2UABGMasebEKyqCadOSEkL79u2/vT1t2jRefPFF3nzzTQoKChg+fDgbNmzY6jlt27b99nZubm5oXVVqcWSZnByo+p4GyEW2y4QJ0PD63gUFwfpm2nHHHVmzZk3Mx1atWkWXLl0oKCjg448/5q233mr267QEJY4s1OGAfYhgRP6rAXKRZikthYkTgxaGWfB34sTmX3Ac6Nq1KwcddBD9+/fnt7/97RaPHXXUUdTU1LDvvvsyfvx4hg0btr01SKmUTaueKkOGDPGKioqww0hr994LB561Fz1+tDcd/vVY2OGIpAVNq54Z06pLSAYOhBmUYDPU4hCR5FPiyEL9+sHMnBLaL5sPK1aEHY6IZBkljiyUnw8rijVALiKpocSRpdrsFxzr7e8pcYjUybQx3eZoiToqcWSp7xy4C4vZjfVvKnGIALRr147ly5dndfJwd5YvX067du1S+jo6ATBL1Q2QH6w5q0QA6NGjB4sWLWLp0qVhh5JS7dq1o0ePHil9DSWOLDVgANzKQI5c8C/YsAFS/AtEJN21adNmiyk+pPnUVZWlOnaEJbuWkBupgY8+CjscEckiShzZbKCOrBKR5FPiyGK7H7Qna+jAxrc0ziEiyaPEkcUGDs7hfQaw4S21OEQkeZQ4slhJSXBtjh0+fR8ikbDDEZEsocSRxXbdFeZ1LCF/41r4/POwwxGRLKHEkeWq+2uAXESSS4kjy3U5uB/V5FHzrgbIRSQ5lDiy3L77tWUWfVn3ulocIpIcShxZrm7qkTYfqsUhIsmhxJHlevWCj9uWULB6CSxZEnY4IpIFlDiyXE4OVPXRALmIJI8SRyuww7ABAET+q8QhIttPiaMV2OuALsyjmLWvKXGIyPZLaeIws6PMbI6ZfWZml8d4vNDMXjaz98zsAzMbkcp4Wqu6AXJmaIBcRLZfyhKHmeUCtwNHA32BUWbWt8FmfwAedveBwKnAHamKpzXr2xc+yB1Ihy8/hbVrww5HRDJcKlscQ4HP3H2uu28CpgDHN9jGgY7R252AxSmMp9Vq0wZWFpWQg8MHH4QdjohkuFQmjj2AhfXuL4quq+9K4DQzWwQ8C/w6VkFmNtbMKsysItsv+5gqbfYrAcDf0ziHiGyfVCYOi7Gu4VXiRwH3uXsPYAQw2cy2isndJ7r7EHcf0r179xSEmv2KDu7JcnZi3Wsa5xCR7ZPKxLEI6Fnvfg+27oo6C3gYwN3fBNoB3VIYU6s1cJAxgxJqKtTiEJHtk8rE8S7Q28x6mVk+weD3kw22WQD8EMDM9iZIHOqLSoF994UZDKRD5Uyorg47HBHJYClLHO5eA5wPPA/MJjh66iMzu9rMRkY3uwQ4x8zeBx4CznT3ht1ZkgQdOsCSXUvIq9kIc+aEHY6IZLC8VBbu7s8SDHrXX/fHerdnAQelMgbZzEsGwnMEU4/07x92OCKSoXTmeCuy8yF9WE871r+pAXIRaT4ljlakZEgeM9mH9W9qgFxEmk+JoxWpm3pkh4/fAw0liUgzKXG0It27Q2WnEnZYvwIWLmz6CSIiMShxtDKb+kevzfGexjlEpHmUOFqZzgfvQwSj+l2Nc4hI8yhxtDJ9h3bgE77HGl2bQ0SaSYmjlakbIG8zU11VItI8ShytTHExfNxuIDt+Mx9WrAg7HBHJQEocrYwZVH0vmGKdGequEpHEKXG0Qu2GBYmjdroSh4gkTomjFep98C4sZjfWvKpxDhFJnBJHK1Q3QK6rAYpIcyhxtEJ77QUzcwfScdEs2LAh7HBEJMMocbRCeXmwoqiEXK+Fjz4KOxwRyTBKHK1U7pBg6hF1V4lIopQ4Wqke39+TNXTQALmIJEyJo5UaODiH9xlA9TtqcYhIYpQ4Wql994UZDKTD3PchEgk7HBHJIEocrVRBASzZrYS2m9bC55+HHY6IZBAljlbMB+jaHCKSOCWOVqzbof2oJo8qXYNcRBKgxNGK7TOkLbPoS9XranGISPyUOFqxuqlH2n2sFoeIxE+JoxXr2hUqOw+kw5olsGRJ2OGISIZQ4mjlNvXVtTlEJDFKHK3cjgcPAGCTTgQUkTgpcbRyex/YhXkUs+Y/GiAXkfgocbRydQPkuTPV4hCR+ChxtHI9e8LHOwyk49efwtq1YYcjIhlAiaOVM4Oq3iXk4PDBB2GHIyIZIKWJw8yOMrM5ZvaZmV0e4/EbzWxGdPnEzFamMh6JLX//YOqR2unqrhKRpqUscZhZLnA7cDTQFxhlZn3rb+PuF7l7ibuXALcCj6YqHmncnt/vwXJ2YtU0DZCLSNNS2eIYCnzm7nPdfRMwBTh+G9uPAh5KYTzSiIGDjBmUENHVAEUkDqlMHHsAC+vdXxRdtxUzKwJ6AS818vhYM6sws4qlS5cmPdDWrk8fmJk3kE4LZkJ1ddjhiEiaazJxmFmumV3UjLItxjpvZNtTganuXhvrQXef6O5D3H1I9+7dmxGKbEtuLqwsKqFN7UaYMyfscEQkzTWZOKI78211MTVmEdCz3v0ewOJGtj0VdVOFKmdwMEDu/9U4h4hsW7xdVa+b2W1mdoiZDapbmnjOu0BvM+tlZvkEyeHJhhuZWR+gC/BmQpFLUh1c8F8iAGPOgOJiKC8POSIRSVd5cW53YPTv1fXWOfCDxp7g7jVmdj7wPJAL3OvuH5nZ1UCFu9clkVHAFHdvrBtLUq28nEOnnLv5V8T8+TB2bHC7tDSsqEQkTVmm7a+HDBniFRUVYYeRXYqLg2TRUFERVFa2dDQikgJmNt3dhySjrLi6qsysk5n9te7IJjO7wcw6JSMACZ/PX5DQehFp3eId47gXWAOcHF1WA39PVVDSsr7ILUxovYi0bvEmju+4+/9ET+ab6+5XAXumMjBpOZfVTmAdBVusq2IHLqudEFJEIpLO4k0c683s4Lo7ZnYQsD41IUlLe72olHOYSCVFRKKn30zlJF4v0sC4iGwt3sRxLnC7mVWaWSVwG/CLlEUlLWrCBHiioJReVJJLhOkMYl/7iAlqcIhIDPGcOZ4D9HH3AcC+wL7uPtDdNQd3ligthYkTg4OozOB+xlDi71G678ywQxORNBTPmeMR4Pzo7dXuvjrlUUmLKy0NjryNRGDtsaOoJo/qex8IOywRSUPxdlX9y8wuNbOeZrZT3ZLSyCQ0p13UnWcZQfXfy6CmJuxwRCTNxJs4fg78CvgPMD266Cy8LHXYYfDvPcZQsGoJvPhi2OGISJqJd4zjNHfv1WDR4bhZygz6XHwMy9mJb268P+xwRCTNxDvG8ZcWiEXSyGlntWVq3ql0+PfjsGpV2OGISBqJt6vqBTM7ycxiXWNDslCnTvDNsWPIr93AuvseCTscEUkj8SaOi4GHgY1mttrM1piZjq7Kckf/cT9msxcrb1Z3lYhsFm/i6AScCVzj7h2BfsDhqQpK0kPJQOOVwjPYY95rRD79POxwRCRNxJs4bgeGEVw7A4IJD29LSUSSVna+5HQiGPOu1jkdIhKIN3Hs7+6/AjYAuPsKID9lUUnaOOYXPXi1zQ/p8OgDwdmBItLqxZs4qs0sl+Cqf5hZd0B7kVagbVv46qgx7FJVydePvhZ2OCKSBuJNHLcAjwE7m9kE4DXg2pRFJWll2J9PZA0dWHiNBslFJM5rjrt7uZlNB34IGHCCu89OaWSSNgr3bs+LPX/C/h88wqaVt5LfuaDpJ4lI1oq3xYG7f+zut7v7bUoarU/H88ewo69h+vjHww5FREIWd+KQ1m3Ixd9nUV4RuWXqrhJp7ZQ4JC45eTksPPR0Bq98kdkvfhF2OCISIiUOidte155BLhE+Hl8edigiEiIlDolbl6G9+bT7Aez1zv2sXuVhhyMiIVHikIS0OWsMe0dm8fy108MORURCosQhCSn+3clstLZsuvsBXI0OkVZJiUMS06ULiweP5MhvHuS1lzaFHY2IhECJQxK22xVj6MZy3v6fZ8MORURCoMQhCWs38khWF+zCd954gC+/DDsaEWlpShySuLw8ak8p5Rh/mvJblocdjYi0MCUOaZYuF5xBPtWsvPMhamrCjkZEWlJKE4eZHWVmc8zsMzO7vJFtTjazWWb2kZk9mMp4JIkGDGBVrwGMXPUATz0VdjAi0pJSljii1++4HTga6AuMMrO+DbbpDVwBHOTu/YALUxWPJN+OvxrDUN7l6f/TnJcirUkqWxxDgc/cfa67bwKmAMc32OYc4PboFQVx969TGI8kWc5po4nk5NL7zfuZMyfsaESkpaQycewBLKx3f1F0XX3fA75nZq+b2VtmdlSsgsxsrJlVmFnF0qVLUxSuJGyXXaj+wVGczmT+dkdt2NGISAtJZeKwGOsanmucB/QGhgOjgLvNrPNWT3Kf6O5D3H1I9+7dkx6oNF/bsWPYg8XMvfsl1q0LOxoRaQmpTByLgJ717vcAFsfY5gl3r3b3ecAcgkQimeK446jp0JmfVN3PQw+FHYyItIRUJo53gd5m1svM8oFTgScbbPM4cBiAmXUj6Lqam8KYJNnatSN39CmcZI9y3y2rNX+VSCuQssTh7jXA+cDzwGzgYXf/yMyuNrOR0c2eB5ab2SzgZeC37q4zyjKMnTmGHXw9vWf+g7ffDjsaEUk18wz7iThkyBCvqKgIOwypz51I7z68Xrk7k0ZP44EHwg5IRBoys+nuPiQZZenMcdl+ZuSceQaH1L7C21PmsWxZ2AGJSCopcUhynH46ACdXl9G7N+TkQHExlOsqsyJZR4lDkqOoiCV9D+MMHmDlSscd5s+HsWOVPESyjRKHJM1fvjqD3nzGgbzx7bqqKhg3LsSgRCTplDgkaSYuP4kNtOE5jqKWHOZRzCjKWbAg7MhEJJnywg5AsseZXZ8kb3mEdqwFoJj5TGIs3XYCKA01NhFJHrU4JGmuZRx5bDlnVXuqmODqqxLJJkockjQdvondJ9X+mwVUVbVwMCKSMkockjyFhTFXL6CQU06B6uoWjkdEUkKJQ5JnwgQoKNhynRmLfnoxTz8NZ50FkUg4oYlI8ihxSPKUlsLEiVBUBGaw227Qrh0Hz7iNv172FZMnwyWXoIkQRTKcEockV2kpVFYGTYvFi+HFF+GLL7jw2cO54hffcNNNcN11YQcpIttDiUNS68AD4YknsDlzmDBjBGefsoZx44KGiYhkJiUOSb0f/QgefhirqOBvXx3PCUdt4NxzYerUsAMTkeZQ4pCWcfzxcN995Lwyjan2Uw4ZVs3o0fCvf4UdmIgkSolDWs5pp8Edd5D7z6f5125n0G+vWk48Ed55J+zARCQRShzSss49F/78Z/IfncLr+5zLLjs7I0bA7NlhByYi8VLikJb3u9/BuHEUPHg30w+7lLxc54gj0GSIIhlCiUPC8ac/wa9/Ted7/8qMk/7EmjVwxBGwdGnYgYlIU5Q4JBxmcNNNMGYMu975P0w//Ubmz4cRI2DNmrCDE5FtUeKQ8OTkwN13w0kn8Z3bLuatc+7hvfdg2LDg5HNdflYkPSlxSLjy8oLMcOSRDLjtHO467P8xa1Yw3qHLz6a38vIgsSvBtz5KHBK+tm3h0Ufh4IMZ8+JpjOCZLR7W5WfTT3k5vPizcqbNL6bGc5g2v5gXf1au5NFKKHFIeigogKee4n0G8Bgn8CW76vKzaeztC8q5rXosxcwnB6eY+dxWPZa3L1DmaA2UOCR9dOrEP7qcTR617MpX3+6QJjGW87toh5ROLl4+jvZseXWu9lRx8XI1DVsDJQ5JK+NyrieHLeddb08Vf1g/TtOxp5FCYjcBC1nAK6+0cDDS4pQ4JK00dvnZbusXcOWVLRuLNG5Nwa4x13+ZswfDhwcTBKxa1bIxSctR4pD00sjlZ79p35Orr4ZrrmnheGQrVS+/TV7VKiLYFusd2K3jOm499XUmTYJ+/eCpp8KJUVJLiUPSS6zLzwI7HXsQp58O48fDn/8cQlwSeOklco/8IV+yGwsu+Ovmqz0WFWHXXEPOTl04/5FDqTz/L+zUxRk5EkaP1owAWcfdM2oZPHiwS5YrK3MvKnI3cy8sdN9vP/ecHK954d8+apQ7uN9wQ9hBtkKPP+61+W19Jv394lGLY2+zcqX7SSe5g9cec5z/7+XLvU0b965dg7c1EmnZkGUzoMKTtB8OPREkuihxtEKrV7vvvbd7t25e/fl8/+lPg0/uzTeHHVgrMnmyR3JzfVbHoV7ccbl//fU2to1E3G+5xb1NG/fCQp/70Fs+bFjwno0Y4b5gQYtFLfUkM3GktKvKzI4yszlm9pmZXR7j8TPNbKmZzYguZ6cyHslQO+4Ijz0GGzeSd8pJlN+zgRNPhAsugDvvDDu4VuCOO+D00/l6r0MZuvpFfnf9TnTvvo3tzeDXv4bXXgMzep1xCK//9CZuvsmZNi0Y+7jzzuCy9JKhkpWBGi5ALvA5sCeQD7wP9G2wzZnAbYmUqxZHK/bYY8HP1p//3DduiPhxxwV3J04MO7AsFYm4X3utO/imESO9127rfdAg95qaBMr45hv3kSODN+rEE71yxgo//PDg7iGHuM+Zs2XPZFFRcF+Sj0zoqgIOAJ6vd/8K4IoG2yhxSGLGjQs+tn/7m2/Y4H700cEO5+9/DzuwLBOJuP/ud8H/urTUf3fRJgf3t95qZll/+Yt7Xp57r14eebfC//53986d3XNzgx6tYGayYCkoCDF5ZHEWy5TE8RPg7nr3T2+YJKKJ40vgA2Aq0LORssYCFUBFYWFhsv+fkklqatyPPDLY27z5pq9f73744cH3fPLksIPLEjU17mPHBruH887zDz+o9bw897PP3s5y33jDvUcP9/x899tu8y8XR3yHHbZMGnVLz56JFZ2U/X1ZWZC10iaLJVemJI6fxkgctzbYpivQNnr7XOClpspVi0N8+XL3Xr3c99jDfckSX7fO/bDD3HNy3B96KOzgMtymTe6nnhrsGn7/e4/URnz4cPcuXdyXLk1C+UuXBs1EcD/5ZO/IqpiJA9w7dnTfZx/3Y491/9Wv3P/8Z/cpU9zffNN98WL32tqgyKTs71etct9559iBFBUloeLhy5TE0WRXVYPtc4FVTZX9nF9AAAASVklEQVSrxCHu7v7ee+477OD+/e+7b9rka9cGfea5ue6PPBJ2cBmqqsr9mGOC3cKf/+zu7g8+GNy9664kvk5trft117nn5vrneb39Mq71eRR5LebzKPJRlHnnzu7nn+9+3HHuAwYE3VoN9+f5+e7f+Y5727YJ7u9ra90/+sj9nnvczznHvX//oKnSWAYzS2Llw5MpiSMPmAv0qjc43q/BNrvVu30i8FZT5SpxyLcmTw4+whde6O7BUbsHHhh0pT/+eMixZZpVq4IkbOb+t799u2q33dyHDElwQDxer7ziG3bo5JEGO+q1FPirv9y6ubBqlfvMme5PP+1+++3BEExd42gUZVslHwiGxP5ZtsxXlD/jPn580K/ZsePm1+vc2f2oo9yvvFItjnRIHEGcjAA+iR5dNS667mpgZPT2dcBH0aTyMrBXU2UqccgWfvOb4GNcXu7uwc5l6NBgCOSSS5IzzpnF46WBpUvdBw8OMu6UKd+uvvjioM7vvJPC1+7RI/bOOj8/2KGfcor7L34RZIlrrw0yRnm5+zPPuL/2mvuHH/r4zrf4WrYcLNlAvv+Hg30Ovb9dV0OOz+s8wN8/4Bf+6R/u8w3vf7y5v8vdvazMq/MLto6ltDSF/4CWkzGJIxWLEodsYdOmoI9qhx3c33/f3d1XrHAvLt76+9+ccc6yMvcz22z5a/bMNmWZnzzqZ8O8vGB55plvH545M+j2Gzs2xXFsq4tov/3ce/cOWgL5+Y1v18hSazlefezxPnfsdT7l3Jf9tBPWbJGn2rQJXuL884PG6w03bPleV1Lo79h+wcaTJqX4H5F6yUwcFpSXOYYMGeIVFRVhhyHpZMkSGDQIdtgBKiqgSxd69oRFi7betHNn+O1vYdOmzcvGjVver7/s/K9y7qgdu8W1J9ZRwBVdJ3LLstIWrGQSlZcH1+Otqnc9jbZt4Z57oLQUdxg+HD78ED75BLp2TWEsxcXB9YEbKiqCysot123YACtXBtPu1v97yimxyzaLeZbh4sXw9tvw1lvB34oKWLcudhH5bOT5dicwfOPz8NBDjb9WBjCz6e4+JCmFJSsDtdSiFofE9MYbwU/Io492r63d5g/Z+r8427cPjhjaZZdgWqzvfte9b1/3kpKgy2sehTGfPI+isGvcfEVFsf8h0b78uqGjFjmxMhmHRDVRn6ZUV7vPmNH456SAdUGrNi8vGGDJUKirSiSGO+4IPtLjxze6L+nZ033jxiYm24tE3KdPd//d77YauP22G4QMPtKmsaxq5itXBkl06NAtu/9TansHkZJ0/kVjn5ndd/dg8Gzw4OAQrpdeSiy+NKHEIRJLJOJ+5pnu4C9f9ETi+5KZM93/8IegXx3c8/K8uk27mHuTNV2LWqpWyXXPPbH3jtFf6BdcEOy/33037EATlIQjGGLlHwhyxSOPeHAQQd++7h06NPMU+nApcYg0pqrKfdAg944d/Yn/m9P0vmTOHPerrw52CBCcRfjDHwb9NMuWxTzSZhN5/vToDBsdj0Tc//jHoA79+/tWp2wXFHjltWWek+N+7rlhBxuehvnn5pv925l9L7zQfVPlF+577hn0b0YPxsgUShwi21JZGVwAYvfdg76phplj7lz3668PBjKiXTR+yCHut93mvmTJ1uXV25tEOnRwBz/dJvu//92SldoOGze6jxkT1PVnPwuORGuwh4yUlflBBwX/tuXLww44vWzcuPmo7wMPdF/8+tzgs7XLLu6ffBJ2eHFT4hBpyuWXe8zR8D333Hx///3db7zRfdGi+MvdtMmrDxnuG6yt/6jj215ZmboqJMWqVe4/+lFQ36uuanRw5/77g03uvruF48sgU6YEvVTdurm/fves4EZhofv8+WGHFhclDpGmNDbS2aZN0NqYO7f5ZS9d6pt69PLFtpsf0W+RV1UlLerkWrjQfd99g6OBtjF98IoVwakS++/fggPiGWr2bPd+/YKG2sRzp3ukY8dgTCxWSzXNJDNx6Jrjkp0WLIi9vqYGLrsMevVqftndutHmn0/Svd0a/vTRifzq5+vxdDsd6oMPYNgwmDcPnnkGzjyz0U3/+MfgmuB33AE52iNs0157Bed+jB4NY+8axG/7Pot/8QUccQSsWBF2eC1GHxPJToWFia1PVP/+5E0pZz+r4AdTzuHWW9Ioc7z4IhxySNDGevXVYKfWiBkz4Pbb4Ze/DM6hlKa1bw+TJwdXMbz1vwdxevvHiMz+GI4+GtasCTu8FqHEIdlpwgQoKNhyXUFBsD5ZRo7E/3QNp1HO4ov+j2nTkld0s91/f7ADKywMTo0eMCDmZuXlwcnZAwcG9xvZTBphBueeG1wd97WCIzglMoXIuxX4CScEZ7hnu2T1ebXUojEOiVtLzE4YifimH5/itZiP7vhUeOOkkUhwWDEEhxOvXNnopll+vaIWt3y5+4gR7qfxgDt49dHHBUeupRk0OC6SRtat8/V9B/kqdvSf9P2o5QfLN21yP+us4Ot8xhnB8aPbsJ0zdEgMtbXu11zjfp7d7g6+sPAAX5hb6LWYL8wtijlNfEtLZuJQV5XI9ioooN1zj5PfuYDrZo3k0p9/g7fUkMeaNXDcccEEhePHw333QX5+o5u/8krsOQWh8eMJpGk5OTBuHPz4X+cxNfdkeix4kx61C8jB6VE7n4F3juW188rDDjNplDhEkqFnT9o9+xjFuQs5ccrJ3HlrTWpep7w8mFE2Jwd69ID+/YPB8EmT4Oqrg873BtzhhRfg+98PZr1t7MipZB030Jr98IcwjLe3Wt+eKoonjgshotRQ4hBJlgMOIGfSRH7Ev4lceDGvvprk8uumQ58/P8gGX3wRNBMuvhjOPnurzd3hqaeCo3KPPDI4MvfWW+Huu1N/3EBrtntt7KbbHrXz4YYbgvct0yWrz6ulFo1xSLrb8KuL3cEv3nGiL1yYxILjHJyorQ0m5RswIHi4V69g6q36Qx9Zf1XDEC3MLYr5Pm0guBhVxMx9+PDg4lDffJP6gKJv9mBwT9J+OPREkOiixCFpr7ra1xx0pG+kjY/d+z++fn0Synz//dhJo26ureBlvazMfe+9g9V9+gRTiaThAT5Z7dVflvlatjxsbS0FflWfMv+efeIT2l7py7pGZ2DOz3c/4QT3hx/2lBxVUe8QOiUOkXS3YoWv3v17/hXd/bcnV277+h+NmTcvuM52//6NJw3wSGGR33NPcBGquslvp0xxr6lJdqUkXq/+sswX5hZtdVTVhx+6H3+8O0T8yK7v+owfXOSR3XYL3rgddwwmo3z++c2/AuJtFq5bF8yH8txz7n/7m/sVV7iPHh3MCR/9nCQzcejSsSKpMmcO6wfsz5yNxVTc+BpnX9ih6ecsXQqPPAIPPgivvx6sO+igYI6LnBxqLriEvE2bL/m6MbeA33aayK3flDJ4MPzhDzBypKYOSXdvvAGXXx6c2P+979Ry16nTGL74QewfU2H1aujYMbi0b029gyzatYPzztt8ud26pbIy+NzUl5cHPXrglZXUHS4xBKhw3/roieZIVgZqqUUtDskkNc885zXk+FRO8l13ro3943HNmmDFiBHBhIQQzKR37bVbTMZYVuZ+Zpsyn0fwS3YeRT6KMv/ud92ffbaJqxpK2olE3J95JpiHEoLLyPzrqfUemfqPra+X0nBp1y7oizziCPdzzglOIpk82f3VVz0yf4EvmFfjTz7pPt+K1OIAtTgk87x18l8Z9sglrKQTHVnNAgr5U5urOe03Xfn+wnLsqSfIWV/Fhp17Mu+A0XzQbzRz8vdh2XJj2TK+XT74AGprty6/sLDxczMk/UUiQQNz/Pig8XDYYfDvaTlYrH2zGSxZAt27gxmbNsHs2cGcY++/H/ydMWPzfIujKGcSY2lPVVJbHEocIin2m67l/PWbMeSxea/vgAHL2YlH+CnllPI6B1H/nNxOnaBbt83LM8/ELt8s2PlIZtu4ESZOhD/9Cd5ZWkwxW/8aWNGxiPuvqvw2QcyaBdXVwWPt2sG++wbzjpWUBMupp8LBC8u5lnH8hPlKHCKZotJi7wS+pjv3XLmInXbNp1s36Np1c5Lo2hXatNly+7qu7YaKioJfqpId1qyBi3Yp5+b1QUuhzjoKOIeJPEQpu+66ZYIYMAB69w6GNuqrO/WnqgpgCO4VSUkceU1vIiLbo5DYJ4R1YxlX/E/j04M0NGFC/Z1AQCfuZZ8dd4R7N5RSBVzLOApZwAIK+T0TmGKlLPkSdtklvrJKS4O/48YltztTLQ6RFFvbrZgOy7f+1q7tWkSHZZUJlVVeHuwEFiwIxjYmTNi8c5DskYrWpZlNd/ch2xNXHR20J5JiHW6eQE3+lnN81OQX0OHmxJsKpaXBjiMSCf4qaWSnlriczPZQ4hBJtdJS8u6dGPxcNIOiouC+9vrSiNLSYKC83keGiWn0kVFXlYhIK6CuKhERCY0Sh4iIJESJQ0REEqLEISIiCVHiEBGRhGTcUVVmtgaYE73bCVhV7y8N1je8Hevxba2L57HmbLct3YBlbLsO2Vy/bdV1WzGkQ/3q6tawrHSoXzLfu4blbW/90uG9g9j1y6bvXh9333E7ywgka5rdllqAinq3J9b/23B9rMcSWRfPY83ZLp76basO2Vy/bdV1WzGkQ/1ifTbTpX7JfO+SXb90eO8aq182fveSsWR6V9VTDf42XB/rsUTWxfNYc7ZLtKzWVL9t1XVbMah+8ceTDMmsX7q9d/XLy8bv3nbLxK6qCk/SSSzpSPXLXNlcN1D9Ml0y65eJLY6JYQeQYqpf5srmuoHql+mSVr+Ma3GIiEi4MrHFISIiIVLiEBGRhChxiIhIQjI+cZjZCWY2ycyeMLMjwo4n2cxsTzO7x8ymhh1LsplZezO7P/r+pcmVBlIjm99HaBXfw73N7C4zm2pmvww7nlSIfh+nm9mxTW2blonDzO41s6/N7MMG648yszlm9pmZXQ7g7o+7+znAmcApIYSbsATrN9fdzwon0sQlUjfgx8DU6Ps3ssWDjVOCdYopnd/HJNUvbb+HSarfbHc/FzgZSKtDdpNRv6jLgIfjetFknUmYzAX4PjAI+LDeulzgc2BPIB94H+hb7/EbgEFhx57C+k0NO+5k1w24AiiJbvNg2LEnqU77AE83WHZO5/cxyfVLu+9hsupH8OPmDWB02HVKdv2AHwGnEiT+Y5t6zTzSkLv/x8yKG6weCnzm7nMBzGwKcLyZzQauB/7p7v9t0UCbKZH6AbNaNrrtk2DdFgE9gBmkaesXEquTu18HNNnUTyfJqJ+ZGWn6PUzW++fuTwJPmtkzwIOpizgxSXr/DgPaEySX9Wb2rLtHGnvNtP2yxrAHsLDe/UXRdb8myJY/MbNzwwgsSWLWz8y6mtldwEAzuyKc0LZbY+/do8BJZnYnaTalQhwaq1NMGfg+JlQ/Mu97mOj7N9zMbjGzvwHPpjq4JEiofu4+zt0vJEiIk7aVNID0bHE0wmKsc3e/BbilpYNJgcbqtxzIhC/itjRWt3XAz1o6mCSJWafGNs7A9zHR+mXa9zDR+k0DpqUqmBRIqH7fbuB+XzyFZ1KLYxHQs979HsDikGJJhWyuXzbWLRvrVJ/ql9lSWr9MShzvAr3NrJeZ5RMM5DwZckzJlM31y8a6ZWOd6lP9Mltq6xf2EQGNHCXwEPAlUE2QOc+Krh8BfEJwtMC4sONU/VpH3bKxTqqf6rc9iyY5FBGRhGRSV5WIiKQBJQ4REUmIEoeIiCREiUNERBKixCEiIglR4hARkYQocYgkwMwqzazb9m4jksmUOEREJCFKHCKNMLPHo1dE+8jMxjZ4rNjMPrbgCoYfRK8MV1Bvk1+b2X/NbKaZ7RV9zlAze8PM3ov+7dOiFRJJEiUOkcb93N0HE1zx7Tdm1rXB432Aie6+L7AaOK/eY8vcfRBwJ3BpdN3HwPfdfSDwR+DalEYvkiJKHCKN+42ZvQ+8RTDTaO8Gjy9099ejt8uAg+s99mj073SgOHq7E/BI9BKfNwL9UhG0SKopcYjEYGbDCS5MdIC7DwDeA9o12KzhRG/172+M/q1l83Vv/gS87O79geNilCeSEZQ4RGLrBKxw96roGMWwGNsUmtkB0dujgNfiKPOL6O0zkxKlSAiUOERiew7IM7MPCFoKb8XYZjYwJrrNTgTjGdvyv8B1ZvY6kJvMYEVakqZVF2kGMysGno52O4m0KmpxiIhIQtTiEBGRhKjFISIiCVHiEBGRhChxiIhIQpQ4REQkIUocIiKSECUOERFJyP8HPvhYgz27fZwAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.semilogx(alpha_arr, train_err, 'b-o', label = 'test')\n",
    "plt.semilogx(alpha_arr, test_err, 'r-o', label = 'train')\n",
    "plt.xlim([np.max(alpha_arr), np.min(alpha_arr)])\n",
    "plt.title('Error vs. alpha')\n",
    "plt.xlabel('alpha')\n",
    "plt.ylabel('error')\n",
    "plt.legend()\n",
    "pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.4260869565217391 0.42727272727272725\n"
     ]
    }
   ],
   "source": [
    "min_train_err = np.min(train_err)\n",
    "min_test_err = np.min(test_err)\n",
    "print(min_train_err, min_test_err)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.06951928]\n"
     ]
    }
   ],
   "source": [
    "alpha_opt = alpha_arr[test_err == min_test_err]\n",
    "print(alpha_opt)\n",
    "alpha_opt = alpha_opt[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.4287220026350461\n",
      "0.42727272727272725\n"
     ]
    }
   ],
   "source": [
    "mlp_model = MLPClassifier(alpha = alpha_opt, hidden_layer_sizes = (100,2),\n",
    "                          solver = 'lbfgs', activation = 'logistic', random_state = 42)\n",
    "mlp_model.fit(X_train, y_train)\n",
    "\n",
    "y_train_pred = mlp_model.predict(X_train)\n",
    "y_test_pred = mlp_model.predict(X_test)\n",
    "print(np.mean(y_train != y_train_pred))\n",
    "print(np.mean(y_test != y_test_pred))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[  0   0   0 ...   0   0   0]\n",
      " [  0   0   0 ...   0   0   0]\n",
      " [  0   1 104 ...   0   0   0]\n",
      " ...\n",
      " [  0   0   0 ...  26   0   3]\n",
      " [  0   0   0 ...   0   0   0]\n",
      " [  0   0   0 ...   0   0   0]]\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import confusion_matrix\n",
    "print(confusion_matrix(y_test_pred, y_test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAATgAAAEKCAYAAACGzUnMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3X2cXFWd5/HPN4QQAvLYgYmJTEIWCQjyYL3CkzgZwCxIRnjtC1gZxAyijOsTqIwgugMzjgozPrGOA2ZFiQwqiLgy+BQ2wIorJHQHETRhIcBAA0JaeVQBM/72j3urU+lUV93qvlV97+3v+/XqV+reunXOqarOr8+9557zU0RgZlZFUya6AWZm3eIAZ2aV5QBnZpXlAGdmleUAZ2aV5QBnZpXlAGdmlTUhAU7SsZLuk/SApPMnog1mVn3q9Y2+krYC/h/wRmAQuBM4NSJ+2dOGmFnlTZ2AOhcCD0TEgwCSvgmcAIwa4Pr6+mLu3Lm9aZ3ZJPTwww8zNDSk8ZQhqZPe0o8i4tjx1JfFRAS42cCjDduDwCGtXjB37lz6+/u72iizyaxWq/W6yr5eVDIRAa7ZX4ktIr+ks4CzAPbYY49ut8nMciBl6wT26tLYRAwyDAKvatieAzw+8qCIWBYRtYiozZw5s2eNM7OxmzJlSqafnrWnZzVtciewl6R5kqYBbwFumIB2mFnOJGX66ZWen6JGxEZJ7wV+BGwFfCUifpHltUNDQ8OPjznmGABuvPHG4X3Tp08H4P777x/ed9hhh2322r6+1qf+I49rrLPZa+vPP/vss8P75s+f3+6ttKwTYP369aOW1dimVm3rhtWrVw8/XrhwYVfqyPJd3XbbbcOP582bB2z6/htft27dOgAWLFiQqe7BwcHNyhqtHbfffjuw6fcra7urrNfBK4uJuAZHRHwf+P5E1G1m3ZNXgJP0FWAJ8FRE7Jfu+yfgL4CXgfXAGRHxTMtyyrDgZa1WC4+imnVPrVajv79/XNFpypQpMW3atEzHvvTSSwMRMerQraQ3AC8AX2sIcIuBm9OzwEsAIuK8lm3K2ngzs3byugYXET8GfjNi34qI2Jhu3kEyQNnShJyimln1SOrlCOnbgWvaHeQAZ2a56eAaXJ+kxutOyyJiWcY6PgpsBK5ud6wDnJnlpoMAN9TqGlyL8peSDD4cHRkGEBzgzCw33bxNRNKxwHnAn0XE77K8xoMMZpabvAYZJH0DuB3YW9KgpDOBfwZeAdwk6WeSLm9XjntwZpaLPAcZIuLUJruv6LQcBzgzy41nMphZZTnA5aTTeX+nn346AB/96EeH92Wdn1hXnx/aaMcddxz1+F7OSazPoZwzZ9O9j/XP6MUXX9ziuW4Z+b2cd96mG83POOMMALbffvvhfWNtU/39Niujca5uHu/9nnvuAWD//fffoo5m33F9nuyRRx6Z6fh2853rmn3HI59rNoe2cf7wM88kM5sWL168Wf0bN24kDw5wZlZJRZxs77moZpbLXNStt946dt5550zHbtiwoeVc1Ly4B2dmuenlYpZZOMCZWW6KdoratXAr6SuSnpJ0b8O+XSTdJOn+9N9s/VkzK7ysN/n2Mgh2sz95JTAyLdj5wMqI2AtYmW6bWUVMmgDXbD0nkvyny9PHy4ETu1W/mfVe0QJcr6/B7R4RTwBExBOSdutx/WbWRR5kyMh5Uc3KpYj3wfU6wD0paVbae5sFPDXagenid8sguQ9u5PN5ZDD67ne/C8ARRxwxvO8zn/kMAJ/61KeAzbN2LVmyZIsy6rMbGmc0dNqmVneoT6Rmd9hnveu+0zo6LatxVkmrLGZZZxDceuutAJx00klA65kSI59vddxkU7QA1+v+5A3A0vTxUuC7Pa7fzLqoaNfgujaTIV3PaRHQBzwJXAj8L+BaYA/gEeDkiBg5ELEFz2Qw6648ZjJss802MXv27EzHPvTQQ+WeyTDKek4AR3erTjObWEU7RS3sIIOZlUuPs2pl4gBnZrlxD87MKssBzswqywHOzCrJN/qaWaU5wJlZZXkU1cwqyz24Hli3bt3w404zZ3U6N/Kqq64C4LjjjhveNxFzNPOYm5tFPcMUbJ5lajTt5nSOVat5so2fQacZsfJsW7fLb6yjVZ3d+g5G8jU4M6u0ogU4Z9Uys1zmos6YMSNe/epXZzr27rvvbjkXVdJXgCXAUxGxX7pvF+AaYC7wMHBKRDzdqp5iXRE0s1KbMmVKpp8MriSHlAcOcGaWizyTzuSV8qDy1+CyXHzN+8Lr4YcfDsBPf/rTLZ5bs2YNALvttmm19iz157HQZGMZdcuXLx9+fOKJye/LNttss0W7mn2OvRrY6Jbbbrtt+PGRRx652XONF+anT58OZH+fzRZSrb+23QDY6tWrAVi4cGGmujpVX9hz0aJFXSm/g2twfZIarzstSxe5baXjlAeVD3Bm1jsdBLihXqwH180FL18FfA34E+CPJBH60rFcKPQgg1l35THIsN1228VrXvOaTMfeeeedbRe8lDQXuLFhkOE+YFFDyoNbI2LvVmV08xrcRuBDEbEPcCjwHkn74tyoZpXV5SXLO0550M28qE9ExJr08fPAWmA2zo1qVkn1BS/zGEVNUx7cDuwtaVDSmcDFwBsl3Q+8Md1uqSfX4NKu5kHAKpwb1ayy8rrRN6+UB10PcJK2B74NnBMRz2X9AJwX1ax8ijaToav3wUnamiS4XR0R16e7n0wvENIqN2pELIuIWkTUZs6c2c1mmllOipY2sGsBTsm7uAJYGxGfbXjKuVHNKijPG33z0s1T1COA04F7JP0s3XcByYXBa9OLho8AJ3exDWbWQ0U7Re1mXtSfAKO9W+dGNasgL3hpZpU1aXpwZja5eMFLM6s0BzgzqywHODOrLA8ymFkl+RqcmVWaA5yZVZYDnJlVlgOcmVWWA5yZVVJ9wcsicYAzs9y4B2dmleUAZ2aV5QBnZpVUxBt9u7mi73RJqyXdLekXkv4u3T9P0ipJ90u6RtK0brXBzHqraCv6dnPI4yXgqIg4ADgQOFbSocAlwOfSvKhPA2d2sQ1m1kN5pQ3MrT3dKjgSL6SbW6c/ARwFXJfud15Us4ooVU4GSbu0emFE/KZd4ZK2AgaA/wR8EVgPPBMRG9NDBkmSQXfd6tWrAVi4cGGu5Q4NDQHQ19fXcl+37bDDDgA899xzWzz3gx/8AIDjjjtui+cGBweHH8+ZM6dtPevWrRt+vGDBgo7b2Wv19vayrevXrwdgxx13HN7Xy9+FVkb7fd24ceNoL+lI0a7BtRpkGCDpcTVrcQB7tis8Iv4DOFDSTsB3gH1GKWsLzotqVj6lCXARMS+vSiLiGUm3AocCO0mamvbi5gCPj/KaZcAygFqt1jQIdiLvnltds7/ME/HXulnPra5Zz60uS6+tURl6bY0mor3z58/veZ1Zjfb7OnVqPjdU5BngJH0AeAdJJ+ge4IyIeLGTMtpeg1PirZL+e7q9h6S20ULSzLTnhqRtgWOAtcAtwEnpYc6LalYR9alaeQwySJoNvB+oRcR+wFbAWzptU5ZBhn8BDgP+Mt1+nuR6WjuzgFsk/Ry4E7gpIm4EzgM+KOkBYFeS5NBmVgE5DzJMBbaVNBWYwShne+0KaOeQiDhY0l0AEfF0lnvXIuLnwEFN9j8IdOd80cwmVF6nqBHxmKRPkySH/z2wIiJWdFpOlh7cH9LR0IDk1BP4Y6cVmVn1ddCD65PU3/Bz1ohydgZOAOYBrwS2k/TWTtuTpQf3P0hGQHeX9AmS62cf67QiM6u+DnpwQxFRa/H8McBDEbEhLfd64HDgXztpT9sAFxFXSxoAjk53nRgRazupxMyqL+ebeB8BDpU0g+QU9Wigv9NCso4NzyAZxQhg204rMbPJIa9pWBGxStJ1wBpgI3AX6W1jHbWn3QGS/pZkStUuQB/wVUk+RTWzLeQ5ihoRF0bEgojYLyJOj4iXOm1Plh7cqcBB9RvsJF1MElX/odPKzKzaSjOTocHDwHSgfgfxNiRzSs3MhhVxPbhWk+2/QHLN7SXgF5JuSrffCPykN80zszIpTYBj04jFAMltInW3dq01ZlZqpcmqFRHLe9kQMyu/MvXgAJC0F/ApYF+Sa3EARETb5ZLMbPIo4jW4LP3JrwKXkdyL8ufA14CrutkoMyunoq3omyXAbRsRKwFFxL9HxEUky46bmW2maAEuy20iL0qaAtwv6b3AY8Bu3W2WmZVR0U5RswS4c0imar0f+DhJ721pNxtlZuVTX/CySLJMtr8zffgCcEZ3m2NmZVaaHpykf2OUhDAAEfHmLBWka8n1A49FxBJJ84BvksxtXQOcHhEvd9RqMyuk0gQ44NM51XE2SS6GHdLteuLnb0q6nCTx82U51WVmE6g0AS4i/s94C5c0Bzge+ARJHgaRXMOr53dYDlyEA5xZJRQtwHX7iuDngQ+zaYnzXcmY+FnSWfXljDds2NDlZprZeGW9RaRot4mMiaQlwFMRMSBpUX13k0ObXudrlxe1npE9a17PPDKcN8sKvmJFkgdj8eLFYy43j3asWbMGgIMPPnhMZY0sbzSNv5wDAwPA5om57777bgCOPjpZALqe5R025QvttM5mmpXR6nNpbGMeeWub1ZXluTzKb6bZ72GzMjr9f9Op0o2ijsMRwJslvYlkitcOJD26TImfzax8inaK2rVR1Ij4CPCRtKxFwLkRcZqkb5EkrvkmTvxsVilFC3CKaB7DJP1Z+vC/AH/Cpmw2pwIPR8QFmSvZFOCWSNqTTbeJ3AW8td1SxLVaLfr7O843YWYZ1Wo1+vv7xxWdZs2aFUuXZpsDcMkllwy0yaqVi7ajqJI+HhFvaHjq3yT9uJNKIuJW0nXknPjZrLqK1oPLcg1upqQ908BEeqPuzO42y8zKqIyDDB8AbpX0YLo9F/jrrrXIzEqrdD24iPhhuuhl/f6KdWNJ32Vm1VbEBS+zrOg7A/gg8KcR8U5Je0naOyJu7H7zzKxMihbgsq7o+zJwWLo9iHOimlkTRZvJkCXAzY+IfwT+ABARv6f5jAQzm+TKGOBelrQt6U2/kuaT5Eo1MxtWX/Ayy0/G8naSdJ2kdZLWSjqs/as2l2UU9SLgh8CrJF1NMgXLC1+a2RZy7p1dCvwwIk6SNI1kZfGOZBlFXSFpADiU5NT07IgYavMyM5uE8gpwknYA3gD8FUC6KG7HC+O27StKWhkRv46I70XEjRExJGllpxWZWfXleA1uT2AD8FVJd0n6sqTtOm3PqAFO0nRJuwB9knaWtEv6Mxd4ZacVmVn1dRDg+urrPaY/Z40oaipwMHBZRBwE/BY4v9P2tDpF/WuSjFqvBAbYNHL6HPDFTisys2rrcIR0qM1k+0FgMCJWpdvXkWeAi4hLgUslvS8ivtBpwWY2+eQ1FzUifiXp0XRSwX3A0cAvO25PhmP+KGmn+kZ6uvruTisys+rL+T649wFXS/o5cCDwyU7bkyXAvTMinqlvRMTTwDs7rcjMqi/PABcRP4uIWkS8NiJOTGNPR7LcBzdFkiJdGVNJntNpWQqX9DDwPPAfwMaIqKUDF9eQrEryMHDKWBpuZsVSxMn2WXpwPwKulXS0pKOAb5Dc+JvVn0fEgQ0XFM8HVkbEXsBKxnDh0MyKqWhTtbL04M4jGVH9byQjqSuAL4+jzhOARenj5SQr/Z43jvLMrCBKt+BlRPyRJDHzWJIzB7BCUgBfSlMB7h4RT6RlPyFpt2YvTO+LOQs2T/lmZsVVtFPUVlm1ro2IUyTdQ5PsWhHx2gzlHxERj6dB7CZJ67I2rF1eVDMrliJeg2vVgzs7/XfJWAuPiMfTf5+S9B2SZDNPSpqV9t5mAU+NtXwzK5aiBbhRT5gbTiP/vdlPu4IlbSfpFfXHwGLgXuAGknyo4LyoZpVSmkEGSc/TOvHzDm3K3h34TvpmpgJfT/M73EkyKnsm8AhwcsetNrNCKloPrtVUrXrv6++BXwFXkYyinga8ol3BaZrBA5rs/zXJtAszq5D6gpdFkuU2kf8cEYc0bF8maRXwj11qk5mVVNF6cFnC7X9IOk3SVpKmSDqNZGaCmdlminYNLkuA+0vgFODJ9OfkdJ+Z2WaKFuCy3Oj7MMnsAzOzlkp3iirp1ZJWSro33X6tpI91v2lmViZZe2+F6sEB/xP4G+BLABHxc0lfZ4KTP69evRqAhQsX5lru+vXrAdhxxx0BePHFF4efmzNnzhbHDw1tmX+nr6+vozoHBwcBmD59eqYy7rnnHgBmzZo15jo7aRdseu+N7zePOuuf9/z580c9prHO+vfxtre9bXjfzTffDMDtt98OwGGHbcoul6V82PReX3jhBQAWLFjQ8vgVK1YAsHjx4o7rGq/G7+Wll17qSZ1ZlXEUdUZErB4RdTd2qT1mVmJFO0XNEuCGlCR7rq8HdxLwRFdbZWalVLQAp3Qdy9EPkPYkmfR+OPA08BBwWpbpWnmp1WrR39/fq+rMJp1arUZ/f/+4otO8efPiwgsvzHTsGWecMdAm6UwuWvbgJE0BahFxTDqfdEpEPN/tRplZORWtB9fyimC6Ftx708e/dXAzs1amTJmS6adXslyDu0nSuSR5FH5b3xkRv+laq8ysdMq2Hlzd29N/39OwL4A982+OmZVZ6QJcRMzrRUPMrPxKF+AkTQfeDbyepOd2G3B5RLzY8oVmNumULsABXyPJbfqFdPtUkrXh2i5UKWknkgxc+5EEx7cD95FDXtRWd/OvW7cp9UP9jvT6vu233374uWYzEzp12223AbDPPvts0Y485T2DYKzqnzts+uxbtSePmR7NNJtl0czHPpbMKjznnHOG99VnQ7R6XVE+70b1NjVrT/33u90MjG4rY4DbOyIaF668RdLdGcu/FPhhRJwkaRowA7iAJC/qxZLOJ8mL6rSBZiVXxAUvs9zoeyXJKekd6fYhwNKIeHeb1+0A3A3sGQ2VSLoPWNSQdObWiNi7VVm+0desu/K40Xf+/Plx8cUXZzr2lFNOaXujr6StgH7gsYgYU/KrLD24Q4C3SXok3d4DWFtPJ9gifeCewAbgq5IOAAZIMnVlyotqZuWT8ynq2cBaoF3+l1FlCXDHjqPsg4H3RcQqSZeSnI5m4sTPZuWTV4CTNAc4HvgE8MGxlpPlNpGxzjkdBAYjYlW6fR1JgMuUF9WJn83KpcMbffskNV53Wpb+n6/7PPBhMiS4aiVLD25MIuJXkh6VtHdE3EeSSeuX6c9S4GKcF9WsUjoYZBga7RqcpCXAUxExIGnReNrTtQCXeh9wdTqC+iBwBsn810LkRW017D4eIxfNbCy/W3WOVX3hRth88cZemcjP45Of/OTw4wsuuKAnddZ/N6D5IpUjP4+st8N0qlu3weR0inoE8GZJbwKmAztI+teIeGunBXU1wEXEz4BmUdp5Uc0qKI8AFxEfAT6SlrcIOHcswQ2634Mzs0mirJPtJ9zGjRsZGhqakNOYsXTls6yP3+kd/t06pZiI09JGTzyRLA7d7D11+/S18bS0PiPlyCOPBGDNmjXDzx188MG51dlp7oQ8T0sbdeszzTvARcStwK1jfX0pApyZlYN7cGMwderUrvzFyVJm3vWOtbyiDErkbf/99x/1uV6+53rPrS7PXlsnyv49F22qVikCnJkVn6/BmVmlOcCZWWU5wJlZZTnAmVllOcCZWSUVccFLBzgzy417cGZWWQ5wZlZZDnBj0Mu5qJ3Of6wvZzOeOYN5lNErWefEZv0cs8zJLWKGq+XLlwNw/PHHD+/rtG1l+t6z8I2+ZlZpk2aQQdLeJPlP6/YE/pYkz2pHeVG7NRe1mU7ryeOvb5n+gmf9fPI8rii9tkZLly4ddxll+t6zKloPrmvhNiLui4gDI+JA4HXA74DvkORlWBkRewEr6SARjZkVW/00td1Pr/SqP3k0sD5NYHMCsDzdvxw4sUdtMLMuyhrcehngenUN7i3AN9LHzotqVlGT5hS1Lk0482bgWx2+7ixJ/ZL6N2zY0J3GmVmuJmMP7jhgTUQ8mW5PSF7UkZmuoHsXr6+99loATjnllK6U30z9/W2zzTbD++oXsevPtcriBN35POpLgcOmRSVXr149vG/hwoW51znR6u/zyiuvHN5X/16mT58+vG8iBk/WrVsHwIIFC4b3DQ4O8vLLL+dSftFGUXvRmlPZdHoKcANJPlRwXlSzyijiNThFdC9pvKQZwKPAnhHxbLpvV+BaYA/SvKgR8ZtW5dRqtejv7291iJmNQ61Wo7+/f1yRZ999942rrroqa30DoyV+zlO386L+Dth1xL5f47yoZpVUtEEGz2Qws9w4wJlZZTnAmVklFXHBy2K1xsxKLa9RVEmvknSLpLWSfiHp7LG0xz04M8tNjqeoG4EPRcQaSa8ABiTdFBG/7KQQBzgzy01eAS6dzlmf0vm8pLXAbMABzsx6r1s38UqaCxwErOr0tQ5wZpabDgYZ+iQ13r2/LJ2euRlJ2wPfBs6JiOc6bY8DnJnlpoMe3FC7mQyStiYJbldHxPVjaY8DnJnlJq9TVCUFXQGsjYjPjrUc3yZiZrnIebL9EcDpwFGSfpb+vKnTNrkHZ2a5yXEU9SfAuAtzgDOz3HiqlplVVtGmajnAmVkuipj4uavhVtIH0nlk90r6hqTpkuZJWiXpfknXpDkbzKwCiraib9cCnKTZwPuBWkTsB2xFkl3rEuBzaV7Up4Ezu9UGM+utSRPgUlOBbSVNBWaQzC07Crgufd55Uc0qZNIEuIh4DPg0Sd6FJ4BngQHgmYjYmB42SDKB1swqYNIEOEk7k2Sxnwe8EtiOJIXgSE2z3sh5Uc1Kpb7gZZafXulmTccAD0XEhoj4A3A9cDiwU3rKCjAHeLzZiyNiWUTUIqI2c+bMLjbTzPIyaXpwJKemh0qakc4rO5pkLadbgJPSY5wX1axCJk2Ai4hVJIMJa4B70rqWAecBH5T0AElKwSu61QYz662iBbhu50W9ELhwxO4HgYXdrNfMeq+IN/p6JoOZ5cZTtSbI4OAgAHPmzBl3WUNDQ8OP7733XgAWLVo07nJb1dXX19fyuDzfX5Z6AKZPnw60b9tEWrFiBQCLFy8e3tfss8r6OWfRq++iUX0g7o477hjeN3/+fABWrlw5vO+AAw4AuveduQdnZpVVtACniKa3oRVKrVaL/v7+9gea2ZjUajX6+/vHFZ0OOuiguPnmmzMdu8suuwy0W7I8D+7BmVluitaDc4Azs9w4wJlZJdWnahWJA5yZ5cY9ODOrLAc4M6ssBzgzqywHuIppdgf8yDvZG2c+jPUO8sYZBM3ukG91J36W57K2rd3xI+tq1+6xatWOsXzeWWYf5PE9tpPnjIpmLr/8cgDe9a53De9bv349L7300rjL9lxUM6u0oo2ieiaDmeUyk+F1r3tdNM6FbWXatGk9mclQrHBrZqWW53pwko6VdJ+kBySdP5b2OMCZWS6yBrcsAU7SVsAXSfK47AucKmnfTtvkAGdmucmxB7cQeCAiHoyIl4FvkiSx6ogHGcwsNzkOMswGHm3YHgQO6bSQUgS4gYGBIUm/BYbaHlxcfZS7/VD+9+D2j+5Px1vAwMDAjyRlvb9luqTGkcNlEbGsYbtZN6/jEdFSBLiImCmpvxejLt1S9vZD+d+D299dEXFsjsUNAq9q2B41xWgrvgZnZkV0J7CXpHmSpgFvAW7otJBS9ODMbHKJiI2S3gv8CNgK+EpE/KLTcsoU4Ja1P6TQyt5+KP97cPtLJCK+D3x/PGWUYiaDmdlY+BqcmVVWKQJcHlM2eknSqyTdImmtpF9IOjvdv4ukmyTdn/6780S3tRVJW0m6S9KN6fY8SavS9l+TXvwtJEk7SbpO0rr0ezishJ//B9Lfn3slfUPS9DJ9B0VQ+ACX15SNHtsIfCgi9gEOBd6Ttvl8YGVE7AWsTLeL7GxgbcP2JcDn0vY/DZw5Ia3K5lLghxGxADiA5H2U5vOXNBt4P1CLiP1ILrS/hXJ9BxOu8AGOnKZs9FJEPBERa9LHz5P855pN0u7l6WHLgRMnpoXtSZoDHA98Od0WcBRwXXpIYdsvaQfgDcAVABHxckQ8Q4k+/9RUYFtJU4EZwBOU5DsoijIEuGZTNmZPUFs6JmkucBCwCtg9Ip6AJAgCu01cy9r6PPBh4I/p9q7AMxGxMd0u8vewJ7AB+Gp6iv1lSdtRos8/Ih4DPg08QhLYngUGKM93UAhlCHC5TNmYCJK2B74NnBMRz010e7KStAR4KiIGGnc3ObSo38NU4GDgsog4CPgtBT4dbSa9PngCMA94JbAdyWWakYr6HRRCGQJcLlM2ek3S1iTB7eqIuD7d/aSkWenzs4CnJqp9bRwBvFnSwySXBI4i6dHtlJ4uQbG/h0FgMCJWpdvXkQS8snz+AMcAD0XEhoj4A3A9cDjl+Q4KoQwBLpcpG72UXq+6AlgbEZ9teOoGYGn6eCnw3V63LYuI+EhEzImIuSSf980RcRpwC3BSeliR2/8r4FFJe6e7jgZ+SUk+/9QjwKGSZqS/T/X3UIrvoChKcaOvpDeR9CDqUzY+McFNaknS64HbgHvYdA3rApLrcNcCe5D8Ap8cEb+ZkEZmJGkRcG5ELJG0J0mPbhfgLuCtETH+bCVdIOlAkgGSacCDwBkkf9BL8/lL+jvgv5KMyt8FvIPkmlspvoMiKEWAMzMbizKcopqZjYkDnJlVlgOcmVWWA5yZVZYDnJlVlgPcJJWutvHuLpb/V5L+uc0xF0k6t8NyXxhfy2wycYCbvHYCmga4dAUXs9JzgJu8LgbmS/qZpH+StChdw+7rwD2S5kq6t36wpHMlXZQ+ni/ph5IGJN0maUGriiT9RbqG2V2S/rek3RuePkDSzen6Zu9seM3fSLpT0s/TG17NOlamnAyWr/OB/SLiQBiesbAw3fdQugrKaJYB74qI+yUdAvwLyXzV0fwEODQiQtI7SFYp+VD63GtJ1szbDrhL0veA/YC90vYIuEHSGyLix2N6pzZpOcBZo9UR8VCrA9IVUg4HvpVMkQRgmzblzgGuSSe4TwMa6/huRPwe+L2kW0iC2uuBxSRTkQC2Jwl4DnDWEQc4a/Tbhscb2fwSxvT03ykka5Id2EHrWEQOAAAA7ElEQVS5XwA+GxE3pD3FixqeGzlXMEh6bZ+KiC91UIfZFnwNbvJ6HnhFi+efBHaTtKukbYAlAOm6dg9JOhmSlVMkHdCmrh2Bx9LHS0c8d0Kaa2BXYBHJ6jE/At6e9haRNFtSYRentOJyD26SiohfS/q/6UDCD4DvjXj+D5L+nmQFlIeAdQ1PnwZcJuljwNYkq1vc3aK6i0hOaR8D7iBZxLFudVr3HsDHI+Jx4HFJ+wC3p6fBLwBvpdjrt1kBeTURM6ssn6KaWWU5wJlZZTnAmVllOcCZWWU5wJlZZTnAmVllOcCZWWU5wJlZZf1/BrAm/WIwWeYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "M = confusion_matrix(y_test_pred, y_test)\n",
    "M = np.sqrt(M)\n",
    "plt.imshow(M, interpolation = 'nearest')\n",
    "plt.set_cmap('binary')\n",
    "plt.xlabel(\"true label\")\n",
    "plt.ylabel(\"predicted label\")\n",
    "plt.colorbar()\n",
    "pass"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Стало только хуже :("
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Поскольку в описании к датасету указано, что удалось добиться 80% точности предсказания, первый опыт считаю удачным."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}