churn/3. Feature Engineering.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction: Automated Feature Engineering with Featuretools\n",
    "\n",
    "__Problem:__ we have a set of cutoff times and labels - in a label times table - and we need to build relevant features for each label using only data from before the cutoff time. Traditionally, we would do this by hand, a painstaking and error prone process that makes developing useable machine learning solutions extremely difficult. \n",
    "\n",
    "__Solution__: Use automated feature engineering as implemented in Featuretools to build hundreds or thousands of relevant features from a relational dataset with a reusable framework that also automatically filters the data based on the cutoff times. This approachs overcomes the limitations of manual feature engineering, letting us buidl better predictive models in a fraction of the time. \n",
    "\n",
    "The general process of feature engineering is shown below:\n",
    "\n",
    "![](../images/feature_engineering_process.png)\n",
    "\n",
    "Currently, the only option for automated feature engineering using multiple related tables is [Featuretools](https://github.com/Featuretools/featuretools), an open-source Python library. \n",
    "\n",
    "![](../images/featuretools-logo.png)\n",
    "\n",
    "In this notebook, we'll work with Featuretools to develop an automated feature engineering workflow for the customer churn dataset. The end outcome is a function that takes in a dataset and label times for customers and builds a feature matrix that can be used to train a machine learning model. Because we already partitioned the data into independent subsets (in `Partitioning Data`) we'll be able to apply this function to all of the partitions in parallel using Spark with PySpark.\n",
    "\n",
    "## Featuretools Resources\n",
    "\n",
    "We won't spend too much time on the basics of Featuretools here, so refer to the following sources for more information:\n",
    "\n",
    "* [Featuretools Documentation](https://docs.featuretools.com/)\n",
    "* [Featuretools GitHub](https://github.com/Featuretools/featuretools)\n",
    "* [Introductory tutorial on Featuretools](https://towardsdatascience.com/automated-feature-engineering-in-python-99baf11cc219)\n",
    "* [Why Automated Feature Engineering Will Change Machine Learning](https://towardsdatascience.com/why-automated-feature-engineering-will-change-the-way-you-do-machine-learning-5c15bf188b96)\n",
    "\n",
    "The basics are relatively easy to pick up, and if you're new, you can probably follow along with all the code here! Learning Featuretools requires only a few minutes and it can be applied to any relational dataset.\n",
    "\n",
    "\n",
    "With that in mind, let's get started."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Data science helpers\n",
    "import pandas as pd \n",
    "import numpy as np\n",
    "\n",
    "import featuretools as ft\n",
    "\n",
    "# Useful for showing multiple outputs\n",
    "from IPython.core.interactiveshell import InteractiveShell\n",
    "InteractiveShell.ast_node_interactivity = \"all\"\n",
    "\n",
    "N_PARTITIONS = 1000"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "All of the data is stored on S3. This makes it possible to read and write directly from any computer without needing to worry about losing data if the computer (in this case EC2 instances) is shut down. To access, first configure AWS from the command line using `aws configure`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "PARTITION = '50'\n",
    "BASE_DIR = 's3://customer-churn-spark/'\n",
    "PARTITION_DIR = BASE_DIR + 'p' + PARTITION"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Read in all data\n",
    "members = pd.read_csv(f'{PARTITION_DIR}/members.csv', \n",
    "                      parse_dates=['registration_init_time'], \n",
    "                      infer_datetime_format = True, \n",
    "                      dtype = {'gender': 'category'})\n",
    "\n",
    "trans = pd.read_csv(f'{PARTITION_DIR}/transactions.csv',\n",
    "                   parse_dates=['transaction_date', 'membership_expire_date'], \n",
    "                    infer_datetime_format = True)\n",
    "\n",
    "logs = pd.read_csv(f'{PARTITION_DIR}/logs.csv', parse_dates = ['date'])\n",
    "\n",
    "cutoff_times = pd.read_csv(f'{PARTITION_DIR}/MS-31_labels.csv', parse_dates = ['cutoff_time'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The 3 data tables are represented by the following schema. \n",
    "\n",
    "![](../images/data_schema.png)\n",
    "\n",
    "This schema is all the domain knowledge needed to perform automated feature engineering in Featuretools."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Define Entities and EntitySet\n",
    "\n",
    "The first step in using Featuretools is to make an `EntitySet` and add all the `entitys` - tables - to it. An EntitySet is a data structure that holds the tables and the relationships between them. This makes it easier to keep track of all the data in a problem with multiple relational tables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "import featuretools.variable_types as vtypes\n",
    "\n",
    "# Make empty entityset\n",
    "es = ft.EntitySet(id = 'customers')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Entities\n",
    "\n",
    "When creating entities from a dataframe, we need to make sure to include:\n",
    "\n",
    "* The `index` if there is one or a name for the created index. This is a unique identifier for each observation.\n",
    "* `make_index = True` if there is no index, we need to supply a name under `index` and set this to `True`.\n",
    "* A `time_index` if present. This is the time at which the information in the row becomes known. Featuretools will use the `time_index` and the `cutoff_time` to make valid features for each label.\n",
    "* `variable_types`. In some cases our data will have variables for which we should specify the type. An example would be a boolean that is represented as a float. This prevents Featuretools from making features such as the `min` or `max` of a True/False varaibles.\n",
    "\n",
    "For this problem these are the only arguments we'll need. There are additional arguments that can be used as shown in [the documentation](https://docs.featuretools.com/api_reference.html#entityset-entity-relationship-variable-types). \n",
    "\n",
    "### Members Table\n",
    "\n",
    "The `members` table holds basic information about each customer. The important point for this table is to specify that the `city` and `registered_via` columns are discrete, categorical variables and not numerical and that `registration_init_time` is the `time_index`. The `msno` is the unique index identifying each customer. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>msno</th>\n",
       "      <th>city</th>\n",
       "      <th>bd</th>\n",
       "      <th>gender</th>\n",
       "      <th>registered_via</th>\n",
       "      <th>registration_init_time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>8hW4+CV3D1oNM0CIsA39YljsF8M3m7g1LAX6AQd3C8I=</td>\n",
       "      <td>4</td>\n",
       "      <td>24</td>\n",
       "      <td>male</td>\n",
       "      <td>3</td>\n",
       "      <td>2014-11-04</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>yhcODfebyTYezE6KAPklcV1us9zdOYJ+7eHS7f/xgoU=</td>\n",
       "      <td>8</td>\n",
       "      <td>37</td>\n",
       "      <td>male</td>\n",
       "      <td>9</td>\n",
       "      <td>2007-02-11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>sBlgSL0AIq49XsmBQ2KceKZNUyIxT1BwSkN/xYQLGMc=</td>\n",
       "      <td>15</td>\n",
       "      <td>21</td>\n",
       "      <td>male</td>\n",
       "      <td>3</td>\n",
       "      <td>2013-02-08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Xy3Au8sZKlEeHBQ+C7ro8Ni3X/dxgrtmx0Tt+jqM1zY=</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>9</td>\n",
       "      <td>2015-02-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>NiCu2GVWgT5QZbI85oYRBEDqHUZbzz2azS48jvM+khg=</td>\n",
       "      <td>12</td>\n",
       "      <td>21</td>\n",
       "      <td>male</td>\n",
       "      <td>3</td>\n",
       "      <td>2015-02-12</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                           msno  city  bd gender  \\\n",
       "0  8hW4+CV3D1oNM0CIsA39YljsF8M3m7g1LAX6AQd3C8I=     4  24   male   \n",
       "1  yhcODfebyTYezE6KAPklcV1us9zdOYJ+7eHS7f/xgoU=     8  37   male   \n",
       "2  sBlgSL0AIq49XsmBQ2KceKZNUyIxT1BwSkN/xYQLGMc=    15  21   male   \n",
       "3  Xy3Au8sZKlEeHBQ+C7ro8Ni3X/dxgrtmx0Tt+jqM1zY=     1   0    NaN   \n",
       "4  NiCu2GVWgT5QZbI85oYRBEDqHUZbzz2azS48jvM+khg=    12  21   male   \n",
       "\n",
       "   registered_via registration_init_time  \n",
       "0               3             2014-11-04  \n",
       "1               9             2007-02-11  \n",
       "2               3             2013-02-08  \n",
       "3               9             2015-02-01  \n",
       "4               3             2015-02-12  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "members.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "members['msno'].is_unique"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Entityset: customers\n",
       "  Entities:\n",
       "    members [Rows: 6658, Columns: 6]\n",
       "  Relationships:\n",
       "    No relationships"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create entity from members\n",
    "es.entity_from_dataframe(entity_id='members', dataframe=members,\n",
    "                         index = 'msno', time_index = 'registration_init_time', \n",
    "                         variable_types = {'city': vtypes.Categorical, \n",
    "                                           'registered_via': vtypes.Categorical})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Transactions Table\n",
    "\n",
    "The transactions table contains payments made by the customers. Each row records one payment. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>msno</th>\n",
       "      <th>payment_method_id</th>\n",
       "      <th>payment_plan_days</th>\n",
       "      <th>plan_list_price</th>\n",
       "      <th>actual_amount_paid</th>\n",
       "      <th>is_auto_renew</th>\n",
       "      <th>transaction_date</th>\n",
       "      <th>membership_expire_date</th>\n",
       "      <th>is_cancel</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>5F7G3pHKf5ijGQpoKuko0G7Jm3Bde6ktfPKBZySWoDI=</td>\n",
       "      <td>41</td>\n",
       "      <td>30</td>\n",
       "      <td>99</td>\n",
       "      <td>99</td>\n",
       "      <td>1</td>\n",
       "      <td>2017-02-10</td>\n",
       "      <td>2017-03-10</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>DQMPoCSc6EB39ytgnKCRsUIZnR6ZWSrHeDmX7nbxAKs=</td>\n",
       "      <td>41</td>\n",
       "      <td>30</td>\n",
       "      <td>149</td>\n",
       "      <td>149</td>\n",
       "      <td>1</td>\n",
       "      <td>2016-02-01</td>\n",
       "      <td>2016-03-02</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Lrais3nsgqYwpfpSoyK3fHuPutf6cloTI5T5dQfs4lA=</td>\n",
       "      <td>38</td>\n",
       "      <td>30</td>\n",
       "      <td>149</td>\n",
       "      <td>149</td>\n",
       "      <td>0</td>\n",
       "      <td>2016-02-23</td>\n",
       "      <td>2016-04-23</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>ZPOjgxQw1/J7v5xgBJTCLXWuwq5Xmk33nO6AoUO1+mY=</td>\n",
       "      <td>41</td>\n",
       "      <td>30</td>\n",
       "      <td>149</td>\n",
       "      <td>119</td>\n",
       "      <td>1</td>\n",
       "      <td>2015-09-06</td>\n",
       "      <td>2016-08-01</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>MvR23u4bIiWM+U+VE1Mvw3qqdj/0Ixs1sf7avavjhRs=</td>\n",
       "      <td>38</td>\n",
       "      <td>30</td>\n",
       "      <td>149</td>\n",
       "      <td>149</td>\n",
       "      <td>0</td>\n",
       "      <td>2016-10-28</td>\n",
       "      <td>2016-11-27</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                           msno  payment_method_id  \\\n",
       "0  5F7G3pHKf5ijGQpoKuko0G7Jm3Bde6ktfPKBZySWoDI=                 41   \n",
       "1  DQMPoCSc6EB39ytgnKCRsUIZnR6ZWSrHeDmX7nbxAKs=                 41   \n",
       "2  Lrais3nsgqYwpfpSoyK3fHuPutf6cloTI5T5dQfs4lA=                 38   \n",
       "3  ZPOjgxQw1/J7v5xgBJTCLXWuwq5Xmk33nO6AoUO1+mY=                 41   \n",
       "4  MvR23u4bIiWM+U+VE1Mvw3qqdj/0Ixs1sf7avavjhRs=                 38   \n",
       "\n",
       "   payment_plan_days  plan_list_price  actual_amount_paid  is_auto_renew  \\\n",
       "0                 30               99                  99              1   \n",
       "1                 30              149                 149              1   \n",
       "2                 30              149                 149              0   \n",
       "3                 30              149                 119              1   \n",
       "4                 30              149                 149              0   \n",
       "\n",
       "  transaction_date membership_expire_date  is_cancel  \n",
       "0       2017-02-10             2017-03-10          0  \n",
       "1       2016-02-01             2016-03-02          0  \n",
       "2       2016-02-23             2016-04-23          0  \n",
       "3       2015-09-06             2016-08-01          0  \n",
       "4       2016-10-28             2016-11-27          0  "
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trans.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAsYAAAGSCAYAAADgnwfJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs/XmcznX////fhlSDMmEWylqT/dRqLWU5MbwLb7K0K5/B0EKI6l1nZzpRTsspTZzqRIlKlnESJc6sGWcpQkz5pshek+VEluP3h8scP8c5Y5mZg7HcrpeLy6Vez+fxej1fj3kZ9+N1PF/PIyI9PT2AJEmSdJHLl9cDkCRJks4FBmNJkiQJg7EkSZIEGIwlSZIkwGAsSZIkAQZjSZIkCTAYS+e8AQMGEBUVxcKFC/Pk+AsXLiQqKooBAwaEbG/evDlRUVFs3LgxT8YFsHHjRqKioujatWuejSFcvv/+e+6//34qVqzIVVddRVRUVF4P6ZxwoutPF55q1apRrVq10+5/If3917nDYCydBVFRUSF/oqOjKV++PHXr1qVLly7MmDGDQ4cOnZFjd+3aNU+DdW5k/MPXvHnzvB7KGXXkyBHuu+8+Zs6cSb169ejduzdPP/10tvaRnJwcvL7+/e9/h2VcGW/KJkyYEJb95YX77ruPqKgoypYty4EDB/J6OHmuWrVqOXrTNWHChCx/j1WtWpVHHnmEL7744gyMVjr7LsnrAUgXk4ywc+TIEXbv3k1aWhrTp09n0qRJxMfHM3r0aG688caQ1yQmJtK6dWuuueaavBgyN998M6mpqRQrVixPjn8yJUuWJDU1lSuvvDKvh5IrGzdu5Ntvv6Vhw4aMHj06R/sYN24cERERBAIBxo4dyy233BLmUZ5/tmzZwpw5c4iIiCA9PZ1p06bRvn37vB7Wea1q1arBN6p79+5l+fLlTJkyhenTp/P222/TrFmzHO87JSUlXMOUcsxgLJ1F/fr1y7QtPT2dl156iTfffJOWLVvy6aefct111wXbixUrlqehtGDBglx//fV5dvyTKVCgwDk7tuzYsmULADExMTl6/ZIlS/j222+55557WLp0KVOnTuUvf/nLef+GIbfefvttDh8+TI8ePRg6dCjjxo0zGOdStWrVMv0e69+/P4MHD+aZZ57JVTAuV65cbocn5ZpTKaQ8FhUVxV//+lfatm3Lb7/9xgsvvBDSfqI5xkuWLKF9+/ZUqVKFmJgYrrvuOu68806effZZAoFj3/RerVo1Jk6cCMBdd90V8jFohuOnWkyaNIn69etTsmRJbrvtNuDUczwDgQAjRozglltuITY2lipVqvDss8+yZ8+eLM/1RNMi/vs8J0yYQPXq1QFYvHhxyNgzxnKyOYbbt2+nT58+VK9enZiYGMqVK0e7du1YvHhxpr4Z59i1a1c2btzII488Qvny5YmNjeXOO+9k9uzZWY75ZBYuXEjbtm0pV64cMTExVK9enb59+7Jz584T1mTixImZzvF0jB07FoD777+fDh06sG/fPj744IOTvmbq1Km0bNkyOL6qVavy0EMPsXTpUuDYHPJBgwYB0K1bt5D6Z8wrz7h2sppnfqKfzXfffcef/vQn7rzzTq699trgsR977DF++umn0z7nUzl69Chvv/02BQsWpGfPntSuXZulS5eybt26LPsf//dg8uTJ3HHHHZQoUYKKFSvyzDPPcPDgQQA+++wz/ud//odSpUpRunRpEhMT+eWXX7Lc58qVK3n44YeJj48nOjqaKlWq0L17d3744YcTHj87tTx+zNOnT6dBgwaUKFGCsmXL8sgjj/Dzzz9n2kdGjY//eeZ2qlLnzp0B+OGHH9i1axcAv//+O6NHj+aee+6hatWqxMTEUKZMGe6+++4T/n060RzjPXv28Mwzz1C5cmViY2O59dZbee2114K/56Rw8o6xdI7o168f77//PrNnz2b37t0nvds3d+5c2rZtS+HChUlISODqq68mPT2d77//nlGjRvHiiy9yySWX0LVrV959912++eYbOnToQOnSpU+4z9dee43PPvuMhIQE7rjjDn7//ffTGnffvn35/PPPadWqFVdeeSWffPIJI0eO5PPPP2fWrFlcdtll2a4FHPtHskuXLrzxxhuUKlWKe++9N9iWEdpP5McffyQhIYHNmzdTt25d/vd//5etW7cybdo0PvnkE0aMGMF9992X6XU//fQTDRs2pGzZsrRr145ff/2VqVOncu+99zJt2jTq1at3WmMfP348TzzxBJGRkbRo0YK4uDiWLVvGG2+8wT//+U/mzJnD1VdfDRybXvPjjz8yceLEkI+pT3WOGX799VdSUlIoVaoU9erVo0yZMgwePJhx48bx6KOPZuofCARISkpi4sSJFC1alGbNmhETE8PPP//MkiVLmD59OrVr1w7We/HixTRr1iwksBQpUuS0xpaVGTNm8NZbb3H77bdTo0YNLr30Ur799lveeecdZs+ezb/+9a9gbXJj7ty5/PTTT7Rr144rrriC++67j6VLlzJ27NiTvukYPXo08+fPp1mzZtSpU4fZs2fz+uuv89tvv9G4cWM6d+5M06ZNefDBB/nss894//33+eWXX5g8eXLIfj755BPuv/9+jhw5wl133UW5cuVYvXo177zzDv/85z9JSUnhD3/4Q67PE+DNN9/ko48+IiEhgbp16/Lvf/+bKVOm8M0337Bw4UIuu+wyihQpwtNPP01ycjK7d+8OmcN+st8Lp+P4gBoREQEcuy779u1LzZo1qV+/PsWLF2fr1q3Mnj2b9u3bM3ToUDp27HjKfR88eJAWLVrw5ZdfUrlyZe655x52797N4MGDs3yTK+WWwVg6R5QrV46rr76azZs389VXX500hI0bN46jR4/yz3/+M9M/rr/88guXXHLsr3ZSUhKrVq3im2++4d577+X2228/4T4XLlzIxx9/nO1/rFNTU1m4cCGlSpUC4Pnnn+eBBx5g1qxZjBw5kp49e2Zrfxn+8Ic/UKRIEd544w1Kly6d5TSUE+nZsyebN2+mb9++9O3bN7i9e/fuNGrUiJ49e3LnnXdmCmCLFi3K9Jp77rmH1q1bM2LEiNMKxps2baJXr14ULFiQuXPnUqlSpWBbxkfOPXv25L333gOOvSFauHAhEydOzPJj6lOZOHEiBw4coEOHDkRERFC2bFnq1KnD4sWL+fLLL7nppptC+o8bN46JEydyww03MG3atJBPD44ePcrWrVuBYw+t/fjjjyxevJjmzZtn+UYiJ9q1a0dSUlKmN0zz5s2jTZs2DB48mKFDh+b6OBl30TPG3bJlS55++mkmTZrEn/70pxO+YVu4cCELFiygfPnywLE3fjfddBMTJ07ko48+YsaMGdx6663Asbuid9xxB3PnzmXlypXBvzv79u2jS5cuHDp0KNMbqvHjx/P444/TpUsXFi9eHAySufHpp58yb948qlSpEtzWqVMnJk+ezKxZs2jVqhVRUVH069ePd999l927d2f7OjuZUaNGAcd+hxUtWhQ4dkd61apVmf6O/fbbbzRt2pQXX3yR9u3bExkZedJ9v/baa3z55Zc0a9aMd955h3z5jn3Q3aNHD+68886wnYOUwakU0jmkRIkSAMGPI08lq39UMv5hyq6HHnooR3ewunTpEgzFAPnz5+fFF18kIiKCd955J0djyY2ff/6ZuXPncvXVV2cK5VWqVOGRRx7h4MGDwWB6vFKlStG7d++QbQ0bNuSaa6457afuJ02axO+//86jjz4aEooBevXqRYkSJZgzZ05wXnFuZTx0d/wd9YwwmBEOj5fxcN+QIUMyrU6QL18+SpYsGZZxnUjJkiWzDKUNGjSgYsWKzJs3L9fHyHjornTp0sE3g4ULF6ZFixb8+uuvTJ8+/YSv7dy5czAUw7G74wkJCRw9epQmTZoEQzHApZdeSsuWLQH45ptvgttnzpzJrl27uPvuuzO9mXrwwQepXr06a9asYfny5bk+14wxHx+KM44DhH21iFWrVjFgwAAGDBjAs88+S+PGjfnrX/9K/vz5efnll4P9Lrvssizv/BcpUoT77ruP9PR0vvzyy1Meb8KECURERPDiiy8GQzEcu8udMYVDCieDsXQOyfhI8lR3ke655x7gWGh74oknmDx5cq7XE7755ptz9Lq6detm2hYfH09MTAwbNmzIcq7xmfT1118DULNmTS699NJM7Rl3mTL6Ha9atWrkz58/0/ZrrrmG9PT0bB0/q7vLl19+ObVq1QKOzT/NrSVLlrBu3Trq1KlD2bJlg9tbtGhB4cKFmTJlSkj99+3bx5o1ayhatGimO8lnSyAQ4L333qNFixZce+21FCtWLDjXdc2aNWF5w/D2229z5MiR4F30DCd7w5AhqzeHcXFxJ2zLeDN7/Hzek10DcPJrMCduuOGGTNsyVrE53ev2dH3zzTcMGjSIQYMGMXr0aDZt2kSrVq2YM2dOpgfv1q5dS9euXalevTpxcXHBn/Nzzz0HcMqf9Z49e9iwYQOxsbHEx8dnas/qd4+UW06lkM4hGR9jFy9e/KT97r77bt5//31GjhzJxIkTGTduHACVK1fm6aefpkWLFtk+dk5XRDjR66Kjo9m2bRt79uzhiiuuyNG+c2L37t0nHVdsbCxw7CPd/3aiubP58+fn6NGjZ/z42ZUR8I6/WwxQqFAhWrZsyTvvvMPkyZODczkzjpkR5vLCM888Q3JyMnFxcTRs2JASJUpw+eWXA/Duu+/m+gG8o0ePMn78eCIiIujQoUNIW506dShXrhxLliwhLS0ty7CV1dz+jDdLJ2s7fh3ys3kNQNbXbca4jhw5EpZjZOjQoQPJycmn7Ld8+XLuvvtuDh8+zB133EFCQgJXXHEF+fLlY9WqVcyaNSv4QOOJZNQxOjo6y/ac/s6STsZgLJ0jvv/+ezZv3swll1yS5R2g/9a4cWMaN27M/v37+fLLL5k7dy5jxozh4YcfZsaMGaf98FaGnM513L59e5YBY8eOHQAhoTgiIuKE/1CHKyRkhJft27dn2b5t27aQfuF2to5//JSAbt260a1btyz7jR07NhiMMwJUuKZxZHy0ndXPNKuf544dOxg1ahSVK1dmzpw5md4wffjhh7ke0yeffMKmTZuArO+kZhg7dmzIR//hlJNrILu1PNcNHjyY/fv3M2PGjEzPNgwZMoRZs2adch8Z9cn4XfLfTlRfKTcMxtI5ImNprGbNmlG4cOHTfl1kZCR169albt26lClThieffJKZM2cGg3HGnaPTveOZXYsXL870kWZaWhrbt2+nfPnyIeEnKioqGFr+24oVKzJty8nYMz7uXrZsGb///num6RSfffYZcPLQlBvVq1dnxowZLFy4kEaNGoW0HTx4kGXLlgX75ca7777LwYMHqVat2gnP5dNPP+Xrr7/mq6++4oYbbqBQoUJUrlyZNWvWZPlg3n871V3HjDnKmzZtCpmXC1n/PH/44QeOHj1K/fr1M4XizZs3Z7mMWXZl3EX/4x//GJwC8d/effddJk2axPPPP5/jVVNOJuNnu3DhQh555JFM7QsWLABCr8Hs1jKnjv+ZZjVtKFw2bNjAVVddleUDv6e7msQVV1xB+fLl+f/+v/+P7777LmR99+zsR8oO5xhLeSw9PZ2nnnqK999/n6uuuoo//elPp3zNokWLOHz4cKbtGXeiChYsGNyW8TBeONeIPd4bb7wRsu8jR47wwgsvEAgEMq1kcMstt7Bp0yY+/vjjkO3jxo0LBsbjRUVFERERccIwnZWrr76ahg0bsnnzZoYPHx7StnbtWt566y0uu+wy2rZte9r7zI62bdty6aWX8uabb7J+/fqQtiFDhvDzzz/TuHHjXE9nyJg+M2jQIEaMGJHln4x1b4+fU5vxwFLPnj0zzT8NBAIhd5Mzrp0T1T/j2/XGjh0bsmTXxo0bg2/0jpexLNjnn38eErb37t3LE088keU1nR0///wzH3/8MVdeeSVjx449YV0aNWrErl27mDFjRq6OdyLNmzenaNGiTJ8+PVN4mzBhAitWrKBSpUohD/Jlt5Y5daZ/H2QoXbo0v/76a8hDiXBsVY5PP/30tPdz3333EQgEeP7550PeIP/444/B1TCkcPKOsXQWZayfevTo0eBXQi9dupT9+/dToUIFRo0aleluUVb69u3L5s2bqVWrFqVLl+byyy9n9erVfPrppxQtWpSHHnoo2Ld+/fr87W9/489//jNr164N3pn679UXcqpGjRrcfvvtIesYr1mzhptuuonu3buH9H3sscf49NNPuf/++2nZsiXR0dF89dVXfPXVVzRp0oQ5c+aE9C9cuDA1atRg2bJltGvXjurVq1OgQAHq1Klz0gdvhgwZQtOmTXn55ZdZsGABt956a3Ad4wMHDjBs2LAz9hXbpUuXZtCgQfTs2ZP69evTsmVLYmNjWbZsGYsXL+bqq6/mr3/9a66OsXjxYtavX8/1119PnTp1TtivQ4cOvPTSS3z44Yf079+fwoUL8+CDD7J06VImTZrETTfdRPPmzYmOjmbr1q0sXryYhIQEBg4cCBx7eCxfvny88cYb/Prrr8E5nYmJiRQpUoRmzZpx/fXXM2XKFDZv3kyNGjXYunUrH330EU2aNMk0NSI2NpbWrVvz4Ycfcvvtt1O/fn12797N/Pnzufzyy6lWrRqrVq3KcV0yHrpr06YNhQoVOmG/Bx98kDlz5jB27FjatGmT4+OdSKFChXj99dd58MEHadmyJXfffTdly5blm2++4eOPP6ZIkSIkJyeHTF/Kbi1zqn79+nz55Zc88MADNG7cmMsvv5xSpUqF/RsBu3btyqeffkpCQgItW7bkyiuvZMWKFXz++ee0aNHipCuDHK979+7MnDmTWbNmcfvtt9OoUSN2797N1KlTqV27Nh999FFYxy0ZjKWzKOPOT4ECBShcuDAlS5akRYsWNG/enKZNm1KgQIHT2s9TTz3FzJkzWbFiRfCb4kqWLEnXrl1JSkoKCX3169dn4MCBjB07ljFjxgQfeAlXMB44cCApKSmMHz+eH3/8keLFi5OUlES/fv0yfUxdr149Jk6cyCuvvEJKSgqXXnopderU4ZNPPmH69OmZgjEcWyP12WefZenSpXzyySccPXqUp59++qTBuEyZMvzrX/9i8ODBzJ49m88//5xChQpRt25dHn/88ZOu5xwOHTt2pHz58owYMYKZM2eyb98+SpQoQWJiIr169cr1Q0MZd4AzluQ6keLFi9OsWTOmTZvGhx9+yEMPPURERARvvPEGDRs2ZOzYsUyfPp2DBw8SHR3NLbfcQqtWrYKvv/766xk9ejQjRozgnXfeYf/+/cCxu+JFihThsssuY/r06Tz//PPMnTuXr776imuvvZa//OUv3HHHHVmGuREjRlC2bFmmTJnCmDFjKF68OAkJCTzzzDM88MADOa5JxjfdnU5dmjRpQlxcHIsWLeL777/n2muvzfFxT6Rp06Z8/PHHDBkyhM8++4zp06cTHR1Nhw4d6NOnT8gqIkCOapkTTz31FLt37+ajjz5i+PDhHD58mLp164Y9GDdq1IhJkyYxePBgpk6dSr58+bj55puZMWMGP/zww2kH48suu4xp06YxcOBApk6dGlzX/KmnnuKuu+4yGCvsItLT0/1ORUmSJF30nGMsSZIkYTCWJEmSAIOxJEmSBBiMJUmSJMBgLEmSJAEGY0mSJAkwGJ8X0tLS8noI5y1rl3PWLuesXe5Yv5yzdjln7XLuQqqdwViSJEnCYCxJkiQBBmNJkiQJMBhLkiRJgMFYkiRJAgzGkiRJEmAwliRJkgCDsSRJkgQYjCVJkiTAYCxJkiQBBmNJkiQJMBhLkiRJgMFYkiRJAgzGkiRJEgCX5PUAJEmSciLqH5vDtq/lt4VtVzqPecdYkiRJwmAsSZIkAQZjSZIkCTAYS5IkSYDBWJIkSQIMxpIkSRJgMJYkSZIAg7EkSZIEGIwlSZIkwGAsSZIkAQZjSZIkCTAYS5IkSYDBWJIkSQIMxpIkSRJgMJYkSZIAg7EkSZIEGIwlSZIkwGAsSZIkAQZjSZIkCTAYS5IkSYDBWJIkSQLyOBgvXryY9u3bU6lSJaKiopgwYUKw7dChQ7zwwgvUqVOHkiVLUqFCBTp16sRPP/0Uso+DBw/Su3dvypcvT8mSJWnfvj2bN28O6fPTTz/Rrl07SpYsSfny5enTpw+///57SJ9FixZxxx13EBsbS/Xq1XnrrbfO3IlLkiTpnJOnwXjfvn1UrlyZgQMHEhkZGdL2n//8h6+//ppevXrx2Wef8e6777J582batGnD4cOHg/369evHjBkzePPNN5k1axZ79uyhXbt2HDlyBIAjR47Qrl079u7dy6xZs3jzzTdJSUnh2WefDe7jhx9+oG3bttSoUYMFCxbQs2dP+vTpw/Tp089OISRJkpTnLsnLgzdu3JjGjRsDkJSUFNJWpEgRpk2bFrJt6NCh1KpVi3Xr1lGlShV+++033n77bUaOHEn9+vUBGDVqFNWqVeNf//oXDRs2ZN68eaxdu5ZVq1ZxzTXXAPDiiy/y+OOP83//939ceeWV/OMf/yAuLo5XX30VgAoVKvDvf/+b1157jRYtWpzpMkiSJOkccF7NMd6zZw8AUVFRAHz11VccOnSIBg0aBPtcc801VKhQgWXLlgGQmppKhQoVgqEYoGHDhhw8eJCvvvoq2Of4fWT0WbFiBYcOHTqj5yRJkqRzQ57eMc6O33//neeee46mTZty9dVXA7B9+3by589PsWLFQvpGR0ezffv2YJ/o6OiQ9mLFipE/f/6QPnfeeWemfRw+fJhdu3YRFxeX5ZjS0tLCcWqn5Wwe60Jj7XLO2uWctcsd65dzF1ftCoZ1bxdX7cLrfKldfHz8SdvPi2B8+PBhEhMT+e2335g4cWJeDyfoVMUNl7S0tLN2rAuNtcs5a5dz1i53rF/OXXS1W7T51H2y4aKqXRhdSNfdOT+V4vDhwzz66KOsXr2a6dOnU7Ro0WBbTEwMR44cYdeuXSGv2bFjBzExMcE+O3bsCGnftWsXR44cOWmfHTt2cMkll2S6Gy1JkqQL0zkdjA8dOkTHjh1ZvXo1M2bMIDY2NqT9hhtuoECBAsyfPz+4bfPmzaxbt46aNWsCUKNGDdatWxeyhNv8+fO57LLLuOGGG4J9jt9HRp8bb7yRAgUKnKnTkyRJ0jkkT6dS7N27lw0bNgBw9OhRNm3axMqVK7nqqqsoUaIEDz30ECtWrGDixIlERESwbds2AK688koiIyMpUqQIDzzwAC+88ALR0dFcddVVPPvss1SpUiU4Z7hBgwZUqlSJLl260L9/f3799Veef/55HnzwQa688koAOnbsyN///nf69u1Lx44dWbZsGe+++y5jxozJk7pIkiTp7MvTO8YrVqygXr161KtXj/379zNgwADq1avHX/7yFzZv3sysWbPYsmULd955JxUqVAj+mTJlSnAfAwYMoHnz5nTs2JGmTZtSqFAhJk2aRP78+QHInz8/7733HgULFqRp06Z07NiRu+66i/79+wf3UbZsWd5//32WLFnC7bffzuDBgxk0aJBLtUmSJF1E8vSO8e233056evoJ20/WluGyyy7j1VdfDa5BnJVSpUrx3nvvnXQ/t912GwsWLDjl8SRJknRhOqfnGEuSJElni8FYkiRJwmAsSZIkAQZjSZIkCTAYS5IkSYDBWJIkSQIMxpIkSRJgMJYkSZIAg7EkSZIEGIwlSZIkwGAsSZIkAQZjSZIkCTAYS5IkSYDBWJIkSQIMxpIkSRJgMJYkSZIAg7EkSZIEGIwlSZIkwGAsSZIkAQZjSZIkCTAYS5IkSYDBWJIkSQIMxpIkSRJgMJYkSZIAg7EkSZIEGIwlSZIkwGAsSZIkAQZjSZIkCTAYS5IkSYDBWJIkSQIMxpIkSRJgMJYkSZIAg7EkSZIEGIwlSZIkwGAsSZIkAQZjSZIkCTAYS5IkSYDBWJIkSQIMxpIkSRJgMJYkSZIAg7EkSZIE5HEwXrx4Me3bt6dSpUpERUUxYcKEkPZAIMCAAQOoWLEicXFxNG/enLVr14b0SU9PJzExkdKlS1O6dGkSExNJT08P6bN69WqaNWtGXFwclSpVYtCgQQQCgZA+06dPp2bNmsTExFCzZk1mzJhxZk5akiRJ56Q8Dcb79u2jcuXKDBw4kMjIyEztw4cPZ+TIkQwaNIh58+YRHR1Nq1at2LNnT7BPp06dWLlyJZMnT2by5MmsXLmSzp07B9t3795Nq1atiImJYd68eQwcOJARI0bw2muvBfukpqbyyCOPcM8997Bw4ULuueceHn74Yf7973+f2QJIkiTpnHFJXh68cePGNG7cGICkpKSQtkAgQHJyMk8++SQtWrQAIDk5mfj4eCZPnkzHjh1Zt24dc+fOZfbs2dSoUQOAoUOHkpCQQFpaGvHx8XzwwQfs37+f5ORkIiMjqVy5MuvXr+f111+ne/fuREREkJyczO23306vXr0AqFChAgsXLiQ5OZk333zzLFZEkiRJeSVPg/HJbNy4kW3bttGgQYPgtsjISOrUqcOyZcvo2LEjqampFC5cmJo1awb71KpVi0KFCrFs2TLi4+NJTU2ldu3aIXekGzZsyMsvv8zGjRspW7Ysy5cvJzExMeT4DRs2ZPTo0ScdY1paWpjO9tTO5rEuNNYu56xdzlm73LF+OXdx1a5gWPd2cdUuvM6X2sXHx5+0/ZwNxtu2bQMgOjo6ZHt0dDRbtmwBYPv27RQrVoyIiIhge0REBMWLF2f79u3BPiVLlsy0j4y2smXLsm3btiyPk7GPEzlVccMl4+63ss/a5Zy1yzlrlzvWL+cuutot2hzW3V1UtQujC+m6c1UKSZIkiXM4GMfGxgKwY8eOkO07duwgJiYGgJiYGHbt2hWywkQgEGDnzp0hfbLaR0ZbxrFOdhxJkiRd+M7ZYFymTBliY2OZP39+cNuBAwdYunRpcE5xjRo12Lt3L6mpqcE+qamp7Nu3L6TP0qVLOXDgQLDP/PnzKVGiBGXKlAHg1ltvDTlORp/j5y5LkiTpwpanwXjv3r2sXLmSlStXcvToUTZt2sTKlSv56ae+aVZVAAAgAElEQVSfiIiIoGvXrgwfPpyUlBTWrFlDUlIShQoVok2bNsCx1SMaNWpEjx49SE1NJTU1lR49etCkSZPgXJc2bdoQGRlJUlISa9asISUlhWHDhpGUlBScm9ylSxcWLFjA0KFDWb9+PUOGDGHhwoV07do1z2ojSZKksytPg/GKFSuoV68e9erVY//+/QwYMIB69erxl7/8BYAnnniCrl270rt3b+rXr8/WrVuZMmUKV1xxRXAfY8aMoWrVqrRu3ZrWrVtTtWpVRo0aFWwvUqQIU6dOZcuWLdSvX5/evXvTrVs3unfvHuxTs2ZN3nrrLd59913q1q3LpEmTeOutt7jlllvOXjEkSZKUpyLS09MDp+6mvHQhPe15tlm7nLN2OWftcsf65dzFVruof4RvVYrlt/3noqpdOF1I1905O8dYkiRJOpsMxpIkSRIGY0mSJAkwGEuSJEmAwViSJEkCDMaSJEkSYDCWJEmSAIOxJEmSBBiMJUmSJMBgLEmSJAEGY0mSJAkwGEuSJEmAwViSJEkCDMaSJEkSYDCWJEmSAIOxJEmSBBiMJUmSJMBgLEmSJAEGY0mSJAkwGEuSJEmAwViSJEkCDMaSJEkSYDCWJEmSAIOxJEmSBBiMJUmSJMBgLEmSJAEGY0mSJAkwGEuSJEmAwViSJEkCDMaSJEkSYDCWJEmSAIOxJEmSBBiMJUmSJMBgLEmSJAE5CMbVqlWjf//+rF+//kyMR5IkScoT2Q7GVapU4W9/+xu1atWifv36jBo1ip07d56JsUmSJElnTbaD8aRJk1i3bh2vvPIKl156KX379qVSpUq0a9eOadOmcfDgwTMxTkmSJOmMytEc46uuuopOnToxZ84cVqxYQa9evdiwYQOPPPII8fHxPP744yxevDjcY5UkSZLOmFw/fFe2bFmefvppZsyYQcuWLdmzZw9vv/02d911F9WrV2f06NEcPXo0HGOVJEmSzphLcvPiffv2MWPGDN5//30WLFhAREQEjRs35t5776VAgQKMGzeOvn37smbNGoYNGxauMUuSJElhl+07xkePHmXu3LkkJiZSoUIFunbtyo4dO/jzn//M2rVree+992jRogXNmjXjvffe48knn2TKlCk5GtyRI0fo378/f/jDH4iNjeUPf/gD/fv35/Dhw8E+gUCAAQMGULFiReLi4mjevDlr164N2U96ejqJiYmULl2a0qVLk5iYSHp6ekif1atX06xZM+Li4qhUqRKDBg0iEAjkaNySJEk6/2T7jnHFihXZuXMn0dHRPPzww3To0IEqVaqcsH+VKlXYs2dPjgY3bNgwxowZQ3JyMpUrV2b16tV07dqVSy+9lD59+gAwfPhwRo4cyciRI4mPj+eVV16hVatWLF++nCuuuAKATp06sWnTJiZPngzA448/TufOnXnvvfcA2L17N61ataJOnTrMmzePtLQ0unXrRsGCBXnsscdyNHZJkiSdX7IdjG+77TY6dOhAw4YNyZfv1DecW7duTcuWLXM0uNTUVJo2bUpCQgIAZcqUISEhgS+++AI4drc4OTmZJ598khYtWgCQnJxMfHw8kydPpmPHjqxbt465c+cye/ZsatSoAcDQoUNJSEggLS2N+Ph4PvjgA/bv309ycjKRkZFUrlyZ9evX8/rrr9O9e3ciIiJyNH5JkiSdP7IdjN96661sHyR//vzZfg1ArVq1ePPNN1m/fj3XX3893377LQsXLqRHjx4AbNy4kW3bttGgQYPgayIjI6lTpw7Lli2jY8eOpKamUrhwYWrWrBmy30KFCrFs2TLi4+NJTU2ldu3aREZGBvs0bNiQl19+mY0bN1K2bNksx5eWlpaj88qJs3msC421yzlrl3PWLnesX85dXLUrGNa9XVy1C6/zpXbx8fEnbc92MJ4zZw7z589n4MCBWbb369ePBg0a8Mc//jG7u87kySefZO/evdSsWZP8+fNz+PBhevXqRadOnQDYtm0bANHR0SGvi46OZsuWLQBs376dYsWKhdz1jYiIoHjx4mzfvj3Yp2TJkpn2kdF2omB8quKGS8adbWWftcs5a5dz1i53rF/OXXS1W7Q5rLu7qGoXRhfSdZfth++GDRvGb7/9dsL2PXv2hG0FiilTpjBp0iTGjBnDZ599xhtvvMGYMWMYP358WPYvSZIkZch2MF6zZg033XTTCdtvuOEG1qxZk6tBZXj++efp3r07rVu3pkqVKrRv355u3boxdOhQAGJjYwHYsWNHyOt27NhBTEwMADExMezatStkhYlAIMDOnTtD+mS1j4w2SZIkXfiyHYwPHz7MgQMHTti+f//+sH0t9H/+859M85Pz588f/MKQMmXKEBsby/z584PtBw4cYOnSpcE5xTVq1GDv3r2kpqYG+6SmprJv376QPkuXLg05r/nz51OiRAnKlCkTlnORJEnSuS3bwbhSpUrMmjUry7ZAIMDMmTOpUKFCrgcG0LRpU4YNG8acOXPYuHEjM2bMYOTIkfzP//wPcGyucNeuXRk+fDgpKSmsWbOGpKQkChUqRJs2bQCoUKECjRo1okePHqSmppKamkqPHj1o0qRJcD5MmzZtiIyMJCkpiTVr1pCSksKwYcNISkpyRQpJkqSLRLYfvuvcuTOJiYk89NBD9OnTJxiCv/32W1555RVSU1MZOXJkWAb3yiuv8PLLL/PUU0+xc+dOYmNjg8fN8MQTT7B//3569+5Neno6N998M1OmTAmuYQwwZswY+vTpQ+vWrQFISEjglVdeCbYXKVKEqVOn0qtXL+rXr09UVBTdunWje/fuYTkPSZIknfsi0tPTs/31bgMGDGDw4MEEAoHgWsZHjx4lIiKCnj178uyzz4Z9oBezC+lpz7PN2uWctcs5a5c71i/nLrbaRf0jfKtSLL/tPxdV7cLpQrrusn3HGI4tyda2bVtSUlL44YcfAChXrhx33XUX1157bTjHJ0mSJJ0VOQrGANdee23wizYkSZKk812OgzEcWwEiPT09ZCm0DCVKlMjNriVJkqSzKtvB+ODBg7z66qu88847wW+Oy8ovv/ySq4FJkiRJZ1O2g3Hv3r155513aNKkCXXq1KFIkSJnYlySJEnSWZXtYDx9+nTuv/9+/va3v52J8UiSJEl5Ittf8BEIBLjxxhvPxFgkSZKkPJPtYJyQkMCCBQvOxFgkSZKkPJPtYNy3b1++//57evbsyVdffcXOnTv59ddfM/2RJEmSzifZnmN80003ERERwapVqxg7duwJ+7kqhSRJks4n2Q7GTz311JkYhyRJkpSnsh2Mn3vuuTMxDkmSJClPZXuO8fEOHTrE9u3bOXToULjGI0mSJOWJHAXjzz//nObNm1OyZEkqVqzIkiVLANi1axf/+7//y7/+9a9wjlGSJEk647IdjJcsWcLdd9/Njh07ePjhhwkEAsG2YsWKcejQIcaNGxfWQUqSJElnWraD8UsvvUSVKlVYvHgxffv2zdR+22238cUXX4RlcJIkSdLZku1g/PXXX9O+fXsKFChAREREpvYSJUqwffv2sAxOkiRJOluyHYwvueQSjhw5csL2n3/+mcKFC+dqUJIkSdLZlu1gfOutt5KSkpJl2759+5gwYQJ169bN9cAkSZKksynbwbhfv36sWLGCtm3b8sknnwDwzTffMH78eO6880527dpF7969wz5QSZIk6UzK9hd83HLLLbz//vv06NGDLl26AP//L/0oU6YM7733HlWrVg3vKCVJkqQzLNvBGOCOO+7giy++4Ouvv+a7777j6NGjlCtXjptvvpl8+XL1nSGSJElSnshRMAaIiIjghhtu4IYbbgjneCRJkqQ8ke1gvGzZstPqV7NmzWwPRpIkScor2Q7GTZs2zXL94v/2yy+/5GhAkiRJUl7IdjCeNm1apm1Hjhzhxx9/ZOzYseTLl4//+7//C8vgJEmSpLMl28H4jjvuOGHbAw88QNOmTfn888+pX79+rgYmSZIknU1hXUIif/78tG7dmvHjx4dzt5IkSdIZF/a11X777TfS09PDvVtJkiTpjMr2VIotW7Zkuf23335jyZIljBgxwhUpJEmSdN7JdjCuXLnyCVelCAQC3HTTTQwbNizXA5MkSZLOpmwH4+HDh2faFhERQVRUFOXKlfProCVJknReynYwfvDBB8/EOCRJkqQ8FfaH7yRJkqTzUbbvGLdq1SrbB4mIiGDKlCnZfp0kSZJ0tmQ7GO/fv5+tW7eyceNGrrjiCkqVKgXATz/9xJ49eyhbtixxcXFhH6gkSZJ0JmU7GL/44ou0b9+eYcOGce+991KgQAEADh06xIQJE3jxxRcZNWoUNWrUCPtgJUmSpDMl23OMn3vuOe69914eeuihYCgGKFCgAA8//DD33nsvzz33XFgHKUmSJJ1p2Q7G33zzDWXLlj1he5kyZVi1alVuxiRJkiSdddkOxrGxsUybNo0jR45kajty5AhTp04lNjY2LIOTJEmSzpZsB+PHHnuMJUuW0LhxY8aPH8+iRYtYtGgR48eP549//CPLli3jscceC9sAt27dSpcuXbj22muJjY2lZs2aLFq0KNgeCAQYMGAAFStWJC4ujubNm7N27dqQfaSnp5OYmEjp0qUpXbo0iYmJpKenh/RZvXo1zZo1Iy4ujkqVKjFo0CACgUDYzkOSJEnntmw/fPfoo4+SL18++vfvzxNPPBH8euhAIMBVV13Fq6++yqOPPhqWwaWnp9OkSRNq1arF+++/T7Fixdi4cSPR0dHBPsOHD2fkyJGMHDmS+Ph4XnnlFVq1asXy5cu54oorAOjUqRObNm1i8uTJADz++ON07tyZ9957D4Ddu3fTqlUr6tSpw7x580hLS6Nbt24ULFgwrCFfkiRJ565sB2OAjh07cv/997N8+XI2bdoEQKlSpbjllltCHsjLrb/97W/ExcUxatSo4Lbj5zcHAgGSk5N58sknadGiBQDJycnEx8czefJkOnbsyLp165g7dy6zZ88OrpQxdOhQEhISSEtLIz4+ng8++ID9+/eTnJxMZGQklStXZv369bz++ut07949GP4lSZJ04crxN98VKFCAOnXq0LZtW9q2bUvt2rXDGooBZs6cyc0330zHjh257rrruO222xg9enRwisPGjRvZtm0bDRo0CL4mMjKSOnXqsGzZMgBSU1MpXLgwNWvWDPapVasWhQoVCulTu3ZtIiMjg30aNmzIli1b2LhxY1jPSZIkSeemHN0xTk9PJzk5mYULF7Jr1y5ee+01br31Vn755Rf+8Y9/cPfddxMfH5/rwf3www+8+eabJCUl8eSTT7Jq1SqefvppABITE9m2bRtAyNSKjP/fsmULANu3b6dYsWIhd30jIiIoXrw427dvD/YpWbJkpn1ktJ1oFY60tLRcn+PpOpvHutBYu5yzdjln7XLH+uXcxVW7gmHd28VVu/A6X2p3qnya7WD8008/kZCQwI4dO6hQoQJpaWn85z//AaBo0aJMmjSJrVu38uqrr+ZsxMc5evQoN954Iy+88AIA1atXZ8OGDYwZM4bExMRc7z+3whH+T0fGlA9ln7XLOWuXc9Yud6xfzl10tVu0Oay7u6hqF0YX0nWX7akUzz//PIcOHWLp0qVMnTo108oNzZo1Y8GCBWEZXGxsLBUqVAjZdv311wfnNWcsC7djx46QPjt27CAmJgaAmJgYdu3aFTLOQCDAzp07Q/pktY+MNkmSJF34sh2M58+fT2JiIuXLl8/yobSyZcuyeXN43sHVqlWL7777LmTbd999R6lSpYBjXyYSGxvL/Pnzg+0HDhxg6dKlwTnFNWrUYO/evaSmpgb7pKamsm/fvpA+S5cu5cCBAyHnWaJECcqUKROWc5EkSdK5LdvB+ODBgxQtWvSE7Xv27CFfvhw/0xciKSmJ5cuXM3jwYDZs2MC0adMYPXo0nTp1Ao7NFe7atSvDhw8nJSWFNWvWkJSURKFChWjTpg0AFSpUoFGjRvTo0YPU1FRSU1Pp0aMHTZo0Cd72b9OmDZGRkSQlJbFmzRpSUlIYNmwYSUlJrkghSZJ0kch2gq1YsSJLliw5YfusWbOoWrVqrgaV4aabbmLChAlMnTqV2rVr89JLL/HMM88EgzHAE088QdeuXenduzf169dn69atTJkyJbiGMcCYMWOoWrUqrVu3pnXr1lStWjVkCbgiRYowdepUtmzZQv369enduzfdunWje/fuYTkPSZIknfuy/fBdly5d6Nq1K1WqVKFly5bAsTm7GzZsYODAgaSmpvL222+HbYBNmjShSZMmJ2yPiIigX79+9OvX74R9oqKiGD169EmPU6VKFT766KMcj1OSJEnnt2wH43bt2vHjjz/Sv39/XnrpJQBat25NIBAgIiKC559/nubNm4d9oJIkSdKZlKN1jHv37k3btm1JSUnh+++/5+jRo5QrV44WLVpQvnz5cI9RkiRJOuOyFYwPHDjAjBkzuO6667jxxht57LHHztS4JEmSpLMqWw/fXX755XTv3p2vv/76TI1HkiRJyhPZXpXi2muvDX4VsyRJknShyPYc4549e9KvXz9atmyZ6VvpJEk6XtQ/cvKFTwWz/Krf9I5X535AknQS2Q7GqampFCtWjLp161K7dm3KlSvH5ZdfHtInIiKCQYMGhW2QkiRJ0pmW7WD897//PfjfixYtYtGiRZn6GIwlSZJ0vsl2MN65c+eZGIckSZKUp07r4bs+ffrw1VdfAZA/f37y58/PwYMHyZcvX/D///uPJEmSdD45rWD897//nbS0tOD///LLL1xzzTUsWLDgjA1MkiRJOpuyvVxbhkAgEM5xSJIkSXkqx8FYkiRJupAYjCVJkiSysSrFDz/8wBdffAHA7t27AUhLS6Nw4cJZ9r/55pvDMDxJkiTp7DjtYDxgwAAGDBgQsq1Pnz6Z+gUCASIiIvjll19yPzpJkiTpLDmtYDxy5MgzPQ5JkiQpT51WML733nvP9DgkSZKkPOXDd5IkSRIGY0mSJAkwGEuSJEmAwViSJEkCDMaSJEkSYDCWJEmSAIOxJEmSBBiMJUmSJMBgLEmSJAEGY0mSJAkwGEuSJEmAwViSJEkCDMaSJEkSYDCWJEmSAIOxJEmSBBiMJUmSJMBgLEmSJAEGY0mSJAkwGEuSJEmAwViSJEkCDMaSJEkSYDCWJEmSgPMsGA8ZMoSoqCh69+4d3BYIBBgwYAAVK1YkLi6O5s2bs3bt2pDXpaenk5iYSOnSpSldujSJiYmkp6eH9Fm9ejXNmjUjLi6OSpUqMWjQIAKBwFk5L0mSJOW98yYYL1++nLFjx1KlSpWQ7cOHD2fkyJEMGjSIefPmER0dTatWrdizZ0+wT6dOnVi5ciWTJ09m8uTJrFy5ks6dOwfbd+/eTatWrYiJiWHevHkMHDiQESNG8Nprr52185MkSVLeOi+C8W+//cb/+3//j9dee42oqKjg9kAgQHJyMk8++SQtWrSgcuXKJCcns3fvXiZPngzAunXrmDt3LsOGDaNGjRrUqFGDoUOHMmfOHNLS0gD44IMP2L9/P8nJyVSuXJkWLVrwxBNP8Prrr3vXWJIk6SJxXgTjjOBbr169kO0bN25k27ZtNGjQILgtMjKSOnXqsGzZMgBSU1MpXLgwNWvWDPapVasWhQoVCulTu3ZtIiMjg30aNmzIli1b2Lhx45k8NUmSJJ0jLsnrAZzKuHHj2LBhA6NHj87Utm3bNgCio6NDtkdHR7NlyxYAtm/fTrFixYiIiAi2R0REULx4cbZv3x7sU7JkyUz7yGgrW7ZslmPLuON8NpzNY11orF3OWbucs3YZCoZtT9b09FxcdQrf9QUXW+3C63ypXXx8/Enbz+lgnJaWxp///Gdmz55NgQIF8no4mZyquOGSlpZ21o51obF2OWftcs7aHWfR5rDtypqe2kV37YXx+gKvsZy6kK67c3oqRWpqKrt27aJWrVoUK1aMYsWKsXjxYsaMGUOxYsUoWrQoADt27Ah53Y4dO4iJiQEgJiaGXbt2hcwVDgQC7Ny5M6RPVvvIaJMkSdKF75wOxs2bN2fJkiUsXLgw+OfGG2+kdevWLFy4kOuuu47Y2Fjmz58ffM2BAwdYunRpcE5xjRo12Lt3L6mpqcE+qamp7Nu3L6TP0qVLOXDgQLDP/PnzKVGiBGXKlDlLZytJkqS8dE5PpYiKigpZhQKgYMGCXHXVVVSuXBmArl27MmTIEOLj47nuuusYPHgwhQoVok2bNgBUqFCBRo0a0aNHD4YNGwZAjx49aNKkSfC2f5s2bRg0aBBJSUn06tWL7777jmHDhtGnT5+QucmSJEm6cJ3Twfh0PPHEE+zfv5/evXuTnp7OzTffzJQpU7jiiiuCfcaMGUOfPn1o3bo1AAkJCbzyyivB9iJFijB16lR69epF/fr1iYqKolu3bnTv3v2sn48kSZLyxnkXjGfOnBny/xEREfTr149+/fqd8DVRUVFZrmpxvCpVqvDRRx+FZYySJEk6/5zTc4wlSZKks8VgLEmSJGEwliRJkgCDsSRJkgQYjCVJkiTAYCxJkiQBBmNJkiQJMBhLkiRJgMFYkiRJAgzGkiRJEmAwliRJkgCDsSRJkgQYjCVJkiTAYCxJkiQBBmNJkiQJMBhLkiRJgMFYkiRJAgzGkiRJEmAwliRJkgCDsSRJkgQYjCVJkiTAYCxJkiQBBmNJkiQJMBhLkiRJgMFYkiRJAgzGkiRJEmAwliRJkgCDsSRJkgQYjCVJkiTAYCxJkiQBBmNJkiQJMBhLkiRJgMFYkiRJAgzGkiRJEmAwliRJkgCDsSRJkgQYjCVJkiTAYCxJkiQBBmNJkiQJMBhLkiRJwDkejIcMGUL9+vUpVaoU1157Le3atWPNmjUhfQKBAAMGDKBixYrExcXRvHlz1q5dG9InPT2dxMRESpcuTenSpUlMTCQ9PT2kz+rVq2nWrBlxcXFUqlSJQYMGEQgEzvg5SpIk6dxwTgfjRYsW8eijjzJnzhxSUlK45JJLaNmyJb/++muwz/Dhwxk5ciSDBg1i3rx5REdH06pVK/bs2RPs06lTJ1auXMnkyZOZPHkyK1eupHPnzsH23bt306pVK2JiYpg3bx4DBw5kxIgRvPbaa2f1fCVJkpR3LsnrAZzMlClTQv5/1KhRlC5dms8//5yEhAQCgQDJyck8+eSTtGjRAoDk5GTi4+OZPHkyHTt2ZN26dcydO5fZs2dTo0YNAIYOHUpCQgJpaWnEx8fzwQcfsH//fpKTk4mMjKRy5cqsX7+e119/ne7duxMREXHWz12SJEln1zl9x/i/7d27l6NHjxIVFQXAxo0b2bZtGw0aNAj2iYyMpE6dOixbtgyA1NRUChcuTM2aNYN9atWqRaFChUL61K5dm8jIyGCfhg0bsmXLFjZu3Hg2Tk2SJEl57Jy+Y/zf+vbtS7Vq1YJ3frdt2wZAdHR0SL/o6Gi2bNkCwPbt2ylWrFjIXd+IiAiKFy/O9u3bg31KliyZaR8ZbWXLls1yPGlpabk/qdN0No91obF2OWftcs7aZSgYtj1Z09NzcdUpfNcXXGy1C6/zpXbx8fEnbT9vgvEzzzzD559/zuzZs8mfP39eDwc4dXHDJWPKh7LP2uWctcs5a3ecRZvDtitremoX3bUXxusLvMZy6kK67s6LqRT9+vXjww8/JCUlJeTubWxsLAA7duwI6b9jxw5iYmIAiImJYdeuXSErTAQCAXbu3BnSJ6t9ZLRJkiTpwnfOB+Onn346GIqvv/76kLYyZcoQGxvL/Pnzg9sOHDjA0qVLg3OKa9Sowd69e0lNTQ32SU1NZd++fSF9li5dyoEDB4J95s+fT4kSJShTpsyZPD1JkiSdI87pYNyrVy/effdd/v73vxMVFcW2bdvYtm0be/fuBY7NFe7atSvDhw8nJSWFNWvWkJSURKFChWjTpg0AFSpUoFGjRvTo0YPU1FRSU1Pp0aMHTZo0Cd72b9OmDZGRkSQlJbFmzRpSUlIYNmwYSUlJrkghSZJ0kTin5xiPGTMGILgUW4ann36afv36/f/au/+Yquo/juOvOyyGYlwERBHRmSR6B5EgMssSNYpM/B2aK/PHcKbTXKiQbK7WgtQg28jVzDJNSxFLN9QtQ82U/EejyJibqckMkHWZoAji/f5h3jho99I3u+ciz8fG5v2cw937fPbmfl4eDudIkpYsWaKrV69q2bJlstvtiouLU1FRkbp37254n+XLl2vKlCmSpJSUFK1evdq5PSAgQLt27VJGRoaSkpJktVq1cOFCLVq06L8+RAAAAHgJrw7GbZ9OdycWi0VZWVnOoHwnVqtVH374ocv3sdls2rt37z+uEQAAAPcGr76UAgAAAPAUgjEAAAAggjEAAAAgiWAMAAAASCIYAwAAAJIIxgAAAIAkgjEAAAAgiWAMAAAASCIYAwAAAJIIxgAAAIAkgjEAAAAgiWAMAAAASJK6mF0A3Bt2pKt0pPKuvJd9dp+78j4AAO9m/fjurBsSawc6D84YAwAAACIYAwAAAJIIxgAAAIAkgjEAAAAgiWAMAAAASCIYAwAAAJIIxgAAAIAkgjEAAAAgiWAMAAAASCIYAwAAAJIIxgAAAIAkgjEAAAAgiWAMAAAASCIYAwAAAJIIxgAAAIAkqYvZBQAAAKBjsX5c2epVV+lI5d/u6459dp9/X9BdwhljAAAAQARjAAAAQBLBGAAAAJBEMAYAAAAkEYwBAAAASQRjAAAAQBK3awMAANCwI//ulmOtedPtx/DPEIwBAPgXjPdz/XcIVIC5uJQCAAAAEMEYAAAAkEQwBgAAACQRjAEAAABJBGODDRs2KCYmRqGhoXriiSd09OhRs0sCAACAhxCM/1RUVKTMzEy9+uqrOnz4sBISEjRt2jT99ttvZpcGAAAAD7DY7XaH2UV4gzFjxshms+m9995zjg0dOlQTJkzQqlWrTKwMAAAAnsAZY+C0PL0AAAnkSURBVElNTU06efKkRo8ebRgfPXq0vv/+e5OqAgAAgCcRjCXV1taqpaVFISEhhvGQkBBVV1ebVBUAAAA8iWAMAAAAiGAsSQoKCpKPj49qamoM4zU1NerZs6dJVQEAAMCTCMaS7r//fsXGxqqkpMQwXlJSouHDh5tUFQAAADypi9kFeIuFCxdq/vz5iouL0/Dhw7Vx40b9/vvvmj17ttmlAQAAwAM4Y/ynyZMnKycnR2vWrNHIkSNVWlqq7du3KyIiwtS6eOiIa3l5eUpKSlLfvn314IMPKi0tTT///LNhnwULFshqtRq+xo4da1LF3iUnJ+e2uXnooYec2x0Oh3JychQVFaVevXpp3LhxOnXqlIkVe4/o6Ojb5s5qteq5556T5H5uO5PvvvtO06dP1+DBg2W1WvXZZ58Ztrenz+x2u9LT0xUREaGIiAilp6fLbrd78jBM4WrumpubtWrVKo0YMUJhYWEaNGiQ5s2bd9v998eNG3dbL86ZM8fTh+Jx7vquPWvDtWvXtGzZMg0YMEBhYWGaPn26KisrPXkYpnA3d3f67LNarcrIyHDu01HXXoJxK/PmzdOPP/6o6upqHTp0SI8++qip9fDQEfeOHDmiuXPnav/+/dq9e7e6dOmiiRMn6o8//jDsN2rUKFVUVDi/duzYYVLF3icyMtIwN63/87Vu3ToVFBTo7bff1jfffKOQkBBNmjRJly9fNrFi71BSUmKYt0OHDslisWjixInOfVzNbWfS0NCgIUOGKDc3V35+frdtb0+fzZs3T2VlZSosLFRhYaHKyso0f/58Tx6GKVzN3ZUrV/TDDz8oIyNDhw4d0tatW1VZWampU6fq+vXrhn1nzpxp6MX8/HxPHoYp3PWd5H5tyMrK0p49e/TRRx+puLhYly9fVlpamlpaWjxxCKZxN3et56yiokKff/65JBk+/6SOufZyKYUXKygo0PPPP69Zs2ZJktasWaMDBw5o48aNPHTkT0VFRYbXH3zwgSIiIlRaWqqUlBTnuK+vr0JDQz1dXofQpUuXO86Nw+HQ+vXr9corr2jChAmSpPXr1ysyMlKFhYWd/jKj4OBgw+vNmzere/fumjRpknPs7+a2s0lOTlZycrIk6eWXXzZsa0+fVVRU6Ouvv9a+ffuUkJAgScrPz1dKSopOnz6tyMhIzx6QB7mau4CAAH355ZeGsfz8fCUmJqqiokI2m8053rVr107Xi67m7hZXa0NdXZ02b96sgoICJSUlSbq5xkRHR+vgwYMaM2bMf1O4F3A3d23nrLi4WAMHDtRjjz1mGO+Iay9njL0UDx35/9TX1+vGjRuyWq2G8WPHjmngwIGKi4vT4sWLb7sDSWd29uxZRUVFKSYmRnPmzNHZs2clSefOnVNVVZWhB/38/DRixAh6sA2Hw6HNmzcrLS3NcHbl7+YWf2lPnx0/flz+/v6GP4ZOTExUt27d6MU2bp1lb/sZuHPnTg0YMECJiYnKzs7mtz5/crU2nDx5Us3NzYbeDA8P16BBg+i7Vurr61VUVOQ8iddaR1x7OWPspXjoyP8nMzNT0dHRzrNKkjR27FiNHz9e/fr10/nz5/Xmm28qNTVVBw8elK+vr4nVmi8+Pl7vv/++IiMjdenSJa1Zs0bJyckqLS1VVVWVJN2xBy9evGhGuV6rpKRE586d04svvugcczW3PXr0MLFa79KePquurlZQUJAsFotzu8ViUXBwMJ+HrTQ1NSk7O1tPP/20+vTp4xyfNm2a+vbtq169eumXX37R66+/rvLycu3atcvEas3nbm2orq6Wj4+PgoKCDN/HOmxUWFiopqYmzZgxwzDeUddegjHuGa+99ppKS0u1b98++fj4OMenTJni/LfNZlNsbKyio6O1f/9+paammlGq13jyyScNr+Pj4xUbG6utW7dq2LBhJlXV8WzatElDhw5VdHS0c8zV3C5atMjTJeIed/36daWnp6uurk7btm0zbHvppZec/7bZbOrfv7/GjBmjkydPKjY21sOVeg/Whrtj06ZNeuaZZ267vKyjzi+XUngpHjryz2RlZWnnzp3avXu3+vfv73Lf3r17KywsTGfOnPFMcR2Iv7+/oqKidObMGed1YfSgazU1NSouLr7jrxFbaz23+Et7+qxnz56qra2Vw+Fwbnc4HLp06RK9qJuheO7cuSovL9dXX33l9jcSjzzyiHx8fOjFNtquDT179lRLS4tqa2sN+/EZ+JeysjKdOHHC7eef1HHWXoKxl+KhI+23YsUKZyhuz+2wamtrdfHixQ73BwGe0NjYqNOnTys0NFT9+vVTaGiooQcbGxt17NgxerCVrVu3ytfX13B25E5azy3+0p4+S0hIUH19vY4fP+7c5/jx42poaOj0vdjc3KzZs2ervLxce/bsaVd/lZeXq6WlhV5so+3aEBsbq/vuu8/Qm5WVlaqoqOj0fXfLpk2b1K9fP40aNcrtvh1l7eVSCi/GQ0fcy8jI0BdffKEtW7bIarU6r1fs1q2b/P39VV9fr9zcXKWmpio0NFTnz5/XG2+8oZCQED377LMmV2++W9cjhoeHO6+DvXLlimbMmCGLxaIFCxYoLy9PkZGRGjhwoNauXatu3bpp6tSpZpfuFRwOhz799FNNnjxZ/v7+hm2u5razqa+vd54lunHjhi5cuKCysjIFBgaqb9++bvts0KBBGjt2rJYuXap3331XkrR06VI99dRT9/QdKSTXc9e7d2/NmjVLJ06c0LZt22SxWJyfgQ888ID8/Pz066+/avv27UpOTlaPHj1UUVGh7OxsxcTEKDEx0cxD+8+5mrvAwEC3a0NAQIBeeOEFrVq1SiEhIQoMDNTKlStls9naFQQ7Mnc/s9LN2wXu2LFDixcvNlz/f+v7O+raa7Hb7Q73u8EsGzZs0Lp161RVVaXBgwfrrbfeMv3+yt6k7V9e37JixQplZWXp6tWrmjlzpsrKylRXV6fQ0FCNHDlSK1euVHh4uIer9T5z5szR0aNHVVtbq+DgYMXHx2vlypWKioqSdDP45ebm6pNPPpHdbldcXJzWrl2rIUOGmFy5dzh8+LBSU1N14MABxcXFGba5m9vO5Ntvv9X48eNvG58xY4bWr1/frj6z2+1avny59u7dK0lKSUnR6tWr//Yz4F7hau4yMzP18MMP3/H7CgoKNHPmTF24cEHp6ek6deqUGhoa1KdPHyUnJyszM1OBgYH/dfmmcjV3eXl57Vobrl27puzsbBUWFqqxsVGPP/643nnnnXt+/XD3MytJW7Zs0ZIlS/TTTz+pd+/ehv068tpLMAYAAADENcYAAACAJIIxAAAAIIlgDAAAAEgiGAMAAACSCMYAAACAJIIxAAAAIIlgDAAAAEgiGAMAAACSpP8ByLPP8OEK8Y0AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "plt.style.use('fivethirtyeight')\n",
    "plt.rcParams['figure.figsize'] = (10, 6)\n",
    "\n",
    "trans.loc[trans['actual_amount_paid'] < 250, 'actual_amount_paid'].dropna().plot.hist(bins = 30);\n",
    "plt.title('Distribution of Actual Amount Paid');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Domain Knowledge Features\n",
    "\n",
    "Before creating the entity from this dataframe, we can create a few new variables based on domain knowledge. Just because we are automatically going to make hundreds of features doesn't mean we can't use our own expertise. Featuretools will build on top of our knowledge by stacking more primitives on top of any variables that we define."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAioAAAFzCAYAAAAQWSIRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3XuYZVV95vHvCy0gOunGGyKNgrE1wSQqCuJIlEsSRY0Yx6gzKugw41zUwcsMYowRx2iIMSpODOYZEPASL1EUBtFoGojToyCogAiSahWh2wa8QHtBBeQ3f+xVcCiququ6q06trv5+nqee2nvttfdZa+9T57xn7b1PpaqQJEnq0Q6L3QBJkqSZGFQkSVK3DCqSJKlbBhVJktQtg4okSeqWQUWSJHXLoKIFk+QvklyfpJK8OMnxSdZOqfPHSb6V5FdJTlukpi64JAe3/bBysdvSu+meJ4utHbsXLnY7ttRC7tP2t33bQmx7vs1mP/i32h+DimYtyWntD7iS3JrkB0nWJDk2yb2m1H08cBzwUmAP4KPA24EDR+rsCLwP+BjwYOCYcfVlW5XkhUm26S8/msUb212eJ7PY3tokx8+i3vEjz9/bk6xP8uEkD5nFw+wBfHy2bdoWJbl6ZP/8Isk329/25t4nPgrs2XH7Rs3puaU+GFQ0V/+X4UX7IcAhwIeAlwNfTbL7SL1VwO1VdWZVXVdVP6+qn1bVD0bq7AHcGzinqtZX1cYtaVCSHVro0RIwzfNkPl3N8LxbCRwJPA74PzM9f5Ls1Np0XVX9YoHa1JO/ZNg/vwm8FzgBeM10FTO4R/vbvr639k1ngZ9bWiAGFc3VLe1F+3tV9fWqOgl4AnB/hhcN2imcDwA7TH4CauV3DLsmeTFwbdvmF1q9g9uyxyb5XJKfJvl+kjNGP/VObifJ85J8E7gFeHhb9vwkl7RPXFcnecfoaE+S85OcnOQNSa5L8qMk709y79FOtm1/pW3nh0k+k2S3keWvaJ/ofpFkIsnrkyybxf57TJIvt/UuT3LolMd9WJJPJLkpyY1tP/x2W3Zw26+TpyKqjXIdluSWJLu2Zbu07a8Z2e7vtzr3bvP3TnJiG1W4OcnXkjx7Slt2b9v/fpKfJPl/SZ40snxyiPz3k3yhbeeKJIfPYj/MKFOG55OsbPvkB61f307yP9qy84FfB944sk/23sTmfzXy/F0NHA/8NvCwkf3635L8fZKN3HV/33Hqp+2/dyW5Nskv23PtT2a772bo937teXZDe+5flOSpU+pcneR/tmP3owynVt85+txrx/+kJBvbc+gkYOdNPfaIn7b9852qehewGnh22+6Lk9yW5JAkXwN+Cfxephkhy/A3/NkkP259+XKGUdbJ5b/f9snP23Pw1CT33cr27Zbkg0muadu9KslrkmTkcac7/fyKJOva8/cfGUZ31RGDirZaVa1nGFl5doZh2GOAVwK/Yvj0s8c0q30UOKBNH9HqfDHJvsA/A19i+LR7aNvO55PsMrL+g4D/ChwF7AusyxB+TgL+upUdCfwewyevUc8B7gMcDDwfeAbw2smFSV4CfBD4FLAfw8jRZ4Ed2/Ljgf8OvI7hk90xwH8C3ri5fQW8A/ifwGOACxk+ze/Rtrs7sAa4AfhdhiHqq4Dzk9wf+CLD6BXcuV+PaeW3t3UAngj8BNg/d4a0Q4GLquqn7YX7/wCPAp4H/Fbbbx9Jclhryz2B84B/BRze2nsOw3H4zSl9ejvw1ra9C4GPZiTUzYO/BZYzHMvfAI4G1rVlz2YYJflr7twn1959EzP6eft9j5GyNzLs0/2AP526Qtt/ZwPPBF7B8Bw4Evh+Wz6XfTfq1xj+Lg5pj/2PwFlJHj6l3iuADcDj2/TLGf4OJv0F8G9am54A/Ax42SYed1N+zl33zQ4MoxqvZjgWF09dIckjgS8ANzI87x4DvLOtS4ZwfibwEeB3gGcBewNnjIaKLWjfzsDlbXv7Am8G3gS8eKaVkxzR2vYO4NEMp6H/ao5t0EKrKn/8mdUPcBrwTzMs+89AAQ9o8y8GbptS53hg7cj83m2dg6Y8xkemrLczcDPwrJHt3A48eEq9q4H/PKXsSe0xdmvz5wOXTqlzEvClkflrgL+ZoZ+7trY8dUr5kcBNm9h3B7d2HD1Stgz4LvDmkX5dMGW9AN8CXtnmXzj82d5t++cDb2vTbwFOAa6YbCdDgHjzSFt+ASyfso33AZ8aOX7rgGVT6pwLvGtKn549snz3VvaUTeyLuz03NvM8uRQ4fhP1125q+Sa2++C2X64B7tHKCjhlmnULeGGbPqzNP24T/dvkvpvD39ylwOunPMfPmlLnM8CH2/S92rH9j1PqXDza9xke62rgT9v0DsDTGUZNThjpVwG/u6njyTAKdSmwwwyPc/7kNqcciwIevaXtm2GdE4HPb+I5sAb40JR13t7asnIux8qfhfuZzVC1NBuTn4S29kLP/YGHJfnplPJdGK57mXR9VV1zx4MPIw4PAd6R5O3TtOthwEVt+tIp2/4e8JS2nQcAewGfm6F9jwTuCXwid72odUdglyT3r6rvb6J/X5qcqKrbkny5bROGvj92mr7fk7v2fTrnAX/Ypg8F/hfDG9ahSb4IPJbh4ubJx9kJWD/lA+xOwMRInQcCN02pszN3jkJMumSkT9cn+RVDYJkv7wL+rp1SOh/4dFV9YQu39dC2f3dg2K8XAX9UVbeO1PnyZrbxWODGqrrbaEIzl313h/YcfhPD8XsgQ5DdheF5PeqSKfPfA/Zp07/eHueLU+qsYRg53Jw3JDmO4blQwOkMb+6jLpq60hSPBT5bVbfPsHx/4MAkL59m2Sru3r9Zta+N5h7LMEq6kmHf3YPhw8BM9gU+PKVsDXO47kULz6Ci+fJIYCPww63czg4Mn8hOmGbZ6LZ/Ns16MJwKOW+addeNTN8yZVkx+9Ogk/X+GPiXaZb/aJbbmWnbq7nz9M6ozV1ofC7wZ0kezPBGcS7Dp83XMVwAfSt3vnnt0La3/zTbuWWkzpXAH01T5+YZ1hk1b6eVq+rUJJ8FnspwWuQzST5ZVVtyu/C1DCMitwMbqmq64DD1uTVXc9l3o05jGFk4FvgOQ6j5CMOb8qitef5uznsYTrX9nGH/TA0bv6qtv6h48vTRB6ZZdt1WtO81DM/3VwFfYzj9+SqGkRdtwwwq2mpJ9gReAJyxiU9Rs3Uxw3nrb1Ubh52N9kn+WuARVfW/t/TBq+qGJOuAPwDOmqbKNxhGKh5aVedswUMcyHBKhnYB5AHc+YJ9Me20wSbeDG5p6+5YVb8aKb+wtevPgImqui7JeQxvdM8GvlhVvxx5nBXALlV1+QyPczHD6awfV9UNc+7lPKuqDcCpwKlJzgE+nOS/VtWPGfbJbO/6urWqtvb7RL4C7JbkcTOMqmzpvnsScGxVnQXQri96KMN1F7P1LYb98a8ZnquTnjjL9X80T/vnsCQ7zPB6cDHwyC18nE2170kMIznvmyxIsrmRyCsY9tV7Rspmu680JgYVzdVOSR7I8KnovsBBDJ9ibmi/t9ZbGYbeP5jkRIYLFPdmuEDuxKr69ibWfT1wSpIbGS7Wu5XhQsfDq+o/zaENbwJOSnI9w3dn7MDwSf4jVfWDJG8F3tpO/fwTw9/RbwOPqarXzrTR5rgk1zF8Yn41w91Sf9uW/Q3DhaJnJvlzhk//KxkuyPx0VX2xrQfwzAx39Uze9n1Lkv/HcFHlewGq6kdJLme4ruX4kTac29p9RpJjgcuA3RhesH/Rgt6HGD6NfjrJ6xlGj3ZnOC1xZVV9arY7cyZJHj1N8d3elJP8DcPFqFcxDOc/m2Hf/KRV+Q7wxDaadDPDm9nWBuZNOZdhlOqjSV7NsP8eBPxmVZ3Mlu+7q4AXtOO6I8NF13O67b6qfpbkvcCft+fvVQzPqUcw/I2Ow9sYgvOHkvw1w0W1+zEE8C8xhOnPJXkH8H6G47iKYZTy5TOMcs3GVcCLkhwCrGcIi49vjz+Tvwb+oZ2CPYfh9exFW/j4WiDe9aO5+l2GOw6uYbhe4AUMb7D71Tx8l0JVXcnwhnlvhrsergD+N8P1BDdtZt0PAM9lOBf/ZYZz6cczvGjNpQ0nM4xsPIfhfPkXGMLCbW35mxlCxn9kuN5lDcMb09Wz2Px/Z7gb4RKGT25HVNX32navZ7hL4wfAGQwvvB9iuEZhQ6tzEcMFgn/H8MbzNyPbPo8hNJ07Unbu1LI2UvXM9hjvBL4JfJphiPxbrc4vgCczfPo9leHN9gyGEaBNnfOfrR0Zhuen/txvmrphuE7lcoZjcS+G8Dk54vZGhhGiqxiC7YLeXtoe9+kMb2zvbY/7wcm2b8W+ewnDa/KXGe44+yybvx5kOse19T/QtrWCu44YLKiq+jrDhdb3Z7iD7xKG0zK/asvPYwhtv8MQ+C5jeB7+hOHDxZZ6c3u8MxmuBdsNePdm2vrJ1rbJwP4CRu4AVB8yh9F1SZKksXJERZIkdcugIkmSumVQkSRJ3TKoSJKkbm2ztydv3LjRq4AlSVpili9ffpevdHZERZIkdcugIkmSumVQmcbExMTmK23j7OPSYB+XBvu4NNjHhWFQkSRJ3TKoSJKkbhlUJElStwwqkiSpWwYVSZLULYOKJEnqlkFFkiR1y6AiSZK6ZVCRJEndGltQSbIiyceTfDPJlUmekOQ+ST6fZKL93q3VTZJ3J1mb5LIk+42rnZIkqR/jHFE5EfhsVf0G8CjgSuA4YHVVrQJWt3mAw4FV7eelwEljbKckSerEWIJKkuXAk4BTAKrqlqq6CTgCOL1VOx14Vps+Anh/DS4AViTZYxxtlSRJ/RjXiMo+wPeBU5N8LcnJSe4F7F5VG1qd64Dd2/SewLUj669rZZIkaTuSqlr4B0keB1wAPLGqLkxyIvBj4BVVtWKk3o1VtVuSs4ETqmpNK18NvLaqLp6su3Hjxjsavj38x0pJ0vT2X7PrFq970UE3z2NLtKVWrVp1x/Ty5cszumzZmNqwDlhXVRe2+Y8zXI9yfZI9qmpDO7VzQ1u+HthrZP2VrWxaox2cDxMTE/O+zd7Yx6XBPi4N9nErrZnx7WGz5rNNHseFMZZTP1V1HXBtkke0osOAK4CzgKNa2VHAmW36LODIdvfPgcDGkVNEkiRpOzGuERWAVwAfSrIT8G3gJQxB6WNJjga+Czy31T0HeBqwFri51ZUkSduZsQWVqroEeNw0iw6bpm4BL1vwRkmSpK75zbSSJKlbBhVJktQtg4okSeqWQUWSJHXLoCJJkrplUJEkSd0yqEiSpG4ZVCRJUrcMKpIkqVsGFUmS1C2DiiRJ6pZBRZIkdcugIkmSumVQkSRJ3TKoSJKkbhlUJElStwwqkiSpWwYVSZLULYOKJEnqlkFFkiR1y6AiSZK6ZVCRJEndMqhIkqRuGVQkSVK3DCqSJKlbBhVJktQtg4okSeqWQUWSJHXLoCJJkrplUJEkSd0yqEiSpG4ZVCRJUrcMKpIkqVsGFUmS1C2DiiRJ6pZBRZIkdcugIkmSumVQkSRJ3TKoSJKkbhlUJElStwwqkiSpWwYVSZLULYOKJEnq1tiCSpKrk3w9ySVJLm5l90ny+SQT7fdurTxJ3p1kbZLLkuw3rnZKkqR+jHtE5ZCqenRVPa7NHwesrqpVwOo2D3A4sKr9vBQ4acztlCRJHVjsUz9HAKe36dOBZ42Uv78GFwArkuyxGA2UJEmLZ5xBpYDPJflKkpe2st2rakObvg7YvU3vCVw7su66ViZJkrYjqarxPFCyZ1WtT/IA4PPAK4CzqmrFSJ0bq2q3JGcDJ1TVmla+GnhtVV08WXfjxo13NHxiYmIsfZAk9Wf/Nbtu8boXHXTzPLZEW2rVqlV3TC9fvjyjy5aNqxFVtb79viHJJ4EDgOuT7FFVG9qpnRta9fXAXiOrr2xl0xrt4HyYmJiY9232xj4uDfZxabCPW2nNjG8PmzWfbfI4LoyxnPpJcq8k/2pyGvgD4HLgLOCoVu0o4Mw2fRZwZLv750Bg48gpIkmStJ0Y14jK7sAnk0w+5t9X1WeTXAR8LMnRwHeB57b65wBPA9YCNwMvGVM7JUlSR8YSVKrq28Cjpin/IXDYNOUFvGwMTZMkSR1b7NuTJUmSZmRQkSRJ3TKoSJKkbhlUJElStwwqkiSpWwYVSZLULYOKJEnqlkFFkiR1y6AiSZK6ZVCRJEndMqhIkqRuGVQkSVK3DCqSJKlbBhVJktQtg4okSeqWQUWSJHXLoCJJkrplUJEkSd0yqEiSpG4ZVCRJUrcMKpIkqVsGFUmS1C2DiiRJ6pZBRZIkdcugIkmSumVQkSRJ3TKoSJKkbhlUJElStwwqkiSpWwYVSZLULYOKJEnqlkFFkiR1y6AiSZK6ZVCRJEndMqhIkqRuGVQkSVK3DCqSJKlbBhVJktQtg4okSeqWQUWSJHXLoCJJkrplUJEkSd0yqEiSpG6NNagk2THJ15Kc3eb3SXJhkrVJPppkp1a+c5tf25bvPc52SpKkPox7ROUY4MqR+b8E3llVDwNuBI5u5UcDN7byd7Z6kiRpOzO2oJJkJfB04OQ2H+BQ4OOtyunAs9r0EW2etvywVl+SJG1Hxjmi8i7gWOD2Nn9f4Kaquq3NrwP2bNN7AtcCtOUbW31JkrQdWTaOB0nyDOCGqvpKkoPne/sTExPzvckF2WZv7OPSYB+XBvu4NXbd4jXnu00exy2zatWqGZeNJagATwSemeRpwC7ArwEnAiuSLGujJiuB9a3+emAvYF2SZcBy4IczbXxTHdwSExMT877N3tjHpcE+Lg32cSutWb/5OjOYzzZ5HBfGWE79VNXrqmplVe0NPB84t6peAJwHPKdVOwo4s02f1eZpy8+tqhpHWyVJUj8W+3tUXgu8OslahmtQTmnlpwD3beWvBo5bpPZJkqRFNK5TP3eoqvOB89v0t4EDpqnzC+CPx9owSZLUncUeUZEkSZqRQUWSJHXLoCJJkrplUJEkSd0yqEiSpG4ZVCRJUrcMKpIkqVsGFUmS1C2DiiRJ6pZBRZIkdWvWQSXJMUnut5CNkSRJGjWXEZVDgauTnJ3keUl2XqhGSZIkwRyCSlUdATwE+AzwSuC6JCcnedJCNU6SJG3f5nSNSlX9sKreU1VPAJ4M7A+cl+TqJK9Pcu8FaaUkSdouzfli2iSHJTkVOB+4HjgSeBHwGIbRFkmSpHmxbLYVk7wdeD6wEXg/8KdVtX5k+QXAjfPeQkmStN2adVABdgH+qKoumm5hVd2a5HHz0yxJkqS5BZW/AG4eLUiyG3DPqvoeQFV9cx7bJkmStnNzuUblU8DKKWUrgU/OX3MkSZLuNJeg8oiq+vpoQZv/jfltkiRJ0mAuQeWGJA8bLWjzP5zfJkmSJA3mElTeB3wiyTOS7JvkD4GPAycvTNMkSdL2bi4X054A3Aq8HdgLuJYhpLxjAdolSZI0+6BSVbcDf9V+JEmSFtxcRlRI8gjgUcBdviq/qt43n42SJEmCuX0z7Z8AfwZcyl2/T6UYrl+RJEmaV3MZUXklcEBVXbZQjZEkSRo1l7t+fg74zbOSJGls5hJU3gD8ryR7JNlh9GehGidJkrZvczn1c1r7/R9GysJwjcqO89UgSZKkSXMJKvssWCskSZKmMZfvUfkuQDvVs3tVbViwVkmSJDGHa1SSrEjy98AvgLWt7JlJ/nyhGidJkrZvc7kQ9r3ARuAhwC2t7EvA8+a7UZIkSTC3a1QOAx5UVbcmKYCq+n6SByxM0yRJ0vZuLiMqG4H7jRYkeTDgtSqSJGlBzCWonAx8IskhwA5JngCcznBKSJIkad7N5dTPXzJ8O+17gHsw/H+fvwNOXIB2SZIkzen25GIIJQYTSZI0FnP578mHzrSsqs6dn+ZIkiTdaS6nfk6ZMn9/YCdgHfDQeWuRJElSM5dTP3f5Cv0kOwJ/CvxkvhslSZIEc7vr5y6q6lfAW4BjN1c3yS5Jvpzk0iTfSPKmVr5PkguTrE3y0SQ7tfKd2/zatnzvLW2nJEnadm1xUGl+H7h9FvV+CRxaVY8CHg08NcmBDHcSvbOqHgbcCBzd6h8N3NjK39nqSZKk7cxc/tfPtUmuGfn5AfAPwHGbW7cGP22z92g/BRwKfLyVnw48q00f0eZpyw9Lktm2VZIkLQ1zuZj2hVPmfwb8S1X9eDYrt2tavgI8jOG7WL4F3FRVt7Uq64A92/SewLUAVXVbko3AfYEfzKG9kiRpG5fh61HG+IDJCuCTwBuA09rpHZLsBXymqn4ryeXAU6tqXVv2LeDxVXVHUNm4ceMdDZ+YmBhnFyRJHdl/za5bvO5FB908jy3Rllq1atUd08uXL7/LGZS5fI/KBxhO12xSVR25meU3JTkPeAKwIsmyNqqyEljfqq0H9gLWJVkGLAd+ONM2Rzs4HyYmJuZ9m72xj0uDfVwa7ONWWrN+83VmMJ9t8jgujLlcTHsTwzUkOzKcptmB4VqSmxhO40z+3E2S+7eRFJLck+Ei3CuB84DntGpHAWe26bPaPG35uTXuoR9JkrTo5nKNysOBp1fV/50sSHIQ8Iaqespm1t0DOL1dp7ID8LGqOjvJFcBHkvw58DXu/FK5U4APJFkL/Ah4/hzaKUmSloi5BJUDgQumlF3IcApnk6rqMuAx05R/GzhgmvJfAH88h7ZJkqQlaC6nfr4GvLWdupk8hfMW4JKFaJgkSdJcgsqLgScCG5NcD2wEDuLOa0kkSZLm1Vz+18/VwL9utxE/CNhQVdcsVMMkSZLm9BX6Se4LHAw8uaquSfKgJCsXpGWSJGm7N5ev0H8ycBXwAoYvawNYBZy0AO2SJEma010/7wKeV1Wrk9zYyi5kmrt2tncrTt3yLx+66SV7br6SJEnbibmc+tm7qla36ckvX7uFuYUdSZKkWZtLULkiydQvdvs94Ovz2B5JkqQ7zGU05DXA2Uk+Ddwzyd8Bf8jwNfqSJEnzbtYjKlV1AfA7wDeA9wHfAQ6oqosWqG2SJGk7N6sRlfY/elYDT6mqty1skyRJkgazGlGpql8B+8y2viRJ0nyYS/B4E3BSkock2THJDpM/C9U4SZK0fZvLxbQnt99HcuftyWnTO85noyRJkmAWQSXJA6vqOoZTP5IkSWMzmxGVfwF+raq+C5DkjKp69sI2S5IkaXbXqGTK/MEL0A5JkqS7mU1Qqc1XkSRJmn+zOfWzLMkh3DmyMnWeqjp3IRonSZK2b7MJKjcwfBPtpB9OmS/gofPZKEmSJJhFUKmqvcfQDkmSpLvxy9okSVK3DCqSJKlbBhVJktQtg4okSeqWQUWSJHXLoCJJkrplUJEkSd0yqEiSpG4ZVCRJUrcMKpIkqVsGFUmS1C2DiiRJ6pZBRZIkdcugIkmSumVQkSRJ3TKoSJKkbhlUJElStwwqkiSpWwYVSZLULYOKJEnqlkFFkiR1y6AiSZK6NZagkmSvJOcluSLJN5Ic08rvk+TzSSba791aeZK8O8naJJcl2W8c7ZQkSX0Z14jKbcBrqmpf4EDgZUn2BY4DVlfVKmB1mwc4HFjVfl4KnDSmdkqSpI6MJahU1Yaq+mqb/glwJbAncARweqt2OvCsNn0E8P4aXACsSLLHONoqSZL6MfZrVJLsDTwGuBDYvao2tEXXAbu36T2Ba0dWW9fKJEnSdiRVNb4HS+4N/DPwlqo6I8lNVbViZPmNVbVbkrOBE6pqTStfDby2qi6erLtx48Y7Gj4xMTG2PszG/mt23eJ1Lzro5nlsiSQtfb7mbvtWrVp1x/Ty5cszumzZuBqR5B7AJ4APVdUZrfj6JHtU1YZ2aueGVr4e2Gtk9ZWtbFqjHZwPExMTW7fNNTM2dbPmuy8z2eo+bgPs49JgH5eGBe1jJ6+5HseFMa67fgKcAlxZVe8YWXQWcFSbPgo4c6T8yHb3z4HAxpFTRJIkaTsxrhGVJwIvAr6e5JJW9ifACcDHkhwNfBd4blt2DvA0YC1wM/CSMbVTkiR1ZCxBpV1rkhkWHzZN/QJetqCNkiRJ3fObaSVJUrcMKpIkqVsGFUmS1C2DiiRJ6pZBRZIkdcugIkmSumVQkSRJ3TKoSJKkbhlUJElStwwqkiSpWwYVSZLULYOKJEnqlkFFkiR1y6AiSZK6ZVCRJEndMqhIkqRuGVQkSVK3DCqSJKlbBhVJktQtg4okSeqWQUWSJHXLoCJJkrplUJEkSd0yqEiSpG4ZVCRJUrcMKpIkqVsGFUmS1C2DiiRJ6pZBRZIkdcugIkmSumVQkSRJ3TKoSJKkbhlUJElStwwqkiSpWwYVSZLULYOKJEnqlkFFkiR1y6AiSZK6ZVCRJEndMqhIkqRuGVQkSVK3DCqSJKlbBhVJktStsQSVJO9LckOSy0fK7pPk80km2u/dWnmSvDvJ2iSXJdlvHG2UJEn9GdeIymnAU6eUHQesrqpVwOo2D3A4sKr9vBQ4aUxtlCRJnRlLUKmqLwA/mlJ8BHB6mz4deNZI+ftrcAGwIske42inJEnqy2Jeo7J7VW1o09cBu7fpPYFrR+qta2WSJGk7s2yxGwBQVZWktnT9iYmJ+WzOPGxz10V63H4fa7HYx6XBPi4NC9fHfl5zPY5bZtWqVTMuW8ygcn2SPapqQzu1c0MrXw/sNVJvZSub0aY6uCUmJia2bptrNtncTZrvvsxkq/u4DbCPS4N9XBoWtI+dvOZ6HBfGYp76OQs4qk0fBZw5Un5ku/vnQGDjyCkiSZK0HRnLiEqSDwMHA/dLsg54I3AC8LEkRwPfBZ7bqp8DPA1YC9wMvGQcbZQkSf0ZS1Cpqn87w6LDpqlbwMsWtkWSJGlb4DfTSpKkbhlUJElStwwqkiSpWwYVSZLULYOKJEnqlkFFkiR1y6AiSZK6ZVCRJEndMqhIkqRuGVQkSVK3DCqSJKlbBhVJktQtg4okSeqWQUWSJHXLoCJJkrplUJEkSd0yqEiSpG4ZVCRJUrcMKpIkqVsGFUmS1C2DiiRJ6pZBRZIkdcugIkmSumVQkSRJ3TKoSJKkbhlUJElStwwqkiSpWwYVSZLULYOKJEnqlkFFkiR1y6AiSZK6ZVCRJEndMqhIkqRuGVQkSVK3DCqSJKlbBhVJktQtg4okSeqg3JyQAAAGPUlEQVSWQUWSJHXLoCJJkrplUJEkSd0yqEiSpG4ZVCRJUreWLXYDZpLkqcCJwI7AyVV1wrgee/81u8Ka9eN6OEmSNIMuR1SS7Ai8Bzgc2Bf4t0n2XdxWSZKkcet1ROUAYG1VfRsgyUeAI4ArFrVVkiRt41acuuVnDC46aB4bMkupqvE/6mYkeQ7w1Kr6D23+RcDjq+rlk3U2btzYX8MlSdJWWb58eUbnuzz1I0mSBP0GlfXAXiPzK1uZJEnajvR66mcZ8C/AYQwB5SLg31XVNxa1YZIkaay6vJi2qm5L8nLgHxluT36fIUWSpO1Pr6d+qKpzqurhVfXrVfWWhXqcJHslOS/JFUm+keSYVn6fJJ9PMtF+77ZQbRiHJDsm+VqSs9v8PkkuTLI2yUeT7LTYbdxaSVYk+XiSbya5MskTluBxfFV7nl6e5MNJdtnWj2WS9yW5IcnlI2XTHrcM3t36elmS/Rav5bM3Qx//qj1XL0vyySQrRpa9rvXxqiRPWZxWz810fRxZ9pokleR+bX7JHMdW/op2LL+R5G0j5UviOCZ5dJILklyS5OIkB7TysRzHboPKGN0GvKaq9gUOBF7WvrPlOGB1Va0CVrf5bdkxwJUj838JvLOqHgbcCBy9KK2aXycCn62q3wAexdDfJXMck+wJ/DfgcVX1Wwyjjc9n2z+WpwFPnVI203E7HFjVfl4KnDSmNm6t07h7Hz8P/FZV/Q7Dqe7XAbTXn+cDj2zr/G37bqnencbd+0iSvYA/AK4ZKV4yxzHJIQxfn/Goqnok8PZWvpSO49uAN1XVo4E/a/MwpuO43QeVqtpQVV9t0z9heHPbk+GJd3qrdjrwrMVp4dZLshJ4OnBymw9wKPDxVmWb7h9AkuXAk4BTAKrqlqq6iSV0HJtlwD3bdVy7AhvYxo9lVX0B+NGU4pmO2xHA+2twAbAiyR7jaemWm66PVfW5qrqtzV7AcNMADH38SFX9sqq+A6xl+G6prs1wHAHeCRwLjF4QuWSOI/BfgBOq6petzg2tfCkdxwJ+rU0vB77XpsdyHLf7oDIqyd7AY4ALgd2rakNbdB2w+yI1az68i+GF4vY2f1/gppEXyXUM4Wxbtg/wfeDUdorr5CT3Ygkdx6paz/Bp7RqGgLIR+ApL71jCzMdtT+DakXpLpb//HvhMm14yfUxyBLC+qi6dsmjJ9BF4OPC77fTrPyfZv5UvpT6+EvirJNcyvAa9rpWPpY8GlSbJvYFPAK+sqh+PLqvh1qj+bo+ahSTPAG6oqq8sdlsW2DJgP+CkqnoM8DOmnObZlo8jQLtO4wiGUPYg4F5MM9S+1Gzrx21zkrye4RT0hxa7LfMpya7AnzCcKljKlgH3Ybh04H8AH2uj1kvJfwFeVVV7Aa+ijVyPi0EFSHIPhpDyoao6oxVfPzmE1X7fMNP6nXsi8MwkVwMfYThNcCLDEN3kXV9L4Xtq1gHrqurCNv9xhuCyVI4jwO8B36mq71fVrcAZDMd3qR1LmPm4LanvWEryYuAZwAvqzu+KWCp9/HWGUH1pe/1ZCXw1yQNZOn2E4bXnjHb648sMI9f3Y2n18SiG1xuAf+DOU1hj6eN2H1Ra8j0FuLKq3jGy6CyGg0P7fea42zYfqup1VbWyqvZmuLDr3Kp6AXAe8JxWbZvt36Squg64NskjWtFhDP8bakkcx+Ya4MAku7bn7WQfl9SxbGY6bmcBR7a7DQ4ENo6cItqmZPgP8ccCz6yqm0cWnQU8P8nOSfZhuFDxy4vRxq1RVV+vqgdU1d7t9WcdsF/7W10yxxH4FHAIQJKHAzsBP2CJHMfme8CT2/ShwESbHs9xrKrt+gc4iGFY+TLgkvbzNIbrOFa3A/JPwH0Wu63z0NeDgbPb9EMZ/mjWMiTknRe7ffPQv0cDF7dj+Slgt6V2HIE3Ad8ELgc+AOy8rR9L4MMM19zcyvBmdvRMxw0Iw39W/xbwdYY7oBa9D1vYx7UM5/cnX3feO1L/9a2PVwGHL3b7t7SPU5ZfDdxvCR7HnYAPtr/JrwKHLrXj2N4nvwJcynAN52PHeRy7/GZaSZIk8NSPJEnqmEFFkiR1y6AiSZK6ZVCRJEndMqhIkqRuGVQkSVK3DCqSJKlbBhVJktSt/w9dQ2e96r4ZnAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 576x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Difference between listing price and price paid\n",
    "trans['price_difference'] = trans['plan_list_price'] - trans['actual_amount_paid']\n",
    "\n",
    "# Planned price per day\n",
    "trans['planned_daily_price'] = trans['plan_list_price'] / trans['payment_plan_days']\n",
    "\n",
    "# Actual price per day\n",
    "trans['daily_price'] = trans['actual_amount_paid'] / trans['payment_plan_days']\n",
    "\n",
    "trans.loc[trans['price_difference'] > 0, 'price_difference'].plot.hist(bins = 30, \n",
    "                                                                       figsize = (8, 6));\n",
    "plt.title('Dfiference between List Price and Price Paid');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There is no `index` in this dataframe so we have to specify to make an index and pass in a name. There is a `time_index`, the time of the transaction, which will be critical when filtering data based on cutoff times to make features. Again, we also need to specify several variable types.\n",
    "\n",
    "There is one slight anomaly with the transactions where some membership expire dates are after the transactions date, so we will filter those out."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Filter anomalies\n",
    "trans = trans[trans['membership_expire_date'] > trans['transaction_date']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Entityset: customers\n",
       "  Entities:\n",
       "    members [Rows: 6658, Columns: 6]\n",
       "    transactions [Rows: 22329, Columns: 13]\n",
       "  Relationships:\n",
       "    No relationships"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create entity from transactions\n",
    "es.entity_from_dataframe(entity_id='transactions', dataframe=trans,\n",
    "                         index = 'transactions_index', make_index = True,\n",
    "                         time_index = 'transaction_date', \n",
    "                         variable_types = {'payment_method_id': vtypes.Categorical, \n",
    "                                           'is_auto_renew': vtypes.Boolean, 'is_cancel': vtypes.Boolean})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Logs\n",
    "\n",
    "The `logs` contain user listening behavior. As before we'll make a few domain knowledge columns before adding to the `EntitySet`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>msno</th>\n",
       "      <th>date</th>\n",
       "      <th>num_25</th>\n",
       "      <th>num_50</th>\n",
       "      <th>num_75</th>\n",
       "      <th>num_985</th>\n",
       "      <th>num_100</th>\n",
       "      <th>num_unq</th>\n",
       "      <th>total_secs</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>6+/V1NwBbqjBOCvRSDueeJZ58F4DY7h7fG6fSZtHaAE=</td>\n",
       "      <td>2017-03-04</td>\n",
       "      <td>29</td>\n",
       "      <td>28</td>\n",
       "      <td>18</td>\n",
       "      <td>11</td>\n",
       "      <td>111</td>\n",
       "      <td>79</td>\n",
       "      <td>34727.142</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>E2aBGFTKR6jzp+1knh7JOOF39gLuu+CoZMWaAL/DA0M=</td>\n",
       "      <td>2017-03-27</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>184</td>\n",
       "      <td>173</td>\n",
       "      <td>33408.719</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>g7exJzakJlHXwzUydnShY5w24WXSwJyS6QqgoFeyr7g=</td>\n",
       "      <td>2017-03-15</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>21</td>\n",
       "      <td>21</td>\n",
       "      <td>4951.000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>X+i9OmM3P42cETt5gPkOnz8vXGViQL5/M/NMiMQ+Olc=</td>\n",
       "      <td>2017-03-13</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>33</td>\n",
       "      <td>27</td>\n",
       "      <td>8755.599</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>tbl8blAVl6j4A8zW1Gnyg78Hc0LAQzzcYesmzgJ7ofs=</td>\n",
       "      <td>2017-03-27</td>\n",
       "      <td>6</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>6</td>\n",
       "      <td>1035.853</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                           msno       date  num_25  num_50  \\\n",
       "0  6+/V1NwBbqjBOCvRSDueeJZ58F4DY7h7fG6fSZtHaAE= 2017-03-04      29      28   \n",
       "1  E2aBGFTKR6jzp+1knh7JOOF39gLuu+CoZMWaAL/DA0M= 2017-03-27       1       0   \n",
       "2  g7exJzakJlHXwzUydnShY5w24WXSwJyS6QqgoFeyr7g= 2017-03-15       0       0   \n",
       "3  X+i9OmM3P42cETt5gPkOnz8vXGViQL5/M/NMiMQ+Olc= 2017-03-13       3       1   \n",
       "4  tbl8blAVl6j4A8zW1Gnyg78Hc0LAQzzcYesmzgJ7ofs= 2017-03-27       6       5   \n",
       "\n",
       "   num_75  num_985  num_100  num_unq  total_secs  \n",
       "0      18       11      111       79   34727.142  \n",
       "1       2        0      184      173   33408.719  \n",
       "2       0        0       21       21    4951.000  \n",
       "3       0        0       33       27    8755.599  \n",
       "4       0        0        2        6    1035.853  "
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "logs.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbIAAAEGCAYAAAAAKBB/AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3XuYXFWd7//3t6pvuV8IQgho4tDIJOowjoOA0UFxJKA/ow5ocuanqPFxUJgzHkcdGGdAmJPfyKjDHA+oz28EwRuBwVvGE+WAjAJnuAQQEYKhOgmYhISQW+fal+r6nj/2qs7uXbuqdqcv1Z3+vJ6nnq699tprr726ur691l57b3N3RERExqtcoysgIiIyFApkIiIyrimQiYjIuKZAJiIi45oCmYiIjGsKZCIiMq4pkB1jzOwWM7tnhMr+kJkVqy2PwP4+b2YdI1X+YJnZa8zsETPrMrPnGl2fNGZ2qZkdaHQ96jGz083Mzez1ja5LnJnNNrN/N7N9oX4nNrpOUp8C2TgQgpOHV6+Z7TSzB8zss2Y2JZH9r4CLB1F20cw+lDH77cC8rGUPog6Lw7HNT6z6EnDWcO9vCP4J2AecDvxxcqWZ/SL2e6r2mp9lR2a2xcyuGNbaHyl7VajLtYn0U0P6WGrz0faXwBnA2cBcYEdaJjM718zuCX+Lh83sOTO7w8yG/e9D6lMgGz/uJ/rDegXwFuC7wOXA42Z2QjmTu3e6+57h3LFFmt39sLu/OJxl1+LuB9x952jtL4N24Jfu/py7v5Sy/r1Ev6PyC6LfUTxt82hUNIMu4K/N7KRGV2S4mVnzEDZvB55096fdfbu7l1LK/wPgLuA3RH+LC4GPAFuBqUPYtxwtd9drjL+AW4B7UtLnAbuBb1bLCywi+qPbCxwEngE+ENY9B3j8FdI/BBSJ/kh/BfQAF5TTY2WX870NeJroy/Fh4IxknkS9Tw77OxeYn6wD8IuQ7/NAR2LbS4B1oU5bgP8ONMXW/wL4BvD3wPbQPt8CptZp47nAqtBOh0M5rw/r0ur4+Qy/Nwf+35R0A64M7d8DdACXxdY/lLK/E4Fm4CZgY6jjBuAaoDm27aXAgTr1WgWsAZ4Aboqlnxr2dVZYPj0svz6x/RbgivC+LeS5FPg+0WdsE7AUmA3cARwIx/j/xMool70stHVXyPPexL5OAr4D7CTqDd8PnBNbvySUcz7wINANfLjKcbcCXwZeCO3+G+Di2PrtiTb/WZVyrgC2ZPj9LwJ+FtpkP/AjYH7yd0X0d/Br4BDwCPCHiXIuIPrMdxH9Pb411O+i2Ofp6vB56ibqRf6U2N/Fsf5Sj2wcc/etRD2z95pZtd/lbcAu4BzgNcCngHKP7Y+BPuCTDOxFQNRbvy7kPx14tEr5OaIht08AZwIvAf/LzCZlPIzNRF96hO3nEvVsKpjZO4CbgW8Drwb+GriM6I847iKiL9Fzib4o3wn8TbUKmJkRfcmcHvKeCbwI3G1mc0Id5xJ9gV8X3n8p4/Gl+RTwd0RBaBHwL8D1ZvbnYf2FwDZgJUd+LzuAPNF//cuA3wc+TdTunz6KOpSAzwAfMrPXHPWRHHEV8AOiYbl7iYLP94DVIe3nwHfMbEZiuy8BXwP+IGx/h5ktBDCzqcAviY777cAfhbJ/bma/lyjny8A/EP0O76pSxy8BHyDqJb+GKPDebmaLw/rXAD8G7iFq8+VVytkGHG9mb6uyvlz3u4kCzmKi4DMHWGNmTbGsrUT/sH08HN9+YFX57zkMRf+IKNj/IdHn+PrE7pYT/Q1/gqhHeX7Y98TR6EiqV/0XVXpkYd2lRH8sL0vLC3QCH6pRdjG5nqgX5cCbUtKTPTIHzoulzSL6L3NF2jYhrb9HFpYXh+X5iXyfJ9YjI/pv/I5Enr8i6p20hOVfAL9O5Pka8GCNNjgv7H9hLK2V6Avrqljac8DfDeL3Vq1H9hJwbUod18WW+3s9dfZxJfCbxOchS4/sJ+H9T4G7wvuh9Mi+EFt/Skj7Yixtbkh7W6LszyXKfgz419ixbARyiTz/Wd4fR3pkF9c55plAL/CRRPpPgTVpbVOjrCaiXr6H3+Uaon8K5sXyXEbUg5yZ+Nz3AO9L/O3GP3d/EtJeEZa/DDwbbwPg3QzskV0JPMUE6oElX+qRjX8WfnqV9V8CvhEmInzezF43iLLXZsz3YPmNR+fnniHqaQy3RcB9ibRfEn2Zxv9D/3UizwvACVS3CNjl7uvKCe7eTTRMOqzHYWYvI/rPPO042uud3zGzT5jZWjPbEWYnXk103vRofRZ4m5m9fQhlwMA23x5+PpmS9rLEdg8mlv+TI23+x8DLgX1mdqD8Cuntie0eqVO/04gCUFq7D+p37O5Fd/8g0dD+J4kCzWXAb83snJBtEdG5tr2x7bYQBeb4/rqJ/l7KXgg/y5/XhcDDPvBcXbLNbgNmAM+Z2c1m9l9SJoEd0xTIxr9FRL2uXWkr3f0fiP6I7yAajnvIzP57hnL73L1rGOpXcbKc6FzPSOpJLDvHwGfdzD4A/DPR0OoFRENN1wEtR1umu/8G+CbwRSrbqPy7s0R62u+vN15sMs1D1yFlH7XkiM7jnZF4/T7R8GDcwUGUOyzc/QV3/667f5Koh7md6NzsYBRjbQNH2i6XklatHs8RBfaPEZ0TvhZ4xszm1truWDLu/7gnsjDV98+BH3jK7Koyd9/o7l9194uIzmV8PLa6h+gcxFD0T9c2s5lEXzTl3s0OIB+fWQkke4XlwFOvHk8Db06k/QlHJj4craeB48rnZgDMrBV4A9GQzbBx9x1EExfSjuNZdy9/+af9Xt5M9N/5V9z9MXcvAAuGoVp/T9Sj/VAivTz1vH9mY/jMJXtVQ5Gc6n8ORz47jxJ9Qe92947Ea9sg9/Ms0TB6WrsP+Xcc/ul7jiNt8zTw2vD3AICZnQy8cpD7Wwe8IZzHLau4PMLdu9x9jbt/muhc3xyi870TQlP9LDJGtISLM3PAcUTnla4k+rK5Mm2DcML5OqKT2puIzhMs4cgXBSH9LWb2U6DHBz/d3YF/MrPyJJKVRCesvxfWPxKWv2Bm/x/RF+ZViTKeJ/rv/0Izux3odvfOlH39I/Dv4fqq8sSCzwNfdvdkL2ww7g31/J6ZXUbUw/17oiHLrw2h3Gr+EfgHM9sEPEB0cn4F8OFYnk3Am8KXXxdRj3s9sCxMellPdK5kyF9W7r7NzL5MdJ4nnr7XzB4DrjCzjUTt8Y+hPsPl4+Gi9yeIprCfQTQzFeBW4L8CPzGzvyf6Z+VEolmyv3L3/5V1J+FYvkb0OdxDFGiWE7V9MrjVZGZ/SfTP2o9CnVqAPyOa0FGeeHQr8DngNjP7W6Lv2uuJZmb+cBC7+59Evc//aWY3EJ1/vKZ8WKE+f0EUpNcSfXaXEP2unqko7VjV6JN0etV/EU3gKE8JLhJ9qT1AdH5jSkree8L7NqKAsonoy2cH0UXNp8TyLyH6wPeQmH6fUo8B6RyZfv/2UEY3UUB4XWK7d4T1h4H/Q/Tl0T/ZI+T5LNGMvD7qT78v13crUeCsmH6f2ObvgOfqtHFy+v0vqZzk8BzDM9nDgL+lyvT7kOdsoi/3Lo5Mv28lmrW5h+gL61tEMyC7YtsNarJHLG0K0eSW/skeIX1h+KwdIgqe7yR9ssdFsW2aQtqyau3Bkckey4km8XQRBYWLE9scD/xrqFv5kos7gdfEPr8OzMnw+yhPvy+X9ZuU/WWZ7PHHRMO7G8JnZTdRELkUsFi+8qUv5en3PyZl+n2i7AETbkJaefp9N9H0+3eGPO8I699PdMnG3vB7+jXwweH+HhrLLwsNISIi40CYmHMXcJpHw8sTngKZiMgYZmaXE50v3E40Yet/AJvd/dxG1mss0TkyEZGx7ZVEF0K/jGhY9GdEdxeRQD0yEREZ1zT9XkRExrVjamixs7NT3UsRkWPcjBkzBlykrx6ZiIiMawpkIiIyrimQpSgUdGnGYKi9slNbZae2ym6it5UCmYiIjGsKZCIiMq4dU7MWRUSOZe7OgQMHKJUGPuyira2Nzs60+2yPT7lcjqlTpzLwpv/VKZCJiIwTBw4coLW1lZaWgY+ga21tpa2trUG1Gn49PT0cOHCAadOmZcqvoUURkXGiVCpVBLFjUUtLS0WvsxYFskHY0Fnko7/czcfu282WA8VGV0dERFAgy8zdWfHL3dy58TB3bDjMx+7b0+gqiYiMqr179/KNb3yjZp7nn3+ef/u3f6tb1vPPP8/ZZ589LPXKFMjMbImZrTezjvB03uT6VjO7Pax/2Mzmx9ZdGdLXm9n5Ie0UM/sPM1tnZk+b2V/F8s82s7vNrBB+zgrpZmZfCWU9aWavG+rBD8ae7hJP7OrtX/7PF4fyQGIRkfGns7OTm266qWae3/3ud9x5552jVKNI3ckeZpYHbgT+lOjprGvNbLW7r4tlWwHscfdTzWwZcB3wfjNbCCwjelLqScA9ZnYa0VOF/9rdHzezacBjZnZ3KPMK4Ofu/oUQNK8geoTBBUB7eL2B6BH0bxiGNshk26HK8dq+kpPPZZtVIyIy3GZ+c+uwlrf3w/Nqrr/mmmvYtGkTixcv5i1veQsA99xzD2bGpz/9ad773vdyzTXX8Oyzz7J48WKWL1/OO9/5Ti699FIOHjwIwBe/+EXe8Ibh/erOMmvxTKLHzW8EMLNVwFKiR2+XLSV6LD1EjyG/waJ5k0uBVe7eDWwysw7gTHd/kOi5Orj7fjN7BpgXylwKnBvKupXo0fV/E9K/5dFzZx4ys5lmNtfdtx3NgQ/WtkN9FWk9JZikwVkRmSCuvvpqnnnmGR544AF+/OMf881vfpMHHniAXbt28da3vpVzzjmHq6++mhtuuIHbb78dgEOHDvHDH/6QtrY2NmzYwIoVK/jFL34xrPXKEsjmAZtjy1uo7An153H3opl1AseF9IcS2w4I+WEY8g+Bh0PSCbHgtB04oUY95hEC4kh7ISWQdfc5k5rUIxORieehhx7iz/7sz8jn87zsZS/jnHPO4fHHH6+YMt/b28tnPvMZnnrqKXK5HBs2bBj2ujT0OjIzmwp8H/iku+9Lrnd3N7OjejTLUO89ltz+N79rBpoHpK3v2MDsY38mbCYT/V5vg6G2yk5tNVBbWxutra0jVn5XV1fN9d3d3ZRKJbq6uujr66O3t7d/m1KpRG9vLz09PfT19fWnf+UrX2H27Nncc889lEolXvGKV9DV1TWgrDT79u1jx44d/cvt7e1V65UlkG0FToktnxzS0vJsMbMmYAawq9a2ZtZMFMS+6+4/iOV5sTxkaGZzgfKRZKlHv1oHXU+hUKjYvuvFPcChAWknz38l86bkj3o/x4q09pJ0aqvs1FaVOjs7B1z4XD6n1dXVNSoXRM+ZM4eDBw/S1tbG4sWLueWWW7jkkkvYs2cPDz30ECtXrmTbtm0cOnSovz6HDh1i3rx5TJ48me985zv09fX1B+RcLle13tOnT+eUU05JXZeU5QzPWqDdzBaYWQvR5I3ViTyrgUvC+4uAe8O5rNXAsjCrcQHRRI1Hwvmzm4Bn3P2fa5R1CfDjWPoHw+zFs4DO0To/BlXOkfXpOZ4iMnHMnj2bs846i7PPPpu1a9eyaNEiFi9ezLve9S6uvfZaTjjhBBYtWkQ+n+eNb3wjN954Ix/96Ee57bbbeOMb30ihUGDKlCnDXi+L4k2dTGYXAv8C5IGb3X2lmV0LPOruq82sDfg20bmu3cCy2OSQzwEfIZqp+El3/6mZLQbuB34DlKcD/q27rzGz44A7gJcDzwPvc/fdIfjdACwh6hp92N0fjddzuJ4Qnfaf4Dk/fJF1ewdeBP3we17Gq2YOHG6ciPSfc3Zqq+zUVpU6OzuZMWNGRfpo9chGU7VjhconRGc6R+bua4A1ibSrYu+7gIurbLsSWJlIewBInSXh7ruA81LSHbgsS31HwtYqsxZFRKSxNHk8gwO9JTp7Kjt7GloUEWk8BbIM0s6PAfSUFMhERBpNgSyDFw6mjyH2pMc3EREZRQpkGaRdDA3qkYnI6MrlcvT0HPv3ee3p6SGXyx6e9GDNDF44WCWQ6RyZiIyiqVOncuDAAQ4fPjwgfd++fUyfPr1BtRp+5SdEZ6VAlkG1HlmvZi2KyCgys9SnJu/YsSPzxcPHIg0tZrC1So+sW0OLIiINp0CWQdVZixpaFBFpOAWyDKqdI9PQoohI4ymQ1dHd57zUVWX6vYYWRUQaToGsjmrDiqChRRGRsUCBrI5qw4qgey2KiIwFCmR11OyRaWhRRKThFMjqqNkj09CiiEjDKZDVkfb4ljL1yEREGk+BrI5aPbJe3TRYRKThFMjqONBbvdelO3uIiDRepkBmZkvMbL2ZdZjZFSnrW83s9rD+YTObH1t3ZUhfb2bnx9JvNrMdZvZUoqzbzeyJ8HrOzJ4I6fPN7HBs3deP9qAHo9ZpMJ0jExFpvLo3DTazPHAj8KfAFmCtma1293WxbCuAPe5+qpktA64D3m9mC4FlwCLgJOAeMzvN3fuAW4AbgG/F9+fu74/t+8tAZ2z1Bnc/Y/CHefSKXj1Y6c4eIiKNl6VHdibQ4e4b3b0HWAUsTeRZCtwa3t8JnGdmFtJXuXu3u28COkJ5uPt9wO5qOw3bvw+4bRDHM+z6agSrbvXIREQaLstjXOYBm2PLW4A3VMvj7kUz6wSOC+kPJbadl7FubwJedPdCLG2Bmf0K2Af8nbvfX23jQqFQbVUm5e0PHG4F8ql59u4/SKFQNRZPKENt74lEbZWd2iq7Y72t2tvbq64by88jW87A3tg24OXuvsvM/gj4kZktcvd9aRvXOuh6CoVC//ZNz+wAelPzNU+aTHv7y496P8eKeHtJbWqr7NRW2U30tsoytLgViD+x7eSQlprHzJqAGcCujNtWCGW8F7i9nBaGJ3eF948BG4DTMtR/SGqNHmpoUUSk8bIEsrVAu5ktMLMWoskbqxN5VgOXhPcXAfe6u4f0ZWFW4wKgHXgkwz7fBvzW3beUE8zs+DDxBDN7ZShrY4ayhqSvxhT7Xk2/FxFpuLpDi+Gc1+XAXUQni25296fN7FrgUXdfDdwEfNvMOogmcCwL2z5tZncA64AicFmYsYiZ3QacC8wxsy3A1e5+U9jtMionebwZuNbMeoEScKm7j/gJqmKt6featSgi0nCZzpG5+xpgTSLtqtj7LuDiKtuuBFampC+vsb8PpaR9H/h+lvoOp2KNXpeuIxMRaTzd2aOOmhdEa2hRRKThFMjqqB3IRq8eIiKSToGsDg0tioiMbQpkddSe7KFAJiLSaApkddTqkekxLiIijadAVketTpd6ZCIijadAVke968i8xt3xRURk5CmQ1VFraBH0KBcRkUZTIKsj2SNrTdwIX8OLIiKNpUBWg7tXnCNry9uAZU3BFxFpLAWyGpIxKmfQmgxkGloUEWkoBbIaiokg1WTQkksGMvXIREQaSYGshmJiRmJTzmhJtJiGFkVEGkuBrIbUHpmGFkVExhQFshpKiR5ZLm1oUT0yEZGGUiCrITn1vilntGj6vYjImJIpkJnZEjNbb2YdZnZFyvpWM7s9rH/YzObH1l0Z0teb2fmx9JvNbIeZPZUo6/NmttXMngivC+uVNVLShhabKyZ7jHQtRESklrqBzMzywI3ABcBCYLmZLUxkWwHscfdTgeuB68K2C4FlwCJgCfDVUB7ALSEtzfXufkZ4rclQ1ohIm+yRnH7fq6FFEZGGytIjOxPocPeN7t4DrAKWJvIsBW4N7+8EzjMzC+mr3L3b3TcBHaE83P0+YPcg6lq1rJGSHDXMGxWzFrs1tCgi0lBZAtk8YHNseUtIS83j7kWgEzgu47ZpLjezJ8Pw46xB1GNYJe+zmE8bWtSjXEREGqqp0RVI8TXgHwAPP78MfGSwhRQKhSFVolAosPGQAZP600rFXnoOdRNvts0vbKOgB5MNub0nErVVdmqr7I71tmpvb6+6Lksg2wqcEls+OaSl5dliZk3ADGBXxm0HcPcXy+/N7F+BnwyiHv1qHXQ9hUKB9vZ2unf3wuM7+tMnt7Ywe0Yz7Dzcnzbr+BNob59y1Ps6FpTbS+pTW2WntspuordVlqHFtUC7mS0wsxaiCRerE3lWA5eE9xcB93r0oK7VwLIwq3EB0A48UmtnZjY3tvgeoDyrcdBlDVXF0GLOKq4j02NcREQaq26PzN2LZnY5cBeQB25296fN7FrgUXdfDdwEfNvMOogmcCwL2z5tZncA64AicJm79wGY2W3AucAcM9sCXO3uNwH/ZGZnEA0tPgf8Rb2yRkraZI/KmwZrsoeISCNlOkcWpsCvSaRdFXvfBVxcZduVwMqU9OVV8n+gRj1SyxopldPvoTk5a1HT70VEGkp39qih8oJoDS2KiIw1CmQ1JG9Rlc+l3TRYPTIRkUZSIKshedPgqEc2MI9m3ouINJYCWQ3JocV8ymNcdGcPEZHGUiCrIW2yhx7jIiIytiiQ1VDZI6t8jIsme4iINJYCWQ3JzlY0/V5DiyIiY4kCWQ19FTcNTpl+r6FFEZGGUiCrofIJ0dBa8YTo0auPiIhUUiCrIctjXHRnDxGRxlIgq6GyR5Z2Zw8FMhGRRlIgqyEZo5pMQ4siImONAlkNlUOLlvKEaPXIREQaSYGsBt1rUURk7FMgqyHZI2syKu61qKFFEZHGUiCrofKCaKvskWloUUSkoTIFMjNbYmbrzazDzK5IWd9qZreH9Q+b2fzYuitD+nozOz+WfrOZ7TCzpxJlfdHMfmtmT5rZD81sZkifb2aHzeyJ8Pr60R50VskYlbeUey1qaFFEpKHqBjIzywM3AhcAC4HlZrYwkW0FsMfdTwWuB64L2y4ElgGLgCXAV0N5ALeEtKS7gVe7+2uBZ4ErY+s2uPsZ4XVptkM8epVDi3qMi4jIWJOlR3Ym0OHuG929B1gFLE3kWQrcGt7fCZxnZhbSV7l7t7tvAjpCebj7fcDu5M7c/X+7ezEsPgScPMhjGjZZJnvoXosiIo2VJZDNAzbHlreEtNQ8IQh1Asdl3LaWjwA/jS0vMLNfmdkvzexNgyjnqCTvtdikoUURkTGnqdEVqMbMPgcUge+GpG3Ay919l5n9EfAjM1vk7vvSti8UCkPaf6FQ4KVdzUBzf9re3bvYvOlFYHJ/WnexNOR9HQvUBtmprbJTW2V3rLdVe3t71XVZAtlW4JTY8skhLS3PFjNrAmYAuzJuW8HMPgS8EzjPPXq6pbt3A93h/WNmtgE4DXg0rYxaB11PoVCgvb2d6Xs6YeuB/vQTjp/D7582FR58oT+t121I+zoWlNtL6lNbZae2ym6it1WWocW1QLuZLTCzFqLJG6sTeVYDl4T3FwH3hgC0GlgWZjUuANqBR2rtzMyWAJ8F3uXuh2Lpx5cnipjZK0NZGzPU/6glnxCdzxl5g/jgYskrhyBFRGT01A1k4ZzX5cBdwDPAHe7+tJlda2bvCtluAo4zsw7gU8AVYdungTuAdcDPgMvcvQ/AzG4DHgReZWZbzGxFKOsGYBpwd2Ka/ZuBJ83sCaIJJZe6e8VkkeHUl7jYucnAzHS/RRGRMSTTOTJ3XwOsSaRdFXvfBVxcZduVwMqU9OVV8p9aJf37wPez1He4JHtkTSHst+SMrthFZj0lZxIDJ4GIiMjo0J09aqi8IDoKVrpxsIjI2KFAVkMxMWRYvoRMQ4siImOHAlkNlUOL6T0yPVxTRKRxFMhqSJvsASl399DQoohIwyiQ1VB9ssfAfBpaFBFpHAWyGpI9slyY7JHskfWqRyYi0jAKZDUkbxrcP7SY042DRUTGCgWyGipuGhwCWMXQoh7lIiLSMApkNVT0yMrnyJJDi+qRiYg0jAJZDWlPiIbK6featSgi0jgKZDUknxBdvrNH8oLoXs1aFBFpGAWyGpIdrfi9FuP0cE0RkcZRIKsh2SNrqnKvRQ0tiog0jgJZDVUneyRaTUOLIiKNo0BWQ7KjlatyiyoNLYqINI4CWQ0V15GV7+yhx7iIiIwZCmQ1VL+ObGC67rUoItI4mQKZmS0xs/Vm1mFmV6SsbzWz28P6h81sfmzdlSF9vZmdH0u/2cx2mNlTibJmm9ndZlYIP2eFdDOzr4SynjSz1x3tQWdVbbKHZi2KiIwddQOZmeWBG4ELgIXAcjNbmMi2Atjj7qcC1wPXhW0XAsuARcAS4KuhPIBbQlrSFcDP3b0d+HlYJuy/Pbw+Bnwt2yEevYoLoqvd2UNDiyIiDZOlR3Ym0OHuG929B1gFLE3kWQrcGt7fCZxnZhbSV7l7t7tvAjpCebj7fcDulP3Fy7oVeHcs/VseeQiYaWZzsxzk0Ure/f7InT0GpuumwSIijZMlkM0DNseWt4S01DzuXgQ6geMybpt0grtvC++3AycMoh7DqtoToluTT4jWTYNFRBqmqdEVqMXd3cyOqrtTKBSGtO9CoUB37yTgSND63aaNHGiBPbvyQGt/+kt7OikUXhrS/sa7obb3RKK2yk5tld2x3lbt7e1V12UJZFuBU2LLJ4e0tDxbzKwJmAHsyrht0otmNtfdt4Whwx2DqEe/WgddT6FQoL29HX/kBeBIHD3t1N9jVmuOkzkIHXv709umTqO9ffZR72+8K7eX1Ke2yk5tld1Eb6ssQ4trgXYzW2BmLUSTN1Yn8qwGLgnvLwLudXcP6cvCrMYFRBM1Hqmzv3hZlwA/jqV/MMxePAvojA1Bjojkqa/yiGJrxWNcRrIWIiJSS90embsXzexy4C4gD9zs7k+b2bXAo+6+GrgJ+LaZdRBN4FgWtn3azO4A1gFF4DJ37wMws9uAc4E5ZrYFuNrdbwK+ANxhZiuA54H3haqsAS4kmjByCPjwcDRALcVEgCo/IbotEcgOJi84ExGRUZPpHJm7ryEKJPG0q2Lvu4CLq2y7EliZkr68Sv5dwHkp6Q5clqW+w6XaZI9piWmL+9UlExFpGN3Zo4ZqPbLpLQN7ZPt71CMTEWkUBbIqSu4kw1P5HNn0RI9sn3pkIiINo0BWRcVdPQws3KJqmnpkIiJjhgJZFRXDirGWSuuRuSuYiYg0ggJZFRUTPexIL6w1P/A2Vb0l6NLdPUREGkKBrIqK+yzGWsrMKnp1DwzzAAAWU0lEQVRlmrkoItIYCmRV9CV6ZIlLxypmLu7TQ8lERBpCgayKyqn3AwNX5bVkOkcmItIICmRVVHs6dJl6ZCIiY4MCWRXJp0Pn6/TI9qlHJiLSEApkVSRvGKxzZCIiY5MCWRWV91kcuL5y1qJ6ZCIijaBAVkW9yR7qkYmIjA0KZFUkJ3vkEy1VcY5Mt6kSEWkIBbIq+krV7+wBML1FF0SLiIwFCmRVVNw0uO70e/XIREQaIVMgM7MlZrbezDrM7IqU9a1mdntY/7CZzY+tuzKkrzez8+uVaWb3m9kT4fWCmf0opJ9rZp2xdVcxgpLT75sSsxb1cE0RkbGh7hOizSwP3Aj8KbAFWGtmq919XSzbCmCPu59qZsuA64D3m9lCYBmwCDgJuMfMTgvbpJbp7m+K7fv7wI9j+7nf3d95tAc7GJUXRNeb7KEemYhII2TpkZ0JdLj7RnfvAVYBSxN5lgK3hvd3AudZ9PCupcAqd+92901ARyivbplmNh14K/Cjozu0oUnOWkxeR6YemYjI2JAlkM0DNseWt4S01DzuXgQ6geNqbJulzHcDP3f3fbG0s83s12b2UzNblKHuR61UcdNg9chERMaiukOLDbQc+EZs+XHgFe5+wMwuJOqptVfbuFAoDGnnz2/ZCrT1L3cfPjSgzH1FgMn9y3u7i0Pe53g2kY99sNRW2amtsjvW26q9verXfaZAthU4JbZ8ckhLy7PFzJqAGcCuOttWLdPM5hANP76nnBbvmbn7GjP7qpnNcfedaZWuddD1FAoFTph7Eqzb3Z82Y+pk2ttf3r/cV3J46IX+5YN9xu+deiq5RM9tIigUCkNq74lEbZWd2iq7id5WWYYW1wLtZrbAzFqIJm+sTuRZDVwS3l8E3OvuHtKXhVmNC4h6UI9kKPMi4Cfu3lVOMLMTw3k3zOzMUPddgzvc7CoviLaK5SmJqYy6TZWIyOir2yNz96KZXQ7cBeSBm939aTO7FnjU3VcDNwHfNrMOYDdRYCLkuwNYBxSBy9y9DyCtzNhulwFfSFTlIuDjZlYEDgPLQrAcEcknRCen30N0nuxgLOLt7ykxo0WX5omIjKZM58jcfQ2wJpF2Vex9F3BxlW1XAiuzlBlbd25K2g3ADVnqOxwqnxBdGcmmNefYxpGIp0e5iIiMPnUfqqj3YE2onLm4XzcOFhEZdQpkVVQ+WLMyT/JRLuqRiYiMPgWyKpL3Wkze2QNgmh7lIiLScApkVSQne2TpkWnWoojI6FMgq6LyCdHqkYmIjEUKZFXUu9ci6ByZiMhYoEBWRWWPrDLPtJbkU6LVIxMRGW0KZFVUXhBd2SWb3qw7e4iINJoCWRUVT4hOvbOHemQiIo2mQFZFcmgxea9FqOyRKZCJiIw+BbIqkpM90u+1qOn3IiKNpkBWRfJei6nT79UjExFpOAWyKjJNv1ePTESk4RTIqqi4RVVKIKvokfWqRyYiMtoUyKqouGlwytDi5CYb0FPr7oPuZAQUEZERpUBWRZYemZlV9Mr2q1cmIjKqFMiqSPbI0iZ7QMp5sh71yERERlOmQGZmS8xsvZl1mNkVKetbzez2sP5hM5sfW3dlSF9vZufXK9PMbjGzTWb2RHidEdLNzL4S8j9pZq8byoHXk+WCaKg8T9apmYsiIqOqbiAzszxwI3ABsBBYbmYLE9lWAHvc/VTgeuC6sO1CYBmwCFgCfNXM8hnK/Iy7nxFeT4S0C4D28PoY8LWjOeCskk+IzldpqYq7e2jmoojIqMrSIzsT6HD3je7eA6wClibyLAVuDe/vBM4zMwvpq9y92903AR2hvCxlJi0FvuWRh4CZZjY3Q/2PSl9yaDHlXouQNrSoHpmIyGhqypBnHrA5trwFeEO1PO5eNLNO4LiQ/lBi23nhfa0yV5rZVcDPgSvcvbtKPeYB29IqXSgU6h5YLXv27SfePDt3bKfgfRX5rKtlQL71m7dxWk9lvmPdUNt7IlFbZae2yu5Yb6v29vaq67IEstF2JbAdaAH+f+BvgGsHW0itg66nUCgwacpU2NnVnzZv7om0L5hckXf+7r3w0sH+5eYZx9PePu2o9z0eFQqFIbX3RKK2yk5tld1Eb6ssQ4tbgVNiyyeHtNQ8ZtYEzAB21di2apnuvi0MH3YD3yQahsxaj2FTOdkjfWhxTlt+wPJLXRpaFBEZTVkC2Vqg3cwWmFkL0eSN1Yk8q4FLwvuLgHvd3UP6sjCrcQHRRI1HapVZPu8VzrG9G3gqto8PhtmLZwGd7p46rDgckpM90h6sCTCnbeCKnQpkIiKjqu7QYjjndTlwF5AHbnb3p83sWuBRd18N3AR828w6gN1EgYmQ7w5gHVAELnOPTjSllRl2+V0zOx4w4Ang0pC+BriQaMLIIeDDQz76GrJO9jguEch2dU2882MiIo2U6RyZu68hCiTxtKti77uAi6tsuxJYmaXMkP7WKuU4cFmW+g4H9chERMYH3dmjimSPrNoF0ccrkImINJQCWRWVF0Rnm+yhQCYiMroUyKroy/CEaIAZLTZg3cGiczgZBUVEZMQokFVRzPCEaIjugJ+c8LFTEz5EREaNAlkVWZ4QXVY5c1HDiyIio0WBrIo+zzbZA+B4nScTEWkYBbIqKh6sWWVoESqn4OvuHiIio0eBrIrk0GK1yR5QObSoc2QiIqNHgayKrJM9oLJHpnNkIiKjR4GsiuTQYo04pnNkIiINpEBWReW9FqvnrRxaVCATERktCmRVVN5rMfvQos6RiYiMHgWyKiome9RoKd04WESkcRTIqkhO9qj2YE3QZA8RkUZSIKuiNIg7e8xszQ1Yv7/X6U7OFhERkRGhQFZF5fT76nlzqfdbVK9MRGQ0ZApkZrbEzNabWYeZXZGyvtXMbg/rHzaz+bF1V4b09WZ2fr0yzey7If0pM7vZzJpD+rlm1mlmT4TXVYygyguia3TJgDmtmvAhItIIdQOZmeWBG4ELgIXAcjNbmMi2Atjj7qcC1wPXhW0XAsuARcAS4Ktmlq9T5neB04HXAJOAj8b2c7+7nxFe1x7NAWeV9QnRZeqRiYg0RpYe2ZlAh7tvdPceYBWwNJFnKXBreH8ncJ6ZWUhf5e7d7r4J6AjlVS3T3dd4ADwCnDy0Qxy8UsrprVy9HpkuihYRaYgsgWwesDm2vCWkpeZx9yLQCRxXY9u6ZYYhxQ8AP4sln21mvzazn5rZogx1PyoVNwyuHcMAmDNJPTIRkUZoanQFavgqcJ+73x+WHwde4e4HzOxC4EdAe7WNC4XCUe84Gcjy5nXLs4NNQMuR/b+wk0LLtqOuw3gzlPaeaNRW2amtsjvW26q9verXfaZAthU4JbZ8ckhLy7PFzJqAGcCuOttWLdPMrgaOB/6inObu+2Lv15jZV81sjrvvTKt0rYOu51fPDPxANOdydct7Vd8B+F1n/3Jp8gza22cddR3Gk0KhMKT2nkjUVtmprbKb6G2VZWhxLdBuZgvMrIVo8sbqRJ7VwCXh/UXAveEc12pgWZjVuICoB/VIrTLN7KPA+cByd+8fnzOzE8N5N8zszFD3XUdz0PVU3DA4Qyslz5G9dFhDiyIio6Fuj8zdi2Z2OXAXkAdudvenzexa4FF3Xw3cBHzbzDqA3USBiZDvDmAdUAQuc/c+gLQywy6/DjwPPBji1g/CDMWLgI+bWRE4DCwLwXLYJUNQvan3UDlrUXf3EBEZHZnOkbn7GmBNIu2q2Psu4OIq264EVmYpM6Sn1sndbwBuyFLfoap8OnT9bY7XjYNFRBpCd/ZIUfSBPbBMsxaTgaxbPTIRkdGgQJaiYtZiradqBrNacwMevrmvxzmUvD2IiIgMOwWyhK6iUzg4sFlq3TC4LGfGy6cOnPCxdkfPcFZNRERSjOXryEZVseSc+YMXee5AHyVvHbAuy2QPgMUntvLc/kP9y/dt6+ZPTmob1nqKiMhA6pEFTTmj6Om3p8oy2QPgzXMHBsBfbusehpqJiEgtCmQx7TPSO6hZhhahMpA9vrOXzh6dJxMRGUkKZDHVA1m2SHbi5DynzzxSRsnh/2xXr0xEZCQpkMVUC2RZhxahsld2n4YXRURGlAJZTPuM5tT0rJM9ICWQvaBAJiIykhTIYk6rNrQ4iFZafGLrgOvJ1u0tsuOw7vIhIjJSFMhiTpiUY1pzZe8ry509yma25jjjuIE9u/s1vCgiMmIUyGLMLPU8WZY7e8Qlhxd/uOnwkOolIjKebD5Q5JpHO/mXJ/dzsHfkZ24rkCWcmhLIBtMjA3jLSQMD2U9+18WPFMxEZALoLTnvu3sX1//mAJ9/bB9//eDeEd+nAlnCaSkTPgbbI1t8Yiuvnj2wnP/24B62H9K5MhE5tt21uYtn9hb7l+/YeJgXDo7sd58CWULa0OJge2T5nPH1N82iJda6e7qdv3xgD8W0W4eIiIxTLxzs49+fP8zWEKy+XTg0YH3J4Xsdh9I2HTa612JC6jmyQQYygFfPbuZzr5vO1Y/u60+7e2s377lrJzcsnsWvdvbywPZuDHjVzCZeNbOZ1x/fwqTBRk0RkQZ5YmcP7/nfO9nT7cxsMW5YPIu7t3RV5PtO4SCfeu1UcoO4lGkwFMgSXjmtskmO9i5Tly+ays82d/Hgi0fugn//9h7+4M4XU/NPbzH+y6mTWXH6lKrXtImIjLSSO892FjlxUp6ZrdHQ0v3buvnsQ3vZ013iU6+dxvL2yXzkF7vZ0x2NMu3tcT74H7tT71f73P4+HtjeUzERbrhkCmRmtgT4H0Ae+Ia7fyGxvhX4FvBHwC7g/e7+XFh3JbAC6AP+q7vfVatMM1sArAKOAx4DPuDuPbX2MZzaUnpEz+8vpuSsrzzEeOGanWzNcH5sX4/z9XUH+fq6g/zJ3FZWnD6FOW057n2hmw2dReZOyfHmua2cMqWJh3f08NjOHiY3Ge94eRtvCtevbT9cYmdXiQXT8kxtjj6A3X3O7w4UmdGS4/i2HGaGu/Pi4RLFknPSlPyI/ackIoPn7nT1QVs+mk1dTtvVXaI1b0wLf9vuzpaDfXQcNE4pOm1N0d/2pv19FDqLnDg5x6tnNZMz+O3eIg+EW+a9eW4rp81o4lc7e7n12YM821nkTXNbueS0KRQ6e/nbhztZt7dIcw4+9vtTWTAtzxUPd1IMQeqzD3dy49MHeP7AwO+1WmdOvvPswRELZOZe+5yNmeWBZ4E/BbYAa4Hl7r4ulucTwGvd/VIzWwa8x93fb2YLgduAM4GTgHuA08JmqWWa2R3AD9x9lZl9Hfi1u3+t2j7ide3s7ByWE1Azv7l1wHJrHl784LyjLu+lw3185Be7uX/7yD2f7KTJOYoOOw5H3UcjGiZtMni2s9j/AZzZYpw0Jc/mA33s740SpzYZr5rZRM7gxcMl9vaUmNWS44RJedqajN3dJXZ39dGUM+a05ZjZkuNQ0ensKdFTclpKPbxs2mQc2N9b4mCv05o3prcYrTnjYNHZH6bgTmvOMaXJ6Ck5B4tOd9GZ3GxMaYoeTHqoGKU352Byk9GWNw4XnUNFp89hSpMxuckoAYeLTlef05ozJjUZTbko7XCfkwMmhe17SnC4WKLo0JY3JuUNB7r6ou2bzWhrMlpyUVp3n+McyVv0KL2nz2kJaTk7kjdvRms+2r6nFP3j0OdRG7TljVJ5+5LT193FzKmTaTLoDnkN+rcv76uvBC35KN0o7z+6OL81bzTbkX050JKL6tsXtu8tOc25aP/V6trr0fP34nV1jhxrU9i+yY60lWG05qN9lbfvLR1pF7Pod9Adti/vqztsX/Ko/pOaQl2LTnfJaQm/w7wd+R32dHUxa+ok2vJGV1/0GegtRZ+LyU1RXQ/1Oof6nJZc9NloyRuHis6B3qhdpjYZU5qN7r4orbvPmdRkTG3OkTfY11PiQK/TlIPpLTkm5aPPa2dPid6SM6Mlx/SWHL0lZ293if29zpQmY2Zrjpacsae7xO7uEjmD2a05Zrbm2N8T/TN5qOjMbs0xZ1KOksP2Q33s6ioxuck4YXKe6c3G9sMlXjjYR9GdkybnmTs5T2dPief393Gg6ExvMV4xtYnmHHR0FtkX/mZPmpxj3pQ8HfuK/T2ivMGrZjSxu7vE9sNHhpGmNxszWnNsTgSdOW05dnYNHG7KWe1gNBRtefjt++f29/CGYsaMGQP+884SyM4GPu/u54flKwHc/R9jee4KeR40syZgO3A8cEU8bzlf2KyiTOALwEvAie5ejO+72j48dgDDFchERGTsSgayLKFxHrA5trwlpKXmcfci0Ek0NFht22rpxwF7QxnJfVXbh4iITGCafi8iIuNalskeW4FTYssnh7S0PFvCsN8MogkZtbZNS98FzDSzptDriuevto9+ye6miIgc+7L0yNYC7Wa2wMxagGXA6kSe1cAl4f1FwL3h3NVqYJmZtYbZiO3AI9XKDNv8RyiDUOaP6+xDREQmMnev+wIuJJpluAH4XEi7FnhXeN8G/BvQQRSoXhnb9nNhu/XABbXKDOmvDGV0hDJb6+1jOF/AklDXDuCKkdjHWH8BNwM7gKdiabOBu4FC+DkrpBvwldBeTwKvi21zSchfAC5p9HGNUFudQvTP1zrgaeCv1F5V26ot/O3+OrTVNSF9AfBwaJPbgZaQ3hqWO8L6+bGyrgzp64HzG31sI9hmeeBXwE/UVjXaqdEVGEuv8KHZEIJpS/iDW9joejWgHd4MvC4RyP6pHNiJZqNeF95fCPw0fEGfBTwc0mcDG8PPWeH9rEYf2wi01dxyMAKmEf1ztlDtldpWBkwN75vDF+5ZwB3AspD+deDj4f0ngK+H98uA28P7heFvszV8sW8A8o0+vhFqs08B34sFMrVVykuTPQY6E+hw943u3kN0YfbSBtdp1Ln7fcDuRPJS4Nbw/lbg3bH0b3nkIaJznHOB84G73X23u+8h6pUsGfnajy533+buj4f3+4FniGbYqr0SwjEfCIvN4eXAW4E7Q3qyrcpteCdwnkVXBy8FVrl7t7tvIuptnDkKhzCqzOxk4B3AN8KyobZKpUA2UJZLDSaqE9x9W3i/HTghvB/sJRbHLDObD/whUU9D7ZXCzPJm9gTR0PXdRD2EwV5yMyHaCvgX4LNA+arlo7k8aUK0lQKZDJpHYxaaaBNjZlOB7wOfdPd98XVqryPcvc/dzyCakXwmcHqDqzQmmdk7gR3u/lij6zIeKJANlOVSg4nqxTAERvi5I6RXa7MJ05Zm1kwUxL7r7j8IyWqvGtx9L9EkmbMJl9yEVWmX3DCIy3qOFW8E3mVmzxGd4ngr0b1p1VYpFMgGynKpwUQVv/wheVnEBy1yFtAZhtTuAt5uZrPMbBbw9pB2TAnnIW4CnnH3f46tUnslmNnxZjYzvJ9EdK/VZxj8JTfVLus5Zrj7le5+srvPJ/oeutfd/xy1VbpGzzYZay+qXBYwkV5EN3reBvQSjamvIBpv/znR1PB7gNkhrwE3hvb6DfD6WDkfITq53AF8uNHHNUJttZho2PBJ4InwulDtldpWryWaSv4k8BRwVUgf9CU3VLms51h8AedyZNai2irlVfemwSIiImOZhhZFRGRcUyATEZFxTYFMRETGNQUyEREZ1xTIRERkXFMgExGRcU2BTERExjUFMhERGdf+LzCxQKKOKw8aAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Make a few features by hand\n",
    "logs['total'] = logs[['num_25', 'num_50', 'num_75', 'num_985', 'num_100']].sum(axis = 1)\n",
    "logs['percent_100'] = logs['num_100'] / logs['total']\n",
    "logs['percent_unique'] = logs['num_unq'] / logs['total']\n",
    "logs['seconds_per_song'] = logs['total_secs'] / logs['total'] \n",
    "\n",
    "import seaborn as sns\n",
    "\n",
    "sns.kdeplot(logs['total']);\n",
    "plt.title('Distribution of Total Number of Songs');"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbkAAAEGCAYAAAD4yOuIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl4XGXZ+PHvPTNZm7VNl3RNC4GWshVKW7rQsqhlEYogoiwvvrihbCovoqLwCviiIIqogAKyKAiIsll+yNJd2lKghW6QQku3dMu+Z5bn98c5SWbOTJJJJslMZu7PdeVK5szJOc9Z7/OsR4wxKKWUUsnIFe8EKKWUUv1Fg5xSSqmkpUFOKaVU0tIgp5RSKmlpkFNKKZW0NMgppZRKWn0a5ETkURF5vS+XGbTsK0TE19nnfljfrSKyrb+W31MicoyIrBWRZhHZEe/0qHAi8kUR+VhE/CLyaLzTEw/9fV0OFBExInJpvNORSERkqYg81AfLKbH379y+SFd3ug1yduAy9o9XRA6JyEoRuVFEhjhmvw74YrQrFxGfiFwR5exPA2OiXXYP0jDX3rYSx1d3A7P6en0x+CVQC0wGToo0g32DMUE/+0XkZRE5ZkBTGoMujkdCExE38AjwDDAe61qINF+WiNwmImUi0iQilSLytohcO5Dp7S0RWWAfn7GdzNKj61REXh+MDwQicnNfPmyKiEdErrEfZOtEpFZE3hORH4tIYV+tZ6CJyDYRudUxeRdQDKwZiDR4opxvBXARVlAcBswFfghcKSKnGGP2Axhjavo6gSIigMcY0wQ09fXyO2OMqQfqB2p9USgFHjPG7OhmPj/QdgOaCNwLvCoiU3p7fEQk3RjT2pv/TSHFQA6w2Bizp4v57gdOxQqCG4A8YBpWYBz0Bvo6TQYikga8DJwM/AxYBhwEjgKuAhqA38QtgX3MGOMH9g3kCrv8AR4FXo8wfQxQCfy5s3mBqcCrQDXWgdoCXGZ/twMwwT/29CsAH9aN4D2gFTizbXrQstvmOwPYBDRjPRkc75zHke6x9voWACXONABL7fluBbY5/ve/gM12mnYDt2MF4LbvlwIPAT/BOoiVwONATjf7uBj4m72fmuzlTLe/i5TGWztZTqTtnWP/z+fsz2n2tm2399km4JuO/zHAtcCTQA3wtD19BPBnYL/9vx8C/x30f4cDz9nbUQX8GzgmwjGbA7wLNALvACd1sa1tx+ME4BXgANbDx9vAQke6hwHPYp1r+4HbgMdwnL/ANcBWexvKgB8HH8dO9u0sYLl9fKrsfTMiaLuc6V7QyXKqgau7WZcANwCf2Ofax8D1jnl2YN0Q77XPs/3Arwk9H7OAP9rHsAr4A/B/BJ3XdHGNdpK2Bfb2jY3mHMQK4n/Guh5asJ7i7wm6X0Tcb8BI+/uDQB2wCjglQjo+Yx+XRqxr80xHerpcjj3PqcD79vnwvv3ZAJd2sY0Rr0kgF3jQXl8LsA74bDfH+/tAADi5k+8Le3gPetiefsA+rndgZVB+ap8nB4E7IpxPd2Ddv2qBQ8DPAZfz/hbttWTP79xPJXRc53ODlnMk8C+sa7seeAk4PNp7R5f7t9sZOgly9nf3YV1Arkjz2ifMk1hPJJOwgtU59nfD7URfB4wCRgVtTABYa59sk+x5ryA8yAXsDZ4PHIv1NLQHyIp0wdnTgoOcGzjX/nySnY6h9ny3EnozOBsrl/RD4AjgS1g3jtscJ0E11s1mMvBZrBvQbV3sX8EKzuuxcsjHYBX5VAFFdhpHYd0c7rT/jhg0O9neE+ztOyfoGL1vp22ivR3VwJVB/2OACuBq4DCsXGQW1g3wXawHi0n2Mi4Oupnsw8qpHIN10t5nL2e445gtB+bZ++gVrIDr6eZ4LLD/f6q9/2/HutCPCEr3i8BHWOfNVKybaw2h5+StwKfA+fb2nwXs7OYYjcK68J+0t22uvQ+XBwWTk+x0n2vPn97JsrZgnadDu1jfd7CC6Tfsff8trJtI8DHaYZ8jN9nzXAR4HfP8Fuumdq59PP7P3h/B53Wn12gnaVtAz4Lcb7FyrDOxcquzga/b3+Xb58LT9j4bBaTb+3Mz1gPTdKyHpx9jBY0pjnRsABba++DP9nEqDDou3S1nNFZw/7O9Dz5j75OuglwW1rW4KyjdOfZ3z9rH5nPAFKyHkFZgchf7dD2d3GMd80V7D6oBfmHP89/2tryCVeVxBFagNAQ9ENhprsV6cDoSuMzeL9c5lv1Q0Odb6eJaAoZiXdt3B+0nN44gZ+/PT4E3gBPtnyXANuzriG7uHV3utyh27KOdHQCsi8/Q8UQbMq+9s6/oYtk+5/d0PCXN6+biaZvv9KBphVhPAVd2cdNvD3L257n25xLHfLcSejNYATzjmOc6rJtR24FYCmxwzHM/8FYX++B0e/1HBU3LAMqBnzpOwpu7OVbOfTQc64moBisXNtE+USY7/u+nwPqgzwZ42DHPlVg32s5ubrcCqx3ThKBcSNAxOyFonpn2tCO7Oh6drHMD8GP779II50Ma1o3odftzNtYToDMHeDlQ3cV6bsN6ak4Pmnacvb5T7M8lOJ5OO1nWHKwL2o91M/0jsAiQoHl2Ab90/N+vgU8c58OLjnleAZ6y/x6CdTO/0jHPasd53eU1GiH9C+hZkHsBeLSL5b3u/N5exm4cNy/gTeA3jnR8Iej7kYSWWkSznNvt4xGcGzqHLoKcPc/NwA7HtMPt/zvLMf1d4JEultUI/DaKfR/tPWi9Y55NwAcRrp27HefTCsc8Pwd2BX1eih3kiPJawgpUtzrmKSE0yF1pL6vIcSybgMuDjmWX947OfmJtXSn2b9PJ93cDD9mtcm4VkRN6sOy3o5zvrbY/jDFVWE/KU3uwnmhNxXqKCLYMyMTK7bTZ4JhnL9YB62q5FcaYzW0TjDEtWLm73myHW0TqRaQeq7jicOBCY8wBrKdZAda1zWPP9yOsIBFsrePzicBmY8zuTtZ7EnCiY7l1WCd08LLbnr7b7LV/d7WPEJHhIvIHEdkqItX28qcCE+xZjrJ/r25fkTFerOKiNlOxnhqfc6TzQSBfRIZ3svqpWAG8vV7SGLMBK0D06BgZY1ZhnS/zsIpSRwJ/B14USx7Wg1ikc61ERLKDpq13zBN8rh2OlSta7ZjnLcfnWK7RaPwBuFBENorIvSJypoh0d99py8VXO47TPMLP0/Z9YKy2AX469kE0yzkKWGuMCW4RurIX29m2LAg/dsvp+jyRLr4L1tt70D6sByrntBGOac5zYxUw1j4nI6WlN9dSJFOx7i2H2ibYx/JDQvdbr+4d0TY86SpxNVhFUmGMMbeJyF+xihNOA34kIr80xtzczXL9xpjmGNMGVq7FKa0PltsVZwMNw8D2R/QDx9vrPWCMqQv6ri0ds7GenII5H1QaerheF1Zxw9URvgtu8BIwVsWzc73d7aNHsYq7bsQqomjCqsdMd8zX2QNX8Dq+iFWs6VTZTRr6hH1D/Y/98yu7qfoTwClY9dDRiuZc62p/xHKNRsUY86qIjMcqvlsA/AX4QEROd5wHwVxYD6vnR/jOed5GahDlCvod7XLi6UM6AmRf8Do+m06mxXJfise11Kt7R683UkTGAJcA/zDGRAomViqM+cQY8wdjzIVYxWJXBX3dilVGG4v2Zv4iUoBVDt6WKzqAlbMJjvTOJ9W2i6S7dGzCugkFm491s/24JwmOsNxhItJ+kotIBlZWfGNvFmiM2WaM+dgR4MCqqAUYb88T/NPdNrwDHNVF0/F1WA89uyMs+2APkt/Z8TgF+IMx5kVjzAdYxbmTgr5vO+Ynt00QEQ9WDrRNWwOlSRHSuK2Lm+4mYJaItAdUETkOq06pV8fIYYv9e4QxphariC3SubbdGBPtzXkb1r482TE9rFtMN9dozIwxlcaYp4wx38SqV5pPx0090j1gHdaxrY1wjPYSvWiWsxmYYXcBaTMnimVHSvcm+7fz2J1C1+fJX4DTRMR5rAAI6kLQX/egNs5zYzawxz4nnaK9lqK5x2/CurcUtU2w79lH0gfXV7RBLl1ERonIaLE6JV+FlbU9gFUJGkZEckTk9yJymohMFJFpWE+Lm4Nm2w6cai+3KNJyumGAX4rIKXZfsMexisietL9fa3++U0RKRWQh1kUc7FOsHN9ZIjJCRPI7Wdf/AReIyE0icoSIXIRVD/UrE1vz+jftdD4pInNE5Gh7OzKx6vP6jDFmG1Zfrj+JyGUicriIHCci/y0iP+jm35/C2lcvisgZ9jE9XUS+ZH//O6yT+QURmSdWh8+5InKHiMzuQTI7Ox4fApfY59/xdnraLx5jTBlW/ePvRWS+/dDwIFbrPmPPU49Vz/BzEfmOiBwpIlNF5GIR+UUXafqdvZxHReRosTqxPoFVh7GiB9uGiCwTkW+JyHQRmSAip2MV6VVjVbaDda5dIyJft8/bb2IFnp9Hux5jTIO9/beLyDn2OXsH1kOgsdMSzTXamaNE5HjHT7ZzJvv4f8He16VYD8b1WA0UwLoHnCgih4lIkVjN6f9qT/+XiHzWPpdmisgPRWRRtPsgyuXcj1V3/UcRmWIfjzuiWPZ2YJSInGynO9t+UHwW+IOIfE5EJovIvcDRwF1dLOterFKQV0XkhqBzY6GIPI9VzwX9dw9qc7xYRdZHiMhXsOr7fhVpxh5cS9uBOSIy3t5PkWLOk1gtPp8WkRNE5ESsUpo9WI2SYtNVhZ1dufcoHc0/fVhFkyuxio2GRJi3rZI/0078dqyIf8BO8Lig+RdiPcW24uhCECEdIdPbPmO18NuCVcm+lqCKSXu+s+3vm7DKmD+Ho4m3vS17sIr6ltrTbiVyF4K29O7BuhjCuhA4/iesgjrCtjm7ECzD7kIQNM8OetjwpJN53Pb2brW345C9vi8GzROx0h2rfuNx+3+a7WVcEfT9BKwbS1vz6U+xnlIndpY+HA2Bujgex2AV7zXZ++LbOBotYHUh+DtWUdQBrJZizwIvOdb5Nay6nGas1mlrgKu62W/BXQiqCepCYCJUpnexnJuwGhAcsNe/095HwQ2PBPgfrGvHi9WVIFIXgpsd0x5q21/257YuBLV2mv+A1d/qg2iv0QjpX0B4s/C2n1nOY4zVnWYjVmCrsc+14Kbjk+z9Wh98HtjH8n77PGi73v4JTHOkY6wjfSGN2bpbjj3P6cAHWOfsRqxi2+4anqTZ+66S0C4EefSwC4H9fx6soLIOq6qgFqvo+kdAQYz3oEiNe/4f8BfH+XQHHS1UK7BakHbXhaDLawmrHcC7WNeNoesuBIvp6ELwMhG6EHR374j0I/bMSiUduwhqK1YrxO/HOz2JQETeBKqMMRfEOy0qcYg1estDxpjb452WvhZrwxOlEoaInILVYuw9rE6538V6anw0fqmKH7sI/wSsqoV0rL5Pp2L1hVMqJWiQU8nEjVU8fDhWMd9G4FRjNVRJRQarLu+3WPXvW4HzjTH/L66pUmoA9bq4UkQyscrSM7CC5d+NMbc45rkCq8K1bSy/3xljYh7FWimllIpGLDm5FuA0Y0y93SJqpYi8Yoxxdj592hgTqe+UUkop1a96HeSMlQVsG6U/zf7pcbawpqZGW74opVSSy8/Pj3Zklz4V00gcIuIWkfVYTY9fM8ZEej/QBSLyvoj8XUTGxbI+pZRSqidiCnLGGL8x5nis/goz7I7MwV7CGmj3WOA1rLH6lFJKqQHRJ2MqGmPaRmtY6JheYazBhsHqqHqi838TVVlZWbyTMKBSbXsh9bY51bYXUm+bU217oxHL2JXDxRorEhHJwnoP01bHPMVBH8+lY4w+pZRSqt/F0rqyGHjMHlXChfWeo5dF5GfAOmPMi8C1InIu1lA7lVhDsyillFIDIpbWle8D0yJM/2nQ3z+kkwGco1g+9fX1BAKdvuCgX2VmZlJTU9P9jEki0va6XC5ycnIQiUujKKWUilnCjnhSX19PRkYG6enO14UNjIyMDDIzM+Oy7niItL2tra3U19eTm5sbp1QppVRsBvJlnj0SCATiFuCUJT09PW45aaWU6gsJm5NTSinVtWaf4c71tXxU4+MbU4YwJt4JSkAa5JRSapD6xfpafvOBNfDUivIWnpkGpXFOU6JJ2OJK1eHll19m69atXc7z/PPPM2vWLAoLC3nvvfdCvrvnnnuYNm0a06dP54033mif/vrrrzN9+nSmTZvGfffd1y9pV0r1j4AxPLmtsf1zndewusodxxQlpkGTkyv4857uZ+qB6q/2b8be5/Ph8fTN7v3Xv/7FwoULmTx5cqfzTJkyhSeeeILrr78+ZPrWrVt57rnnWL16NeXl5SxatIh33nkHgBtuuIHnn3+e0aNHs2DBAj7/+c93uQ6lVOJ4v8LL/qbQOvONdZpvcdI90oVPP/2Uk046ia9//evMmDGDyy+/nMbGRtavX89ZZ53F/Pnz+cIXvsC+ffsAOPvss7nppptYsGAB999/PwcOHOCSSy5hzpw5zJkzhzVrrKE9n376aU477TTmzp3L9ddfj9/vB2DMmDHcdtttzJkzhzPOOIMDBw6wZs0aXnnlFX7yk58wd+5ctm/fHjGtRx55JKWl4QUVixcv5oILLiAjI4OSkhImTZrEO++8wzvvvMOkSZMoKSkhPT2dRYsWsXjx4n7ak0qpvvbv3c1h0z6o05yckwa5bpSVlXHllVeydu1acnNzeeihh7jxxht5/PHHWbZsGZdeeim33XZb+/xer5elS5dyzTXX8IMf/IA5c+awatUqli9fzuTJk/nwww/5xz/+wauvvsrKlStxu90888wzADQ0NDB9+nRWrVrF7Nmzeeyxx5g5cyZnnnkmt912GytXrmTixIk9Sn95eTljxnTkWkePHk15eXnY9OLiYsrLy2PcW0qpgfJahCC3rUFo8GqL6GCDprgyXsaOHcusWbMAuOiii7jnnnvYsmULixYtAqyuDiNHjmyf//zzz2//e/ny5TzwwAMAuN1u8vPzefrpp9mwYQOnnnoqAM3NzRQVFQFWk/2FC63hP48//niWLFnS/xuolBp0DjX7WXfQGzY9gLC+wsucURlxSFViGjRBrr/r0KKVk5PD5MmTee211yJ+P2TIkC7/3xjDl7/8ZW655Zaw79LS0tpHF3G73fh8vpjTW1xczJ49HfWZe/fupbjYGlI0eHp5eXn7dKVUYntjT0unL+9cd7BVg1wQLa7sxu7du1m7di0Af//73znppJM4dOhQ+zSv18uWLZHHnZ4/fz4PP/wwAH6/n5qaGubPn88LL7zAwYMHAaiqqmLnzp1dpiEnJ4e6urpepf/MM8/kueeeo6WlhR07dvDxxx9z4okncsIJJ/Dxxx+zY8cOWltbef755znzzDN7tQ6l1MD6967woso2bx9oHcCUJD4Nct0oLS3loYceYsaMGVRXV/ONb3yDxx57jFtuuYU5c+Ywb9689oDndOedd7JixQpmz57N/Pnz+fDDD5k8eTI333wz559/PrNnz2bRokXs37+/yzRccMEF3HfffcybN6/ThicvvfQSRx11FG+//TYXXXQRX/jCFwCr1eX555/PzJkzufDCC7n77rtxu914PB7uuusuLrjgAmbMmMG5557LlClTYttZSql+5wsY3tjTeZBbd7AVYzrL56UeiffOqKmpiZiAmpoa8vPzBzo57Zqbm9m/fz8XX3wxb731VtzSMVCam5sjjtUZ7+PQn8rKyiK2SE1Wqba9kJzb/Nb+Fs5cfKj9c2GG0OQzNPs75tn4xZGMzUms2qj8/Py4jPSuOTmllBpE3jkYWhx5+phMphWFjvMbqVFKqkqsUJ9gJkyYkHC5uBtuuIHVq1eHTPvWt77FpZdeGqcUKaUG0s56f8jnY4amUZzt5q39HcHv7YOtLJqYNdBJS0ga5AaZu+++O95JUErFkTPIjc9xU5Ibeitfd1Abn7TRIKeUUoPIzvrQrkXjczyMyg4d6WRjpRZXtknYOjmXy0Vrqz6NxFNraysuV8KeIkqlHGMMux05uXE5bkZnu0gLulQbfIZGn458Agmck8vJyaG+vp6mpqa4rL+2tpa8vLy4rDseIm2vy+UiJycnTilSSjnVtBpqvR0N0jPdMDzThYgwLMPFvqABmyubA2Tn6ENqwgY5ESE3Nzdu6z9w4ADjxo2L2/oHWqptr1KDUaSiyrZRkoZmhga5ipYAY/UZNXGLK5VSSoVyNjoZl9NRFzcsI/R2XtGsxZWgQU4ppQaNSC0r2xRlhjY+0SBniSnIiUimiKwVkQ0isklE/jfCPBki8rSIbBORNSJSEss6lVIqVe2KUFzZZlimIyfXokEOYs/JtQCnGWOOA44HForILMc8VwJVxpjDgV8Dv4hxnUoplZK6Kq4c6gxympMDYgxyxlJvf0yzf5xjUZ4HPGb//XfgdGmrKVVKKRW1roortU4uspjr5ETELSLrgQPAa8aYNY5ZxgC7AIwxPqAGGBbrepVSKtX0rLgyNCCmqpi7EBhj/MDxIlIA/FNEjjbGbOzNssrKymJNTp9KtPT0t1TbXki9bU617YXk2eZ6H1S3Zrd/ThNDze5PqLPLxZqrXUDHm0R2VzVQVlY5wKkMlQhvgOizfnLGmGoRWQIsBIKD3B5gHLBbRDxAPlARaRmJsEPaJOMrOrqSatsLqbfNqba9kFzbvKnSi1VgZhmf6+HIIzq2ramiFTYe7PgsGZSWjh/IJCakWFtXDrdzcIhIFvAZYKtjtheB/7L/vhB408T7JXZKKTXIODuCj3O8L26YowvBIW1dCcSekysGHhMRN1bAfMYY87KI/AxYZ4x5EXgYeEJEtgGVwMUxrlMppVJOV41OIHLDE2MMqd7OL6YgZ4x5H5gWYfpPg/5uBr4Yy3qUUirV7QoLcqG370yPkO02NPqtoOY31liXBRmpHeR0xBOllBoEwosr3WHzFHhCa4IqtchSg5xSSg0G3RVXAuSnhQa5Q83ajUCDnFJKDQLdFVcCFKSFftYO4RrklFIq4TV4AyFjUXoERmWF374LHcWVOn6lBjmllEp4uxtCc3Fjhrhxu8IblBQ4iisrNSenQU4ppRKdM8hFanQC4UHukAY5DXJKKZXodjvq48YOiRzknA1PtLhSg5xSSiW8XY6c3NgIjU4AChyTteGJBjmllEp4u5195DrJyRVqnVwYDXJKKZXgIjU8iSS8Tk77yWmQU0qpBOcMcmOjbHiidXIa5JRSKqEFjGFPlDm5PA8EdyyoaTV4A6n90hcNckoplcAONAXwBmXICtKF3LTIt263QKHjbQRVKZ6b0yCnlFIJLLyosuuXxwzLDL2tp3pfOQ1ySimVwJxFlZ31kWsT6b1yqUyDnFJKJbBdUXYfaDPUkZNL9dftaJBTSqkEFm33gTZFmZqTC6ZBTimlEljYkF6ddB9o4yyuTPW+chrklFIqgYU1POlhcaXm5JRSSiWsngY5Z05O6+SUUkolpCafCekC4BYYld11kMtPD72t17ZqkFNKKZWA9jSEtqwsznbjifCy1GB5ziDn1RFPekVExonIEhHZLCKbROS6CPMsEJEaEVlv//w0tuQqpVTqiPZlqcHy00ODYE2KF1d23XW+az7g+8aYd0UkF3hHRF4zxmx2zLfCGHNODOtRSqmUtKu+Z90HQHNyTr3OyRljyo0x79p/1wFbgDF9lTCllEp1PR3tBLROzqlP6uREpASYBqyJ8PXJIrJBRF4Rkal9sT6llEoFPW1ZCZCbFlpcWec1BEzq5ubExLjxIpIDLAPuMMb8w/FdHhAwxtSLyFnAvcaY0uB5ampq2hNQVlYWU1qUUiqZfGdjBmurOwLbPUc1M29o9zmz+W9l0ejvCHZvzmokN5bKqV4qLe243efn53fdYqafxLTZIpIGPAf81RngAIwxtUF/LxaRP4hIkTHmUKTlBe+QeCsrK0uo9PS3VNteSL1tTrXthcG/zfvX7wM6cnMzSsdTOjSt0/nbtrfg3XIaGzuCYdG4iYzv5u0FySqW1pUCPAxsMcbc08k8o+z5EJEZ9voqertOpZRKFU0+w86ghicCTMrrvrgSIjQ+aU3d4spYQvsc4DLgAxFZb0/7ETAewBjzAHAhcJWI+IAm4GITa/moUkqlgI9rfQTfLMfluMn2RJcvyUvTxidteh3kjDErCX3TeqR5fgf8rrfrUEqpVFVW4w35fER+9LfrsL5yKRzkdMQTpZRKQB/VhI52ckRB9EFO+8p10CCnlFIJqMwZ5PI7b3DilOfIyaVycaUGOaWUSkAfVYcGudIeFFeG18lpTk4ppVSCCBjDtlpnTq4HdXKO1+1onZxSSqmEsafBT6OvI/dVkC4UZUZ/u85L0+LKNhrklFIqwUSqj7O7HEdFG5500CCnlFIJxtmysrQHLStBG54E0yCnlFIJJjwn17Mg53wTgdbJKaWUShgfVYd2BO9Jy0rQ1pXBNMgppVSCiTUnF1Zc6dWcnFJKqQRQ0xpgX1NHUEpzwYQevicn/MWpmpNTSimVALY5cnGTcj2kuXr2KracNAkZWLjBZ/AFUjPQaZBTSqkE4iyq7Gl9HIBLhFxtYQlokFNKqYTy3qHWkM+9CXIQofFJivaV0yCnlFIJZFl5S8jn6cPTe7Ucfd2ORYOcUkoliPJGP1uDBmZ2C8wtzujVsvTt4BYNckoplSCW7g3NxZ1YlB7WUjJaziCnOTmllFJxtXRvc8jn+aN7l4sDyNdBmgENckoplRCMMSxz5OQWxBLkdJBmQIOcUkolhA9rfCGdwLM9wkm9bHQCOkhzGw1ySimVAJbsCc3FzRmZTrq7Z53Ag4XXyWlOTimlVJwsdXQdWDAmM6blhQ/SrDm5HhGRcSKyREQ2i8gmEbkuwjwiIr8VkW0i8r6InBBbcpVSKvm0+g2rnEGul10H2jj7yaXqIM2960pv8QHfN8a8KyK5wDsi8poxZnPQPGcCpfbPTOB++7dSSinbX8saqfd1FCcOz3RxVGEst2ftJ9em1zk5Y0y5MeZd++86YAswxjHbecDjxrIaKBCR4l6nVimlkkyjL8Av1teGTDuvJAuR3tfHQXjDE+0nFwMRKQGmAWscX40BdgV93k14IFRKqZT1wOaGkFaVmW743rG5MS9Xc3KW2PLrG6rlAAAgAElEQVTDgIjkAM8B1xtjarubvytlZWWxJqdPJVp6+luqbS+k3jan2vZCYm9zjRfuWZ8FQS/GuWiUl4a9n9DbVLdtb2WLAFnt0yubvAO+L0pLSwd0fZHEFOREJA0rwP3VGPOPCLPsAcYFfR5rT4soEXZIm7KysoRKT39Lte2F1NvmVNteSOxtDhjD1Surqfc3tk/LTxd+Nn88BRm9K2QL3t7R3gC8Xd7+XWNAEnZf9KdYWlcK8DCwxRhzTyezvQhcbreynAXUGGPKO5lXKaVSgi9gBbgntzWGTL/+mNxeBzinbI8Q3M2u2Q8t/tQrsowlJzcHuAz4QETW29N+BIwHMMY8ACwGzgK2AY3AV2NYn1JKDXq1rQGuWVXFCztCx6kcO8TNN48a0mfrERHy0oWqlo7AVtsaYHiWu8/WMRj0OsgZY1YCXTb/McYY4Du9XYdSSiWTlz5t4sbV1ZQ3hrZ0HJnl4pnPDCPb07fjc+Snu6hq8bd/rm01DM/q4h+SUMwNT5RSSnVtT4OfG1dX86+dzWHfjR3i5sWFRUzK6/vbsTXqSVCQS8EO4RrklFKqn/gDhoe3NnDbu7XURXgLwBH5Hp777DDG5fTPrVj7ymmQU0qpfrGx0sv1/6li3UFv2HcegWuPyeF/jssjyxNbp++u6CDNGuSUUqpPNfsMv1hfy30b6/FFiCnTh6dx7+xCpg5N6/e0hL1TTnNySimleuv9ila+tbyKzdW+sO9y04SfnpjHfx85BLer/3JvwfKcbwdPwRenapBTSqkY+QKGez+o5871tURq23HO+Ex+OauA0UMGtvl+eHGl5uSUUkr1wMc1Pq5aUcXag61h343OdvHLWQWcMyE+7fb17eAa5JRSqleMMTxR1shNa2pojFD5dvFhWfxiVkFYvdhACq+T0+JKpZRS3Wj0BbjhrZqwYbkAhmW4+PXsAs4tiX+va214okFOKaV6ZEedj6+8UcHmqvDGJQvHZfLbOQWMSJChs7ThiQY5pZSK2qp9LVz2ZiWVLaE5omyPcOfMfC4rzY75Zad9SRueaJBTSqmoPPFRA997qzqs9eQR+R4eO3UoUwr7v99bT+VrwxMNckop1RV/wHDrO1bnbqdFJVncN7eA3LT4NS7pir4dXIOcUkp1qs4b4GvLqnh1V/jAyj+clsuNx+UmVPGkU54j+NZ6AxhjEjrNfU2DnFJKRfBpnY8vv14RNnpJphvun1fI+ROz45Sy6GV6hHQXtJVSegPQ5Ddk9+N4mYlGg5xSSjms3t/CpW9Wcqg5tA5rVJaLJ08fxgnD0+OUsp7LT3dxMGg7alsN2Sl050/MgmSllIqTp7Y1cu7/OxQW4I4blsabnx8xqAIc6KgnKRTPlVKqc/6A4ZZ1tfxuU3gDk/NKMrl/XmGfv7l7IFiNT4JfnJpajU80yCmlUl51S4CvLavk9T0tYd/9z3G5/HBaLq5B2ljD2fgk1frKaZBTSqW0bTVevvxGJWU14Q1M7ptTyBcPS/wGJl1J9b5yGuSUUinr9d3NXLmsMuyN2cXZLv562uBqYNKZVO8rp0FOKZVyvAHDHe/W8psPwuvfTixK46+nD2NUdmKMPxmrVG94ElMtqog8IiIHRGRjJ98vEJEaEVlv//w0lvUppVSsdtT5OHPxwYgB7qLDsvjXmcOTJsBBpDo5zcn1xKPA74DHu5hnhTHmnBjXo5RSMXt+exPXrqoKa2HoErjlxDyuPTon6UYDcb5upybSq8uTWExBzhizXERK+iYpSinVPxp9AX60poZHPwp//9vobBd/mj+UOaMy4pCy/pfqxZUDUSd3sohsAPYCNxhjNg3AOpVSCoAtVV7+e2klW6ojv//tD3MLGJqZPMWTTqne8ESMiW2D7Zzcy8aYoyN8lwcEjDH1InIWcK8xpjR4npqamvYElJWVxZQWpZQK9sI+N3d9kk5LIDQ3kyaGayd6+VKxjyQrnQzzdrWLb2/MbP88Lc/PH48N7w/YH0pLO273+fn5cdnT/ZqTM8bUBv29WET+ICJFxphDkeYP3iHxVlZWllDp6W+ptr2QetucStvb4jf8YHU1j24LL548LM/NIwuGctywwd89wCnSMW441AobD7Z/9noyKS0dP9BJi5t+DXIiMgrYb4wxIjIDqzVnRX+uUymV2sob/Vz+ZgVvH/SGfXfxYVncdXLivv+tPzgbnmidXA+IyFPAAqBIRHYDtwBpAMaYB4ALgatExAc0ARebWMtHlVKqE5sqvVz0WgV7Gv0h07M9wq9OLuDLhw/u0Ut6QxuexMAY8+Vuvv8dVhcDpZTqV0v2NHP5kkrqHN0DJua6+ctpw5g6NC1OKYuvsIYnXpNSL05NnTy7UiopGWP405Z6vvhaRViAm1XgZ8nnR6RsgANIc0nIS1IDBup9qVOgpsN6KaUGrRa/4Ya3qnmiLLyBydcmD+HKYQcpyNBn+bw0oTEosNW2GnJTJO7r0VdKDUprD7RwxssHwwKcALeflMdds/LxpEaJXLfC+8qlTr2c5uSUUoPKxzU+fvV+HU9G6B6Q4xEePKWQsydkxSFlicvZ+CSV3imnQU4plfC8AcPinc08srWBZeWROzJPynXz5BnDmFyQIuVwPRDejUDr5JRSKu72Nvh5ZGsDT5Q1sL+p89zHhZOyuHtWgda/dcL5JoLaFBqkWYOcUirh+AOGB7c0cMe7tTR00RJwcoGHX84q4JTi5Bxcua+E95XTnJxSSsXF1mov31pexfqK8BFL2pw0PI2vHjmELx6WTZpLW5d0RxueKKVUAli5r4WvvF4R9r43gNw04aLDsrniyCEck8L93noj7J1yGuSUUmpgvbijia8vr6QldEQu8tKFH03L49LSbHJSaMzJvpSX5iiujPAQkaw0yCml4u6pbY18e0UVzlvvopIs7pyZz6js5H3f20DQ4kqllIqTf25v5DsrwwPcrSfmcd0xOSkzxmJ/SuVBmjXIKaXiZvHOJr6+rIpAUIRzC/x2TgGXlA6JX8KSTHidnBZXKqVUv3plZxNXLKnE5whwf14wlHNLdMSSvpTKxZVai6uUGnDPb2/isjcrCb7XCvDAvEINcP1AG54opVQPlDf6WVHewvLyFg40+ZlckMbZ4zM5aUQ6ri7q0IwxPPZRI997qzqkiBLg3jkFfPGw1Hup6UBI5beDa5BTSkWt1W/4n9XVPP5RY0hDkX/vbuG3G+sZmeXisiOG8PXJQxjpaBF5qNnPdauq+dfO5pDpghXgLj9C6+D6S26EnJw/YHCnQEd6DXJKqag0+wz/taSCV3dHHiAZYH9TgLs31PHbD+o4c3wmxw5NZ3iWi1X7Wnh1VzPVjgYPLoH75xXyJc3B9Su3S8hLl5DhvKpbAwzLTP6uGRrklFLdavQFuOSNSpbs7TzABWsNwAs7mnlhR3On82S5hQdOKeQ8rYMbEEUZLmpbO3raVzRrkFNKKYwxfG1ZVViAG5fj5kuTsinJc7N0r5VTq4uyQcOJRWk8cEohpfk6PNdAGZbp4pO6jiB3qDnAEXFMz0DRIKeU6tLDWxtY7KhHm1zg4YXPFbXXu11aOoR6b4C/ljVy/+Z6dtT5Iy2KvHThmqk5fPfYXDwpUB+USKxcW8eg1xUtqdH4RIOcUqpTW6q83Px2Tci0qYUeXlhYRJGjqCsnzcU3j8rha5OHsO5gKxurvGyp8rG/yc8R+R5OG5PJjBHp+taAOBmWGdrCsqJZg1y3ROQR4BzggDHm6AjfC3AvcBbQCFxhjHk3lnUqpQZGs89w5bJKmoMyZblpwl9PHxYW4IK5XcLMkRnMHKnveEskRRmpGeRi7Qz+KLCwi+/PBErtn28A98e4PqXUALllXQ2bq3wh0351cgEluVoANBgVOXJyh5ojFyknm5iCnDFmOVDZxSznAY8by2qgQESKY1mnUqr//XtXMw9uaQiZdtGkLC7Spv6D1lBncaXWyfWJMcCuoM+77WnlkWYuKyvr5+T0TKKlp7+l2vZC6m1zNNtb0Qrfei8Lq5u2ZXRGgKtGVFBWVtGPqesfeowtrZUuILP9886K+n4/nqWlpf26/GgkVLlDIuyQNmVlZQmVnv6WatsLqbfN0WyvP2D44esVVHo7ugu4BR49YwTTRgy+OjY9xh1qDrbC5oPtn5vdmZSWjh+opMVNfw/QvAcYF/R5rD1NqZTkCxhW7mvhH580UudNrOIiX8DwzRVVvL4ntD/cjcfnMmMQBjgVKrxOLrHOv/7S3zm5F4GrReRvwEygxhgTsahSqWT2aZ2P33xQx0ufNrffXEZlufjbGcM4vig9zqkDb8DwjWVV/HNHU8j0k0em8/1jc+OUKtWXhjpaV1ZqnVz3ROQpYAFQJCK7gVuANABjzAPAYqzuA9uwuhB8NZb1KTUYbanycubig2HjNu5rCnD2K4f484KhfHZcZif/3b+MMSzZ28L/vlPLhgpvyHdjh7j50ymF2mk7SeSmCeku2l9v1OgzNPoCZHuS+41rMQU5Y8yXu/neAN+JZR1KDWb+gOGaVVVhAa5Ng89w8RsVAzpIcZPP8N6hVlbsa+H13c28fdAbNs/4HDcvLSxibE5CVdurGIgIwzJdlDd25OAONQcYn6NBTinVSw9uaWCdI4gM8QgNQa/DDhi4blUVJxSl9ctYji1+w9K9LTyzLY2PNh9gS5U35G3cTiW5VoAbpwEu6QzLdIcEucrmAONz4pigAaBnsVL9ZEedj9vfrQ2Zdtb4TB5dMJRHP2zgprU17S8ObfbDt1dU8cpZw/useHBDRSu/31TPKzvbBk5OI3jswkgWlWRx58x8RmUn/+j0qWhYRuo1PtEgp1Q/+d5/qmkMyjLlpQv3nFxAulv4xlE5ZHqEa1dVt3//9kEv922s57sxNvRYs7+FX26o44090b0WB2B+cQa3nJjHCcPj3whG9R9nC8tU6BCuQU6pfrBmfwtvOl5Nc/tJoTmky0qzeWVnM6/s6hjh/+fv1fKZsZkcPbTnxZb7G/38dF0NT3/c1O28E3LczByRztziDE4pztChulKEc9QTzckppXrltxvrQz7PHpnOZaWhDUtEhN/MLmDN8wfam3N7A/DVpZW8+fnh5KZF1yDAFzA8tLWBn79bS20n73MbmeVibn4LXzp6FCcOT0uJl2WqcM6cXGUKjF+Z3M1qlIqDshpv2PvXfnB8HtZLOUKNzHZzz8kFjv/3cf2qaqzGyV1bs7+FBS8d5KY1NRED3KwR6fztjKFs+dIobjrcy2fHZWqAS2Gp2CFcc3JK9bHfbawnONwcNyyNU4o7r+taNDGLS/Zk89eyxvZpz21v4uSR6XxtSuSmbweb/Nz6Tm3I/wSbWujhzpkFzCvWkUpUh2EZoQ84qfC6HQ1ySvWh/Y1+ntoWGniuOzonYi4u2F2z8nnvUGvIq21uXFPD3kY/Nx2fR7rb+v/qlgB/KWvgrg111EToe5ebJvxoWh5fnzJEO3GrMGEvTtWGJ0qpnnhwS337iBJgNfA4tySr2//L9rh4/NShLHjxIPV2i8yAgXver2fxzmaOHZpGs9/w793NdFaNctGkLH52kjb/V51LxbeDa5BTqo9UtwT4k+MdbFcfnRN1jurw/DR+P6+QK5ZUhhR3bq32sbXa1+n/TSnwcNfJBcwdpUWTqmupWCenDU+U6iMPbK63O11bhmW4uKS0Z0N1nVeSxT8/N4zR2d1fmjke4baT8lh+3ggNcCoqhY7O4FUtAfyB7hs4DWYa5JTqA7WtAe7fHNpt4Oqjc3o1+O2C0ZmsWjSSCyZGLuYcn+Pmx9NyeeeCkVxzdC5pWvemopTmEgrSO84XA1S1JnduTosrHWpbAzT6DFG03laq3Z+2NIQ0BClIF742ZUivl1eY4eLhBUP58Qk+NlZ6afQZmv2Gw/M9zB6ZjqubhixKdWZYpovq1o6K3YrmAEVJ3K1Eg1yQBzbXc9OaGgBGpGcyb28lX5iYxdkTum84oFJXvTfA7zeF5uKumpoTdWfurkzK8zApTy9T1XeKMt18XNsR5A41Bzgyjunpb1pcaSur8fKjtTXtnw+0unhuexOXvFnJsx9H7oukFFhDcQW/gDIvTfhmJ/3blIo358tTk73xiQY526821NFZ/esvN9QR0PJLFcHy8hb+sCm0ReU3puRQkKGXlkpM4UN7aZBLettrfTz7SeeD2pbV+HizByO6q9RQ0xrg2yuqQqaNy3Fz7TGai1OJy9lX7lCSj1+pQQ645/06/EEZtSPyPZw2LLRfkrPlnEptrX7Ddauq2d3QcYMQ4P55heSl62WlEleqjXqS8lfjp3W+sGGYbjgul8vHhga5N/a0sLW66xdOqtSwr9HP5//fIZ7fEZr7/87UHO2vphKe88WpyT7qScoHud9vqifovZYclufmCxOzmJobYIbjBZIPam4upX1a5+PuDXXMf/EAaw60hnw3pcDDzSfkxSllSkVveFZod4H9Tckd5FK6bbIxhpc/DX0a/96xue3DMF01dQhrl3bczP62rYmfnpgfNmqASg7VLQF2N/jZ3eBjd72fPQ1++7OfXfX+kKLJYBNy3Dxx2lAyPdp3TSW+cTmhQe7Tus6HjEsGKR3kNlX52NvY8RST7REumNgxDNPnJ2Qxdkht+82tyW946dMmLj+i9518VWJo8AZYUeni0bU1bKz0srnKy8FeFNucOjqDRxYM1QcfNWiMdwS53Q1+vAGTtCPnxHRlishCEflQRLaJyE0Rvr9CRA6KyHr752uxrK+vvb479MWW84ozQp7GPS7hUsfYg684XoapBo9mn+HvnzRy4b8PMempcr63OZPfb6pnWXlLjwNcmgu+f2wOz35mmAY4Nahke1yMzOo4Z/0G9nRSSpEMep2TExE38HvgM8Bu4G0RedEYs9kx69PGmKtjSGO/eW1PaMD6zJjwRgNnT8jizvV17Z+X7m2h0Rfo1ZiEqneqWwK8squZT+t8ZLiFTLcwLsfN7JHpDI1iOKL3K1p5oqyRZz9upDrCO9iiJcD04Wl86bBszp+YpW/YVoNWSa6H/U0dVTE76nyU5CZnwV4sWzUD2GaM+QRARP4GnAc4g1xCqm0NsGZ/aOOBM8Zmhs13dKGHcTludtV3FFku3dvCWeN1qK/+9p99LfxxSwOv7GqipZMHzamFHuYVZzB3VAYzRqST5RECBjZWelm1r4V/7WxmQ0V0rWIz3DA+x8OYIW7G2j9jhrgZl2P9PXqIWx9uVFKYkOtmzYGOz5/WaU4ukjHArqDPu4GZEea7QEROAT4CvmuM2RVhHgDKyspiSE7PvHnIjc905NwmZAXw7ttO2b7w9MzOTePp+rT26U9vPEBpS2iATAYDuf+70uyH+3ak8Ux5WrfzbqrysanKxwObG7qd12l0RoDZQ/2ckBegdEiAcVkGd6RqiQYwDbCnx2tIPIlyjAdSqm1zNNub25oGdFxf7+48yGzX3j5PS2lpaZ8vs6f6O3/6EvCUMaZFRL4JPAac1tnMA7lD7jtQBXT0jzt7Ui6lpePaP5eVlbWn58tDmnm6vKL9u//UpDPpsPG4k6iiNnh74+mDSi9fX1bZ5UtCYzE0w8VFh2VxSekQMg7t4Igj4r/NAyVRjvFASrVtjnZ7T6CBh3dVt3+u9eRRWjq0P5MWN7EEuT3AuKDPY3E87BpjKoI+PgT8Mob19RljTFijk8+MCS+qbDN7ZAZ5aUKt/ULMg80B3jnUyowR2vG3L/1zeyPfXlFNkz+83mxCjpuzJ2TiEaHOG2DdQS8bK71EU8MmwGljMrisdAhnjs8kw86ulVV0/X9KJStn/duO+uTtRhBLkHsbKBWRiVjB7WLgK8EziEixMabc/ngusCWG9fWZSF0HZncxUkW6WzhjbCb/2N7Rp+6Vnc0a5PpIwBh+/l4dd2+oC/uuJNfNb2YXML84A3G8Q62qJcCqfS2s3NfCyn2t7KzzETAQwBqEdtbIdOaOyuC00RmMzUnOSnWlemNCWF85rZMLY4zxicjVwKuAG3jEGLNJRH4GrDPGvAhcKyLnAj6gEriiD9Icszf3hHcdyIhYGdPhrPGhQW7xzmZumZ7fL+lLJXsa/Hx7RRXLysMHwP7K4dn8YlZ+p+9lK8xwcc6ELM7R9/0p1SPF2W7SXdD2UvDKlgA1rQHyk3Dc1Zgeb40xi4HFjmk/Dfr7h8APY1lHf3hzb+gN9YwIXQeczhiTiUdoHwLswxofH1Z7ObKg+8YRKlzAGJ79pIkbV1eHvFEbrD5o95xcwGXa6V6pfuF2CeNzPGyr7Sim/LTOx7HD0rv4r8Ep+cJ2N5p8hrf2hwa500Z3Xh/XpiDDxbzi0GD4wo7OX8+jIqttDfCXsgZm/vMA31xeFRbghme6eGlhkQY4pfpZSa6jyLI+OYssU66i4q39LSF9rsbluJmUF12n3kUlWSwJygW+sKOJG4/XQXmNMextDLC/0U91a4DqlgAtAfAGDA1eQ0VzgAPNftYf8rKxytvpy2lPKc7g/nmFjBminayV6m8Tcj1Ax/1sR5KOYZlyQW7JXmcuLrxBQ2fOnpDJd9+i/Sa9qcrHthovh+enVpFlq9+w+kArr+1uZt3BVjZVeamNYSSRdBfcMj2fq44agivKY6GUik1JijQ+Sbkg52x0cmoURZVtijLdzB2VwfLy4NxcM98/LvmDXHmjn9d3N/Pa7maW7G2hztv7oNbGLXDhpCy+f2wuR2jdplIDaoKzG4Hm5Aa//Y1+NlV1HEgB5o/uWTeARSVZjiDXxPePy+2rJCaEymY/6yu8vHfIy3uHWllf4e30NTO9cWS+hzPGZvLNo4YwXpv2KxUXE7ROLvksdTRTn1aU1uMR5M+ZkMkNqzuKLN+v9LK91sfEvMG5K6tbAmyo8PL6bg+7dlfy3qHWXp3sOR6hJM/D0AwXBelClkdIdwkZbmFopouiDBdjc9zMHJGuAxsrlQCcHcI/rfMRMCbpqgwG5525l5xFldG0qnQakWWNfr9yX8fYlc/vaOK7xw6O3NyBJj8ry1tYYXegLqtpy9mmA9G3Fh2R5eKMMZl8ZmwG04rSGZ/jTrqLQ6lklp/uojBDqGqxnthbA1DeGEi6hl8pE+SMsd4eEGxBFP3jIjmvJCskyD28tYGrj85JyJcOHmr2s7K8lZX7WlhR3sKHNb0rd09zwfHD0jhjbCafHZvJccPSNKgpNciV5Hqoaul4S8eOOp8GucHq3UNe9jd1DOU1xCPMGN67jo+LSrK4+e2a9q4Iuxv8PPtxI18pjX/frspmPyv3tbJiXwurylvY3IuBjj0CUwrTmFaUxrRh6UwrSmNKYVq3o8IopQaXCTke3jvUEeS21fiY08UQh4NRygS557Y3hnw+dXQG6b28aQ/PcnNZ6RAe2trxepdff1DPlw7LHvA3E1S3BOyxG62cWnDDmmi4BCbne5iU3sz8ScOYVpTO1MI0sjwa0JRKdlMKPTy/o+Pzqn0t/NeR8X9Y70spEeQCxvDP7aH1TRdMim28w2uOzuHPHzbQNmB+WY2Pl3c2c15J/46jWNMa4D/77Dq18lY+iHIk/jYusYod543KYF5xBjNHppOb5rJf0ZHTb+lWSiWeU4oz+L/3OgZGX1begjEm6r7Dg0FKBLm39rdS3hhaVPm5cT1vdBJsQq6HCydl8fTHHcHznvfrOHdCZp+eILWtAd7a31Gn9n5l5yOGROISOHZoWvvbs08emU5eEg7CqpTquenD08nxCPX2oLz7mwJsrfYxpTB5+q2mRJD7hyMXd9b4TLI9sd/ov3tsbkiQ21Dh5d4P6rk+hpaW9d4Aq/e3sqLcKoJcX+ElwuvVOiXA0UPTmFeczrxRGZw8MoOCHnaTUEqlhjSXMGdUOq/u7miUt3Rviwa5wcQXMDzvCHJfmNg3RYqTC9I4e3wm/9rZ0TXh1ndqGZ7l4pIoG6HUtgZYe6CV/+y3ih/fPdTa/qaDaE0t9DDXLn6cMyqjx33/lFKpa/7ozJAgt6y8haumJk/VRdIHuWXlLVS0dBRV5qcLp3XxFvCeunV6HsvLQ4e5unZVNZXNAS49YkhIwAkYw656P+8cbOWtA62s3m+N+9iT4keAKQUe5trFj3NHaedqpVTvzXe8XWXVvha8AZOQXaJ6I+mD3NPbQltVnjshq0+bwpfmp/HX04dx4b8Ptb+A0G/gJ+tquf29Wo4uTMMl0OSHT2p9NPY0m4Y1DNbc4gzmjcpgzqh0hmdpUFNK9Y2jCj0Mz3RxsNm6gdV5De8ebGXmyOToSpDUQW5zlZe/93GrykhOKc7gT/OHcsWSypCWji1+eCeoD0q0Ds/zMK843c6pZTAyW4OaUqp/iAgLRmfw7Ccd98pl5S0a5AaDW96uCSkKPDzPw7x+6uh4XkkWv5tbwPfeqg55X100Jhd4OHlkOiePtILa6CQbcUApldhOKQ4Nckv3tnDj8XFMUB9K2iC3ZE8zr+0JHcbrf6fn9Wtn7UtKh3DamEye2tbIEx81sD3C+5kK0oUphWnMGJ7OrJHpzByRzlCtU1NKxdECx9tY1h5o5cNqL0cmwSuwkjLI+QOGn6yrDZk2e2Q6Z43vuwYnnSnOdvO9Y3O5/pgcymp81LYaDAZBmJDrZnimK6k6WiqlBr9xOR6OzPe0j23rM3DNympeOatowEdx6mtJ19a8xW/41ooqNlaG1oXdflL+gAYXlwhHFqRx0oh0ZozI4KQR6YzIcmuAU0olpB8cH9q/d+3BVh7c0tDJ3INHUgW5nfU+Lvz3oZCyZbDePn1CLwdjVkqpVHD+xCzOdIwEdfu7tYP+jeExBTkRWSgiH4rINhG5KcL3GSLytP39GhEpiWV9ndlQ0crUp/dx7LP7WRH0Chyw3nv2v9Pz+2O1SimVNESEe2YXkJfeUdp0SnHGoH/7SK+DnIi4gd8DZwJHAV8WkaMcs10JVBljDgd+Dfyit+vryvgcD3sawxt5HJbn5t9nD0+69yMppVR/KMrOIj0AAAQhSURBVM52c8dJ+QzLcPHQ/EKeOn0oxYO8C5MY0/POyQAicjJwqzHmc/bnHwIYY/4vaJ5X7XneEhEPsA8YboJWWlNT07sEKKWUGjTy8/PjkiWMpbhyDLAr6PNue1rEeYwxPqAGGBbDOpVSSqmoJVXDE6WUUipYLP3k9gDjgj6PtadFmme3XVyZD1QEzxCvLKxSSqnkF0tO7m2gVEQmikg6cDHwomOeF4H/sv++EHjT9LYSUCmllOqhXgc5u47tauBVYAvwjDFmk4j8TETOtWd7GBgmItuA7wFh3QziLVG6QQyUKLb3eyKyWUTeF5E3RGRCPNLZV7rb3qD5LhARIyLTBzJ9/SGabRaRi+zjvElEnhzoNPalKM7p8SKyRETes8/rs+KRzr4iIo+IyAER2djJ9yIiv7X3x/sicsJApzGhGGNS9gdwAx8Dk4B0YANwlGOebwMP2H9fDDwd73T38/aeCmTbf1+V7Ntrz5cLLAdWA9Pjne4BOMalwHtAof15RLzT3c/b+0fgKvvvo4Ad8U53jNt8CnACsLGT788CXgEEmAWsiXea4/mT6g1PZgDbjDGfGGNagb8B5znmOQ94zP7778DpMnjH5up2e40xS4wxbS/hW41V1zpYRXN8AW7D6sPZHOG7wSaabf468HtjTBWAMebAAKexL0WzvQbIs//OB/YOYPr6nDFmOVDZxSznAY8by2qgQESKByZ1iSfVg1yqdYOIZnuDXYn1RDhYdbu9dlHOOGPMvwYyYf0ommN8BHCEiKwSkdUisnDAUtf3otneW4FLRWQ3sBi4ZmCSFjc9vc6TWlK+hUDFTkQuBaYD8+Odlv4iIi7gHuCKOCdloHmwiiwXYOXUl4vIMcaY6rimqv98GXjUGPMrexCLJ0TkaGNMIN4JU/0v1XNyPekGQWfdIAaRaLYXETkD+DFwrjGmxfn9INLd9uYCRwNLRWQHVv3Fi4O88Uk0x3g38KIxxmuM2Q58hBX0BqNotvdK4BkAY8xbQCZQNCCpi4+orvNUkepBLtW6QXS7vSIyDXgQK8AN5roa6GZ7jTE1xpgiY0yJMaYEqw7yXGPMuvgkt09Ec04/j5WLQ0SKsIovPxnIRPahaLZ3J3A6gIhMwQpyBwc0lQPrReByu5XlLKDGGFMe70TFS0oXVxpjfCLS1g3CDTxi7G4QwDpjzItY3SCesLtBVGJdRINSlNt7F5ADPGu3r9lpjDm304UmsCi3N6lEuc2vAp8Vkc2AH/gfY8ygLJ2Icnu/D/xJRL6L1QjlikH8oIqIPIX1kFJk1zPeAqQBGGMewKp3PAvYBjQCX41PShNDrwdoVkoppRJdqhdXKqWUSmIa5JRSSiUtDXJKKaWSlgY5pZRSSUuDnFJKqaSlQU4ppVTS0iCnlFIqaWmQU0oplbT+P+F/csHpThvzAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.kdeplot(logs['percent_100']);\n",
    "plt.title('Distribution of Percentage of Songs Listened to Completion');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There is `time_index` in the logs although no `index` present."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Entityset: customers\n",
       "  Entities:\n",
       "    members [Rows: 6658, Columns: 6]\n",
       "    transactions [Rows: 22329, Columns: 13]\n",
       "    logs [Rows: 424252, Columns: 14]\n",
       "  Relationships:\n",
       "    No relationships"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "es.entity_from_dataframe(entity_id='logs', dataframe=logs,\n",
    "                         index = 'logs_index', make_index = True,\n",
    "                         time_index = 'date')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Making features by hand may seem counterintuitive if we are using automated feature engineering, but the benefits of doing this before using Featuretools is that these features can be stacked on top of to build deep features. Automated feature engineering will therefore take our existing hand-built features and extract more value from them by combining them with other features.\n",
    "\n",
    "Another method to improve the power of deep feature synthesis is through interesting values, which specify conditional statements used to build features."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Interesting Values\n",
    "\n",
    "In order to create conditional features, we can set interesting values for existing columns in the data. The following code will be used to build features conditional on the value of `is_cancel` and `is_auto_renew` in the transactions data. The primitives used for the conditional features are specified as `where_primitives` in the call to Deep Feature Synthesis. For example, if we used a `mean` primitive along with the following interesting values, we will get a mean of transactions where the transaction was cancelled, as well as the mean of transactions where the transaction was not cancelled. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "es['transactions']['is_cancel'].interesting_values = [0, 1]\n",
    "es['transactions']['is_auto_renew'].interesting_values = [0, 1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Relationships\n",
    "\n",
    "Table relationships should be familiar to anyone who has worked with relational databases and the idea is the same in Featuretools. We use relationships to specify how examples in one table relate to examples in other tables. The entityset structure for this problem is fairly simple as there are only three entities with two relationships.  `members` is the parent of `logs` and `transactions`. In both relationships, the parent and child variable is `msno`, the customer id.\n",
    "\n",
    "The two relationships are: one linking `members` to `transactions` and one linking `members` to `logs`. The order for relationships in featuretools is parent variable, child variable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Entityset: customers\n",
       "  Entities:\n",
       "    members [Rows: 6658, Columns: 6]\n",
       "    transactions [Rows: 22329, Columns: 13]\n",
       "    logs [Rows: 424252, Columns: 14]\n",
       "  Relationships:\n",
       "    transactions.msno -> members.msno\n",
       "    logs.msno -> members.msno"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Relationships (parent, child)\n",
    "r_member_transactions = ft.Relationship(es['members']['msno'], es['transactions']['msno'])\n",
    "r_member_logs = ft.Relationship(es['members']['msno'], es['logs']['msno'])\n",
    "\n",
    "es.add_relationships([r_member_transactions, r_member_logs])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Cutoff Times\n",
    "\n",
    "`cutoff_times` are a critical piece of any time based machine learning problem. The label times dataframe has columns of member id, cutoff time, and label. __For each cutoff time, only data from before the cutoff time can be used to build features for that label.__ This is one of the greatest advantages of Featuretools compared to manual feature engineering: __Featuretools automatically filters our data based on the cutoff times to ensure that all the features are valid for machine learning.__ Normally, we would have to take extreme care to make sure all of our features are valid, but Featreutools is able to implement the filtering logic behind the scenes for us.\n",
    "\n",
    "All we have to do is make sure to pass in the correct label times for the prediction problem we want to solve."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>msno</th>\n",
       "      <th>cutoff_time</th>\n",
       "      <th>label</th>\n",
       "      <th>days_to_churn</th>\n",
       "      <th>churn_date</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>++bK0FRJecXCogqXdjtO2Kyb3jq7uLM7qThTk+nN8tE=</td>\n",
       "      <td>2015-08-01</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>++bK0FRJecXCogqXdjtO2Kyb3jq7uLM7qThTk+nN8tE=</td>\n",
       "      <td>2015-09-01</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>++bK0FRJecXCogqXdjtO2Kyb3jq7uLM7qThTk+nN8tE=</td>\n",
       "      <td>2015-10-01</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>++bK0FRJecXCogqXdjtO2Kyb3jq7uLM7qThTk+nN8tE=</td>\n",
       "      <td>2015-11-01</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>++bK0FRJecXCogqXdjtO2Kyb3jq7uLM7qThTk+nN8tE=</td>\n",
       "      <td>2015-12-01</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                           msno cutoff_time  label  \\\n",
       "0  ++bK0FRJecXCogqXdjtO2Kyb3jq7uLM7qThTk+nN8tE=  2015-08-01    0.0   \n",
       "1  ++bK0FRJecXCogqXdjtO2Kyb3jq7uLM7qThTk+nN8tE=  2015-09-01    0.0   \n",
       "2  ++bK0FRJecXCogqXdjtO2Kyb3jq7uLM7qThTk+nN8tE=  2015-10-01    0.0   \n",
       "3  ++bK0FRJecXCogqXdjtO2Kyb3jq7uLM7qThTk+nN8tE=  2015-11-01    0.0   \n",
       "4  ++bK0FRJecXCogqXdjtO2Kyb3jq7uLM7qThTk+nN8tE=  2015-12-01    0.0   \n",
       "\n",
       "   days_to_churn churn_date  \n",
       "0            NaN        NaN  \n",
       "1            NaN        NaN  \n",
       "2            NaN        NaN  \n",
       "3            NaN        NaN  \n",
       "4            NaN        NaN  "
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cutoff_times = cutoff_times.drop_duplicates(subset = ['msno', 'cutoff_time'])\n",
    "cutoff_times.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Deep Feature Synthesis\n",
    "\n",
    "With the entities and relationships fully defined, we are ready to run [Deep Feature Synthesis (DFS)](https://www.featurelabs.com/blog/deep-feature-synthesis/). This process applies feature engineering building blocks called [feature primitives](https://docs.featuretools.com/automated_feature_engineering/primitives.html) to a dataset to build hundreds of features. Feature primitives are basic operations of two types - transforms and aggregations - that stack to build deep features (for more information see the previous linked resources). These includes many operations that we would traditionally carry out by hand, but automated feature engineering saves us from having to implement these features one at a time. \n",
    "\n",
    "The call to `ft.dfs` needs the entityset which holds all the tables and relationships between them, the `target_entity` to make features for, the specific primitives, the maximum stacking of primitives (`max_depth`), the `cutoff_times`, and a number of optional parameters.\n",
    "\n",
    "To start, we'll use the default aggregation and transformation primitives as well as two `where_primitives` and see how many features this generates. To only generate the definitions of the features, we pass in `features_only = True`.\n",
    "\n",
    "For full details on Deep Feature Synthesis, take a look at [the documentation](https://docs.featuretools.com/api_reference.html#deep-feature-synthesis). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "feature_defs = ft.dfs(entityset=es, target_entity='members', \n",
    "                      cutoff_time = cutoff_times,\n",
    "                      where_primitives = ['sum', 'mean'],\n",
    "                      max_depth=2, features_only=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This will generate 188 features.\n"
     ]
    }
   ],
   "source": [
    "print(f'This will generate {len(feature_defs)} features.')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<Feature: MEAN(transactions.price_difference WHERE is_cancel = 1)>,\n",
       " <Feature: MAX(logs.num_75)>,\n",
       " <Feature: SUM(logs.num_75)>,\n",
       " <Feature: COUNT(logs)>,\n",
       " <Feature: MEAN(logs.num_985)>,\n",
       " <Feature: MIN(logs.percent_unique)>,\n",
       " <Feature: MAX(logs.percent_unique)>,\n",
       " <Feature: MAX(logs.num_25)>,\n",
       " <Feature: NUM_UNIQUE(transactions.DAY(membership_expire_date))>,\n",
       " <Feature: SUM(transactions.price_difference WHERE is_cancel = 1)>]"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import random; random.seed(42)\n",
    "\n",
    "random.sample(feature_defs, 10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that Featuretools has built almost 200 features automatically for us using the table relationships and feature primitives. If built by hand, each of these features would require minutes of work, totaling many hours to build 188 features. Moreover, although the features are not necessarily intuitive, they are easy to explain in natural language because they are simple operations stacked on top of each other."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Specify Primitives \n",
    "\n",
    "Now we'll do a call to `ft.dfs` specifying the primitives to use. Often, these will depend on the problem and can involve domain knowledge. The best way to choose primitives is by trying out a variety and seeing which perform the best. Like many operations in machine learning, choosing primitives is still largely an empirical, rather than theoretical, practice."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Aggregation Primitives"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>name</th>\n",
       "      <th>type</th>\n",
       "      <th>description</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>mean</td>\n",
       "      <td>aggregation</td>\n",
       "      <td>Computes the average value of a numeric feature.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>num_unique</td>\n",
       "      <td>aggregation</td>\n",
       "      <td>Returns the number of unique categorical variables.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>last</td>\n",
       "      <td>aggregation</td>\n",
       "      <td>Returns the last value.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>percent_true</td>\n",
       "      <td>aggregation</td>\n",
       "      <td>Finds the percent of 'True' values in a boolean feature.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>max</td>\n",
       "      <td>aggregation</td>\n",
       "      <td>Finds the maximum non-null value of a numeric feature.</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           name         type  \\\n",
       "0          mean  aggregation   \n",
       "1    num_unique  aggregation   \n",
       "2          last  aggregation   \n",
       "3  percent_true  aggregation   \n",
       "4           max  aggregation   \n",
       "\n",
       "                                                description  \n",
       "0          Computes the average value of a numeric feature.  \n",
       "1       Returns the number of unique categorical variables.  \n",
       "2                                   Returns the last value.  \n",
       "3  Finds the percent of 'True' values in a boolean feature.  \n",
       "4    Finds the maximum non-null value of a numeric feature.  "
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_p = ft.list_primitives()\n",
    "trans_p = all_p.loc[all_p['type'] == 'transform'].copy()\n",
    "agg_p = all_p.loc[all_p['type'] == 'aggregation'].copy()\n",
    "\n",
    "pd.options.display.max_colwidth = 100\n",
    "agg_p.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Specify aggregation primitives\n",
    "agg_primitives = ['sum', 'time_since_last', 'avg_time_between', 'all', 'mode', 'num_unique', 'min', 'last', \n",
    "                  'mean', 'percent_true', 'max', 'std', 'count']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Transform Primitives"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>name</th>\n",
       "      <th>type</th>\n",
       "      <th>description</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>57</th>\n",
       "      <td>multiply</td>\n",
       "      <td>transform</td>\n",
       "      <td>Creates a transform feature that multplies two features.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>58</th>\n",
       "      <td>weekend</td>\n",
       "      <td>transform</td>\n",
       "      <td>Transform Datetime feature into the boolean of Weekend.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59</th>\n",
       "      <td>and</td>\n",
       "      <td>transform</td>\n",
       "      <td>For two boolean values, determine if both values are 'True'.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>60</th>\n",
       "      <td>add</td>\n",
       "      <td>transform</td>\n",
       "      <td>Creates a transform feature that adds two features.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>61</th>\n",
       "      <td>seconds</td>\n",
       "      <td>transform</td>\n",
       "      <td>Transform a Timedelta feature into the number of seconds.</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        name       type  \\\n",
       "57  multiply  transform   \n",
       "58   weekend  transform   \n",
       "59       and  transform   \n",
       "60       add  transform   \n",
       "61   seconds  transform   \n",
       "\n",
       "                                                     description  \n",
       "57      Creates a transform feature that multplies two features.  \n",
       "58       Transform Datetime feature into the boolean of Weekend.  \n",
       "59  For two boolean values, determine if both values are 'True'.  \n",
       "60           Creates a transform feature that adds two features.  \n",
       "61     Transform a Timedelta feature into the number of seconds.  "
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trans_p.tail()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Specify transformation primitives\n",
    "trans_primitives = ['weekend', 'cum_sum', 'day', 'month', 'diff', 'time_since_previous']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Where Primitives\n",
    "\n",
    "These primitives are applied to the `interesting_values` to build conditional features. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Specify where primitives\n",
    "where_primitives = ['sum', 'mean', 'percent_true', 'all', 'any']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Custom Primitives\n",
    "\n",
    "[Custom primitives](https://docs.featuretools.com/automated_feature_engineering/primitives.html#defining-custom-primitives) are one of the most powerful options in Featuretools. We use custom primitives to write our own functions based on domain knowledge and then pass them to `dfs` like any other primitives. Featuretools will then stack our custom primitives with the other primitives, again, in effect, amplifying our domain knowledge.\n",
    "\n",
    "For this problem, I wrote a custom primitive that calculates the sum of a value in the month prior to the cutoff time. This is actually a primitive I [wrote for another problem](https://github.com/Featuretools/Automated-Manual-Comparison/tree/master/Retail%20Spending) but I can apply it to this problem because primitives are data agnostic. That's one of the benefits of feature primitives: they can work for any problem and writing a custom primitive will pay off many times over."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "from featuretools.primitives import make_agg_primitive\n",
    "\n",
    "def total_previous_month(numeric, datetime, time):\n",
    "    \"\"\"Return total of `numeric` column in the month prior to `time`.\"\"\"\n",
    "    df = pd.DataFrame({'value': numeric, 'date': datetime})\n",
    "    previous_month = time.month - 1\n",
    "    year = time.year\n",
    "   \n",
    "    # Handle January\n",
    "    if previous_month == 0:\n",
    "        previous_month = 12\n",
    "        year = time.year - 1\n",
    "        \n",
    "    # Filter data and sum up total\n",
    "    df = df[(df['date'].dt.month == previous_month) & (df['date'].dt.year == year)]\n",
    "    total = df['value'].sum()\n",
    "    \n",
    "    return total"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>value</th>\n",
       "      <th>date</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10</td>\n",
       "      <td>2018-01-01 00:00:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>12</td>\n",
       "      <td>2018-01-07 13:20:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>14</td>\n",
       "      <td>2018-01-14 02:40:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>15</td>\n",
       "      <td>2018-01-20 16:00:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>19</td>\n",
       "      <td>2018-01-27 05:20:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>22</td>\n",
       "      <td>2018-02-02 18:40:00</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   value                date\n",
       "0     10 2018-01-01 00:00:00\n",
       "1     12 2018-01-07 13:20:00\n",
       "2     14 2018-01-14 02:40:00\n",
       "3     15 2018-01-20 16:00:00\n",
       "4     19 2018-01-27 05:20:00\n",
       "5     22 2018-02-02 18:40:00"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "70"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "numeric = [10, 12, 14, 15, 19, 22, 9, 8, 8, 11]\n",
    "dates = pd.date_range('2018-01-01', '2018-03-01', periods = len(numeric))\n",
    "pd.DataFrame({'value': numeric, 'date': dates}).head(6)\n",
    "total_previous_month(numeric, dates, pd.datetime(2018, 2, 1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>value</th>\n",
       "      <th>date</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10</td>\n",
       "      <td>2018-01-01 00:00:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>12</td>\n",
       "      <td>2018-01-12 19:12:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>14</td>\n",
       "      <td>2018-01-24 14:24:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>5</td>\n",
       "      <td>2018-02-05 09:36:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>7</td>\n",
       "      <td>2018-02-17 04:48:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>8</td>\n",
       "      <td>2018-03-01 00:00:00</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   value                date\n",
       "0     10 2018-01-01 00:00:00\n",
       "1     12 2018-01-12 19:12:00\n",
       "2     14 2018-01-24 14:24:00\n",
       "3      5 2018-02-05 09:36:00\n",
       "4      7 2018-02-17 04:48:00\n",
       "5      8 2018-03-01 00:00:00"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "12"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "numeric = [10, 12, 14, 5, 7, 8]\n",
    "dates = pd.date_range('2018-01-01', '2018-03-01', periods = len(numeric))\n",
    "pd.DataFrame({'value': numeric, 'date': dates}).head(6)\n",
    "total_previous_month(numeric, dates, pd.datetime(2018, 3, 1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Custom Primitive Implementation\n",
    "\n",
    "Making a custom primitive is simple: first we define a function (`total_previous_month`) and then we `make_agg_primitive` with `input_type[s]`, a `return_type`, and whether or not the primitive requires the `cutoff_time` through `uses_calc_time`. \n",
    "\n",
    "This primitive is an aggregation primitive because it takes in multiple numbers - transactions for the previous month - and returns a single number - the total of the transactions. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Takes in a number and outputs a number\n",
    "total_previous = make_agg_primitive(total_previous_month, input_types = [ft.variable_types.Numeric,\n",
    "                                                                         ft.variable_types.Datetime],\n",
    "                                    return_type = ft.variable_types.Numeric, \n",
    "                                    uses_calc_time = True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now just have to pass this in as another aggregation primitive for Featuretools to use it in calculations.\n",
    "\n",
    "\n",
    "The second custom primitive finds the time since a previous true value. This is originally intended for the `is_cancel` variable in the `transactions` dataframe, but it can work for any Boolean variable. It simply finds the time between True examples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "def time_since_true(boolean, datetime):\n",
    "    \"\"\"Calculate time since previous true value\"\"\"\n",
    "    \n",
    "    if np.any(np.array(list(boolean)) == 1):\n",
    "        # Create dataframe sorted from oldest to newest \n",
    "        df = pd.DataFrame({'value': boolean, 'date': datetime}).\\\n",
    "                sort_values('date', ascending = False).reset_index()\n",
    "\n",
    "        older_date = None\n",
    "\n",
    "        # Iterate through each date in reverse order\n",
    "        for date in df.loc[df['value'] == 1, 'date']:\n",
    "\n",
    "            # If there was no older true value\n",
    "            if older_date == None:\n",
    "                # Subset to times on or after true\n",
    "                times_after_idx = df.loc[df['date'] >= date].index\n",
    "\n",
    "            else:\n",
    "                # Subset to times on or after true but before previous true\n",
    "                times_after_idx = df.loc[(df['date'] >= date) & (df['date'] < older_date)].index\n",
    "            older_date = date\n",
    "            # Calculate time since previous true\n",
    "            df.loc[times_after_idx, 'time_since_previous'] = (df.loc[times_after_idx, 'date'] - date).dt.total_seconds()\n",
    "\n",
    "        return list(df['time_since_previous'])[::-1]\n",
    "    \n",
    "    # Handle case with no true values\n",
    "    else:\n",
    "        return [np.nan for _ in range(len(boolean))]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[]"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "booleans = []\n",
    "dates = []\n",
    "df = pd.DataFrame({'value': booleans, 'date': dates})\n",
    "time_since_true(df['value'], df['date'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0.0,\n",
       " 509760.00000000006,\n",
       " 1019520.0000000001,\n",
       " 1529280.0,\n",
       " 2039040.0000000002,\n",
       " 0.0,\n",
       " 0.0,\n",
       " 0.0,\n",
       " 509760.00000000006,\n",
       " 1019520.0000000001,\n",
       " 1529280.0]"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "booleans = [1, 0, 0, 0, 0, 1, 1, 1, 0, 0, 0]\n",
    "dates = pd.date_range('2018-01-01', '2018-03-01', periods = len(booleans))\n",
    "df = pd.DataFrame({'value': booleans, 'date': dates})\n",
    "time_since_true(df['value'], df['date'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0.0, 2548800.0, 5097600.0]"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "booleans = [1, 0, 0]\n",
    "dates = pd.date_range('2018-01-01', '2018-03-01', periods = len(booleans))\n",
    "time_since_true(booleans, dates)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[nan, nan]"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "booleans = [0, 0]\n",
    "dates = pd.date_range('2018-01-01', '2018-03-01', periods = len(booleans))\n",
    "time_since_true(booleans, dates)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is a transformation primitive since it acts on multiple columns in the same table. The returned list is the same length as the original column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "from featuretools.primitives import make_trans_primitive\n",
    "\n",
    "# Specify the inputs and return\n",
    "time_since = make_trans_primitive(time_since_true, \n",
    "                                  input_types = [vtypes.Boolean, vtypes.Datetime],\n",
    "                                  return_type = vtypes.Numeric)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's add the two custom primitives to the respective lists. In the final version of feature engineering, I did not use the `time_since` primitive. I ran into problems with the implementation but would encourage anyone to try and fix it or build their own custom primitive[s]."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "agg_primitives.append(total_previous)\n",
    "# trans_primitives.append(time_since)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Deep Feature Synthesis with Specified Primitives\n",
    "\n",
    "We'll again run Deep Feature Synthesis to make the feature definitions this time using the selected primitives and the custom primitives. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "feature_defs = ft.dfs(entityset=es, target_entity='members', \n",
    "                      cutoff_time = cutoff_times, \n",
    "                      agg_primitives = agg_primitives,\n",
    "                      trans_primitives = trans_primitives,\n",
    "                      where_primitives = where_primitives,\n",
    "                      chunk_size = len(cutoff_times), \n",
    "                      cutoff_time_in_index = True,\n",
    "                      max_depth = 2, features_only = True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This will generate 255 features.\n"
     ]
    }
   ],
   "source": [
    "print(f'This will generate {len(feature_defs)} features.')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<Feature: MIN(logs.num_unq)>,\n",
       " <Feature: NUM_UNIQUE(logs.MONTH(date))>,\n",
       " <Feature: LAST(transactions.payment_method_id)>,\n",
       " <Feature: SUM(logs.num_100)>,\n",
       " <Feature: SUM(logs.num_985)>,\n",
       " <Feature: MIN(logs.total_secs)>,\n",
       " <Feature: MAX(logs.num_unq)>,\n",
       " <Feature: MAX(logs.percent_unique)>,\n",
       " <Feature: STD(transactions.planned_daily_price)>,\n",
       " <Feature: LAST(logs.MONTH(date))>,\n",
       " <Feature: SUM(logs.num_75)>,\n",
       " <Feature: TOTAL_PREVIOUS_MONTH(transactions.planned_daily_price, transaction_date)>,\n",
       " <Feature: MAX(logs.num_25)>,\n",
       " <Feature: ALL(transactions.is_cancel WHERE is_auto_renew = 0)>,\n",
       " <Feature: SUM(transactions.actual_amount_paid WHERE is_auto_renew = 0)>]"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "random.sample(feature_defs, 15)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that our custom primitive `TOTAL_PREVIOUS_MONTH` has been applied to create more features. The benefit of custom primitives are that they can be used to encode specific domain knowledge into the feature engineering process. Moreover, we don't get just the custom primitive itself, we also get features that are stacked on top of the primitive."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Run Deep Feature Synthesis\n",
    "\n",
    "Once we're happy with the features that will be generated, we can run deep feature synthesis to make the actual features. We need to change `feature_only` to `False` and then we're good to go."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Built 255 features\n",
      "EntitySet scattered to workers in 2.067 seconds\n",
      "Elapsed: 03:47 | Remaining: 00:00 | Progress: 100%|██████████| Calculated: 29/29 chunks\n",
      "230 seconds elapsed.\n"
     ]
    }
   ],
   "source": [
    "from timeit import default_timer as timer\n",
    "\n",
    "start = timer()\n",
    "feature_matrix, feature_defs = ft.dfs(entityset=es, target_entity='members', \n",
    "                                      cutoff_time = cutoff_times, \n",
    "                                      agg_primitives = agg_primitives,\n",
    "                                      trans_primitives = trans_primitives,\n",
    "                                      where_primitives = where_primitives,\n",
    "                                      max_depth = 2, features_only = False,\n",
    "                                      verbose = 1, chunk_size = 1000,  \n",
    "                                      n_jobs = -1,\n",
    "                                      cutoff_time_in_index = True)\n",
    "end = timer()\n",
    "print(f'{round(end - start)} seconds elapsed.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `chunk_size` is a parameter that may need to be adjusted to optimize the calculation. I suggest playing around with this parameter to find the optimal value. Generally I've found that a large value makes the calculation proceed quicker although it depends on the machine in use and the number of unique cutoff times. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>bd</th>\n",
       "      <th>city</th>\n",
       "      <th>registered_via</th>\n",
       "      <th>gender</th>\n",
       "      <th>SUM(logs.num_25)</th>\n",
       "      <th>SUM(logs.num_50)</th>\n",
       "      <th>SUM(logs.num_75)</th>\n",
       "      <th>SUM(logs.num_985)</th>\n",
       "      <th>SUM(logs.num_100)</th>\n",
       "      <th>SUM(logs.num_unq)</th>\n",
       "      <th>...</th>\n",
       "      <th>WEEKEND(LAST(transactions.membership_expire_date))</th>\n",
       "      <th>DAY(LAST(logs.date))</th>\n",
       "      <th>DAY(LAST(transactions.transaction_date))</th>\n",
       "      <th>DAY(LAST(transactions.membership_expire_date))</th>\n",
       "      <th>MONTH(LAST(logs.date))</th>\n",
       "      <th>MONTH(LAST(transactions.transaction_date))</th>\n",
       "      <th>MONTH(LAST(transactions.membership_expire_date))</th>\n",
       "      <th>label</th>\n",
       "      <th>days_to_churn</th>\n",
       "      <th>churn_date</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>msno</th>\n",
       "      <th>time</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>+9v4Rbyc+58MyKbt1wrCskWClJadOJh7CapZa9CYXUM=</th>\n",
       "      <th>2015-01-01</th>\n",
       "      <td>24.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>female</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>+FMjiiorqZQ3ZzNNmgO0vZM2yh8IHPvWSvwy2fSBMLU=</th>\n",
       "      <th>2015-01-01</th>\n",
       "      <td>27.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>male</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>111.0</td>\n",
       "      <td>106.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>+V3HOZsK34UPrNOYg6IhG8sP1dY6w5LG8J98eodnBBk=</th>\n",
       "      <th>2015-01-01</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>+ikgRAmrCW349x39kQ0nOqh9jvajPXJFZkI9Q6omEMs=</th>\n",
       "      <th>2015-01-01</th>\n",
       "      <td>0.0</td>\n",
       "      <td>14.0</td>\n",
       "      <td>9.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>463.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>+kbXNszLheADYStfNoRwa9q9sZykS5Tfk044GMwOw1o=</th>\n",
       "      <th>2015-01-01</th>\n",
       "      <td>29.0</td>\n",
       "      <td>15.0</td>\n",
       "      <td>9.0</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>54.0</td>\n",
       "      <td>55.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 258 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                           bd  city  \\\n",
       "msno                                         time                     \n",
       "+9v4Rbyc+58MyKbt1wrCskWClJadOJh7CapZa9CYXUM= 2015-01-01  24.0   5.0   \n",
       "+FMjiiorqZQ3ZzNNmgO0vZM2yh8IHPvWSvwy2fSBMLU= 2015-01-01  27.0   6.0   \n",
       "+V3HOZsK34UPrNOYg6IhG8sP1dY6w5LG8J98eodnBBk= 2015-01-01   NaN   NaN   \n",
       "+ikgRAmrCW349x39kQ0nOqh9jvajPXJFZkI9Q6omEMs= 2015-01-01   0.0  14.0   \n",
       "+kbXNszLheADYStfNoRwa9q9sZykS5Tfk044GMwOw1o= 2015-01-01  29.0  15.0   \n",
       "\n",
       "                                                         registered_via  \\\n",
       "msno                                         time                         \n",
       "+9v4Rbyc+58MyKbt1wrCskWClJadOJh7CapZa9CYXUM= 2015-01-01             7.0   \n",
       "+FMjiiorqZQ3ZzNNmgO0vZM2yh8IHPvWSvwy2fSBMLU= 2015-01-01             7.0   \n",
       "+V3HOZsK34UPrNOYg6IhG8sP1dY6w5LG8J98eodnBBk= 2015-01-01             NaN   \n",
       "+ikgRAmrCW349x39kQ0nOqh9jvajPXJFZkI9Q6omEMs= 2015-01-01             9.0   \n",
       "+kbXNszLheADYStfNoRwa9q9sZykS5Tfk044GMwOw1o= 2015-01-01             9.0   \n",
       "\n",
       "                                                         gender  \\\n",
       "msno                                         time                 \n",
       "+9v4Rbyc+58MyKbt1wrCskWClJadOJh7CapZa9CYXUM= 2015-01-01  female   \n",
       "+FMjiiorqZQ3ZzNNmgO0vZM2yh8IHPvWSvwy2fSBMLU= 2015-01-01    male   \n",
       "+V3HOZsK34UPrNOYg6IhG8sP1dY6w5LG8J98eodnBBk= 2015-01-01     NaN   \n",
       "+ikgRAmrCW349x39kQ0nOqh9jvajPXJFZkI9Q6omEMs= 2015-01-01     NaN   \n",
       "+kbXNszLheADYStfNoRwa9q9sZykS5Tfk044GMwOw1o= 2015-01-01    male   \n",
       "\n",
       "                                                         SUM(logs.num_25)  \\\n",
       "msno                                         time                           \n",
       "+9v4Rbyc+58MyKbt1wrCskWClJadOJh7CapZa9CYXUM= 2015-01-01               0.0   \n",
       "+FMjiiorqZQ3ZzNNmgO0vZM2yh8IHPvWSvwy2fSBMLU= 2015-01-01               0.0   \n",
       "+V3HOZsK34UPrNOYg6IhG8sP1dY6w5LG8J98eodnBBk= 2015-01-01               0.0   \n",
       "+ikgRAmrCW349x39kQ0nOqh9jvajPXJFZkI9Q6omEMs= 2015-01-01               0.0   \n",
       "+kbXNszLheADYStfNoRwa9q9sZykS5Tfk044GMwOw1o= 2015-01-01              22.0   \n",
       "\n",
       "                                                         SUM(logs.num_50)  \\\n",
       "msno                                         time                           \n",
       "+9v4Rbyc+58MyKbt1wrCskWClJadOJh7CapZa9CYXUM= 2015-01-01               0.0   \n",
       "+FMjiiorqZQ3ZzNNmgO0vZM2yh8IHPvWSvwy2fSBMLU= 2015-01-01               0.0   \n",
       "+V3HOZsK34UPrNOYg6IhG8sP1dY6w5LG8J98eodnBBk= 2015-01-01               0.0   \n",
       "+ikgRAmrCW349x39kQ0nOqh9jvajPXJFZkI9Q6omEMs= 2015-01-01               0.0   \n",
       "+kbXNszLheADYStfNoRwa9q9sZykS5Tfk044GMwOw1o= 2015-01-01               4.0   \n",
       "\n",
       "                                                         SUM(logs.num_75)  \\\n",
       "msno                                         time                           \n",
       "+9v4Rbyc+58MyKbt1wrCskWClJadOJh7CapZa9CYXUM= 2015-01-01               0.0   \n",
       "+FMjiiorqZQ3ZzNNmgO0vZM2yh8IHPvWSvwy2fSBMLU= 2015-01-01               0.0   \n",
       "+V3HOZsK34UPrNOYg6IhG8sP1dY6w5LG8J98eodnBBk= 2015-01-01               0.0   \n",
       "+ikgRAmrCW349x39kQ0nOqh9jvajPXJFZkI9Q6omEMs= 2015-01-01               0.0   \n",
       "+kbXNszLheADYStfNoRwa9q9sZykS5Tfk044GMwOw1o= 2015-01-01               5.0   \n",
       "\n",
       "                                                         SUM(logs.num_985)  \\\n",
       "msno                                         time                            \n",
       "+9v4Rbyc+58MyKbt1wrCskWClJadOJh7CapZa9CYXUM= 2015-01-01                0.0   \n",
       "+FMjiiorqZQ3ZzNNmgO0vZM2yh8IHPvWSvwy2fSBMLU= 2015-01-01                0.0   \n",
       "+V3HOZsK34UPrNOYg6IhG8sP1dY6w5LG8J98eodnBBk= 2015-01-01                0.0   \n",
       "+ikgRAmrCW349x39kQ0nOqh9jvajPXJFZkI9Q6omEMs= 2015-01-01                0.0   \n",
       "+kbXNszLheADYStfNoRwa9q9sZykS5Tfk044GMwOw1o= 2015-01-01                3.0   \n",
       "\n",
       "                                                         SUM(logs.num_100)  \\\n",
       "msno                                         time                            \n",
       "+9v4Rbyc+58MyKbt1wrCskWClJadOJh7CapZa9CYXUM= 2015-01-01                0.0   \n",
       "+FMjiiorqZQ3ZzNNmgO0vZM2yh8IHPvWSvwy2fSBMLU= 2015-01-01              111.0   \n",
       "+V3HOZsK34UPrNOYg6IhG8sP1dY6w5LG8J98eodnBBk= 2015-01-01                0.0   \n",
       "+ikgRAmrCW349x39kQ0nOqh9jvajPXJFZkI9Q6omEMs= 2015-01-01                0.0   \n",
       "+kbXNszLheADYStfNoRwa9q9sZykS5Tfk044GMwOw1o= 2015-01-01               54.0   \n",
       "\n",
       "                                                         SUM(logs.num_unq)  \\\n",
       "msno                                         time                            \n",
       "+9v4Rbyc+58MyKbt1wrCskWClJadOJh7CapZa9CYXUM= 2015-01-01                0.0   \n",
       "+FMjiiorqZQ3ZzNNmgO0vZM2yh8IHPvWSvwy2fSBMLU= 2015-01-01              106.0   \n",
       "+V3HOZsK34UPrNOYg6IhG8sP1dY6w5LG8J98eodnBBk= 2015-01-01                0.0   \n",
       "+ikgRAmrCW349x39kQ0nOqh9jvajPXJFZkI9Q6omEMs= 2015-01-01                0.0   \n",
       "+kbXNszLheADYStfNoRwa9q9sZykS5Tfk044GMwOw1o= 2015-01-01               55.0   \n",
       "\n",
       "                                                            ...      \\\n",
       "msno                                         time           ...       \n",
       "+9v4Rbyc+58MyKbt1wrCskWClJadOJh7CapZa9CYXUM= 2015-01-01     ...       \n",
       "+FMjiiorqZQ3ZzNNmgO0vZM2yh8IHPvWSvwy2fSBMLU= 2015-01-01     ...       \n",
       "+V3HOZsK34UPrNOYg6IhG8sP1dY6w5LG8J98eodnBBk= 2015-01-01     ...       \n",
       "+ikgRAmrCW349x39kQ0nOqh9jvajPXJFZkI9Q6omEMs= 2015-01-01     ...       \n",
       "+kbXNszLheADYStfNoRwa9q9sZykS5Tfk044GMwOw1o= 2015-01-01     ...       \n",
       "\n",
       "                                                         WEEKEND(LAST(transactions.membership_expire_date))  \\\n",
       "msno                                         time                                                             \n",
       "+9v4Rbyc+58MyKbt1wrCskWClJadOJh7CapZa9CYXUM= 2015-01-01                                                 0.0   \n",
       "+FMjiiorqZQ3ZzNNmgO0vZM2yh8IHPvWSvwy2fSBMLU= 2015-01-01                                                 0.0   \n",
       "+V3HOZsK34UPrNOYg6IhG8sP1dY6w5LG8J98eodnBBk= 2015-01-01                                                 NaN   \n",
       "+ikgRAmrCW349x39kQ0nOqh9jvajPXJFZkI9Q6omEMs= 2015-01-01                                                 0.0   \n",
       "+kbXNszLheADYStfNoRwa9q9sZykS5Tfk044GMwOw1o= 2015-01-01                                                 0.0   \n",
       "\n",
       "                                                         DAY(LAST(logs.date))  \\\n",
       "msno                                         time                               \n",
       "+9v4Rbyc+58MyKbt1wrCskWClJadOJh7CapZa9CYXUM= 2015-01-01                   NaN   \n",
       "+FMjiiorqZQ3ZzNNmgO0vZM2yh8IHPvWSvwy2fSBMLU= 2015-01-01                   1.0   \n",
       "+V3HOZsK34UPrNOYg6IhG8sP1dY6w5LG8J98eodnBBk= 2015-01-01                   NaN   \n",
       "+ikgRAmrCW349x39kQ0nOqh9jvajPXJFZkI9Q6omEMs= 2015-01-01                   NaN   \n",
       "+kbXNszLheADYStfNoRwa9q9sZykS5Tfk044GMwOw1o= 2015-01-01                   1.0   \n",
       "\n",
       "                                                         DAY(LAST(transactions.transaction_date))  \\\n",
       "msno                                         time                                                   \n",
       "+9v4Rbyc+58MyKbt1wrCskWClJadOJh7CapZa9CYXUM= 2015-01-01                                       NaN   \n",
       "+FMjiiorqZQ3ZzNNmgO0vZM2yh8IHPvWSvwy2fSBMLU= 2015-01-01                                       NaN   \n",
       "+V3HOZsK34UPrNOYg6IhG8sP1dY6w5LG8J98eodnBBk= 2015-01-01                                       NaN   \n",
       "+ikgRAmrCW349x39kQ0nOqh9jvajPXJFZkI9Q6omEMs= 2015-01-01                                       NaN   \n",
       "+kbXNszLheADYStfNoRwa9q9sZykS5Tfk044GMwOw1o= 2015-01-01                                       NaN   \n",
       "\n",
       "                                                         DAY(LAST(transactions.membership_expire_date))  \\\n",
       "msno                                         time                                                         \n",
       "+9v4Rbyc+58MyKbt1wrCskWClJadOJh7CapZa9CYXUM= 2015-01-01                                             NaN   \n",
       "+FMjiiorqZQ3ZzNNmgO0vZM2yh8IHPvWSvwy2fSBMLU= 2015-01-01                                             NaN   \n",
       "+V3HOZsK34UPrNOYg6IhG8sP1dY6w5LG8J98eodnBBk= 2015-01-01                                             NaN   \n",
       "+ikgRAmrCW349x39kQ0nOqh9jvajPXJFZkI9Q6omEMs= 2015-01-01                                             NaN   \n",
       "+kbXNszLheADYStfNoRwa9q9sZykS5Tfk044GMwOw1o= 2015-01-01                                             NaN   \n",
       "\n",
       "                                                         MONTH(LAST(logs.date))  \\\n",
       "msno                                         time                                 \n",
       "+9v4Rbyc+58MyKbt1wrCskWClJadOJh7CapZa9CYXUM= 2015-01-01                     NaN   \n",
       "+FMjiiorqZQ3ZzNNmgO0vZM2yh8IHPvWSvwy2fSBMLU= 2015-01-01                     1.0   \n",
       "+V3HOZsK34UPrNOYg6IhG8sP1dY6w5LG8J98eodnBBk= 2015-01-01                     NaN   \n",
       "+ikgRAmrCW349x39kQ0nOqh9jvajPXJFZkI9Q6omEMs= 2015-01-01                     NaN   \n",
       "+kbXNszLheADYStfNoRwa9q9sZykS5Tfk044GMwOw1o= 2015-01-01                     1.0   \n",
       "\n",
       "                                                         MONTH(LAST(transactions.transaction_date))  \\\n",
       "msno                                         time                                                     \n",
       "+9v4Rbyc+58MyKbt1wrCskWClJadOJh7CapZa9CYXUM= 2015-01-01                                         NaN   \n",
       "+FMjiiorqZQ3ZzNNmgO0vZM2yh8IHPvWSvwy2fSBMLU= 2015-01-01                                         NaN   \n",
       "+V3HOZsK34UPrNOYg6IhG8sP1dY6w5LG8J98eodnBBk= 2015-01-01                                         NaN   \n",
       "+ikgRAmrCW349x39kQ0nOqh9jvajPXJFZkI9Q6omEMs= 2015-01-01                                         NaN   \n",
       "+kbXNszLheADYStfNoRwa9q9sZykS5Tfk044GMwOw1o= 2015-01-01                                         NaN   \n",
       "\n",
       "                                                         MONTH(LAST(transactions.membership_expire_date))  \\\n",
       "msno                                         time                                                           \n",
       "+9v4Rbyc+58MyKbt1wrCskWClJadOJh7CapZa9CYXUM= 2015-01-01                                               NaN   \n",
       "+FMjiiorqZQ3ZzNNmgO0vZM2yh8IHPvWSvwy2fSBMLU= 2015-01-01                                               NaN   \n",
       "+V3HOZsK34UPrNOYg6IhG8sP1dY6w5LG8J98eodnBBk= 2015-01-01                                               NaN   \n",
       "+ikgRAmrCW349x39kQ0nOqh9jvajPXJFZkI9Q6omEMs= 2015-01-01                                               NaN   \n",
       "+kbXNszLheADYStfNoRwa9q9sZykS5Tfk044GMwOw1o= 2015-01-01                                               NaN   \n",
       "\n",
       "                                                         label  days_to_churn  \\\n",
       "msno                                         time                               \n",
       "+9v4Rbyc+58MyKbt1wrCskWClJadOJh7CapZa9CYXUM= 2015-01-01    0.0            NaN   \n",
       "+FMjiiorqZQ3ZzNNmgO0vZM2yh8IHPvWSvwy2fSBMLU= 2015-01-01    0.0            NaN   \n",
       "+V3HOZsK34UPrNOYg6IhG8sP1dY6w5LG8J98eodnBBk= 2015-01-01    0.0            NaN   \n",
       "+ikgRAmrCW349x39kQ0nOqh9jvajPXJFZkI9Q6omEMs= 2015-01-01    NaN          463.0   \n",
       "+kbXNszLheADYStfNoRwa9q9sZykS5Tfk044GMwOw1o= 2015-01-01    0.0            NaN   \n",
       "\n",
       "                                                         churn_date  \n",
       "msno                                         time                    \n",
       "+9v4Rbyc+58MyKbt1wrCskWClJadOJh7CapZa9CYXUM= 2015-01-01         NaN  \n",
       "+FMjiiorqZQ3ZzNNmgO0vZM2yh8IHPvWSvwy2fSBMLU= 2015-01-01         NaN  \n",
       "+V3HOZsK34UPrNOYg6IhG8sP1dY6w5LG8J98eodnBBk= 2015-01-01         NaN  \n",
       "+ikgRAmrCW349x39kQ0nOqh9jvajPXJFZkI9Q6omEMs= 2015-01-01         NaN  \n",
       "+kbXNszLheADYStfNoRwa9q9sZykS5Tfk044GMwOw1o= 2015-01-01         NaN  \n",
       "\n",
       "[5 rows x 258 columns]"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "feature_matrix.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can save these feature definitions as a binary file which will allow us to make the same exact features for another entityset of the same format. This is useful when we have multiple partitions and we want to make the same features for each. Instead of remaking the feature definitions, we pass in the same feature definitions to a call to `calculate_feature_matrix`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "ft.save_features(feature_defs, '/data/churn/features.txt')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAboAAAEGCAYAAAAT/1CLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3XuYHFWd//H3h2CAqCQBFEOCEmSEX0AFhBCFRSQaAqJhV4S4KgFxdRUVL/tTvMIqQVEUURFXIRJQgRhREFEMNxU14SLXEGBGCCYxIUhIWLkHvvvHOR1qOnPrdPd0p+bzep5+purUqepTp6vnW+fUqWpFBGZmZmW1SasLYGZm1kwOdGZmVmoOdGZmVmoOdGZmVmoOdGZmVmoOdGZmVmoOdNYvSa+UdL2kJyQtbnV5+iJpsaTPtboc1SR9WdIDkkLS0a0uTytIOlfSla0uRz0kHZA/w3GtLosNnAPdIJK0haQvSeqU9LikVZJukPSRVpetH18FHgF2AfbuKYOko/M/gAU9LOuSdFJzi9i+JO0DnAC8DxgDXNRLvpdJmi1piaQnJa2QdKWkNw1meTeUpJMkdfWR5Xjg7TVsb+3GeFKQP7NzG7St/SVdIun+/P3q8SRO0j6S/pRPRpfnE6thVXnGSJoj6ZH8ulDSi/t5/1dLulHSo5KukfTSquWnSTqr/j1tLge6wXUWcBTw/4EJwBuAM4FRrSzUAHQAv4uIxRHxYB/5Ani1pOmDVK5BI2l4Hat3AM9GxCURsSIiHu9h+88DrgS2B/4deAXwVuC3wNZ1vHfbiIg1EfFwq8uxkXkBcCfwSWBFTxkkbQ/MA+4GXgN8AHg/MLOQZxPgMmA88CZgCukY+4Uk9fH+5wB/BF4NrAK+XtjmRODfctnaW0T4NUgvYDXwoX7ynAtcWZX2rvRRrZs/CegCjgA6gceAXwBbkg68u4H/BeYCI/t5vzHAhblsjwPXAnvlZTuQglfxdVIv2zkaWAt8DbgXGF5Y1lVcD1gMfK5q/bOBawvz15K+ZCcDK3P5ZpJOzr4APAA8CMys2s7inO9sUiv0H8ApwCaFPM/LdXgf8ASwEHh/1XYC+AjwE2ANcFEfdTiD9M/oKWBpLvOmhc+zWx32so3d8/Ld+vm8Xgj8T973J4EbgSmF5ZXP7AjSP7bH8udxdNV2xpOC6BPAEuC4XOdnF/JMA27O21gNXA/s0UfZTgK6BnpsA7sCV+RtPwosAt5d+Bx7rDfSP/PfAv/M9XAx8LIevh/TgLvytq8FOqrK0+d2cp4P58/0sVzWo3J5xvWxj9XfmQPysp2BX+X3+yfwS2CnGv5/LKbqe5PTT8llLB7jx+X9fn6en5LLsnNV/a8rXy/v+SiwS54+GFiYp4cDdwAHDbT8rXy5RTe4lgNTJW3VgG2NIf2DfRvpANyXFNjeS/ondzDwL8BnettAPpP7BalL8lBgIimAzJO0Dekf4BjSl+jUPH1aP+WaSQq4H97A/So6nBSU9gM+TtqXX5HOcv8F+C/gM5IOrlrvw8DfSd2sHyN1mRXL8wPSCcH7gf8HfBE4VdKxVds5EfgTsCfQW5fRm4FZwPnAbsAnSP9kTsxZjgc+CjxDqr8xvezrSuBZ4PB+Wo+zgINIJz+7k862L5O0S1W+rwDnAa8incicLekVucwCfg6MBPYH3gK8GdijsF8vAX4KXED6h/ha4Jukk5lGuQB4CHgd8ErSZ1xp8e1NqrOPUqg3SROA3wF/BvYCDsz55knavLDtMaSWzTvz9l9IqjsGuh1J04DTgW+Q6noO6USuL8cDf8h5K+X+k6QtSEF1c+D1+fUC4Dd19hZA+u7/NiKeLaT9BhjBc5/pvsB9EXF3JUNELCR9t/frY9u3AgfnFuHBeR7Syeb1EXFFnWUfHK2OtEPpRTrY7id9oW4Dvg8cBqiQ51wG1qJbC2xTSDszb/dFhbQzgBv7KM9k0hndhELaZqSA/IVC2mJ6OJOs2tbRwNo8fTypm2OrPL+hLbpbqvIsBG6vSrsVOK1q23+oynMKsCRPjycFlF2q8nyh+H65Xs4ZwGf6B2BOVdrxpNbx8Oq66Wdb/0k603+cFMBOBfYuLN8pl+uQqvX+AszK0zvkPB8vLB9GauG/P8+/KefZqZBnK1Kr5ew8v0fOs0MNx/dJ1NaiW0NVS7Mq/9rq5XkbF1albZbLfljV96P4XTgyf+6b17Cd64AfV+U5jT5adDnPlcC5VWnH5m0Xv7Pb5s/6qAHW72J6btHdA5xSlfb8XM635/nvA3/qYd0bgDP7eM9dgKuAvwGXkAL37rksLyKdBHQB1wCvGOixMtgvt+gGUUT8EXg5qTUym3SgzwUu7aefvCfLIuIfhfkVwIrofg1tBdDXxeZdgYci4s5CGZ8EFuRlG+q7pEBX7+jHW6vmV5BOEKrTqvfxz1XzfwTGSdqSdPYu4EZJ/6y8SK3Fjqr1rh9AGXcFfl+V9jvSmfvLB7D+OhHxPeAlpFb6PNJZ/wJJn8pZJuS/1e/3e9b/vG4pbPcZUotx28J2/hERXYU8q0hd3hW3kbrq7pD0c0nH52tBjXQaqaV5bR7IsucA1tkb+Neqz+4hUn0XP7+/V30X/k763F9cw3YmkFr0RdfVsoMFuwJ3Fr+zEfEAqc7r+a41VUTcFRGTI+KlETGN1MU7i9RD8nZS9++upP9j57eupH1zoBtkEbE2Iv4UEV/PB87RpG7D/XOWZ0lfyKLn9bCpp6s33UvaoH/GEfE0aZThcZJ27CFLK/exkvd1pDPTyms3Ujdf0aM1bLchIuKfEXF5RJwUEZNI/1S+uAHdW09Vb5ru9RT9lOMZUlfVgaSz/rcB90g6tMZy9PUeXyINiJhDqv/5kk7uZ7VNSP9Qd696vYLUK1DR0/5X1q9lOxuD5aQTpKJtC8t6y1PJt7yH9N6cACyKiF8CbyT1ZjxJOnGfKOmFNWxr0DjQtd6i/LdyprkS2K4qz0DOdDfEQmDrfL0CAEmbAfuQLjRvsIiYC9xEulZUrad93KOHfBtqUtX860gt4EdymQBeGhFdVa+/bsB7LeS5k5SK15O6pDZke9UWkS78j8zvRQ/vtz+1fV53Ai+StK7FKWk06Z/8OpFcHxGnRMT+pJbqMTWWv08RcW9EfDciDid1H3+gsPgpUrdr0Y2kE5K/9vD51TKicyDbuZN07BTtO4Bt91TuhcCEfO0bAEnbkgao1PVdI/VYvClfR6uYSuoqvbmQZ7ykda3e/L3fngG2UnP+95EGaUGKH5UT1OGFtLbTloUqK0m/k/SfkvbK90xNJnXzrSb1cUPq399F0nGSXi7pP0iDS5rhalL33E8k7StpN9IAhs1Jt0LU6xOkASVjq9KvBI6UNEXSzpJOB17WgPer2D13hb1C0r+Trpl9HSB3180CfiDp3ZJ2yvcKvafQRViLLwNvk3RCfr8jSNeIvh4R1a2KXknaQ9IvJR0haTdJO0o6kjR0+48R8WAOxD8FvivpIEm7SDqD1Brqb5BE0ZWkbuHzJe0t6dWk1s1acstH0uskfT7fn/XSfKy+ivTPvy/DJe1e9apuKSPpBZLOlHSgpPGS9iD9cy5u/z7gDZK2KwSIU0gDiH4kaWJe9w2Szuil96A3A9nO10nH6fGSOiQdA7x7ANu+D3hN/v5uo3TryE9I3X4XSdpT0mtIg4SW0ct9lbCunnaXtDspmLwkz+9UyHYW6UToB5J2lfRW4EvAtyOi0itxJelabmV/9yF91+eTTmD6pHRP3izStd+HcvLvgfdI2pXU0rs1ItYMoH4GX6svEg6lF+lg+AOpRfME6QLvjygMBsn5Pkv6AvyTNDLtOHq4vaBqnc8Bi3t4v6X9lKn69oLfkW8vKORZTA2DUarSf0rVbQmkEXDnk0bYrcz709NglLOrttXTRf7fAD+qKutM4Iek2wseIrUqi0Ovh5ECyF2ks+9/5P1+eyFPAO8a4Oc6g9Tyeip/bjPJtxf0VTdV29iGNMLvFtIgjUdJgwy+Sh7Uk/NtycBuL9ivavvVA4LGk64DFm8vuJ70zxHSdZfLSddAnyQNovoahdtGetiHk1h/aH0AT+Tl55IHo5BOpn7Cc7d4rCT9w9++sL2phXotHv+vJA2MeJh0zHaRBltsVShH9fdjP6oG1/S3nZzn+PyZPk46/mbQ/2CUHUlB4J+sf3vB5Tx3e8Fl9HN7AXBAL3V6bVW+SaTriU/kz+zLwLAevus/JQ1MeiTX94sHeIz/FzC3Km1zUpflGlLL8VUD2VYrXsoFNrMhLF9bWUo6ofl2q8tj1kibtroAZjb4cvfWWlKL6cWk+/6CNDDErFQG5RqdpFmSVkq6o5D2NUl3SbotD18eVVj2aaXnI94t6aBC+tSc1iXphEL6eEkLcvpFGzBCzWyoGUEa3r+Q1IW2Cam784GWlsqsCQal61LS/qQ+6fMiYrecNgW4OiLWSjoVICI+lUf2XEB6Ssd2pH7xymiwe0g3uy4lDXl+R0TcKWkOcHFEXCjpe6SLom3/oFEzM2u+QWnRRcTvSTcQF9N+GxGVxwnNByo/ezGN9MSCJyPiPtIF4on51RVpOPJTpAEU0ySJdK/P3Lz+bNLTRszMzNrmGt17eG6I7VhS4KtYynPD05dUpe9DerL76kLQLOZfZ82aNR51Y2ZWciNHjlzvKVMtv49O0mdJF8V/3OqymJlZ+bS0Raf0o4qHApPjuYuFy0h361eMy2n0kv4QMErSprlVV8xvZmZDXMtadJKmkm7afWtEPFZYdCkwXdJmksaTHrB6PWnwSUceYTkcmA5cmgPkNaQncEC6ofOSZpW7s7OzWZveaLlOunN9dOf6WJ/rpLtm18dg3V5wAemJ8jtLWqr0u1/fIT0hY56kW/JoSSL9RtIc0qOAfgMcFxHP5Nbah0hPVF9Eepho5dl/nwI+LqmLdM3unMHYLzMza3+D0nUZEe/oIbnXYBQRMyn8DHwh/XLSI3Sq0+8ljco0MzPrpuWDUczMzJrJgc7MzErNgc7MzErNgc7MzErNgc7MzEqtXR4BttHY+7oRcF3r70dffcx6TzkzM7MeuEVnZmal5kBnZmal5kBnZmal5kBnZmal5kBnZmal5kBnZmal5kBnZmal5kBnZmal5kBnZmal5kBnZmal5kBnZmal5kBnZmal5kBnZmal5kBnZmal5kBnZmal5kBnZmal5kBnZmal5kBnZmal5kBnZmal5kBnZmal5kBnZmalNiiBTtIsSSsl3VFI20rSPEmd+e/onC5J35LUJek2SXsW1pmR83dKmlFIf42k2/M635KkwdgvMzNrf4PVojsXmFqVdgJwVUR0AFfleYCDgY78eh9wFqTACJwI7ANMBE6sBMec5z8K61W/l5mZDVGDEugi4vfAqqrkacDsPD0bOKyQfl4k84FRksYABwHzImJVRDwMzAOm5mVbRsT8iAjgvMK2zMxsiGvlNbptI2J5nl4BbJunxwJLCvmW5rS+0pf2kG5mZsamrS4AQESEpBis9+vs7Kxj7RENK0c96tuHxmu38rSa66M718f6XCfd1VMfHR0dfS5vZaB7QNKYiFieux9X5vRlwPaFfONy2jLggKr0a3P6uB7y96q/SunTdX1uetDUtQ8N1tnZ2VblaTXXR3euj/W5Trprdn20suvyUqAycnIGcEkh/ag8+nISsCZ3cV4BTJE0Og9CmQJckZc9ImlSHm15VGFbZmY2xA1Ki07SBaTW2DaSlpJGT34FmCPpWOB+4Iic/XLgEKALeAw4BiAiVkn6EnBDzvfFiKgMcPkgaWTnFsCv88vMzGxwAl1EvKOXRZN7yBvAcb1sZxYwq4f0G4Hd6imjmZmVk5+MYmZmpeZAZ2ZmpeZAZ2ZmpeZAZ2ZmpeZAZ2ZmpeZAZ2ZmpeZAZ2ZmpeZAZ2ZmpeZAZ2ZmpeZAZ2ZmpeZAZ2ZmpeZAZ2ZmpeZAZ2ZmpeZAZ2ZmpeZAZ2ZmpeZAZ2ZmpeZAZ2ZmpeZAZ2ZmpeZAZ2ZmpeZAZ2ZmpeZAZ2ZmpeZAZ2ZmpeZAZ2ZmpeZAZ2ZmpeZAZ2ZmpeZAZ2ZmpeZAZ2ZmpeZAZ2ZmpdbyQCfpY5IWSrpD0gWSNpc0XtICSV2SLpI0POfdLM935eU7FLbz6Zx+t6SDWrU/ZmbWXloa6CSNBT4C7BURuwHDgOnAqcDpEbET8DBwbF7lWODhnH56zoekCXm9XYGpwHclDRvMfTEzs/bU8hYdsCmwhaRNgRHAcuBAYG5ePhs4LE9Py/Pk5ZMlKadfGBFPRsR9QBcwcZDKb2ZmbWzTVr55RCyTdBrwN+Bx4LfATcDqiFibsy0FxubpscCSvO5aSWuArXP6/MKmi+usp7Ozs45Sj6hj3capbx8ar93K02quj+5cH+tznXRXT310dHT0ubylgU7SaFJrbDywGvgpqeuxqfqrlD5dt6xxBalDXfvQYJ2dnW1VnlZzfXTn+lif66S7ZtdHq7su3wjcFxEPRsTTwMXAvsCo3JUJMA6oRJdlwPYAeflI4KFieg/rmJnZENbqQPc3YJKkEfla22TgTuAa4PCcZwZwSZ6+NM+Tl18dEZHTp+dRmeOBDuD6QdoHMzNrY62+RrdA0lzgL8Ba4Gbg+8CvgAslnZzTzsmrnAOcL6kLWEUaaUlELJQ0hxQk1wLHRcQzg7ozZmbWlloa6AAi4kTgxKrke+lh1GREPAG8vZftzARmNryAZma2UWt116WZmVlTOdCZmVmpDTjQSTpe0jbNLIyZmVmj1dKiOxBYLOkySUdK2qxZhTIzM2uUAQe6iJgGvAz4NfBRYIWksyXt36zCmZmZ1auma3QR8VBEnBkRrwVeD+wNXCNpsaTPSnpBU0ppZma2gWoejCJpsqQfAtcCDwBHAe8G9iC19szMzNrGgO+jyw9fng6sAc4DPhcRywrL55N+UsfMzKxt1HLD+ObAv0bEDT0tjIinJe3VmGKZmZk1Ri2B7svAY8WE/OsDW0TE3wEi4q4Gls3MzKxutVyj+wXpVwGKxgE/b1xxzMzMGquWQLdzRNxeTMjzuzS2SGZmZo1TS6BbKWmnYkKef6ixRTIzM2ucWgLdLOBnkg6VNEHSW4C5wNnNKZqZmVn9ahmM8hXgaeA00q95LyEFuW80oVxmZmYNMeBAFxHPAl/LLzMzs41CTT+8Kmln4NVAt0d9RcSsRhbKzMysUWp5MspngC8At9L9frogXb8zMzNrO7W06D4KTIyI25pVGDMzs0arZdTl44CffGJmZhuVWgLd54FvSxojaZPiq1mFMzMzq1ctXZfn5r/vLaSJdI1uWKMKZGZm1ki1BLrxTSuFmZlZk9RyH939ALmrctuIWN60UpmZmTXIgK+vSRol6SfAE0BXTnurpJObVTgzM7N61TKQ5HukXxd/GfBUTvszcGSjC2VmZtYotVyjmwxsl39JPAAi4kFJL25O0czMzOpXS4tuDbBNMUHSS4G6rtXlLtG5ku6StEjSayVtJWmepM78d3TOK0nfktQl6TZJexa2MyPn75Q0o54ymZlZedQS6M4m/UzPG4BNJL0WmE3q0qzHGcBvImIX0nM0FwEnAFdFRAdwVZ4HOBjoyK/3AWcBSNoKOBHYB5gInFgJjmZmNrTVEuhOBS4CzgSeR3q+5SWkQLVBJI0E9gfOAYiIpyJiNTCNFETJfw/L09OA8yKZD4ySNAY4CJgXEasi4mFgHjB1Q8tlZmblUcvtBUEKahsc2HowHngQ+KGkVwM3AcfT/faFFcC2eXos6XfwKpbmtN7Se9TZ2VlHkUfUsW7j1LcPjddu5Wk110d3ro/1uU66q6c+Ojo6+lxey68XHNjbsoi4uoYyVb//nsCHI2KBpDN4rpuysu2oDH5plP4qpU/XLWtcQepQ1z40WGdnZ1uVp9VcH925PtbnOumu2fVRy6jLc6rmXwQMJ7WedtzA918KLI2IBXl+LinQPSBpTEQsz12TK/PyZaRfN68Yl9OWAQdUpV+7gWUyM7MSGfA1uogYX3wBI4GZwHc29M0jYgWwJP+gK6RbGO4ELgUqIydnkK4FktOPyqMvJwFrchfnFcAUSaPzIJQpOc3MzIa4mn5hvCginpE0k9Qq+0YdZfgw8GNJw4F7gWNIAXiOpGOB+4Ejct7LgUNIT2Z5LOclIlZJ+hJwQ873xYhYVUeZzMysJDY40GVvAp6tZwMRcQuwVw+LJveQN4DjetnOLPxL52ZmVqWWwShLSD/JUzEC2Bz4YKMLZWZm1ii1tOjeVTX/KHBPRDzSwPKYmZk1VC330f2umQUxMzNrhlq6Ls+ne9dljyLiqLpKZGZm1kC1PAJsNelRXMNIIy03IT2SazXw18LLzMysbdRyje4VwJsj4g+VBEn7AZ+PiIMaXjIzM7MGqKVFNwmYX5W2AHht44pjZmbWWLUEupuBUyRtAZD/zgRuaUbBzMzMGqGWQHc0sC+wRtIDpB9i3Y/nHtVlZmbWdmq5vWAx8DpJ2wPbAcsj4m/NKpiZmVkj1NKiQ9LWpF8JeH1E/E3SdpLGNaVkZmZmDTDgQCfp9cDdwDuBz+fkDuCsJpTLzMysIWpp0X0TODIipgJrc9oCYGLDS2VmZtYgtQS6HSLiqjxdeULKU9T/CwhmZmZNU0ugu1NS9Y3hbwRub2B5zMzMGqqW1tgngMsk/QrYQtL/AG8hPQbMzMysLQ24RRcR84FXAQtJP3B6HzAxIm7oc0UzM7MWGlCLTtIw4CrgoIj4anOLZGZm1jgDatFFxDPA+IHmNzMzaxe1BK7/Bs6S9DJJwyRtUnk1q3BmZmb1qmUwytn571E8d3uB8vSwRhbKzMysUfoNdJJeEhErSF2XZmZmG5WBtOjuAbaMiPsBJF0cEf/W3GKZmZk1xkCur6lq/oAmlMPMzKwpBhLoov8sZmZm7WkgXZebSnoDz7XsqueJiKubUTgzM7N6DSTQrSQ9CaXioar5AHZsZKHMzMwapd9AFxE7NLsQ+ckrNwLLIuJQSeOBC4GtgZuAd0fEU5I2A84DXkMKuEfmXz5H0qeBY4FngI9ExBXNLreZmbW/drnZ+3hgUWH+VOD0iNgJeJgUwMh/H87pp+d8SJoATAd2BaYC383B08zMhriWBzpJ44A3k29IlyTgQGBuzjIbOCxPT8vz5OWTc/5pwIUR8WRE3Ad04R+ENTMz2iDQkX65/JPAs3l+a2B1RFR+xXwpMDZPjwWWAOTla3L+dek9rGNmZkNYS38dXNKhwMqIuEnSAYP1vp2dnXWsPaJh5ahHffvQeO1WnlZzfXTn+lif66S7euqjo6Ojz+UtDXTAvsBbJR0CbA5sCZwBjJK0aW61jQOW5fzLgO2BpZI2BUaSBqVU0iuK66ynv0rp03W9bnZQ1bUPDdbZ2dlW5Wk110d3ro/1uU66a3Z9tLTrMiI+HRHj8sjO6cDVEfFO4Brg8JxtBnBJnr40z5OXXx0RkdOnS9osj9jsAK4fpN0wM7M21uoWXW8+BVwo6WTgZuCcnH4OcL6kLmAVKTgSEQslzQHuBNYCx+Xf0DMzsyGubQJdRFwLXJun76WHUZMR8QTw9l7WnwnMbF4JzcxsY9QOoy7NzMyaxoHOzMxKzYHOzMxKzYHOzMxKzYHOzMxKzYHOzMxKzYHOzMxKzYHOzMxKzYHOzMxKzYHOzMxKzYHOzMxKzYHOzMxKzYHOzMxKzYHOzMxKzYHOzMxKzYHOzMxKrW1+eNVqM+qHy1pdhHVu2K/VJTAz651bdGZmVmoOdGZmVmoOdGZmVmoOdGZmVmoOdGZmVmoOdGZmVmoOdGZmVmoOdGZmVmoOdGZmVmoOdGZmVmotfQSYpO2B84BtgQC+HxFnSNoKuAjYAVgMHBERD0sScAZwCPAYcHRE/CVvawbwubzpkyNi9mDuy1C293Uj4LrWP5Js9TFjW10EM2tDrW7RrQU+ERETgEnAcZImACcAV0VEB3BVngc4GOjIr/cBZwHkwHgisA8wEThR0ujB3BEzM2tPLQ10EbG80iKLiP8FFgFjgWlApUU2GzgsT08DzotkPjBK0hjgIGBeRKyKiIeBecDUQdwVMzNrU61u0a0jaQdgD2ABsG1ELM+LVpC6NiEFwSWF1ZbmtN7SzcxsiGuLn+mR9ALgZ8BHI+KRdCkuiYiQFI18v87OzjrWHtGwclhj1fe5NlY7laUduD7W5zrprp766Ojo6HN5ywOdpOeRgtyPI+LinPyApDERsTx3Ta7M6cuA7Qurj8tpy4ADqtKv7e09+6uUPrXBoAvrWV2fawN1dna2TVnagetjfa6T7ppdHy3tusyjKM8BFkXENwqLLgVm5OkZwCWF9KOUTALW5C7OK4ApkkbnQShTcpqZmQ1xrW7R7Qu8G7hd0i057TPAV4A5ko4F7geOyMsuJ91a0EW6veAYgIhYJelLwA053xcjYtXg7IKZmbWzlga6iLgOUC+LJ/eQP4DjetnWLGBW40pnZmZl0DajLs3MzJrBgc7MzEqt1dfozBpm1A/bY0TsDfu1ugRmVuQWnZmZlZoDnZmZlZoDnZmZlZoDnZmZlZoHo5g1mH+fz6y9uEVnZmal5kBnZmal5kBnZmal5kBnZmal5sEoZiXlJ8WYJW7RmZlZqTnQmZlZqbnr0syayvcVWqu5RWdmZqXmQGdmZqXmrkszGxLaZRQqeCTqYHOLzszMSs0tOjOzQeYBOoPLLTozMys1BzozMys1d12amQ1R7TJAp9mDc9yiMzOzUnOgMzOzUnOgMzOzUnOgMzOzUitVoJM0VdLdkrokndDq8piZWeuVJtBJGgacCRwMTADeIWlCa0tlZmatVqbbCyYCXRFxL4CkC4FpwJ2NfJOh8iQBM7OyKE2LDhgLLCnML81pZmY2hJUp0JmZma2nTF2Xy4DtC/PjchoAI0eO1KCXyMzMWq5MLbobgA5J4yUNB6YDl7a4TGZm1mKlCXQRsRb4EHAFsAiYExELG/keQ+X2BUnbS7pG0p2SFko6PqdvJWmepM78d3ROl6Rv5Xq5TdKehW3NyPk7Jc1o1T41gqRhkm6WdFmeHy9pQd7vi/IJFpI2y/NdefkOhW18OqffLemg1uxJY0gaJWmupLskLZL02qF8jEj6WP6+3CHpAkmbD6VjRNIsSSsl3VHZyCQNAAADwklEQVRIa9jxIOk1km7P63xL0sB76SLCrwG8gGHAX4EdgeHArcCEVperSfs6BtgzT78QuId0y8ZXgRNy+gnAqXn6EODXgIBJwIKcvhVwb/47Ok+PbvX+1VEvHwd+AlyW5+cA0/P094AP5OkPAt/L09OBi/L0hHzcbAaMz8fTsFbvVx31MRt4b54eDowaqscIaeDbfcAWhWPj6KF0jAD7A3sCdxTSGnY8ANfnvMrrHjzQspWmRTcI1t2+EBFPAZXbF0onIpZHxF/y9P+SWshjSfs7O2ebDRyWp6cB50UyHxglaQxwEDAvIlZFxMPAPGDqIO5Kw0gaB7wZODvPCzgQmJuzVNdHpZ7mApNz/mnAhRHxZETcB3SRjquNjqSRpH9s5wBExFMRsZohfIyQxjxsIWlTYASwnCF0jETE74FVVckNOR7ysi0jYn6kqHdeYVv9cqAbuCF5+0LuUtkDWABsGxHL86IVwLZ5ure6KVOdfRP4JPBsnt8aWB2pyxy679u6/c7L1+T8ZaqP8cCDwA9zd+7Zkp7PED1GImIZcBrwN1KAWwPcxNA+RqBxx8PYPF2dPiAOdNYrSS8AfgZ8NCIeKS7LZ1XRkoINMkmHAisj4qZWl6WNbErqpjorIvYAHiV1Ta0zxI6R0aRWynhgO+D5bLwt06Zo5fHgQDdwfd6+UDaSnkcKcj+OiItz8gO5C4H8d2VO761uylJn+wJvlbSY1GV9IHAGqbulcotOcd/W7XdePhJ4iPLUB6Qz6qURsSDPzyUFvqF6jLwRuC8iHoyIp4GLScfNUD5GoHHHw7I8XZ0+IA50Azdkbl/I1wrOARZFxDcKiy4FKqOgZgCXFNKPyiOpJgFrcnfFFcAUSaPzGe+UnLZRiYhPR8S4iNiB9LlfHRHvBK4BDs/ZquujUk+H5/yR06fnEXfjgQ7SBfaNTkSsAJZI2jknTSY9bm9IHiOkLstJkkbk70+lPobsMZI15HjIyx6RNCnX71GFbfWv1SN1NqYXaaTQPaSRUJ9tdXmauJ/7kboYbgNuya9DSNcQrgI6gSuBrXJ+kR6o/VfgdmCvwrbeQ7qg3gUc0+p9a0DdHMBzoy53JP0T6gJ+CmyW0zfP8115+Y6F9T+b6+luahg11o4vYHfgxnyc/II0Sm7IHiPAfwN3AXcA55NGTg6ZYwS4gHR98mlSi//YRh4PwF65bv8KfAfQQMumvAEzM7NSctelmZmVmgOdmZmVmgOdmZmVmgOdmZmVmgOdmZmVmgOdmZmVmgOdmZmVmgOdmZmV2v8BjgG0Bm8gL6EAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "feature_matrix.loc[feature_matrix['SUM(logs.num_100)'] < 10000, 'SUM(logs.num_100)'].plot.hist();\n",
    "plt.title('Sum of Number of Songs Listened to 100 %');"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbsAAAEGCAYAAAD8PTu1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xu8XFV99/HP13BNkSRcREzQhHLERlofUSEUHoxEQ1Bq0GKLVQk2rX0UFatPFUSFiihaKoIV1EK4WOQilpJHUQw3KUq4yTXhModwSUIglJCAcg38nj/WmrCzz8zJTM7MmZOd7/v1mtfMXnvtPWv/9t7z25c1M4oIzMzMquwVvW6AmZlZtznZmZlZ5TnZmZlZ5TnZmZlZ5TnZmZlZ5TnZmZlZ5TnZjQCSJkoKSfv0ui1FkraWdLGkVbl9E4fhPY+V1N/t97FE0mGSVve6HUOVt88PD9N7VSJmI1k3Pgc2+mQn6ay8o3yrVD4hl0/tUdNGgo8DewH7ADsCi8sVJE3NcZrQYNxZki5v8z1PBKasT2O7QdKWko6TVJP0jKQVkm6U9Olet60Vg62f7AJgfBvzu1zSWR1p3DCS9CVJD3Rodm3FrJn8gR758ZKkpZLOk/S6DrSx/h6nS7q6hXr17eQ5SduVxm0q6dFuHFAM5+fsRp/ssmeBT3dyIxspJG06hMn7gAURcUdEPBIRL3aqXc1ExO8j4n+6/T5tOA04FPgnYDLwDuB7wNheNqpTIuKZiHi01+3YkHQ4Zg+QDiQnkLaztwL/T9KoDs2/XY/kdhS9D3imB23prIjYqB/AWcAVwPXAuYXyCUAAU/PwxDy8T2n6fuDYwnAAnyId/f0BeAg4GBgDnAs8BSwC/rIwTX3eH85teSbXOaT0Xjvk9j6W5/MbYN/C+Kl5Pu8BriUl8Y83We5NgROApcDzwELgbwrjH8jzqj+ubjKf+ntOaBLbywvDVwOnA18m7VQrgHOArQp1jgX6S/P5FLAEeBq4jLQzrnlP4DBgdWmatdZfLtsF+CmwEngC+BXwp+vYPlYCn1xHHQH/N6+z54H7gM+U6jwAfBU4OS/3o8BJwCaFOlsCPwRW5fadCnyjGA/gjTkGK/P2dRfwkUHa1nT9NIodsDVwZl4/z5HO5r9dWJ9RetT3j1a3zXcB1+R1uRA4oJ1tPNd5B3A7afu+PQ8H8OFBlrHc7mPzuFcCP8jv9xxwEzB9Heu7HLPDgNXA3sDv8rLdDLxtHfM5loHb+ody+3bNw0cAtwK/z+vkfGDH0n78bdL+8RywDDi/MP/ych+2ju3kK8DC0rjLSfvsWjEmJenzSdviM6T9+63trPMG7XugGBtgJnA3aVu/GugbLKaDxnt9J6zKI+9YlwP/G3ipvrIYWrJ7BJhF+nA9NW8Iv8g7xS7Ad/PK27Y074fzxr4r8DXgReDNuc6WeUP5Kenobxfg6LyB/0lp47ob+AtgEs0/5P4FeBz4APB64It5+afl8duTEvY1wKuBbdaxk7Sa7FaSPuTfAEwnffAfV6hzLGt/uM8kfZB8NrdzNilRtJXsSB+ij5DO1P40x/i7OQbbD7J93AX8rNny5zqH53X8MdLZ8P8hfRDPLtR5gJTAjsx1/gp4oVTnlLxs783t+wYp8RXjcTvwY9JZ5s7AAcCBg7St6fppFLvchtuAPYHXAn8O/H0eNyZvDxfkbeLVwGa0t23eBszIMTgTeBIY18Y2/hrSvnNmjsG7ckwGS3Zbkg7sFhfavVUe95O8bvYH/oR0MPI88IZBYlqO2WGkfeca0ufIG0j7+/0UDmYazOdYBia79+dl2S0PHwG8k7Qv7wX8Fvh1of5nSYlual5fbyMfaAFbkQ6wf1tY7i3XsZ28nrTN7ZPL/5i0nY4vxph0gHc9KRHvQ9qnLiBt49u1sc7fnOu8P7dv+0Js/gD8EngL8CbSAcR/r/dn/fpOWJUHhQ9k4GLyGQxDS3bfKQxvn8u+Wygbl8sOLM37uNK8fwv8qLBDLSnvPMCV9fcrbFxNj/RzvdGkD5BPlMovBq5sFJtB5lV/z1aT3W2lOqcB1xWGj2XtD/drKZxx57ITaT/ZHQvML9URDc7CSnX2Bh4kHXjcTjrzOghQoc5i4Ful6U4CFhWGHwDmlur8Ajgvv/6jvE5ml+rML8VjFU2OzttdP41iB1wCnDXI/C4vj29z23x/YfwOuWz/Nubztbw+imfEBzJIsst1vkQ+ayiU7ZKne3ep/HfAnEHmVY7ZYXk+uxfK9qRwhtZkPseW1u1rSQnkIWDTJtPUk8P4PHxyjo+a1D+dJldlmm0npAP0s3P5CfXtlrWT3bQ8PLkwj81JZ5ZfaWOdD7gCU4jNagoHosBfkw4qtmh1+y8+fM9ubV8A9pb03iHO57b6i4h4jJc/KOtlT5COHl9Vmu660vBvSJetIB2xvRpYKen39QfpSLKvNN0N62jfLqQj8mtK5b8uvF+33FYafpi0AzQzmZT0i65dj/d9G/CWUuyeIh1olOO3RkT8hnR0+7+Bs3NbLwLmKtmatMM2iuVESaMLZbeW6hSXvb5O5pfqlLeJE4HTJV2dOzjs3qzt6+lU4GBJd0o6WdIBktb1OdHOtrkmBpHue73IyzFoZT6TgRsiotgbcn22h/q8YOC6u4b294P6GUzdw/l5sG0bYOe8nE+TkriA90XEC7Cm48hlkhZLeoqXl/V1+flM0llVv6TvS/pLSZu12fayHwIfkLQ9KZH/e4M6bwQej4iF9YKIeI6UrMuxG2ydD+bh/Pm5ZpgUn/LnZks2WZ+Jqioi7pX0A+CbpMtDRS/lZ5XKG3UAeaGFsqC9DkKvIF1Se1+DcU+Xhv/QxnyHalV+HkM6Ki8aS7qcV/R8abjdODTyUoOy8np5Bel+6Ccb1F3VoGyN/MH62/z419wj7UfAvsAtbbSzlWWPdbTlOEnnki4L7Qd8UdK3IuJLbbRjsPlfJum1pMt6U4H/AO6QNC2ad1BqZ9ssx6A+fbvzGWleKsWnvh7XtW0vJp0lvQQsi4g1HUHyeriUtK19Ffgf0oHV5aQDIyLiVkmTSJdz30E60ztO0pSIeHJ9FiTP807gPNLZ1aXrM5+CwdZ5O9O1GtOGfGY30D+T7gt8rFReP8J4Tb1A0qvoQBfkgnKX+z8n3cOAdON8Z+DJiOgvPR6mPf2kS2b7lsrfDtzZ5rxqpB11j2KhpE1Il1zubnN+ZQtJcSjauzS8HBglqXi0WD7juYl0xLmkQfweoz135edX5Q+UJTSO5f0R0eqHdD9p596rVD7gaxgRsSgiTo2Ig0kdCj7ecstbEBErIuK8iPgHUment/PyWdDzQLmnYKe2zVbmsxDYo9Rbsbw9NNKo3Qvyc3nd7Uv7+8H6eiEv36JiosveRrrf+JmI+E1E3EODM6JIPZgvjohPk+51/glpnUHj5W7FD0hJeE6Tg5wFwLaS6tsFkjYnXb5tJ3b1hNb13qc+syuJiMcknUDqfVQsf0bSb4DPS7qbFLvjSUmjU2bned9E6pm5F6knIqQbzf8I/FzS0cC9pA1/P+CuiPivVt8kIp6WdArpCPAx0uWXg0mdQd7VToMj4veSTgO+IekZ4EbSGd0/AtuRdpqh+FfgJ5JuIB1h7gN8pFTnBtIlyRMkfZ102fErpTr/Rurccomkr5GOqCeQzuB/HhHlS6UASPo16Qj3JtIBzy7A10kdba7K1b5BOuOrke5L7kdKQIe3upAR8Yd8VeFrkh4lrd9ZpA+ux3JbtiJddfgpqfPDWNIZ3sKGM13b5PL3p/J7rEXS8aSOAAtIBzEfIvUEfChXuR94h6Q/Jp0Rr6Jz22Yr8zmN1Cnjh5JOJB18Ht/CvO8HXi1pL9IB2tMRcZ+knwCnSvoH0mXEjwO7AX/TYpu7qUY6m/lcPpt/E6XtWtI/kS7v3Uo6+/0g6TJhfd3eT7ok+UZS56en8uXGdTmLdP+22VWPK0n73Y8lHZ7rfRnYgrSOWvU/pO1ruqQFwHP5Nk/nrc+Nvio9aNAJg7TCHmJg1/XXk+7F/IG0Ib6fxh1UPlya32pKnQpIl/f+Lr+emKf7COnD8lnSRvo3pWm2JW1I9a8LLCV1Kqn32JzKIJ0RSvMa9KsHzWLTZF6bke533pFj8zBpR9mtVO9q4PRS2VodB2jcQ+2I3M5nSJdwZpWXk3QGcleu8xvSZbjy+nsd6QO13s38QdJlukmDLNuRwH+Tzh6fzdvFf7D2jXmRvod3P+ly9SIaf/XgS6WytToP8PJXD54kJdNTge8AdxS2yx/n93k2t+kCYKdB2l/fJho9pjCws8WXSUfmvyd9gP2aQqcs0pnXNXl8sQPQem2blPaNdc0n15lG2taey23dj3V3UNk0x24Fa3/1YGs69NWDUp2GHS9KdY6ltK03qHM46cDsGdL9uhmluP8D6eDkybxObgRmFqbfhnSQuIrWvnrQ9LOjHGMGfvXg1zT+6sG61vmhpG16NaWvHpSm2yfPb+K6PpMaPZRnYrbByL+2cBXpQ758n7BSJF0JPBERf9nrtphtyHwZ02yEkPSnpHuN15HOlj9C6nRQ7ixlZm1ysjMbOYJ0z+gUUuexu0nd0H/Z01aZVYAvY5qZWeX5qwdmZlZ5G81lzFWrVvkU1sys4saMGVP+4Q/AZ3ZmZrYRcLIzM7PKc7JrU61W63UTRiTHZSDHpDHHZSDHpLFOxsXJzszMKs/JzszMKs/JzszMKs/JzszMKs/JzszMKs/JzszMKs/JzszMKm+j+bmwTnnbtaPh2qW9bgYrPzq+100wM9tg+MzOzMwqz8nOzMwqz8nOzMwqz8nOzMwqz8nOzMwqz8nOzMwqb1iSnaQ5kpZLurPBuM9JCknb5WFJOkVSv6TbJe1eqDtLUi0/ZhXK3yLpjjzNKZIa/lOtmZltnIbrzO4sYEa5UNJOwHTgoULxAUBffnwMOC3X3QY4BtgT2AM4RtK4PM1pwN8XphvwXmZmtvEalmQXEdcAKxqMOgn4PBCFspnAOZHMB8ZK2hHYH5gXESsi4glgHjAjj9s6IuZHRADnAAd1c3nMzGzD0rN7dpJmAksj4rbSqPHA4sLwklw2WPmSBuVmZmZAj34uTNJo4IukS5jDbmh/9T66Y+0Yik7+XX2njMQ29Zpj0pjjMpBj0lgrcenr61tnnV79NuYfA5OA23JfkgnA7yTtASwFdirUnZDLlgJTS+VX5/IJDeo31UpgmhoBv4sJQ1yGLqjVaiOuTb3mmDTmuAzkmDTWybj05DJmRNwREa+KiIkRMZF06XH3iHgEmAscmntlTgFWRcQy4DJguqRxuWPKdOCyPO5JSVNyL8xDgUt6sVxmZjYyDddXD84DrgN2lbRE0uxBql8KLAL6gX8HPgEQESuA44Ab8+OruYxc5/Q8zX3AL7qxHGZmtmEalsuYEfHBdYyfWHgdwOFN6s0B5jQovwnYbWitNDOzqvIvqJiZWeU52ZmZWeU52ZmZWeU52ZmZWeU52ZmZWeU52ZmZWeU52ZmZWeU52ZmZWeU52ZmZWeU52ZmZWeU52ZmZWeU52ZmZWeU52ZmZWeU52ZmZWeU52ZmZWeU52ZmZWeU52ZmZWeU52ZmZWeU52ZmZWeUNS7KTNEfSckl3Fsr+RdLdkm6XdLGksYVxR0nql3SPpP0L5TNyWb+kIwvlkyRdn8svkLTZcCyXmZltGIbrzO4sYEapbB6wW0T8GXAvcBSApMnAIcAb8zSnSholaRTwPeAAYDLwwVwX4JvASRGxC/AEMLu7i2NmZhuSYUl2EXENsKJU9quIWJ0H5wMT8uuZwPkR8VxE3A/0A3vkR39ELIqI54HzgZmSBOwHXJSnPxs4qKsLZGZmG5SRcs/ub4Ff5NfjgcWFcUtyWbPybYGVhcRZLzczMwNgk143QNLRwGrg3OF6z1qtNoSpR3esHUMxtGXojpHYpl5zTBpzXAZyTBprJS59fX3rrNPTZCfpMOBAYFpERC5eCuxUqDYhl9Gk/HFgrKRN8tldsX5DrQSmqWsHnfWwGdIydEGtVhtxbeo1x6Qxx2Ugx6SxTsalZ5cxJc0APg+8NyKeLoyaCxwiaXNJk4A+4AbgRqAv97zcjNSJZW5OklcBB+fpZwGXDNdymJnZyDdcXz04D7gO2FXSEkmzgX8DXgnMk3SrpO8DRMQC4EJgIfBL4PCIeDGftX0SuAy4C7gw1wX4AvBZSf2ke3hnDMdymZnZhmFYLmNGxAcbFDdNSBFxPHB8g/JLgUsblC8i9dY0MzMbYKT0xjQzM+saJzszM6s8JzszM6s8JzszM6s8JzszM6s8JzszM6s8JzszM6s8JzszM6s8JzszM6s8JzszM6s8JzszM6s8JzszM6s8JzszM6s8JzszM6s8JzszM6s8JzszM6s8JzszM6s8JzszM6s8JzszM6u8YUl2kuZIWi7pzkLZNpLmSarl53G5XJJOkdQv6XZJuxemmZXr1yTNKpS/RdIdeZpTJGk4lsvMzDYMw3VmdxYwo1R2JHBFRPQBV+RhgAOAvvz4GHAapOQIHAPsCewBHFNPkLnO3xemK7+XmZltxIYl2UXENcCKUvFM4Oz8+mzgoEL5OZHMB8ZK2hHYH5gXESsi4glgHjAjj9s6IuZHRADnFOZlZmbW03t2O0TEsvz6EWCH/Ho8sLhQb0kuG6x8SYNyMzMzADbpdQMAIiIkxXC9X61WG8LUozvWjqEY2jJ0x0hsU685Jo05LgM5Jo21Epe+vr511ullsntU0o4RsSxfilyey5cCOxXqTchlS4GppfKrc/mEBvWbaiUwTV076KyHzZCWoQtqtdqIa1OvOSaNOS4DOSaNdTIuvbyMOReo96icBVxSKD8098qcAqzKlzsvA6ZLGpc7pkwHLsvjnpQ0JffCPLQwLzMzs9aTnaQjJG23Pm8i6TzgOmBXSUskzQZOAN4lqQa8Mw8DXAosAvqBfwc+ARARK4DjgBvz46u5jFzn9DzNfcAv1qedZmZWTe1cxtwPOF7S1cCPgP+KiOdamTAiPthk1LQGdQM4vMl85gBzGpTfBOzWSlvMzGzj0/KZXUTMBF5HOmv6DPCIpNMl7dutxpmZmXVCW/fsIuLxiPheROwFvB14G3CVpAckHS1pq6600szMbAja7qAiaZqkM0k9IR8ldQj5CPBmfK/MzMxGoJbv2Uk6ETgEWEX6lZIvRcTSwvj5wBMdb6GZmdkQtdNBZQvgfRFxY6OREfGCpLd2pllmZmad006y+wbwdLEgf99ty4h4GCAi7u5g28zMzDqinXt2/8Xav1RCHr64c80xMzPrvHaS3a4RcUexIA+/obNNMjMz66x2kt1ySbsUC/Lw451tkpmZWWe1k+zmAD+VdKCkyZL+AriI9DNdZmZmI1Y7HVROAF4ATiT9K8FiUqL7dhfaZWZm1jEtJ7uIeAn4l/wwMzPbYLT1f3aSdgXeBKz1s2D5B5rNzMxGpHZ+QeWLwFeA21j7+3ZBg38iMDMzGynaObP7DLBHRNzercaYmZl1Qzu9MZ8B/AspZma2wWkn2X0Z+K6kHSW9ovjoVuPMzMw6oZ3LmGfl578rlIl0z25UpxpkZmbWae0ku0lda4WZmVkXtXwJMiIejIgHSV8mf74+nMvWm6R/lLRA0p2SzpO0haRJkq6X1C/pAkmb5bqb5+H+PH5iYT5H5fJ7JO0/lDaZmVm1tJzsJI2V9GPgWaA/l71X0tfW980ljQc+Dbw1InYjXQ49BPgmcFJE7EL6Q9jZeZLZwBO5/KRcD0mT83RvBGYAp0rypVUzMwPa66DyfdK/lL8OeD6XXQf89RDbsAmwpaRNgNHAMmA/0u9uApwNHJRfz8zD5PHTJCmXnx8Rz0XE/aRkvMcQ22VmZhXRzj27acBr8j+SB0BEPCbpVev75hGxVNKJwEOkrzb8CrgZWBkRq3O1JcD4/Ho86TIqEbFa0ipg21w+vzDr4jRmZraRayfZrQK2I515ASDptcXhduV/Op9J6vyyEvgJ6TJkV9VqtSFMPbpj7RiKoS1Dd4zENvWaY9KY4zKQY9JYK3Hp6+tbZ512kt3ppL/4ORp4haS9gK+TLm+ur3cC90fEYwCS/hPYGxgraZN8djcBWJrrLyX948KSfNlzDOn/9OrldcVpBmglME1d23S2w2pIy9AFtVptxLWp1xyTxhyXgRyTxjoZl3bu2X0TuAD4HrAp6fcwLwFOHsL7PwRMkTQ633ubBiwErgIOznVm5fcBmJuHyeOvjIjI5Yfk3pqTgD7ghiG0y8zMKqSdv/gJUmIbSnIrz/N6SRcBvwNWA7cAPwR+Dpyfe3reApyRJzkD+JGkfmAFqQcmEbFA0oWkRLkaODwiXuxUO83MbMPWzr8e7NdsXERcub4NiIhjgGNKxYto0JsyIp4FPtBkPscDx69vO8zMrLrauWd3Rml4e2AzUs/HnTvWIjMzsw5r5zLmWj8Xlr+0/SXgqU43yszMrJPW+x8L8j2x44HPd645ZmZmnTfUv+d5F/BSJxpiZmbWLe10UFlM+jufutHAFsAnOt0oMzOzTmqng8qHS8N/AO6NiCc72B4zM7OOa6eDyq+72RAzM7Nuaecy5o9Y+zJmQxFx6JBaZGZm1mHtdFBZSfqrnVGk79a9gvQjziuB+woPMzOzEaWde3avB94TEf9dL5C0D/DliPA/g5uZ2YjVzpndFNb+zziA64G9OtccMzOzzmsn2d0CfF3SlgD5+Xjg1m40zMzMrFPaSXaHkf5rbpWkR0l/5roPL//ljpmZ2YjUzlcPHgD+XNJOwGuAZRHxULcaZmZm1ilt/VyYpG2BqcDbI+IhSa+RNKErLTMzM+uQlpOdpLcD9wAfAr6ci/uA07rQLjMzs45p58zuO8BfR8QM0r+BQ+qNOeBPVs3MzEaSdpLdxIi4Ir+u/5LK87T3XT0zM7Nh106yWyip/OXxdwJ3dLA9ZmZmHdfOWdnngJ9J+jmwpaQfAH9B+skwMzOzEavlM7uImA/8GbAAmAPcD+wRETcOpQGSxkq6SNLdku6StJekbSTNk1TLz+NyXUk6RVK/pNsl7V6Yz6xcvybJ3/0zM7M1WjqzkzQKuALYPyK+1eE2nAz8MiIOlrQZ6U9hvwhcEREnSDoSOBL4AnAAqQdoH7AnqSfonpK2AY4B3kq6n3izpLkR8USH22pmZhugls7sIuJFYFKr9VslaQywL3BGfp/nI2Il6dLo2bna2aR/WyCXnxPJfGCspB2B/YF5EbEiJ7h5wIxOttXMzDZc7dyz+2fgNEnHkP7iZ81/20XES+v5/pOAx4AzJb0JuBk4AtghIpblOo8AO+TX44HFhemX5LJm5Q3VarX1bC6kE8/eG9oydMdIbFOvOSaNOS4DOSaNtRKXvr6+ddZpJ9mdnp8P5eVEp/x6VBvzKb//7sCnIuJ6SSeTLlmuEREhaZ1/GtuOVgLT1LVLO9eQIRjSMnRBrVYbcW3qNcekMcdlIMeksU7GZZ3JTtKrI+IR0llYpy0BlkTE9Xn4IlKye1TSjhGxLF+mXJ7HLwV2Kkw/IZctJf2MWbH86i6018zMNkCt3IO7FyAiHoyIB4GT6q8LZeslJ9HFknbNRdOAhcBcXv43hVnAJfn1XODQ3CtzCrAqX+68DJguaVzuuTk9l5mZmbV0GVOl4akdbsOngHNzT8xFwEdJSfhCSbOBB4G/ynUvBd4N9ANP57pExApJxwH1r0F8NSJWdLidZma2gWol2XX0ftmAmUfcSvrKQNm0BnUDOLzJfOaQvv9nZma2llaS3SaS3sHLZ3jlYSLiym40zszMrBNaSXbLWfuM6fHScAA7d7JRZmZmnbTOZBcRE4ehHWZmZl3T0V9EMTMzG4mc7MzMrPKc7MzMrPKc7MzMrPKc7MzMrPKc7MzMrPKc7MzMrPKc7MzMrPKc7MzMrPKc7MzMrPKc7MzMrPKc7MzMrPKc7MzMrPJa+YsfG4HGnrm0101YY+VHx/e6CWZmg/KZnZmZVZ6TnZmZVd6ISHaSRkm6RdLP8vAkSddL6pd0gaTNcvnmebg/j59YmMdRufweSfv3ZknMzGwkGhHJDjgCuKsw/E3gpIjYBXgCmJ3LZwNP5PKTcj0kTQYOAd4IzABOlTRqmNpuZmYjXM+TnaQJwHuA0/OwgP2Ai3KVs4GD8uuZeZg8flquPxM4PyKei4j7gX5gj+FZAjMzG+lGQm/M7wCfB16Zh7cFVkbE6jy8BKh39xsPLAaIiNWSVuX644H5hXkWpxmgVqsNobmjhzBtNdXjObS4VpNj0pjjMpBj0lgrcenr61tnnZ4mO0kHAssj4mZJU4frfVsJTFPXjpwu/yNFX18ftVptaHGtIMekMcdlIMeksU7GpddndnsD75X0bmALYGvgZGCspE3y2d0EoJ5hlgI7AUskbQKMAR4vlNcVpzEzs41cT+/ZRcRRETEhIiaSOphcGREfAq4CDs7VZgGX5Ndz8zB5/JUREbn8kNxbcxLQB9wwTIthZmYjXK/P7Jr5AnC+pK8BtwBn5PIzgB9J6gdWkBIkEbFA0oXAQmA1cHhEvDj8zTYzs5FoxCS7iLgauDq/XkSD3pQR8SzwgSbTHw8c370WmpnZhqrnXz0wMzPrNic7MzOrPCc7MzOrPCc7MzOrPCc7MzOrPCc7MzOrPCc7MzOrPCc7MzOrPCc7MzOrPCc7MzOrPCc7MzOrPCc7MzOrPCc7MzOrPCc7MzOrPCc7MzOrPCc7MzOrPCc7MzOrPCc7MzOrPCc7MzOrvJ4mO0k7SbpK0kJJCyQdkcu3kTRPUi0/j8vlknSKpH5Jt0vavTCvWbl+TdKsXi2TmZmNPL0+s1sNfC4iJgNTgMMlTQaOBK6IiD7gijwMcADQlx8fA06DlByBY4A9gT2AY+oJ0szMrKfJLiKWRcTv8uungLuA8cBM4Oxc7WzgoPx6JnBOJPOBsZJ2BPYH5kXEioh4ApgHzBjGRTEzsxFsk143oE7SRODNwPXADhGxLI96BNghvx4PLC5MtiSXNStvqFarDaGlo4cwbTXV4zm0uFaTY9KY4zKQY9JYK3Hp6+tbZ50RkewkbQX8FPhMRDwpac24iAhJ0cn3ayUwTV27tHMNqYi+vj5qtdrQ4lpBjkljjstAjkmKOKWfAAAFxklEQVRjnYxLr+/ZIWlTUqI7NyL+Mxc/mi9Pkp+X5/KlwE6FySfksmblZmZmPe+NKeAM4K6I+HZh1Fyg3qNyFnBJofzQ3CtzCrAqX+68DJguaVzumDI9l5mZmfX8MubewEeAOyTdmsu+CJwAXChpNvAg8Fd53KXAu4F+4GngowARsULSccCNud5XI2LF8CyCmZmNdD1NdhFxLaAmo6c1qB/A4U3mNQeY07nWmZlZVfT8np2ZmVm39foyplXA2DOXAqN73lN15UebftvEzDZyPrMzM7PKc7IzM7PKc7IzM7PKc7IzM7PKc7IzM7PKc7IzM7PKc7IzM7PKc7IzM7PKc7IzM7PKc7IzM7PKc7IzM7PK829jWmWk3+jsPf9Gp9nI4zM7MzOrPCc7MzOrPCc7MzOrPCc7MzOrPHdQMeuwlzvK9PYPbd1RxuxllTqzkzRD0j2S+iUd2ev2mJnZyFCZMztJo4DvAe8ClgA3SpobEQt72zKz3vBXMcxepojodRs6QtJewLERsX8ePgogIr4BsGrVqmosqJmZNTVmzBg1Kq/SZczxwOLC8JJcZmZmG7kqJTszM7OGKnPPDlgK7FQYnpDLgOantmZmVn1VOrO7EeiTNEnSZsAhwNwet8nMzEaAyiS7iFgNfBK4DLgLuDAiFnTyPTamrzZImiNpuaQ7C2XbSJonqZafx+VySTolx+V2SbsXppmV69ckzerFsnSKpJ0kXSVpoaQFko7I5Rt7XLaQdIOk23Jc/jmXT5J0fV7+C/JBKJI2z8P9efzEwryOyuX3SNq/N0vUOZJGSbpF0s/ysGMiPSDpDkm3Sropl3V/H4oIP1p4AKOA+4Cdgc2A24DJvW5XF5d3X2B34M5C2beAI/PrI4Fv5tfvBn4BCJgCXJ/LtwEW5edx+fW4Xi/bEGKyI7B7fv1K4F5gsuOCgK3y602B6/PyXggcksu/D3w8v/4E8P38+hDggvx6ct6vNgcm5f1tVK+Xb4ix+SzwY+BnedgxgQeA7UplXd+HKnNmNwz2APojYlFEPA+cD8zscZu6JiKuAVaUimcCZ+fXZwMHFcrPiWQ+MFbSjsD+wLyIWBERTwDzgBndb313RMSyiPhdfv0U6QrCeByXiIjf58FN8yOA/YCLcnk5LvV4XQRMk6Rcfn5EPBcR9wP9pP1ugyRpAvAe4PQ8LDbymAyi6/uQk13r/NUG2CEiluXXjwA75NfNYlPZmOXLTG8mncVs9HHJl+tuBZaTPnjuA1ZGur0Aay/jmuXP41cB21K9uHwH+DzwUh7eFscE0oHQryTdLOljuazr+1CVemPaMIqIkLRRflFf0lbAT4HPRMST6QA82VjjEhEvAv9L0ljgYuANPW5ST0k6EFgeETdLmtrr9oww+0TEUkmvAuZJurs4slv7kM/sWjfoVxs2Eo/mSwjk5+W5vFlsKhczSZuSEt25EfGfuXijj0tdRKwErgL2Il1yqh9QF5dxzfLn8WOAx6lWXPYG3ivpAdItj/2Ak9m4YwJARCzNz8tJB0Z7MAz7kJNd6/zVhrS89V5Ps4BLCuWH5p5TU4BV+ZLEZcB0SeNy76rpuWyDlO+hnAHcFRHfLoza2OOyfT6jQ9KWpN+nvYuU9A7O1cpxqcfrYODKSL0O5gKH5J6Jk4A+4IbhWYrOioijImJCREwkfVZcGREfYiOOCYCkP5L0yvpr0rZ/J8OxD/W6Z86G9CD1DLqXdD/i6F63p8vLeh6wDHiBdD18NukewhVADbgc2CbXFelHuO8D7gDeWpjP35JuqvcDH+31cg0xJvuQ7jfcDtyaH+92XPgz4JYclzuBr+TynUkfzP3AT4DNc/kWebg/j9+5MK+jc7zuAQ7o9bJ1KD5Tebk35kYdk7z8t+XHgvrn6HDsQ5X5IWgzM7NmfBnTzMwqz8nOzMwqz8nOzMwqz8nOzMwqz8nOzMwqz8nOzMwqz8nOzMwqz8nOzMwq7/8D5NVGWdkXKhEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "feature_matrix['TOTAL_PREVIOUS_MONTH(logs.num_unq, date)'].plot.hist()\n",
    "plt.title('Number of Unique Songs Listened to in Past Month');"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count    22022.000000\n",
       "mean       458.542639\n",
       "std        559.855146\n",
       "min          0.000000\n",
       "25%         68.250000\n",
       "50%        284.000000\n",
       "75%        632.000000\n",
       "max       4920.000000\n",
       "Name: TOTAL_PREVIOUS_MONTH(logs.num_unq, date), dtype: float64"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "feature_matrix['TOTAL_PREVIOUS_MONTH(logs.num_unq, date)'].describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbMAAAHBCAYAAADjB1VNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xu0XWV57/HvkwQSQBsSwUADFinbKtZRLwh4qYFQY/CopB02qBWitcO22mqlrdW0R46oHG05erS1XirYgFaNVmPwAkFiNnAGeGm9laDuFJQkAgES4gUCJHnOH2tus/ZO9ppz7ay1Z2bW9zPGGmu971x7rWc7tvnxzvnO943MRJKkJptWdwGSJO0vw0yS1HiGmSSp8QwzSVLjGWaSpMabUXcBvbZ9+3anZ0rSQW727NnR3nZkJklqPMNMktR4hpkkqfEMM0lS4xlmkqTGM8wkSY1nmEmSGs8wkyQ1nmEmSWo8w0yS1HhTFmYR8aOI+F5EfDsivln0zY2IayJipHieU/RHRLwvIjZExHcj4qltn7OseP9IRCybqvolSQeuqR6ZnZmZT87MU4r2m4BrM3MIuLZoA5wNDBWPVwMfgFb4ARcCpwGnAheOBqAkaXDVfZrxHGBF8XoFsKSt//JsuQk4MiKOBZ4HXJOZWzNzG3ANsHiqi5YkHVimctX8BNZERAIfyswPA/My847i+J3AvOL1fGBj289uKvom6t+nkZGRHpUuSarb0NDQhMemMsyenZmbI+LRwDUR8f32g5mZRdD1TKdfXJJ08Jiy04yZubl43gJ8jtY1r7uK04cUz1uKt28Gjm/78eOKvon6JUkDbErCLCKOiIhHjr4GFgH/BawGRmckLgM+X7xeDZxfzGo8HdhenI68GlgUEXOKiR+Lij5pIG3dupXly5ezbdu2ukuRajVVI7N5wA0R8R3g68AXM/Mq4J3AcyNiBPidog3wJeBWYAPwL8BrADJzK/A24BvF46KiTxpIK1eu5JZbbuFTn/pU3aVItYrMnl6mqt327dsPrl9ImsDWrVv5kz/5Ex566CEOPfRQPvShDzFnjneqaDDMnj072tt1T82XNEkrV65k9+7dAOzevdvRmQaaYSY11PDwMDt37gRg586dDA8P11yRVB/DTGqoBQsWENE60xIRLFiwoOaKpPoYZlJDLVq0iNFr3pnJ4sUuhqPBZZhJDbVmzZoxI7Orrrqq5oqk+hhmUkMNDw+PGZl5zUyDzDCTGmrBggVMnz4dgOnTp3vNTAPNMJMaaunSpWNGZueee27NFUn1McwkSY1nmEkNtXLlyjETQLxpWoPMMJMaanh4mF27dgGwa9cuJ4BooBlmUkMtWLCAGTNaWxLOmDHDCSAaaIaZ1FBLly5l2rTW/4WnTZvmBBANNMNMaqi5c+eycOFCIoKFCxe6Yr4G2oy6C5A0eUuXLuX22293VKaB535mkqTGcT8zSdJBxzCTJDWeYSY12NatW1m+fDnbtm2ruxSpVoaZ1GArV67klltucfUPDTzDTGqorVu3snbtWjKTtWvXOjrTQDPMpIZauXIlu3fvBmD37t2OzjTQDDOpoYaHh9m5cycAO3fudG1GDTTDTGooN+eU9jDMpIZyc05pD8NMktR4hpnUUG7OKe1hmEkN5eac0h6GmdRQbs4p7WGYSQ3l5pzSHoaZ1FBuzint4eacUoO5OafU4uackqTGcXNO6SDiFjBSi2EmNZhbwEgthpnUUG4BI+1hmEkNtXLlyjE3TTs60yAzzKSGcgUQaQ/DTGqo0047bUz79NNPr6kSqX6GmdRQo4sMT9SWBolhJjXUTTfdNKZ944031lSJVD/DTGood5qW9jDMpIZaunTpmDBzSSsNMsNMaqi5c+fytKc9DYBTTjnFhYY10AwzqcFuvfXWMc/SoDLMpIa69dZbueuuuwC48847+dGPflRvQVKNDDOpoS655JIx7X/4h3+oqRKpfoaZ1FA/+clPxrQ3b95cUyVS/QwzSVLjGWZSQ82aNatjWxokhpnUUDt27OjYlgaJYSY11BFHHNGxLQ0Sw0xqqIcffrhjWxokhpnUUPPmzevYlgaJYSY11N13392xLQ0Sw0xqqDPOOKNjWxokhpnUUIsWLRrTXrx4cU2VSPUzzKSGWrNmzZj2VVddVVMlUv0qhVlEHB8RL4iIlxfPx/e7MEmdrVu3rmNbGiQzJjoQEYcAf1w8TgQ2AD8DHgmcFBG3AR8EPpyZD01BrZLaHH300WzcuHFMWxpUE4YZ8B1gLa0w+1pm7ho9EBHTgVOBPwC+BTyxn0VK2puzGaU9OoXZGZm5ZV8HimC7EbgxIvzPQakGZ5xxxpjrZM5m1CCb8JrZvoIsIqZFxLHj3ud/Dko1cDajtEfVCSBHRsS/ATtoXTsjIl4UEW/vZ3GSJnbllVeOaa9evbqmSqT6VZ2a/0FgO/BrwOhkjxuBc/tRlKRy11133Zj28PBwTZVI9et0zazdWcCvZubDEZHQOr0YEY/uX2mSJFVTdWS2HTiqvSMiHgPc0fOKJFXiQsPSHlXD7CPAv0fEmcC0iHgGsILW6UdJNbjnnns6tqVBUvU047uAB4D3A4cAlwEfAt7bp7oklZgxYwYPPfTQmLY0qKr+9c/LzPcyLrwi4hjgzp5XJanU/fff37EtDZKqpxl/OEH/+l4VIknSZFUNs9irI+JXgN3dfFlETI+Ib0XEF4r2YyPiaxGxISI+FRGHFv0zi/aG4vgJbZ/x5qL/BxHxvG6+XzqYzJo1q2NbGiQdwywiNkbE7cBhEXF7+4PWTMZVXX7f64Fb2trvAt6TmScB24BXFf2vArYV/e8p3kdEnAy8hNZakIuBfy7WiZQGzo4dOzq2pUFSNjJ7OXA+rRulz2t7vBx4amb+UdUviojjgP9Ba2YkERHAQuAzxVtWAEuK1+cUbYrjZxXvPwf4ZGY+mJm30VqN5NSqNUiSDk4dJ4Bk5jBARByVmft7dfn/Am+ktYUMwKOA+zJzZ9HeBMwvXs8HNhY17IyI7cX75wM3tX1m+89IkgZUpdmMmXl/RDwZ+G1aN09H27G3lP18RLwA2JKZ/xERZ0yy1q6NjIxM1VdJU27mzJk8+OCDY9r+zetgNjQ0NOGxSmEWEa+mde1qDXA28GVgEfD5ijU8C3hRRDwfmAX8Cq1p/kdGxIxidHYcsLl4/2bgeGBTRMwAZgP3tvWPav+ZvXT6xaWDkX/zGlRVZzO+EVicmb8LPFA8vxh4uMoPZ+abM/O4zDyB1gSOtZn5B8BXi88BWMaecFxdtCmOr83MLPpfUsx2fCwwBHy94u8gHVTmzp07pv2oRz2qpkqk+lW9afrRmXl98Xp3REzLzC9HxMf38/v/BvhksZXMt4BLi/5LgSsiYgOwlVYAkpk3R8RKWve37QRe274DtjRI7rrrrjHtO+90/QINrqphtikiTsjMH9G6gfqciLiHPdvBVJaZ64B1xetb2cdsxMzcAfz+BD//DuAd3X6vdLBpTfCduC0NkqqnGf8eeELx+iLgY8Ba4K39KEpSudNOO21M+/TTT6+pEql+VWcz/mvb6y9HxBzg0Mz8eb8Kk9TZzJkzO7alQTLhyCwipk30oHW96v7itaQa3HTTTWPaN954Y02VSPXrFEY7ac1WLHtIqsGCBQuYPr21mtv06dNZsGBBzRVJ9ekUZo8FTiwefw4M01oP8QnF81eBP+t3gZL2benSpWPC7Nxzz625Iqk+E14zy8wfj76OiAuAUzLzvqLrhxHxTeCbwAf6W6KkfZk7dy4LFy7k6quvZuHChcyZM6fukqTaVL3mNRs4fFzf4UW/pJosWrSIww47jMWLF9ddilSrqmG2AvhKRLw6Is4ulre6mj0r20uqwZVXXsn999/P6tWr6y5FqlU3y1m9DzgXeDetFTn+qeiXVIOtW7cyPDwMwLp169i2bVvNFUn1qRRmmbk7Mz+YmWdl5hMyc2HRdikpqSZXXHEFu3e3NnvfvXs3V1xxRc0VSfXxPjGpoa677rox7dFRmjSIDDNJUuMZZlJDzZs3r2NbGiSGmdRQ9957b8e2NEgmHWYR8ZaIWNjLYiRVN34zzqOOOqqmSqT67c/I7EzgwxFxQ6+KkVSdm3NKe1TdnHMvmXkmQETM7105kqratWtXx7Y0SPb7mllmbu5FIZK6M23atI5taZBUGplFxPVAlr0vM5+z3xVJquToo48ec6rx0Y9+dI3VSPWqeprxq8Af0lqL8cfAY4BlwGXArf0pTVInd99995j2li1baqpEql/VMFsEPC8zbx7tiIh/Ay7LzNP7UpmkjiKiY1saJFVPsj8B+O9xfbcBj+9tOZKqeuITnzim/Zu/+Zs1VSLVr2qYDQP/GhFDEXFYRDwOuBS4vn+lSepkZGRkTPuHP/xhTZVI9asaZq8onm8Gfg78FxDAK/tQk6QKHnjggY5taZBUumaWmVuBl0TENOBo4O7M3N3XyiRJqqjyTdMRMRv4DeARRRuAzFzbl8okSaqo6n1mrwDeT+sU4/1thxI4sfdlSZJUXdWR2TuAF2fml/tZjCRJk1F1AsgMYE0/C5HUncMPP7xjWxokVcPsXcDfFRNAJB0ATj755DHt8fedSYOk6mnGNwDHAG+MiDE7AGbmY3pelaRS3/ve98a0v/vd79ZUiVS/qmH28r5WIalrM2bM4MEHHxzTlgZV1fvMhvtdiKTu/OIXv+jYlgZJ1an5F010LDPf0rtyJEnqXtXzEsePax8DLAA+19tyJEnqXtXTjHutwRgRi4GX9rwiSZK6tD9T7dcAS3pViKTuTJ8+vWNbGiRVr5mNX7LqcOBlwMaeVySpkmnTprFr164xbWlQVb1mtoHWOoyjW9neD3wLWNaPoiSVO+aYY9i4ceOYtjSoql4zK/1Pvog4LjM37X9JkqrYsmVLx7Y0SHp5XmJ9Dz9LUonxpxU9zahB1su//ih/i6RecadpaY9ehln28LMkSarM8xKSpMYzzKSGOvLII8e058yZU1MlUv28ZiY11H333TemvW3btpoqkerXyzA7ufwtkiT1XqUwi4jfioi1EbE1Ih4qHg9HxEOj78lMVwORJNWi6gognwD+HXgd4PxfSdIBpWqYHQO8JTOdfi9JOuBUvWa2gtbCwpIkHXCqjszeCdwYEcuBu9oPZObCnlclSVIXqobZZ4DbaO0s7TUzSdIBpWqYPRl4VGY+VPpOSZKmWNVrZtfjfWSSpANU1ZHZbcCaiPgce18ze0vPq5IkqQtVw+xw4IvAocDx/StHkqTuVd1p+pX9LkSSpMmqFGYRceJExzLz1t6VI6mqww47bMyGnIcddliN1Uj1qnqacQOtzTfbV8YfXQ1kek8rklTJjh07OralQVL1NOOYWY8RcQxwIa1ZjpJqMH51OVeb0yCb1BYwmXkn8BfA/+5tOZIkdW9/9jP7DVqzHCVJqlXVCSDXs+caGbRC7InARf0oSpKkblSdAPKRce1fAN/JzJEe1yNJUteqTgBZ0e9CJEmarKqnGQ8FXkFrweFHtB/LzPN7X5YkSdVVPc24Avgt4ErGrc0oSVLdqobZYuCxmXlfP4uRJGkyqk7Nvx2Y2c9CJEmarKojs8uBz0fEe9l7C5i1Pa9KkqQuVA2zPyueLx7Xn8CEixCPiohZwHW0RnczgM9k5oUR8Vjgk8CjgP8AzsvMhyJiJq0AfRpwL3BuZv6o+Kw3A68CdgGvy8yrK/4OkqSDVNWp+Y/dz+95EFiYmT+PiEOAGyLiy8AFwHsy85MR8UFaIfWB4nlbZp4UES8B3gWcGxEnAy+hdcP2rwJfiYjHZeau/axPktRg+7OcVWXZ8vOieUjxSGAh8JmifwWwpHh9TtGmOH5WRETR/8nMfDAzb6O1mv+pU/ArSJIOYPsVZhHxxS7eOz0ivg1sAa4B/hu4LzN3Fm/ZBMwvXs8HNgIUx7fTOhX5y/59/IwkaUBVvWY2kRuqvrE4FfjkiDgS+Bzw+P387lIjI662pcHi37wOZkNDQxMe268wy8yut4DJzPsi4qvAM4AjI2JGMfo6DthcvG0zcDywKSJmALNpTQQZ7R/V/jN76fSLSwcj/+Y1qCqdZoyIkyNiXvH6ERHx1oi4MCIqbQETEUcXIzIi4jDgucAtwFeBFxdvWwZ8vni9umhTHF+brZ0HVwMviYiZxUzIIeDrVWqQJB28qo7MPgEspXWP2SW09jLbAXwIOK/Czx8LrIiI6bQCdGVmfiEi1gOfjIi3A98CLi3efylwRURsALbSmsFIZt4cESuB9cBO4LXOZJQkRZWt1iNie2bOLmYU3gWcDDwA3JaZj+5zjV3Zvn27e8drICxZsmSvvlWrVtVQiTT1Zs+eHe3tqiOzHRHxSFohdntm3lNcy5rV6wIlSepW1TD7N2At8Ejgn4q+pwK39aMoSZK6UXUFkDdExCLg4cz8atG9G3hD3yqTJKmiylPzM3PNuPY3e1+OJEndq7rT9PW0lp8a70Faq3B8NjOv7GVhkiRVVXU5q3XACcAw8LHi+deAb9Ka3XhZRLyxD/VJklSq6mnGRcDzMvOW0Y6I+DiwIjNPi4jP0roX7e/7UKMkSR1VHZk9Hrh1XN+Pad08TWZ+HZjXw7okSaqsaphdB3w0Ik6KiFkRcRLwLxQLDUfEk4A7+lSjJEkdVQ2zZcV71wP3F8/TgVcUxx8CXtrr4iRJqqLqfWZbaS3wOw04Grg7M3e3Hf9Bn+qTJKlU5fvMImKI1uhrPrA5Ij6RmW6eJEmqXdUtYF4I/AetiSBbaU38+GZEvKiPtUmSVEnVkdnFwDltS1kREWfQWqdxdR/qkiSpsqoTQI4Drh/Xd0PRL0lSraqG2beBvxzXd0HRL0lSraqeZvxT4MqIeD2wETie1hT9F/arMEmSqqo6Nf/7EfEE4BnAscBPgK9l5sP9LE6SpCq62QJmJ3tfN5MkqXYThllEbGTf276MkZmP6WlFkiR1qdPI7OVTVoUkSfthwjDLzOGpLESSpMmacGp+RLwuImZ2+uGImBkRr+t9WZIkVdfpNOMxwIaI+BKtnaV/APwMeCTwOOAM4Gzg8j7XKElSR51OMy6PiHfT2ublVcCTgCOBbcB3gS8ByzPz3imoU5KkCXWcmp+Z9wCXFA9Jkg5IVZezkiTpgNVxZBYR11Nyr1lmPqenFUmS1KWyFUA+0vY6gPcDr+lfOZIkda/smtmK9nZEvHt8nyRJdfOamSSp8QwzSVLjlU0AWTj+/RFxJq3rZwBk5tp+FCZJUlVlE0AuHde+F7isrZ3AiT2tSJKkLpVNAHnsVBUiSdJkdbxmFhE/napCJEmarLIJIFFyXJKk2pWFWelO05Ik1a1sAsgREXF7pzdk5mN6WI8kSV0rC7MHgfOmohBJkiarLMx2ZubwlFQiSdIkOQFEktR4ZWH2x1NShSRJ+6HsNONJEfGWDsczM9/Wy4IkSepWWZgNdTh2NjAHMMwkSbUqW85qr5mMEfECWgF2N/CnfapLkqTKykZmv1SsoP92YB7wVuBjmbm7X4VJklRVaZhFxOnAxcDjgHcAH8nMh/tdmCRJVZXtZ/YF4DTg74EXAg8U/b+cBenoTJJUt7Kp+c8HHgW8C/gp8HDbY2fxLElSrcpOM7qfmSTpgFc2m/HHo68jYiYwF9iWmTv6XZgkSVWVnWYkIk6LiHXAz4FNwM8i4rpiYogkSbUr22n6GcBXgPXAc4GTi+ebgWuK45Ik1arsmtnbgeWZ+Y9tfT8A1kXEelpT9Rf2qzhJkqooO814CnDZBMc+WhyXJKlWZWGWwCETHDukOC5JUq3KwmwYuGCCYxcA1/W2HEmSuld2zezNwA0R8VTg08AdwLHA7wPPBJ7d3/IkSSrXcWSWmeuBpwPbgHcCXyyetwGnFsclSapV6ULDmfnfwF5bwUiSdKAoW2j4OWUfkJleN5Mk1apsZLYO2AI8BMQ+jifwmB7XJElSV8rC7PPA6cCVwOWZ+bX+lyRJUnfKJoD8LvBEWstZvS8ivh8RfxsRx09JdZIkVVC60HBmbs3M92fmacA5wDHArRHxzL5XJ0lSBaWzGQEiIoBFwDJaazF+DLi1j3VJklRZ2WzGJwHnA+fSOtV4OfCqzHxgCmqTJKmSspHZd2itkv9h4CfALOClrYFaS2ZOtBCxJElToizMrqM1/f6sCY4nE6+qL0nSlOgYZpl5RtUPiohnZeb/2++KJEnqUulsxi58uYefJUlSZb0Ms32tENI6EHF8RHw1ItZHxM0R8fqif25EXBMRI8XznKI/IuJ9EbEhIr5brNo/+lnLivePRMSyHtYvSWqoXoZZp406dwJ/mZkn01pR5LURcTLwJuDazBwCri3aAGcDQ8Xj1cAHoBV+wIXAacCpwIWjAShJGly9DLMJZeYdmfmfxeufAbcA82ndhL2ieNsKYEnx+hxay2dlZt4EHBkRxwLPA64pbuTeBlwDLJ6K30GSdOCqdNN0L0XECcBTgK8B8zLzjuLQncC84vV8YGPbj20q+ibq36eRkZGe1Cw1hX/zOpgNDQ1NeKyXYTbhNbNfviHiEcC/A3+RmT8dd79aRkSnU5Vd6/SLSwcj/+Y1qLo+zRgRz42IN4xfmzEzH1nyc4fQCrKPZ+Zni+67itOHFM9biv7NQPtixscVfRP1S5IGWMcwi4hPRMQftbXfCHwBeBlwTURU2oG6WNvxUuCWzHx326HVtNZ7pHj+fFv/+cWsxtOB7cXpyKuBRRExp5j4sajokyQNsLKR2bNoBQsRMQ34a+Blmfl04MXAX1X8nmcB5wELI+LbxeP5wDuB50bECPA7RRvgS7QWMt4A/AvwGmit4A+8DfhG8bio6JMkDbCya2ZHZuboqb+n0FqbcVXRvgr4RJUvycwbmPia2l5LZWVmAq+d4LMuwyW0JEltykZm9xSzDwHOBG7MzF1F+whg175+SJKkqVQ2MvsI8MWIuJrWVjB/3nbsObTuF5MkqVZlCw1fHBGbgVOA12dm+2nFo4H/08/iJEmqovQ+s8xcwZ5VOsb3S5JUu7Kdpq9g7zUXHwZ+DHw6Mz3NKEmqXdnIbMM++g4BfgO4KSJelplf7H1ZkiRVV3bN7K0THYuI0fvCDDNJUq32Z9X8a4Ff71UhkiRN1v6E2a8C9/WqEEmSJqtsAsiJ++g+BDgB+DtgZR9qkiSpK1UmgCRjl6LaBdwOfAq4qE91SZJUWdkEkCnZiVqSpP1RaXPOYlPNZwBHAXcDX8vMn/WzMEmSqioNs4j4C1rbrswC7qEVaDsi4sJxe5NJklSLss05XwG8CXgVMCszj6UVan8EvDEiXtn3CiVJKlE2MnsDsCwzf7mbc7EFzKci4j7gEuCjfaxPkqRSZRM8fh34ygTHrgX2NXVfkqQpVRZmPwPmT3BsfnFckqRalYXZKuCfI2JWe2dEHAa8H/hcvwqTJKmqsmtmb6J1OvFHEfFl4A7gWOBsYBPw8v6WJ0lSuY4js8zcTuv+sr+lNYvx6cXz3wLPzEzXZpQk1a7KTtMPA5cWD0mSDjhlCw3/YdkHZOZlvStHkqTulY3Mzis5noBhJkmqVdlCw2dOVSGSJE1W2XJWX4+I10TEnKkqSJKkbpXdZ/Zx4JXAHRHx2Yh4UURUWmlfkqSpUjY1/72Z+XTgqcD3gX8EfhIR74uIp01FgZIklam0+WZmrs/M5cAJwEuARwDXRsT3+libJEmVdLWTdGYm8HPgAWAncHg/ipIkqRuVwiwijo+I5RHxfWANMBP4vcz89b5WJ0lSBWU3Tb8COB94NrCO1o7Tn83MB/pemSRJFZXNTPwbYAVwXmZunoJ6JEnqWtlN00/odDwijsrMe3pbkiRJ3YnWnI4JDkZszcy5be1rM/OstvZPM/NX+lxjV7Zv3z7xL6TGWLJkSd0l6CC0atWquktQj8yePTva22UTQA4Z137KuHYgSVLNysKsbJTjKEiSVLuu7jOTJOlAVDabcVZEXN7WPmJce2YfapK8tlHB+OuKJ510EpdccklN1Uj1Kguzd4xrX1zSllQTg0yDrGxq/lunqhBJkiZr0tfMIuKFEfH1XhYjSdJklG3OeUxEXBER3yv2MzsuIk6PiP8EPgh8ZmrKlCRpYmXXzN5Pa7uXfwZ+D1gNzKF1LW1FZj7c3/IkSSpXFma/DZyUmT+NiJXA3cCTMvPm/pcmSVI1ZdfMZmXmTwEy815gu0EmSTrQlI3MDomIV7Jn2apDIuIP29+QmZf1pTJJkioqC7Ov0drPbNQ3gPPa2gkYZpKkWpXdZ3bGFNUhSdKklU3NPzwiLo6I1RHxvyLC5askSQecsgkg7wdeCHwfeDHgejmSpANOWZgtBhZl5huBs4EX9L8kSZK6UxZmR2TmHQCZuRGY3f+SJEnqTtlsxhkRcSZ7puaPb5OZa/tVnCRJVZSF2RbGTr2/d1w7gRN7XZQkSd0om5p/QtUPiojjMnPTflckSVKXJr0FzD6s7+FnSZJUWS/DLMrfIklS7/UyzLKHnyVJUmW9DDNJkmphmEmSGs9rZpKkxpt0mEXEoRFxe1vXyT2oR5Kkru3PyCyA40YbxXJXkiRNuf09zegMRklS7ZwAIklqvI7LWUXEFUw8+pre+3IkSepe2ULDG0qOX9SrQiRJmqyyhYbf2ul4RBzZ23IkSepe19fMImJ6RLwwIj4N3NGHmiRJ6krlMIuIp0TEe4CfAKuAB4Hn9KswSZKq6hhmETEvIv4yIr4HfB14PPDXwFbggsz8RpUviYjLImJLRPxXW9/ciLgmIkaK5zlFf0TE+yJiQ0R8NyKe2vYzy4r3j0TEskn8vpKkg1DZyGwT8Cbgw8D8zDw7My8Hdnb5Pf8KLB7X9ybg2swcAq4t2gBnA0PF49XAB6AVfsCFwGnAqcCFowEoSRpsZWH2cWAm8FfAGyLiSZP5ksy8jtZort05wIri9QpgSVv/5dlyE3BkRBwLPA+4JjO3ZuY24Br2DkhJ0gDqGGaZ+QrgGOB/Ak8Hvl2ccvwV4Kj9/O55mTk6geROYF7xej7QvjTWpqJvon5J0oAru8+MzLwfuBy4PCIeA5xXPL4dEasyc+n+FpGZGRE9XxprZGSk1x8pHbD8e9fBbmhoaMJjZSuAHJeZm0bbmXmTTlRlAAAEyElEQVQ78A7gHRHxDOD8/ajrrog4NjPvKE4jbin6NwPHt73vuKJvM3DGuP51nb6g0y8uHWz8e9cgK7tmtn6iA5l5Y2b+6X5892pgdEbiMuDzbf3nF7MaTwe2F6cjrwYWRcScYuLHoqJPkjTgyk4z9mTDzYj4BK1R1VERsYnWrMR3Aisj4lXAj4HR05VfAp5Paymt+4FXAmTm1oh4GzB6O8BFmTl+UokkaQCVhVlGRNAh1DJzd9mXZOZLJzh01j7em8BrJ/icy4DLyr5PkjRYysLsEUx8T1nQWlHf1fMlSbUqC7P7gSdORSGSJE1WWZjtzswfT0klkiRNUtlsxp5MAJEkqZ/Kwuzs0RcRMTMijo2IWX2uSZKkrpQtZ3VDRJwWEeuAn9NaQupnEXFdcQ+YJEm1K9sC5hnAV2jdPP1c4OTi+WbgmuK4JEm1KpsA8nZgeWb+Y1vfD4B1EbGe1tJWC/tVnCRJVZRdMzuFiW9S/mhxXJKkWpWFWQKHTHDskOK4JEm1KguzYeCCCY5dAFzX23IkSepe2TWzNwM3RMRTgU8DdwDHAr8PPBN4dn/LkySpXNnU/PW0dpjeRmuV+y8Wz9uAU4vjkiTVqmxzzpdm5ido7SwtSdIBqeya2YempApJkvaDazNKkhqvbALI9Ig4k86bc67tbUmSJHWnLMxmApcycZglcGJPK5IkqUtlYfaLzDSsJEkHtLJrZpIkHfCcACJJaryym6YfOVWFSJI0WZF5cK0VvH379gPqF1qyZEndJUjSpK1ataruEvZp9uzZY84ces1MktR4hpkkqfEMM0lS45XdZ6YeG1m0vO4SJGlCQ2surruESXFkJklqPMNMktR4hpkkqfEMM0lS4xlmkqTGM8wkSY1nmEmSGs8wkyQ1nmEmSWo8w0yS1HiGmSSp8QwzSVLjGWaSpMYzzCRJjWeYSZIazzCTJDWeYSZJajzDTJLUeIaZJKnxDDNJUuMZZpKkxjPMJEmNZ5hJkhpvRt0FDJqhNRfXXYIkHXQcmUmSGs8wkyQ1nmEmSWo8r5lNsZFFy+suQZIm1NTr+o7MJEmNZ5hJkhrPMJMkNZ5hJklqPMNMktR4hpkkqfEMM0lS4xlmkqTGM8wkSY1nmEmSGs8wkyQ1nmEmSWo8w0yS1Hiumj/FmroitSQdyByZSZIazzCTJDVeI8MsIhZHxA8iYkNEvKnueiRJ9YrMrLuGrkTEdOCHwHOBTcA3gJdm5nqA7du3N+sXkiZpyZIlY9qrVq2qqRJp6s2ePTva202cAHIqsCEzbwWIiE8C5wDra61KPTX+H2qV83+zcgb+wauJpxnnAxvb2puKPknSgGriyKyykZGRukuQdADx34RmGxoamvBYE8NsM3B8W/u4om8vnX5xSYPHfxMOXk2cADKD1gSQs2iF2DeAl2XmzeAEEEkaBI2fAJKZOyPiz4CrgenAZaNBJkkaTI0bmZVxZCZJB7/xI7MmzmaUJGkMw0yS1HiGmSSp8QwzSVLjGWaSpMYzzCRJjWeYSZIazzCTJDWeYSZJajzDTJLUeIaZJKnxDDNJUuMZZpKkxjPMJEmNZ5hJkhrPMJMkNZ5hJklqvINup2lJ0uBxZCZJajzDTJLUeIaZJKnxDDNJUuP9fz2QybkC90dIAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize = (6, 8))\n",
    "sns.boxplot(y = feature_matrix['TOTAL_PREVIOUS_MONTH(logs.num_unq, date)']);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Parallelizing Feature Engineering\n",
    "\n",
    "In order to run the feature engineering in parallel, we need to write a function that can handle one partition at a time. \n",
    "\n",
    "Now we'll write a function that takes in the partition number, the feature definitions, and a specific cutoff time file name, reads in the data from S3, calculates the feature matrix, and saves the feature matrix back to S3. Since all of the partitions are independent - the features for one partition do not depend on data in any other partition - we can later use this function to parallelize calculating all of the feature matrices."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "There are 255 features.\n"
     ]
    }
   ],
   "source": [
    "feature_defs = ft.load_features('/data/churn/features.txt')\n",
    "print(f'There are {len(feature_defs)} features.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data Access\n",
    "\n",
    "All of the file reading and writing occurs from S3. This means we can use any Amazon EC2 instance to carry out these calculations, including an ephemeral cluster. Once we shut down the machines, the data is still safely stored in the cloud. We do have to worry about overwriting the data in S3, but if we turn on versioning, then we can go back to any previously calculated version of the features. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [],
   "source": [
    "import s3fs\n",
    "\n",
    "# Credentials\n",
    "with open('/data/credentials.txt', 'r') as f:\n",
    "    info = f.read().strip().split(',')\n",
    "    key = info[0]\n",
    "    secret = info[1]\n",
    "\n",
    "fs = s3fs.S3FileSystem(key=key, secret=secret)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The function below takes in a single partition number, retrieves the data from S3, calculates the feature matrix using `calculate_feature_matrix` with a pre-defined set of features, and saves the feature matrix back to S3. This is a refactoring of the above separate steps."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%capture\n",
    "def partition_to_feature_matrix(partition, feature_defs = feature_defs, \n",
    "                                cutoff_time_name = 'MS-31_labels.csv', write = True):\n",
    "    \"\"\"Take in a partition number, create a feature matrix, and save to Amazon S3\n",
    "    \n",
    "    Params\n",
    "    --------\n",
    "        partition (int): number of partition\n",
    "        feature_defs (list of ft features): features to make for the partition\n",
    "        cutoff_time_name (str): name of cutoff time file\n",
    "        write: (boolean): whether to write the data to S3. Defaults to True\n",
    "        \n",
    "    Return\n",
    "    --------\n",
    "        None: saves the feature matrix to Amazon S3\n",
    "    \n",
    "    \"\"\"\n",
    "    \n",
    "    partition_dir = BASE_DIR + 'p' + str(partition)\n",
    "    \n",
    "    # Read in the data files\n",
    "    members = pd.read_csv(f'{partition_dir}/members.csv', \n",
    "                      parse_dates=['registration_init_time'], \n",
    "                      infer_datetime_format = True, \n",
    "                      dtype = {'gender': 'category'})\n",
    "\n",
    "    trans = pd.read_csv(f'{partition_dir}/transactions.csv',\n",
    "                       parse_dates=['transaction_date', 'membership_expire_date'], \n",
    "                        infer_datetime_format = True)\n",
    "    logs = pd.read_csv(f'{partition_dir}/logs.csv', parse_dates = ['date'])\n",
    "    \n",
    "    # Make sure to drop duplicates\n",
    "    cutoff_times = pd.read_csv(f'{partition_dir}/{cutoff_time_name}', parse_dates = ['cutoff_time'])\n",
    "    cutoff_times = cutoff_times.drop_duplicates(subset = ['msno', 'cutoff_time'])\n",
    "    \n",
    "    # Needed for saving\n",
    "    cutoff_spec = cutoff_time_name.split('_')[0]\n",
    "    \n",
    "    # Create empty entityset\n",
    "    es = ft.EntitySet(id = 'customers')\n",
    "\n",
    "    # Add the members parent table\n",
    "    es.entity_from_dataframe(entity_id='members', dataframe=members,\n",
    "                             index = 'msno', time_index = 'registration_init_time', \n",
    "                             variable_types = {'city': vtypes.Categorical,\n",
    "                                               'registered_via': vtypes.Categorical})\n",
    "    # Create new features in transactions\n",
    "    trans['price_difference'] = trans['plan_list_price'] - trans['actual_amount_paid']\n",
    "    trans['planned_daily_price'] = trans['plan_list_price'] / trans['payment_plan_days']\n",
    "    trans['daily_price'] = trans['actual_amount_paid'] / trans['payment_plan_days']\n",
    "\n",
    "    # Add the transactions child table\n",
    "    es.entity_from_dataframe(entity_id='transactions', dataframe=trans,\n",
    "                             index = 'transactions_index', make_index = True,\n",
    "                             time_index = 'transaction_date', \n",
    "                             variable_types = {'payment_method_id': vtypes.Categorical, \n",
    "                                               'is_auto_renew': vtypes.Boolean, 'is_cancel': vtypes.Boolean})\n",
    "\n",
    "    # Add transactions interesting values\n",
    "    es['transactions']['is_cancel'].interesting_values = [0, 1]\n",
    "    es['transactions']['is_auto_renew'].interesting_values = [0, 1]\n",
    "    \n",
    "    # Create new features in logs\n",
    "    logs['total'] = logs[['num_25', 'num_50', 'num_75', 'num_985', 'num_100']].sum(axis = 1)\n",
    "    logs['percent_100'] = logs['num_100'] / logs['total']\n",
    "    logs['percent_unique'] = logs['num_unq'] / logs['total']\n",
    "    logs['seconds_per_song'] = logs['total_secs'] / logs['total'] \n",
    "    \n",
    "    # Add the logs child table\n",
    "    es.entity_from_dataframe(entity_id='logs', dataframe=logs,\n",
    "                         index = 'logs_index', make_index = True,\n",
    "                         time_index = 'date')\n",
    "\n",
    "    # Add the relationships\n",
    "    r_member_transactions = ft.Relationship(es['members']['msno'], es['transactions']['msno'])\n",
    "    r_member_logs = ft.Relationship(es['members']['msno'], es['logs']['msno'])\n",
    "    es.add_relationships([r_member_transactions, r_member_logs])\n",
    "    \n",
    "    # Calculate the feature matrix using pre-calculated features\n",
    "    feature_matrix = ft.calculate_feature_matrix(entityset=es, features=feature_defs, \n",
    "                                                 cutoff_time=cutoff_times, cutoff_time_in_index = True,\n",
    "                                                 chunk_size = 1000)\n",
    "\n",
    "    if write:\n",
    "        # Save to Amazon S3\n",
    "        bytes_to_write = feature_matrix.to_csv(None).encode()\n",
    "\n",
    "        with fs.open(f'{partition_dir}/{cutoff_spec}_feature_matrix.csv', 'wb') as f:\n",
    "            f.write(bytes_to_write)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "950 seconds elapsed.\n"
     ]
    }
   ],
   "source": [
    "from timeit import default_timer as timer\n",
    "\n",
    "start = timer()\n",
    "partition_to_feature_matrix(800, feature_defs, cutoff_time_name = 'MS-31_labels.csv', write = False)\n",
    "end = timer()\n",
    "print(f'{round(end - start)} seconds elapsed.')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>msno</th>\n",
       "      <th>time</th>\n",
       "      <th>city</th>\n",
       "      <th>bd</th>\n",
       "      <th>registered_via</th>\n",
       "      <th>gender</th>\n",
       "      <th>SUM(logs.num_25)</th>\n",
       "      <th>SUM(logs.num_50)</th>\n",
       "      <th>SUM(logs.num_75)</th>\n",
       "      <th>SUM(logs.num_985)</th>\n",
       "      <th>...</th>\n",
       "      <th>WEEKEND(LAST(transactions.membership_expire_date))</th>\n",
       "      <th>DAY(LAST(logs.date))</th>\n",
       "      <th>DAY(LAST(transactions.transaction_date))</th>\n",
       "      <th>DAY(LAST(transactions.membership_expire_date))</th>\n",
       "      <th>MONTH(LAST(logs.date))</th>\n",
       "      <th>MONTH(LAST(transactions.transaction_date))</th>\n",
       "      <th>MONTH(LAST(transactions.membership_expire_date))</th>\n",
       "      <th>label</th>\n",
       "      <th>days_to_churn</th>\n",
       "      <th>churn_date</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>++XKXYly5nP1VAHkIY54gLEceN1Cgws3dKreSAY1KrM=</td>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>9.0</td>\n",
       "      <td>29.0</td>\n",
       "      <td>9.0</td>\n",
       "      <td>male</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>449.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>+2QAkq/St/jSrjbs+BT2wsfowpmZgAmCbKixGiWUqxE=</td>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>426.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>+CTxP5RzNS7ei80eNdPTf/BuVVI5MIuTNhdRNq+6Dqc=</td>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>22.0</td>\n",
       "      <td>56.0</td>\n",
       "      <td>9.0</td>\n",
       "      <td>female</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>+GgwZWpoBR5Hs+5LDJktlBwb6SvvSZWMl1hPrhv4DP8=</td>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>15.0</td>\n",
       "      <td>23.0</td>\n",
       "      <td>9.0</td>\n",
       "      <td>female</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>222.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>+QcKo0T43zDktjv3MNH+/k7+RMr2m2ZtCZJqING+w/c=</td>\n",
       "      <td>2015-01-01</td>\n",
       "      <td>5.0</td>\n",
       "      <td>41.0</td>\n",
       "      <td>9.0</td>\n",
       "      <td>male</td>\n",
       "      <td>17.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 253 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                           msno        time  city    bd  \\\n",
       "0  ++XKXYly5nP1VAHkIY54gLEceN1Cgws3dKreSAY1KrM=  2015-01-01   9.0  29.0   \n",
       "1  +2QAkq/St/jSrjbs+BT2wsfowpmZgAmCbKixGiWUqxE=  2015-01-01   1.0   0.0   \n",
       "2  +CTxP5RzNS7ei80eNdPTf/BuVVI5MIuTNhdRNq+6Dqc=  2015-01-01  22.0  56.0   \n",
       "3  +GgwZWpoBR5Hs+5LDJktlBwb6SvvSZWMl1hPrhv4DP8=  2015-01-01  15.0  23.0   \n",
       "4  +QcKo0T43zDktjv3MNH+/k7+RMr2m2ZtCZJqING+w/c=  2015-01-01   5.0  41.0   \n",
       "\n",
       "   registered_via  gender  SUM(logs.num_25)  SUM(logs.num_50)  \\\n",
       "0             9.0    male               0.0               0.0   \n",
       "1             7.0     NaN               0.0               0.0   \n",
       "2             9.0  female               0.0               0.0   \n",
       "3             9.0  female               0.0               0.0   \n",
       "4             9.0    male              17.0               7.0   \n",
       "\n",
       "   SUM(logs.num_75)  SUM(logs.num_985)     ...      \\\n",
       "0               0.0                0.0     ...       \n",
       "1               0.0                0.0     ...       \n",
       "2               0.0                0.0     ...       \n",
       "3               0.0                0.0     ...       \n",
       "4               5.0                3.0     ...       \n",
       "\n",
       "   WEEKEND(LAST(transactions.membership_expire_date))  DAY(LAST(logs.date))  \\\n",
       "0                                                 0.0                   NaN   \n",
       "1                                                 0.0                   NaN   \n",
       "2                                                 0.0                   NaN   \n",
       "3                                                 0.0                   NaN   \n",
       "4                                                 0.0                   1.0   \n",
       "\n",
       "   DAY(LAST(transactions.transaction_date))  \\\n",
       "0                                       NaN   \n",
       "1                                       NaN   \n",
       "2                                       NaN   \n",
       "3                                       NaN   \n",
       "4                                       NaN   \n",
       "\n",
       "   DAY(LAST(transactions.membership_expire_date))  MONTH(LAST(logs.date))  \\\n",
       "0                                             NaN                     NaN   \n",
       "1                                             NaN                     NaN   \n",
       "2                                             NaN                     NaN   \n",
       "3                                             NaN                     NaN   \n",
       "4                                             NaN                     1.0   \n",
       "\n",
       "   MONTH(LAST(transactions.transaction_date))  \\\n",
       "0                                         NaN   \n",
       "1                                         NaN   \n",
       "2                                         NaN   \n",
       "3                                         NaN   \n",
       "4                                         NaN   \n",
       "\n",
       "   MONTH(LAST(transactions.membership_expire_date))  label  days_to_churn  \\\n",
       "0                                               NaN    0.0          449.0   \n",
       "1                                               NaN    0.0          426.0   \n",
       "2                                               NaN    0.0            NaN   \n",
       "3                                               NaN    0.0          222.0   \n",
       "4                                               NaN    0.0            NaN   \n",
       "\n",
       "   churn_date  \n",
       "0           0  \n",
       "1           0  \n",
       "2         NaN  \n",
       "3           0  \n",
       "4         NaN  \n",
       "\n",
       "[5 rows x 253 columns]"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "feature_matrix = pd.read_csv('s3://customer-churn-spark/p800/MS-31_feature_matrix.csv', low_memory = False)\n",
    "feature_matrix.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that the function works for a single feature matrix (I'd already written all of the feature matrices so I didn't want to rewrite the older data). Later we'll implement a parallel calculation in Spark, but for now, we can run several operations in parallel using multiple processes (cores) on our machine with the built-in `multiprocessing` library."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3668 seconds elapsed.\n"
     ]
    }
   ],
   "source": [
    "from multiprocessing import Pool\n",
    "\n",
    "# Multiprocessing implementation of making 14 feature matrices\n",
    "start = timer()\n",
    "pool = Pool(7)\n",
    "r = pool.map(partition_to_feature_matrix, range(0, 1000, 1000 // 14))\n",
    "pool.close()\n",
    "pool.join()\n",
    "\n",
    "end = timer()\n",
    "print(f'{round(end - start)} seconds elapsed.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Given the time to calculate just one feature matrix, about 15 minutes, calculating all 1000 would take several days if done sequentially. Fortunately, because we partitioned the data into independent subsets, we can calculate the feature matrices in parallel using a distributed framework such as Dask or Spark.\n",
    "\n",
    "(A tutorial on how to distribute the feature engineering in Spark with PySpark is in the `Featuretools on Spark` notebook. This approach works on both a single machine and a cluster)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Conclusions\n",
    "\n",
    "Automated feature engineering is a significant improvement over manual feature engineering in terms of both time and modeling performance. In this notebook, we implemented an automated feature engineering workflow with Featuretools for the customer churn problem. Given customer data and label times, we can now calculate a feature matrix with several hundred relevant features for predicting customer churn while ensuring that our features are made with valid data for each cutoff time. \n",
    "\n",
    "Along the way, we implemented a number of Featuretools concepts:\n",
    "\n",
    "1. An entityset and entities\n",
    "2. Relationships between entities\n",
    "3. Cutoff times\n",
    "4. Feature primitives\n",
    "5. Custom primitives\n",
    "6. Deep feature synthesis\n",
    "\n",
    "These concepts will serve us well in future machine learning projects that we can tackle with automated feature engineering.\n",
    "\n",
    "## Next Steps\n",
    "\n",
    "Although we often hear that \"data is the fuel of machine learning\", data is not exactly a fuel but more like crude oil. _Features_ are the refined product that we feed into a machine learning model to make accurate predictions. After performing prediction engineering and automated feature engineering, the next step is to use these features in a predictive model to estimate the _label_ using the _features_. \n",
    "\n",
    "Generating hundreds of features automatically is impressive, but if those features cannot allow a model to learn our prediction problem then they are not mcuch help! The next step is to use our features and labeled historical examples to train a machine learning model to make predictions of customer churn. We'll make sure to test our model using a hold-out testing set to estimate performance on new data. Then, after validating our model, we can use it on new examples by passing the data through the feature engineering process. \n",
    "\n",
    "\n",
    "If you want to see how to parallelize feature engineering in Spark, see the `Feature Engineering on Spark` notebook. Otherwise, the next notebook is `Modeling`, where we develop a machine learning model to predict churn using the historical labeled examples and the automatically engineered features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}