Class Attendance .ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Pandas Advanced Example - Deriving Class Attendance "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alright! Now, let's look at a more complicated example in pandas - finding class attendance. This following code as a whole can get a bit complex with multiple and nested loops, so don't feel overwhelmed or discouraged if you don't understand each line of code - it took me over a week to figure out this process and that was with the extra hours I had to work with during the summer! One tip: it might be useful to print what's returned after each function/operation in this notebook to track the progress and understand what each function does."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As usual, let's start by importing useful libraries - for our case, we'll be importing pandas, numpy, matplotlib, seaborn, glob, datetime, and other libraries. Although we may not use all these libraries, it is useful to have them imported."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "from matplotlib import pyplot as plt\n",
    "import seaborn as sns\n",
    "import gc, glob, os, datetime, dateutil\n",
    "import datetime"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This code reads the class information data and and can be found in the StudentLife dataset. One important thing to note - in the directory that my jupyter notebook is in on my computer, I simply have to read the file since it's in the same directory. For you, depending on where this notebook lies in your computer, you might have to change the path that is read into pd.read. Nevertheless, let's have a look at the class_info dataframe:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "class_info = pd.read_json('class_info.json')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ANTH 012</th>\n",
       "      <th>BIOL 004</th>\n",
       "      <th>BIOL 006</th>\n",
       "      <th>CHIN 033</th>\n",
       "      <th>CHIN 062</th>\n",
       "      <th>COSC 007</th>\n",
       "      <th>COSC 020</th>\n",
       "      <th>COSC 027</th>\n",
       "      <th>COSC 050</th>\n",
       "      <th>COSC 057</th>\n",
       "      <th>...</th>\n",
       "      <th>MATH 022</th>\n",
       "      <th>MATH 023</th>\n",
       "      <th>MUS 003</th>\n",
       "      <th>MUS 016</th>\n",
       "      <th>NAS 008</th>\n",
       "      <th>NAS 035</th>\n",
       "      <th>PSYC 028</th>\n",
       "      <th>REL 018</th>\n",
       "      <th>SPAN 003</th>\n",
       "      <th>TUCK 003</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>location</td>\n",
       "      <td>silsby-rocky</td>\n",
       "      <td>lsb</td>\n",
       "      <td>lsb</td>\n",
       "      <td>reed</td>\n",
       "      <td>kemeny</td>\n",
       "      <td>baker-berry</td>\n",
       "      <td>moore</td>\n",
       "      <td>sudikoff</td>\n",
       "      <td>kemeny</td>\n",
       "      <td>sudikoff</td>\n",
       "      <td>...</td>\n",
       "      <td>kemeny</td>\n",
       "      <td>kemeny</td>\n",
       "      <td>hopkins</td>\n",
       "      <td>hopkins</td>\n",
       "      <td>moore</td>\n",
       "      <td>carpenterhall</td>\n",
       "      <td>moore</td>\n",
       "      <td>carson-tech_services</td>\n",
       "      <td>dartmouth_hall</td>\n",
       "      <td>carson-tech_services</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>periods</td>\n",
       "      <td>[{'day': 1, 'end': '12:20', 'start': '11:15'},...</td>\n",
       "      <td>[{'day': 2, 'end': '11:50', 'start': '10:00'},...</td>\n",
       "      <td>[{'day': 2, 'end': '11:50', 'start': '10:00'},...</td>\n",
       "      <td>[{'day': 1, 'end': '09:50', 'start': '09:00'},...</td>\n",
       "      <td>[{'day': 2, 'end': '17:50', 'start': '16:00'},...</td>\n",
       "      <td>[{'day': 1, 'end': '14:50', 'start': '13:45'},...</td>\n",
       "      <td>[{'day': 2, 'end': '15:50', 'start': '14:00'},...</td>\n",
       "      <td>[{'day': 1, 'end': '16:50', 'start': '15:00'},...</td>\n",
       "      <td>[{'day': 1, 'end': '13:35', 'start': '12:30'},...</td>\n",
       "      <td>[{'day': 1, 'end': '14:50', 'start': '13:45'},...</td>\n",
       "      <td>...</td>\n",
       "      <td>[{'day': 1, 'end': '12:20', 'start': '11:15'},...</td>\n",
       "      <td>[{'day': 1, 'end': '12:20', 'start': '11:15'},...</td>\n",
       "      <td>[{'day': 2, 'end': '15:50', 'start': '14:00'},...</td>\n",
       "      <td>[{'day': 2, 'end': '11:50', 'start': '10:00'},...</td>\n",
       "      <td>[{'day': 1, 'end': '13:35', 'start': '12:30'},...</td>\n",
       "      <td>[{'day': 1, 'end': '12:20', 'start': '11:15'},...</td>\n",
       "      <td>[{'day': 1, 'end': '14:50', 'start': '13:45'},...</td>\n",
       "      <td>[{'day': 1, 'end': '14:50', 'start': '13:45'},...</td>\n",
       "      <td>[{'day': 1, 'end': '09:50', 'start': '09:00'},...</td>\n",
       "      <td>[{'day': 2, 'end': '15:50', 'start': '14:00'},...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>2 rows × 45 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                   ANTH 012  \\\n",
       "location                                       silsby-rocky   \n",
       "periods   [{'day': 1, 'end': '12:20', 'start': '11:15'},...   \n",
       "\n",
       "                                                   BIOL 004  \\\n",
       "location                                                lsb   \n",
       "periods   [{'day': 2, 'end': '11:50', 'start': '10:00'},...   \n",
       "\n",
       "                                                   BIOL 006  \\\n",
       "location                                                lsb   \n",
       "periods   [{'day': 2, 'end': '11:50', 'start': '10:00'},...   \n",
       "\n",
       "                                                   CHIN 033  \\\n",
       "location                                               reed   \n",
       "periods   [{'day': 1, 'end': '09:50', 'start': '09:00'},...   \n",
       "\n",
       "                                                   CHIN 062  \\\n",
       "location                                             kemeny   \n",
       "periods   [{'day': 2, 'end': '17:50', 'start': '16:00'},...   \n",
       "\n",
       "                                                   COSC 007  \\\n",
       "location                                        baker-berry   \n",
       "periods   [{'day': 1, 'end': '14:50', 'start': '13:45'},...   \n",
       "\n",
       "                                                   COSC 020  \\\n",
       "location                                              moore   \n",
       "periods   [{'day': 2, 'end': '15:50', 'start': '14:00'},...   \n",
       "\n",
       "                                                   COSC 027  \\\n",
       "location                                           sudikoff   \n",
       "periods   [{'day': 1, 'end': '16:50', 'start': '15:00'},...   \n",
       "\n",
       "                                                   COSC 050  \\\n",
       "location                                             kemeny   \n",
       "periods   [{'day': 1, 'end': '13:35', 'start': '12:30'},...   \n",
       "\n",
       "                                                   COSC 057  ...  \\\n",
       "location                                           sudikoff  ...   \n",
       "periods   [{'day': 1, 'end': '14:50', 'start': '13:45'},...  ...   \n",
       "\n",
       "                                                   MATH 022  \\\n",
       "location                                             kemeny   \n",
       "periods   [{'day': 1, 'end': '12:20', 'start': '11:15'},...   \n",
       "\n",
       "                                                   MATH 023  \\\n",
       "location                                             kemeny   \n",
       "periods   [{'day': 1, 'end': '12:20', 'start': '11:15'},...   \n",
       "\n",
       "                                                    MUS 003  \\\n",
       "location                                            hopkins   \n",
       "periods   [{'day': 2, 'end': '15:50', 'start': '14:00'},...   \n",
       "\n",
       "                                                    MUS 016  \\\n",
       "location                                            hopkins   \n",
       "periods   [{'day': 2, 'end': '11:50', 'start': '10:00'},...   \n",
       "\n",
       "                                                    NAS 008  \\\n",
       "location                                              moore   \n",
       "periods   [{'day': 1, 'end': '13:35', 'start': '12:30'},...   \n",
       "\n",
       "                                                    NAS 035  \\\n",
       "location                                      carpenterhall   \n",
       "periods   [{'day': 1, 'end': '12:20', 'start': '11:15'},...   \n",
       "\n",
       "                                                   PSYC 028  \\\n",
       "location                                              moore   \n",
       "periods   [{'day': 1, 'end': '14:50', 'start': '13:45'},...   \n",
       "\n",
       "                                                    REL 018  \\\n",
       "location                               carson-tech_services   \n",
       "periods   [{'day': 1, 'end': '14:50', 'start': '13:45'},...   \n",
       "\n",
       "                                                   SPAN 003  \\\n",
       "location                                     dartmouth_hall   \n",
       "periods   [{'day': 1, 'end': '09:50', 'start': '09:00'},...   \n",
       "\n",
       "                                                   TUCK 003  \n",
       "location                               carson-tech_services  \n",
       "periods   [{'day': 2, 'end': '15:50', 'start': '14:00'},...  \n",
       "\n",
       "[2 rows x 45 columns]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "class_info.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For every class, this dataframe gives us a class location and info on when the class occurs. Since we want to know class attendance, we'll also have to know how much time a student spent in class, and that means checking to see how much time they spent in one of the class locations given above and whether or not they correspond to the actual class times. So, let's extract these class locations:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "#This code extracts the location row of the class_info dataset to create a list of class locations.\n",
    "all_locations = list(class_info.iloc[0,:])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since some classes have the same locations, some locations appear multiple times. This code finds only the unique class locations. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def unique(list1):\n",
    "    unique_locations = []\n",
    "    for i in list1:\n",
    "        if i not in unique_locations:\n",
    "            unique_locations.append(i)\n",
    "    return list(unique_locations)\n",
    "locations = unique(all_locations)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alright! Let's have a look at the list of locations:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['silsby-rocky',\n",
       " 'lsb',\n",
       " 'reed',\n",
       " 'kemeny',\n",
       " 'baker-berry',\n",
       " 'moore',\n",
       " 'sudikoff',\n",
       " 'steele',\n",
       " 'dartmouth_hall',\n",
       " 'carson-tech_services',\n",
       " 'cummings',\n",
       " 'maclean',\n",
       " 'vac',\n",
       " 'batrlett',\n",
       " 'hopkins',\n",
       " 'carpenterhall']"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "locations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Altough this is a solid start, there's one tiny issue: we'll be using the wifi-location data to check if a student is in an academic location, but the locations are formatted as 'in[insert location]' Thus, we have to alter these strings to match the same format as is in the wifi location data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "#To match the location data in the wifi location dataset, the locations were formatted in a certain way. \n",
    "#This code edits the list of unique class locations to match that formatting for further analysis.\n",
    "def location(list2):\n",
    "    class_loc = []\n",
    "    for i in list2:\n",
    "        class_loc.append('in[' + str(i) + ']')\n",
    "    return class_loc\n",
    "class_locations = location(locations)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "#This code reads the wifi location dataset.\n",
    "wifi_locations = pd.read_csv(\"wifi_location.csv\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This code returns the rows of the wifi location data that match classroom locations.\n",
    "We do this since we want to see how much time they spend ONLY in academic locations since those are the most\n",
    "applicable for finding class attendance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "def only_class_locations(data):\n",
    "    df_with_only_class_locations = data[data.location.isin(class_locations)] \n",
    "    return df_with_only_class_locations\n",
    "wifi_locations = only_class_locations(wifi_locations)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following two lines of code convert timestamp data to the pandas datetime format. \n",
    "Then a new column is created to find the day of the year for each timestamp.\n",
    "We do this since we want to analyze class attendance by day, but is good to do anyway."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "def time_day_of_location_timestamps(data):\n",
    "    wifi_locations['time'] = pd.to_datetime(data['time'])\n",
    "    wifi_locations['day'] = data['time'].dt.dayofyear\n",
    "    wifi_locations['index'] = wifi_locations.index\n",
    "    return wifi_locations\n",
    "wifi_locations = time_day_of_location_timestamps(wifi_locations)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>level_0</th>\n",
       "      <th>timestamp</th>\n",
       "      <th>location</th>\n",
       "      <th>uid</th>\n",
       "      <th>time</th>\n",
       "      <th>day</th>\n",
       "      <th>index</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1364396255</td>\n",
       "      <td>in[sudikoff]</td>\n",
       "      <td>u45</td>\n",
       "      <td>2013-03-27 14:57:35</td>\n",
       "      <td>86</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1364397473</td>\n",
       "      <td>in[sudikoff]</td>\n",
       "      <td>u45</td>\n",
       "      <td>2013-03-27 15:17:53</td>\n",
       "      <td>86</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>1364397623</td>\n",
       "      <td>in[sudikoff]</td>\n",
       "      <td>u45</td>\n",
       "      <td>2013-03-27 15:20:23</td>\n",
       "      <td>86</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>1364397661</td>\n",
       "      <td>in[sudikoff]</td>\n",
       "      <td>u45</td>\n",
       "      <td>2013-03-27 15:21:01</td>\n",
       "      <td>86</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>1364397671</td>\n",
       "      <td>in[sudikoff]</td>\n",
       "      <td>u45</td>\n",
       "      <td>2013-03-27 15:21:11</td>\n",
       "      <td>86</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>549125</td>\n",
       "      <td>1893833</td>\n",
       "      <td>1369815184</td>\n",
       "      <td>in[lsb]</td>\n",
       "      <td>u31</td>\n",
       "      <td>2013-05-29 08:13:04</td>\n",
       "      <td>149</td>\n",
       "      <td>1893833</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>549126</td>\n",
       "      <td>1893834</td>\n",
       "      <td>1369815204</td>\n",
       "      <td>in[lsb]</td>\n",
       "      <td>u31</td>\n",
       "      <td>2013-05-29 08:13:24</td>\n",
       "      <td>149</td>\n",
       "      <td>1893834</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>549127</td>\n",
       "      <td>1893835</td>\n",
       "      <td>1369815224</td>\n",
       "      <td>in[lsb]</td>\n",
       "      <td>u31</td>\n",
       "      <td>2013-05-29 08:13:44</td>\n",
       "      <td>149</td>\n",
       "      <td>1893835</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>549128</td>\n",
       "      <td>1893836</td>\n",
       "      <td>1369815244</td>\n",
       "      <td>in[lsb]</td>\n",
       "      <td>u31</td>\n",
       "      <td>2013-05-29 08:14:04</td>\n",
       "      <td>149</td>\n",
       "      <td>1893836</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>549129</td>\n",
       "      <td>1893837</td>\n",
       "      <td>1369815264</td>\n",
       "      <td>in[lsb]</td>\n",
       "      <td>u31</td>\n",
       "      <td>2013-05-29 08:14:24</td>\n",
       "      <td>149</td>\n",
       "      <td>1893837</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>549130 rows × 7 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        level_0   timestamp      location  uid                time  day  \\\n",
       "0             0  1364396255  in[sudikoff]  u45 2013-03-27 14:57:35   86   \n",
       "1             1  1364397473  in[sudikoff]  u45 2013-03-27 15:17:53   86   \n",
       "2             2  1364397623  in[sudikoff]  u45 2013-03-27 15:20:23   86   \n",
       "3             3  1364397661  in[sudikoff]  u45 2013-03-27 15:21:01   86   \n",
       "4             4  1364397671  in[sudikoff]  u45 2013-03-27 15:21:11   86   \n",
       "...         ...         ...           ...  ...                 ...  ...   \n",
       "549125  1893833  1369815184       in[lsb]  u31 2013-05-29 08:13:04  149   \n",
       "549126  1893834  1369815204       in[lsb]  u31 2013-05-29 08:13:24  149   \n",
       "549127  1893835  1369815224       in[lsb]  u31 2013-05-29 08:13:44  149   \n",
       "549128  1893836  1369815244       in[lsb]  u31 2013-05-29 08:14:04  149   \n",
       "549129  1893837  1369815264       in[lsb]  u31 2013-05-29 08:14:24  149   \n",
       "\n",
       "          index  \n",
       "0             0  \n",
       "1             1  \n",
       "2             2  \n",
       "3             3  \n",
       "4             4  \n",
       "...         ...  \n",
       "549125  1893833  \n",
       "549126  1893834  \n",
       "549127  1893835  \n",
       "549128  1893836  \n",
       "549129  1893837  \n",
       "\n",
       "[549130 rows x 7 columns]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#One thing I picked up on when working on this was that the original index was non-uniform \n",
    "#Essentially, some rows would have indices of 1,2,4 and skip 3. Thus, I reset the index.\n",
    "\n",
    "wifi_locations['index'] = wifi_locations.index\n",
    "wifi_locations = wifi_locations.reset_index()\n",
    "wifi_locations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Okay so here's where things get a little complicated - if we look at the wifi locations dataframe above, we see that the location is recorded continously. Thus, if a student is in a location for a prolonged period of time (for lecture, exams, etc.), then the same location will appear several times <b>continously</b>. The goal of the next few functions is to figure out when (i.e. which row index) in the dataframe this location first appears, when it last appears and the corresponding times. We can then subtract the times to figure out how much time a student spent at that specific location! Let's start with our first order of business - finding the indices where a location first appears and when it last <b>continously</b> appears."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To find the time duration spent for each location, this is one of the functions I used, which calculates what I like to call the \"shift number\" (I really don't know why I named it that specifically, but alas). The shift number represents the difference in the index values of the rows where the location first appeared and when the location last continously appeared. To calculate the shift number, I constrained it to locations that appeared consecutively in the same day since some locations appeared continously across multiple days. To really understand the process, look through the code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[4,\n",
       " 2,\n",
       " 4,\n",
       " 1,\n",
       " 0,\n",
       " 2,\n",
       " 1,\n",
       " 8,\n",
       " 9,\n",
       " 48,\n",
       " 4,\n",
       " 0,\n",
       " 4,\n",
       " 8,\n",
       " 2,\n",
       " 0,\n",
       " 351,\n",
       " 2,\n",
       " 372,\n",
       " 9,\n",
       " 15,\n",
       " 0,\n",
       " 2,\n",
       " 369,\n",
       " 48,\n",
       " 2,\n",
       " 7,\n",
       " 1,\n",
       " 0,\n",
       " 6,\n",
       " 0,\n",
       " 1,\n",
       " 1,\n",
       " 274,\n",
       " 7,\n",
       " 0,\n",
       " 12,\n",
       " 1,\n",
       " 1,\n",
       " 3,\n",
       " 38,\n",
       " 3,\n",
       " 1,\n",
       " 2,\n",
       " 1,\n",
       " 18,\n",
       " 0,\n",
       " 0,\n",
       " 0,\n",
       " 0,\n",
       " 1,\n",
       " 1,\n",
       " 4,\n",
       " 0,\n",
       " 12,\n",
       " 1,\n",
       " 0,\n",
       " 202,\n",
       " 528,\n",
       " 8,\n",
       " 4,\n",
       " 3,\n",
       " 1,\n",
       " 31,\n",
       " 1,\n",
       " 1,\n",
       " 3,\n",
       " 2,\n",
       " 0,\n",
       " 9,\n",
       " 0,\n",
       " 35,\n",
       " 0,\n",
       " 0,\n",
       " 21,\n",
       " 0,\n",
       " 7,\n",
       " 1,\n",
       " 162,\n",
       " 0,\n",
       " 19,\n",
       " 4,\n",
       " 195,\n",
       " 2,\n",
       " 0,\n",
       " 46,\n",
       " 0,\n",
       " 17,\n",
       " 0,\n",
       " 54,\n",
       " 18,\n",
       " 0,\n",
       " 0,\n",
       " 0,\n",
       " 1,\n",
       " 259,\n",
       " 0,\n",
       " 0,\n",
       " 9,\n",
       " 193,\n",
       " 0,\n",
       " 184,\n",
       " 3,\n",
       " 1,\n",
       " 10,\n",
       " 0,\n",
       " 0,\n",
       " 248,\n",
       " 180,\n",
       " 1,\n",
       " 715,\n",
       " 109,\n",
       " 0,\n",
       " 4,\n",
       " 279,\n",
       " 3,\n",
       " 2,\n",
       " 1,\n",
       " 4,\n",
       " 1,\n",
       " 12,\n",
       " 1,\n",
       " 0,\n",
       " 0,\n",
       " 52,\n",
       " 3,\n",
       " 14,\n",
       " 3,\n",
       " 1,\n",
       " 5,\n",
       " 0,\n",
       " 2,\n",
       " 0,\n",
       " 2,\n",
       " 1,\n",
       " 3,\n",
       " 1,\n",
       " 1,\n",
       " 1,\n",
       " 8,\n",
       " 24,\n",
       " 6,\n",
       " 0,\n",
       " 1,\n",
       " 15,\n",
       " 26,\n",
       " 0,\n",
       " 0,\n",
       " 17,\n",
       " 32,\n",
       " 0,\n",
       " 3,\n",
       " 0,\n",
       " 0,\n",
       " 1,\n",
       " 1,\n",
       " 0,\n",
       " 1,\n",
       " 1,\n",
       " 37,\n",
       " 119,\n",
       " 0,\n",
       " 0,\n",
       " 0,\n",
       " 6,\n",
       " 30,\n",
       " 0,\n",
       " 0,\n",
       " 0,\n",
       " 1,\n",
       " 1,\n",
       " 1,\n",
       " 0,\n",
       " 0,\n",
       " 0,\n",
       " 2,\n",
       " 5,\n",
       " 7,\n",
       " 1,\n",
       " 6,\n",
       " 0,\n",
       " 0,\n",
       " 16,\n",
       " 0,\n",
       " 14,\n",
       " 0,\n",
       " 0,\n",
       " 2,\n",
       " 0,\n",
       " 1,\n",
       " 7,\n",
       " 7,\n",
       " 4,\n",
       " 1,\n",
       " 61,\n",
       " 1,\n",
       " 1,\n",
       " 2,\n",
       " 1,\n",
       " 0,\n",
       " 12,\n",
       " 0,\n",
       " 4,\n",
       " 128,\n",
       " 0,\n",
       " 0,\n",
       " 0,\n",
       " 0,\n",
       " 16,\n",
       " 4,\n",
       " 4,\n",
       " 1,\n",
       " 1,\n",
       " 51,\n",
       " 94,\n",
       " 0,\n",
       " 15,\n",
       " 1,\n",
       " 2,\n",
       " 32,\n",
       " 3,\n",
       " 3,\n",
       " 2,\n",
       " 1,\n",
       " 0,\n",
       " 4,\n",
       " 0,\n",
       " 5,\n",
       " 34,\n",
       " 4,\n",
       " 0,\n",
       " 1,\n",
       " 2,\n",
       " 14,\n",
       " 2,\n",
       " 38,\n",
       " 0,\n",
       " 18,\n",
       " 0,\n",
       " 0,\n",
       " 2,\n",
       " 0,\n",
       " 1,\n",
       " 16,\n",
       " 2,\n",
       " 40,\n",
       " 4,\n",
       " 1,\n",
       " 7,\n",
       " 1,\n",
       " 0,\n",
       " 7,\n",
       " 0,\n",
       " 0,\n",
       " 1,\n",
       " 0,\n",
       " 1,\n",
       " 3,\n",
       " 0,\n",
       " 3,\n",
       " 1,\n",
       " 19,\n",
       " 1,\n",
       " 1,\n",
       " 8,\n",
       " 0,\n",
       " 1,\n",
       " 0,\n",
       " 3,\n",
       " 8,\n",
       " 0,\n",
       " 3,\n",
       " 2,\n",
       " 1,\n",
       " 5,\n",
       " 137,\n",
       " 0,\n",
       " 2,\n",
       " 2,\n",
       " 0,\n",
       " 0,\n",
       " 33,\n",
       " 0,\n",
       " 1,\n",
       " 1,\n",
       " 7,\n",
       " 0,\n",
       " 1,\n",
       " 1,\n",
       " 0,\n",
       " 0,\n",
       " 6,\n",
       " 1,\n",
       " 11,\n",
       " 0,\n",
       " 1,\n",
       " 0,\n",
       " 0,\n",
       " 5,\n",
       " 3,\n",
       " 0,\n",
       " 0,\n",
       " 4,\n",
       " 6,\n",
       " 0,\n",
       " 4,\n",
       " 3,\n",
       " 0,\n",
       " 12,\n",
       " 3,\n",
       " 3,\n",
       " 0,\n",
       " 11,\n",
       " 10,\n",
       " 4,\n",
       " 4,\n",
       " 19,\n",
       " 4,\n",
       " 9,\n",
       " 22,\n",
       " 3,\n",
       " 1,\n",
       " 12,\n",
       " 12,\n",
       " 19,\n",
       " 2,\n",
       " 14,\n",
       " 0,\n",
       " 2,\n",
       " 0,\n",
       " 3,\n",
       " 1,\n",
       " 28,\n",
       " 24,\n",
       " 2,\n",
       " 11,\n",
       " 1,\n",
       " 8,\n",
       " 0,\n",
       " 11,\n",
       " 0,\n",
       " 2,\n",
       " 1,\n",
       " 1,\n",
       " 4,\n",
       " 0,\n",
       " 7,\n",
       " 1,\n",
       " 42,\n",
       " 5,\n",
       " 5,\n",
       " 6,\n",
       " 0,\n",
       " 2,\n",
       " 0,\n",
       " 0,\n",
       " 3,\n",
       " 0,\n",
       " 0,\n",
       " 1,\n",
       " 3,\n",
       " 4,\n",
       " 1,\n",
       " 139,\n",
       " 271,\n",
       " 3,\n",
       " 4,\n",
       " 4,\n",
       " 3,\n",
       " 7,\n",
       " 4,\n",
       " 0,\n",
       " 4,\n",
       " 4,\n",
       " 4,\n",
       " 12,\n",
       " 1,\n",
       " 5,\n",
       " 3,\n",
       " 1,\n",
       " 5,\n",
       " 2,\n",
       " 0,\n",
       " 4,\n",
       " 5,\n",
       " 1,\n",
       " 0,\n",
       " 8,\n",
       " 5,\n",
       " 12,\n",
       " 2,\n",
       " 7,\n",
       " 1,\n",
       " 2,\n",
       " 0,\n",
       " 1,\n",
       " 6,\n",
       " 4,\n",
       " 0,\n",
       " 0,\n",
       " 6,\n",
       " 2,\n",
       " 3,\n",
       " 1,\n",
       " 4,\n",
       " 0,\n",
       " 9,\n",
       " 10,\n",
       " 0,\n",
       " 1,\n",
       " 6,\n",
       " 5,\n",
       " 0,\n",
       " 2,\n",
       " 0,\n",
       " 10,\n",
       " 29,\n",
       " 0,\n",
       " 4,\n",
       " 0,\n",
       " 30,\n",
       " 1,\n",
       " 2,\n",
       " 9,\n",
       " 1,\n",
       " 8,\n",
       " 1,\n",
       " 10,\n",
       " 12,\n",
       " 4,\n",
       " 1,\n",
       " 4,\n",
       " 4,\n",
       " 9,\n",
       " 5,\n",
       " 3,\n",
       " 1,\n",
       " 5,\n",
       " 0,\n",
       " 4,\n",
       " 0,\n",
       " 6,\n",
       " 4,\n",
       " 12,\n",
       " 1,\n",
       " 0,\n",
       " 0,\n",
       " 6,\n",
       " 12,\n",
       " 2,\n",
       " 0,\n",
       " 0,\n",
       " 4,\n",
       " 0,\n",
       " 1,\n",
       " 7,\n",
       " 7,\n",
       " 1,\n",
       " 8,\n",
       " 7,\n",
       " 1,\n",
       " 13,\n",
       " 7,\n",
       " 0,\n",
       " 12,\n",
       " 0,\n",
       " 1,\n",
       " 2,\n",
       " 1,\n",
       " 0,\n",
       " 4,\n",
       " 0,\n",
       " 6,\n",
       " 2,\n",
       " 6,\n",
       " 1,\n",
       " 8,\n",
       " 0,\n",
       " 4,\n",
       " 4,\n",
       " 2,\n",
       " 21,\n",
       " 0,\n",
       " 4,\n",
       " 2,\n",
       " 7,\n",
       " 11,\n",
       " 13,\n",
       " 4,\n",
       " 1,\n",
       " 0,\n",
       " 2,\n",
       " 1,\n",
       " 0,\n",
       " 42,\n",
       " 1,\n",
       " 1,\n",
       " 22,\n",
       " 1,\n",
       " 6,\n",
       " 0,\n",
       " 9,\n",
       " 1,\n",
       " 2,\n",
       " 4,\n",
       " 3,\n",
       " 2,\n",
       " 11,\n",
       " 3,\n",
       " 7,\n",
       " 6,\n",
       " 1,\n",
       " 2,\n",
       " 1,\n",
       " 3,\n",
       " 4,\n",
       " 0,\n",
       " 6,\n",
       " 3,\n",
       " 0,\n",
       " 30,\n",
       " 14,\n",
       " 0,\n",
       " 2,\n",
       " 0,\n",
       " 7,\n",
       " 11,\n",
       " 1,\n",
       " 28,\n",
       " 3,\n",
       " 8,\n",
       " 1,\n",
       " 0,\n",
       " 2,\n",
       " 2,\n",
       " 24,\n",
       " 1,\n",
       " 13,\n",
       " 2,\n",
       " 3,\n",
       " 0,\n",
       " 4,\n",
       " 5,\n",
       " 6,\n",
       " 3,\n",
       " 0,\n",
       " 0,\n",
       " 1,\n",
       " 0,\n",
       " 0,\n",
       " 0,\n",
       " 11,\n",
       " 0,\n",
       " 19,\n",
       " 4,\n",
       " 2,\n",
       " 342,\n",
       " 2,\n",
       " 7,\n",
       " 50,\n",
       " 58,\n",
       " 345,\n",
       " 8,\n",
       " 3,\n",
       " 7,\n",
       " 121,\n",
       " 0,\n",
       " 68,\n",
       " 9,\n",
       " 150,\n",
       " 0,\n",
       " 24,\n",
       " 0,\n",
       " 191,\n",
       " 1,\n",
       " 1,\n",
       " 11,\n",
       " 9,\n",
       " 1,\n",
       " 0,\n",
       " 0,\n",
       " 2,\n",
       " 0,\n",
       " 1,\n",
       " 17,\n",
       " 3,\n",
       " 0,\n",
       " 0,\n",
       " 0,\n",
       " 5,\n",
       " 0,\n",
       " 24,\n",
       " 2,\n",
       " 2,\n",
       " 2,\n",
       " 3,\n",
       " 11,\n",
       " 4,\n",
       " 3,\n",
       " 8,\n",
       " 1,\n",
       " 5,\n",
       " 3,\n",
       " 10,\n",
       " 2,\n",
       " 6,\n",
       " 6,\n",
       " 2,\n",
       " 20,\n",
       " 3,\n",
       " 0,\n",
       " 34,\n",
       " 4,\n",
       " 28,\n",
       " 17,\n",
       " 17,\n",
       " 4,\n",
       " 13,\n",
       " 1,\n",
       " 0,\n",
       " 1,\n",
       " 36,\n",
       " 0,\n",
       " 7,\n",
       " 6,\n",
       " 9,\n",
       " 1,\n",
       " 6,\n",
       " 0,\n",
       " 0,\n",
       " 59,\n",
       " 11,\n",
       " 3,\n",
       " 0,\n",
       " 0,\n",
       " 10,\n",
       " 0,\n",
       " 3,\n",
       " 0,\n",
       " 0,\n",
       " 0,\n",
       " 1,\n",
       " 8,\n",
       " 3,\n",
       " 3,\n",
       " 11,\n",
       " 1,\n",
       " 7,\n",
       " 1,\n",
       " 34,\n",
       " 2,\n",
       " 4,\n",
       " 1,\n",
       " 0,\n",
       " 13,\n",
       " 38,\n",
       " 1,\n",
       " 8,\n",
       " 1,\n",
       " 0,\n",
       " 4,\n",
       " 9,\n",
       " 2,\n",
       " 6,\n",
       " 4,\n",
       " 18,\n",
       " 22,\n",
       " 3,\n",
       " 3,\n",
       " 0,\n",
       " 15,\n",
       " 9,\n",
       " 1,\n",
       " 30,\n",
       " 13,\n",
       " 1,\n",
       " 3,\n",
       " 43,\n",
       " 5,\n",
       " 1,\n",
       " 6,\n",
       " 0,\n",
       " 15,\n",
       " 10,\n",
       " 1,\n",
       " 2,\n",
       " 29,\n",
       " 0,\n",
       " 11,\n",
       " 3,\n",
       " 7,\n",
       " 0,\n",
       " 0,\n",
       " 1,\n",
       " 0,\n",
       " 0,\n",
       " 1,\n",
       " 0,\n",
       " 9,\n",
       " 4,\n",
       " 4,\n",
       " 7,\n",
       " 5,\n",
       " 5,\n",
       " 1,\n",
       " 1,\n",
       " 11,\n",
       " 4,\n",
       " 3,\n",
       " 0,\n",
       " 0,\n",
       " 3,\n",
       " 2,\n",
       " 3,\n",
       " 4,\n",
       " 0,\n",
       " 1,\n",
       " 0,\n",
       " 2,\n",
       " 0,\n",
       " 0,\n",
       " 1,\n",
       " 12,\n",
       " 4,\n",
       " 2,\n",
       " 0,\n",
       " 183,\n",
       " 5,\n",
       " 5,\n",
       " 9,\n",
       " 0,\n",
       " 2,\n",
       " 0,\n",
       " 14,\n",
       " 1,\n",
       " 0,\n",
       " 3,\n",
       " 7,\n",
       " 0,\n",
       " 1,\n",
       " 13,\n",
       " 2,\n",
       " 0,\n",
       " 1,\n",
       " 0,\n",
       " 0,\n",
       " 12,\n",
       " 0,\n",
       " 4,\n",
       " 18,\n",
       " 4,\n",
       " 0,\n",
       " 4,\n",
       " 9,\n",
       " 0,\n",
       " 0,\n",
       " 0,\n",
       " 6,\n",
       " 0,\n",
       " 7,\n",
       " 8,\n",
       " 0,\n",
       " 1,\n",
       " 1,\n",
       " 5,\n",
       " 2,\n",
       " 3,\n",
       " 0,\n",
       " 16,\n",
       " 0,\n",
       " 1,\n",
       " 1,\n",
       " 20,\n",
       " 2,\n",
       " 0,\n",
       " 3,\n",
       " 2,\n",
       " 0,\n",
       " 11,\n",
       " 18,\n",
       " 14,\n",
       " 1,\n",
       " 1,\n",
       " 16,\n",
       " 11,\n",
       " 0,\n",
       " 1,\n",
       " 30,\n",
       " 11,\n",
       " 0,\n",
       " 19,\n",
       " 16,\n",
       " 3,\n",
       " 1,\n",
       " 0,\n",
       " 2,\n",
       " 0,\n",
       " 3,\n",
       " 2,\n",
       " 185,\n",
       " 1,\n",
       " 1,\n",
       " 18,\n",
       " 16,\n",
       " 21,\n",
       " 12,\n",
       " 62,\n",
       " 23,\n",
       " 2,\n",
       " 12,\n",
       " 11,\n",
       " 0,\n",
       " 11,\n",
       " 20,\n",
       " 18,\n",
       " 3,\n",
       " 9,\n",
       " 0,\n",
       " 36,\n",
       " 1,\n",
       " 3,\n",
       " 0,\n",
       " 10,\n",
       " 8,\n",
       " 7,\n",
       " 10,\n",
       " 0,\n",
       " 0,\n",
       " 0,\n",
       " 4,\n",
       " 5,\n",
       " 0,\n",
       " 5,\n",
       " 4,\n",
       " 2,\n",
       " 2,\n",
       " 4,\n",
       " 0,\n",
       " 1,\n",
       " 2,\n",
       " 24,\n",
       " 2,\n",
       " 0,\n",
       " 1,\n",
       " 6,\n",
       " 1,\n",
       " 7,\n",
       " 5,\n",
       " 3,\n",
       " 7,\n",
       " 0,\n",
       " 3,\n",
       " 0,\n",
       " 0,\n",
       " 4,\n",
       " 10,\n",
       " 1,\n",
       " 7,\n",
       " 8,\n",
       " 0,\n",
       " 1,\n",
       " 1,\n",
       " 0,\n",
       " 20,\n",
       " 1,\n",
       " 3,\n",
       " 5,\n",
       " 3,\n",
       " 1,\n",
       " 4,\n",
       " 1,\n",
       " 7,\n",
       " 24,\n",
       " 1,\n",
       " 4,\n",
       " 0,\n",
       " 8,\n",
       " 1,\n",
       " 1,\n",
       " 2,\n",
       " 0,\n",
       " 17,\n",
       " 2,\n",
       " 4,\n",
       " 1,\n",
       " 10,\n",
       " 7,\n",
       " 1,\n",
       " 7,\n",
       " 2,\n",
       " 15,\n",
       " 6,\n",
       " 2,\n",
       " 0,\n",
       " 8,\n",
       " 0,\n",
       " 1,\n",
       " 7,\n",
       " 1,\n",
       " 2,\n",
       " 8,\n",
       " 5,\n",
       " 12,\n",
       " 3,\n",
       " 0,\n",
       " 0,\n",
       " 0,\n",
       " 1,\n",
       " 52,\n",
       " 1,\n",
       " 1,\n",
       " 24,\n",
       " 3,\n",
       " 0,\n",
       " 0,\n",
       " 21,\n",
       " 0,\n",
       " 39,\n",
       " 69,\n",
       " 1,\n",
       " 6,\n",
       " 2,\n",
       " 5,\n",
       " 8,\n",
       " 1,\n",
       " 0,\n",
       " 35,\n",
       " 3,\n",
       " 1,\n",
       " 5,\n",
       " 5,\n",
       " 0,\n",
       " 23,\n",
       " 0,\n",
       " 3,\n",
       " 0,\n",
       " 4,\n",
       " 9,\n",
       " 0,\n",
       " 0,\n",
       " 344,\n",
       " 45,\n",
       " 4,\n",
       " 8,\n",
       " 5,\n",
       " 1,\n",
       " 6,\n",
       " 0,\n",
       " 0,\n",
       " 7,\n",
       " 0,\n",
       " 23,\n",
       " 11,\n",
       " 20,\n",
       " 7,\n",
       " 3,\n",
       " 892,\n",
       " 691,\n",
       " 3,\n",
       " 4,\n",
       " 0,\n",
       " 0,\n",
       " 3,\n",
       " 5,\n",
       " 9,\n",
       " 8,\n",
       " 3,\n",
       " 1,\n",
       " 0,\n",
       " 3,\n",
       " 1,\n",
       " 0,\n",
       " 11,\n",
       " 9,\n",
       " 1,\n",
       " 1,\n",
       " 24,\n",
       " 1,\n",
       " 0,\n",
       " 3,\n",
       " 0,\n",
       " 1,\n",
       " 0,\n",
       " 8,\n",
       " 0,\n",
       " 131,\n",
       " 2,\n",
       " 0,\n",
       " 1,\n",
       " 29,\n",
       " 1,\n",
       " 12,\n",
       " 9,\n",
       " 6,\n",
       " 8,\n",
       " 8,\n",
       " 1,\n",
       " 0,\n",
       " 4,\n",
       " 7,\n",
       " ...]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def shift_counter(data):\n",
    "    #The three lines below initialize the shift number and lists that contain the shift number.\n",
    "    shift_num = 0\n",
    "    list_shift_num = []\n",
    "    list_id = []\n",
    "    \n",
    "    #This for loop is the most important part of the function: \n",
    "    #We look at instances where the location is the same for continous rows and we count for how many rows.\n",
    "    #Once we hit an exception to this trend, we append the number of rows (shift_num) to our list and reset\n",
    "    #our shift number to 0.\n",
    "    for i in range(0, len(data)):\n",
    "        try: \n",
    "            if data.location[i] == data.location[i+1] and data.day[i] == data.day[i+1]:\n",
    "                shift_num += 1\n",
    "            else:\n",
    "                list_shift_num.append(shift_num)\n",
    "                shift_num = 0\n",
    "\n",
    "        except:\n",
    "            list_shift_num.append(shift_num)\n",
    "            shift_num = 0\n",
    "    return list_shift_num\n",
    "\n",
    "#This list has all of the shift numbers (# of consecutive rows where location and day are the same)\n",
    "list_shift = shift_counter(wifi_locations)\n",
    "list_shift"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "#This function is the same as the previous function.\n",
    "#The only difference is that I tracked the start timestamps, user id, and location associated with shift number.\n",
    "#This is why this function is longer, but it is helpful since it tracks the user, time and location.\n",
    "\n",
    "def shift_counter_id(data):\n",
    "    shift_num = 0\n",
    "    #counter = 0\n",
    "    list_shift_num = []\n",
    "    list_id = []\n",
    "    list_time = []\n",
    "    list_location = []\n",
    "    for i in range(0, len(data)):\n",
    "        try: \n",
    "            if data.location[i] == data.location[i+1] and data.day[i] == data.day[i+1]:\n",
    "                shift_num += 1\n",
    "                uid = data.uid[i]\n",
    "                time = data.timestamp[i]\n",
    "                location = data.location[i]\n",
    "            else:\n",
    "                list_shift_num.append(shift_num)\n",
    "                shift_num = 0\n",
    "                list_id.append(uid)\n",
    "                list_time.append(time)\n",
    "                list_location.append(location)\n",
    "        except:\n",
    "            list_shift_num.append(shift_num)\n",
    "            shift_num = 0\n",
    "            list_id.append(uid)\n",
    "            list_time.append(time)\n",
    "            list_location.append(location)\n",
    "    df = pd.DataFrame({'Shift Num': list_shift_num, 'uid': list_id, 'time': list_time, 'location': list_location}) #'time': list_time, 'location': list_location})\n",
    "    return df\n",
    "\n",
    "#This dataframe contains the shift numbers and their associated times, locations and user ids\n",
    "dataframe = shift_counter_id(wifi_locations)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Shift Num</th>\n",
       "      <th>uid</th>\n",
       "      <th>time</th>\n",
       "      <th>location</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>u45</td>\n",
       "      <td>1364397661</td>\n",
       "      <td>in[sudikoff]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>u45</td>\n",
       "      <td>1364839542</td>\n",
       "      <td>in[sudikoff]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>4</td>\n",
       "      <td>u45</td>\n",
       "      <td>1364842439</td>\n",
       "      <td>in[lsb]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>u45</td>\n",
       "      <td>1364848920</td>\n",
       "      <td>in[baker-berry]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>u45</td>\n",
       "      <td>1364848920</td>\n",
       "      <td>in[baker-berry]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>16143</td>\n",
       "      <td>1</td>\n",
       "      <td>u31</td>\n",
       "      <td>1369317608</td>\n",
       "      <td>in[baker-berry]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>16144</td>\n",
       "      <td>7</td>\n",
       "      <td>u31</td>\n",
       "      <td>1369324215</td>\n",
       "      <td>in[steele]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>16145</td>\n",
       "      <td>171</td>\n",
       "      <td>u31</td>\n",
       "      <td>1369329059</td>\n",
       "      <td>in[sudikoff]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>16146</td>\n",
       "      <td>288</td>\n",
       "      <td>u31</td>\n",
       "      <td>1369438492</td>\n",
       "      <td>in[sudikoff]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>16147</td>\n",
       "      <td>59</td>\n",
       "      <td>u31</td>\n",
       "      <td>1369815244</td>\n",
       "      <td>in[lsb]</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>16148 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       Shift Num  uid        time         location\n",
       "0              4  u45  1364397661     in[sudikoff]\n",
       "1              2  u45  1364839542     in[sudikoff]\n",
       "2              4  u45  1364842439          in[lsb]\n",
       "3              1  u45  1364848920  in[baker-berry]\n",
       "4              0  u45  1364848920  in[baker-berry]\n",
       "...          ...  ...         ...              ...\n",
       "16143          1  u31  1369317608  in[baker-berry]\n",
       "16144          7  u31  1369324215       in[steele]\n",
       "16145        171  u31  1369329059     in[sudikoff]\n",
       "16146        288  u31  1369438492     in[sudikoff]\n",
       "16147         59  u31  1369815244          in[lsb]\n",
       "\n",
       "[16148 rows x 4 columns]"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataframe"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "scrolled": false
   },
   "source": [
    "As you can see from the dataframe above, there are some shift numbers that are zero and since these shift numbers will not result in a time delta, let's take them out and create a new list of non-zero shift numbers using the following function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "def shifts_only(list1):\n",
    "    shifts_only_list = []\n",
    "    for i in list1:\n",
    "        if i != 0:\n",
    "            shifts_only_list.append(i)\n",
    "    return shifts_only_list\n",
    "\n",
    "#This list is a list of non-zero shift numbers.\n",
    "list_shifts = shifts_only(list_shift)\n",
    "#list_shifts"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "11886"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(list_shifts)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As stated before, to find the time spent in a location, we'd need to know the rows where a location first appeared and when it last appeared. To do so, I was able to develop an equation that can find this. It's shown in the cell after the cell below. In essence, one of the parameters of this equation is to find the sum of all the shift numbers, which we'll do in the function below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "#When calculating the time delta, I would need the index of when the location first appeared and the index when the location last consecutively appeared. \n",
    "#I had to find the formula to find the start and ending indices, and it involved, for every element in the list of shift number, the sum of the shift numbers before it.\n",
    "def get_sums(list1):\n",
    "    list_sums_b = []\n",
    "    for i in range(0,len(list1)+1):\n",
    "        new_list = list1[:i]\n",
    "        sums = sum(new_list)\n",
    "        list_sums_b.append(sums)\n",
    "    return list_sums_b\n",
    "\n",
    "#This function returns a list in which every element represents the sum of the shift numbers that came before it.\n",
    "list_sums_before = get_sums(list_shifts)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now this is the function that finds the duration of time spent in a location. The if/elif conditionals contain the equation that I used to find the start and end rows for locations, which I then used to find the time duration. Don't worry if you don't understand the equation and the code below! It took me some time to develop as I looked at the dataframe and looked for patterns. However, it has been an incredibly useful function for us - given what I call \"shift numbers,\" we're essentially able to find the amount of time spent in a particular location/doing a particular activity, such as running or walking. The function below just returns a list of time spent in class locations. I actually used the function after as I created a dataframe that tracked the day, start and end time as well as student id."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "#This function calculates the time deltas for every location. \n",
    "def time_delta(list_shift_num):\n",
    "    time_deltas = []\n",
    "    for i in range(0, len(list_shift_num)):\n",
    "        if i == 0:\n",
    "            time_deltas.append(wifi_locations['time'][list_shift_num[i]] - wifi_locations['time'][0])\n",
    "        elif i != 0:\n",
    "            #To find the time deltas for all the elements after the first one, I used the following formula:\n",
    "            time_deltas.append(wifi_locations['time'][list_shift_num[i]+i+list_sums_before[i]] - wifi_locations['time'][list_sums_before[i]+i])\n",
    "    return time_deltas\n",
    "\n",
    "#This function returns the time deltas.\n",
    "#time_delta(list_shifts)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is the function that returns a dataframe of time spent in classroom locations as well as which student it was for (uid), start/end time and day."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "#This function is the exact same as the previous function but returns a dataframe instead.\n",
    "#The dataframe consists of the time deltas and their associated time, location, and user id.\n",
    "def time_delta(data, list_shift_num):\n",
    "    time_deltas = []\n",
    "    location = []\n",
    "    start_time = []\n",
    "    uid = []\n",
    "    for i in range(0, len(list_shift_num)):\n",
    "        if i == 0:\n",
    "            time_deltas.append(wifi_locations['time'][list_shift_num[i]] - wifi_locations['time'][0])\n",
    "            uid.append(data.uid[i])\n",
    "            start_time.append(data.time[i])\n",
    "            location.append(data.location[i])\n",
    "        elif i != 0:\n",
    "            time_deltas.append(wifi_locations['time'][list_shift_num[i]+i+list_sums_before[i]] - wifi_locations['time'][list_sums_before[i]+i])\n",
    "            uid.append(data.uid[i])\n",
    "            start_time.append(data.time[i])\n",
    "            location.append(data.location[i])\n",
    "        df1 = pd.DataFrame({'Time Delta': time_deltas, 'uid': uid, 'start time': start_time, 'location': location})\n",
    "    return df1\n",
    "\n",
    "#The function returns the new dataframe.\n",
    "variable = time_delta(dataframe, list_shifts)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "#For convenvience, I redefined the dataframe as \"df\"\n",
    "df = variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Time Delta</th>\n",
       "      <th>uid</th>\n",
       "      <th>start time</th>\n",
       "      <th>location</th>\n",
       "      <th>Time Delta (in seconds)</th>\n",
       "      <th>Time Delta (in minutes)</th>\n",
       "      <th>start time in regular notation</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>00:23:36</td>\n",
       "      <td>u45</td>\n",
       "      <td>1364397661</td>\n",
       "      <td>in[sudikoff]</td>\n",
       "      <td>1416.0</td>\n",
       "      <td>23.600000</td>\n",
       "      <td>2013-03-27 15:21:01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>00:20:58</td>\n",
       "      <td>u45</td>\n",
       "      <td>1364839542</td>\n",
       "      <td>in[sudikoff]</td>\n",
       "      <td>1258.0</td>\n",
       "      <td>20.966667</td>\n",
       "      <td>2013-04-01 18:05:42</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>00:03:44</td>\n",
       "      <td>u45</td>\n",
       "      <td>1364842439</td>\n",
       "      <td>in[lsb]</td>\n",
       "      <td>224.0</td>\n",
       "      <td>3.733333</td>\n",
       "      <td>2013-04-01 18:53:59</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>00:00:13</td>\n",
       "      <td>u45</td>\n",
       "      <td>1364848920</td>\n",
       "      <td>in[baker-berry]</td>\n",
       "      <td>13.0</td>\n",
       "      <td>0.216667</td>\n",
       "      <td>2013-04-01 20:42:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>00:00:40</td>\n",
       "      <td>u45</td>\n",
       "      <td>1364848920</td>\n",
       "      <td>in[baker-berry]</td>\n",
       "      <td>40.0</td>\n",
       "      <td>0.666667</td>\n",
       "      <td>2013-04-01 20:42:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>11881</td>\n",
       "      <td>00:01:39</td>\n",
       "      <td>u49</td>\n",
       "      <td>1366374288</td>\n",
       "      <td>in[baker-berry]</td>\n",
       "      <td>99.0</td>\n",
       "      <td>1.650000</td>\n",
       "      <td>2013-04-19 12:24:48</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>11882</td>\n",
       "      <td>00:04:48</td>\n",
       "      <td>u49</td>\n",
       "      <td>1366383400</td>\n",
       "      <td>in[kemeny]</td>\n",
       "      <td>288.0</td>\n",
       "      <td>4.800000</td>\n",
       "      <td>2013-04-19 14:56:40</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>11883</td>\n",
       "      <td>01:17:51</td>\n",
       "      <td>u49</td>\n",
       "      <td>1366384317</td>\n",
       "      <td>in[cummings]</td>\n",
       "      <td>4671.0</td>\n",
       "      <td>77.850000</td>\n",
       "      <td>2013-04-19 15:11:57</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>11884</td>\n",
       "      <td>08:49:00</td>\n",
       "      <td>u49</td>\n",
       "      <td>1366393411</td>\n",
       "      <td>in[sudikoff]</td>\n",
       "      <td>31740.0</td>\n",
       "      <td>529.000000</td>\n",
       "      <td>2013-04-19 17:43:31</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>11885</td>\n",
       "      <td>00:32:09</td>\n",
       "      <td>u49</td>\n",
       "      <td>1366415357</td>\n",
       "      <td>in[hopkins]</td>\n",
       "      <td>1929.0</td>\n",
       "      <td>32.150000</td>\n",
       "      <td>2013-04-19 23:49:17</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>11886 rows × 7 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      Time Delta  uid  start time         location  Time Delta (in seconds)  \\\n",
       "0       00:23:36  u45  1364397661     in[sudikoff]                   1416.0   \n",
       "1       00:20:58  u45  1364839542     in[sudikoff]                   1258.0   \n",
       "2       00:03:44  u45  1364842439          in[lsb]                    224.0   \n",
       "3       00:00:13  u45  1364848920  in[baker-berry]                     13.0   \n",
       "4       00:00:40  u45  1364848920  in[baker-berry]                     40.0   \n",
       "...          ...  ...         ...              ...                      ...   \n",
       "11881   00:01:39  u49  1366374288  in[baker-berry]                     99.0   \n",
       "11882   00:04:48  u49  1366383400       in[kemeny]                    288.0   \n",
       "11883   01:17:51  u49  1366384317     in[cummings]                   4671.0   \n",
       "11884   08:49:00  u49  1366393411     in[sudikoff]                  31740.0   \n",
       "11885   00:32:09  u49  1366415357      in[hopkins]                   1929.0   \n",
       "\n",
       "       Time Delta (in minutes) start time in regular notation  \n",
       "0                    23.600000            2013-03-27 15:21:01  \n",
       "1                    20.966667            2013-04-01 18:05:42  \n",
       "2                     3.733333            2013-04-01 18:53:59  \n",
       "3                     0.216667            2013-04-01 20:42:00  \n",
       "4                     0.666667            2013-04-01 20:42:00  \n",
       "...                        ...                            ...  \n",
       "11881                 1.650000            2013-04-19 12:24:48  \n",
       "11882                 4.800000            2013-04-19 14:56:40  \n",
       "11883                77.850000            2013-04-19 15:11:57  \n",
       "11884               529.000000            2013-04-19 17:43:31  \n",
       "11885                32.150000            2013-04-19 23:49:17  \n",
       "\n",
       "[11886 rows x 7 columns]"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#This code returned two new columns with the time deltas in seconds and minutes.\n",
    "#I converted the time deltas to minutes for convenience.\n",
    "df['Time Delta (in seconds)'] = df['Time Delta'].dt.total_seconds()\n",
    "df['Time Delta (in minutes)'] = df['Time Delta (in seconds)']/60\n",
    "df['start time in regular notation'] = pd.to_datetime(df['start time'], unit = 's')\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The next step of our analysis consists of actually finding the class attendance rates. I devised a methodology for this, and it involved finding the number of classes each student had on a particular weekday. My goal was to then find the number of expected attendances and then compare the number of actual attendances using the time deltas dataframe which we just found. From this, I can find class attendance as a percentage."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>u00</th>\n",
       "      <th>Unnamed: 1</th>\n",
       "      <th>Unnamed: 2</th>\n",
       "      <th>Unnamed: 3</th>\n",
       "      <th>Unnamed: 4</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>u01</td>\n",
       "      <td>ENGS 069</td>\n",
       "      <td>ENGS 022</td>\n",
       "      <td>ANTH 012</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>u02</td>\n",
       "      <td>COSC 077</td>\n",
       "      <td>COSC 098</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>u03</td>\n",
       "      <td>COSC 057</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>u04</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>u05</td>\n",
       "      <td>COSC 050</td>\n",
       "      <td>PSYC 028</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>5</td>\n",
       "      <td>u07</td>\n",
       "      <td>COSC 077</td>\n",
       "      <td>COSC 060</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>6</td>\n",
       "      <td>u08</td>\n",
       "      <td>CHIN 062</td>\n",
       "      <td>COSC 089 1</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>7</td>\n",
       "      <td>u09</td>\n",
       "      <td>ANTH 050</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>COSC 099</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>8</td>\n",
       "      <td>u10</td>\n",
       "      <td>COSC 050</td>\n",
       "      <td>BIOL 004</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>9</td>\n",
       "      <td>u12</td>\n",
       "      <td>COSC 089 1</td>\n",
       "      <td>COSC 050</td>\n",
       "      <td>TUCK 003</td>\n",
       "      <td>COSC 065</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>10</td>\n",
       "      <td>u13</td>\n",
       "      <td>COSC 070</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>11</td>\n",
       "      <td>u14</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>COSC 027</td>\n",
       "      <td>COSC 020</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>12</td>\n",
       "      <td>u15</td>\n",
       "      <td>EARS 003</td>\n",
       "      <td>SPAN 003</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>13</td>\n",
       "      <td>u16</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>COSC 027</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>14</td>\n",
       "      <td>u17</td>\n",
       "      <td>COSC 089 1</td>\n",
       "      <td>MUS 016</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>15</td>\n",
       "      <td>u18</td>\n",
       "      <td>COSC 089 1</td>\n",
       "      <td>CHIN 033</td>\n",
       "      <td>TUCK 003</td>\n",
       "      <td>COSC 065</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>16</td>\n",
       "      <td>u19</td>\n",
       "      <td>COSC 050</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>FILM 051</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>17</td>\n",
       "      <td>u20</td>\n",
       "      <td>COSC 070</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>18</td>\n",
       "      <td>u22</td>\n",
       "      <td>COSC 050</td>\n",
       "      <td>NAS 035</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>19</td>\n",
       "      <td>u23</td>\n",
       "      <td>COSC 070</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>20</td>\n",
       "      <td>u24</td>\n",
       "      <td>M&amp;SS 045</td>\n",
       "      <td>COSC 060</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>21</td>\n",
       "      <td>u25</td>\n",
       "      <td>NAS 008</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>ENGL 028</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>22</td>\n",
       "      <td>u27</td>\n",
       "      <td>ECON 024</td>\n",
       "      <td>JAPN 033</td>\n",
       "      <td>FILM 042</td>\n",
       "      <td>COSC 065</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>23</td>\n",
       "      <td>u30</td>\n",
       "      <td>MUS 003</td>\n",
       "      <td>COSC 050</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>24</td>\n",
       "      <td>u31</td>\n",
       "      <td>MUS 003</td>\n",
       "      <td>COSC 077</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>25</td>\n",
       "      <td>u32</td>\n",
       "      <td>MATH 023</td>\n",
       "      <td>COSC 069</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>26</td>\n",
       "      <td>u33</td>\n",
       "      <td>ENGS 025</td>\n",
       "      <td>ENGS 069</td>\n",
       "      <td>ENGS 093</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>27</td>\n",
       "      <td>u34</td>\n",
       "      <td>COSC 070</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>28</td>\n",
       "      <td>u35</td>\n",
       "      <td>COSC 070</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>29</td>\n",
       "      <td>u36</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>30</td>\n",
       "      <td>u39</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>31</td>\n",
       "      <td>u41</td>\n",
       "      <td>COSC 050</td>\n",
       "      <td>SPAN 002</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>32</td>\n",
       "      <td>u42</td>\n",
       "      <td>COSC 070</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>33</td>\n",
       "      <td>u43</td>\n",
       "      <td>ENGS 031</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>34</td>\n",
       "      <td>u44</td>\n",
       "      <td>COSC 060</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>35</td>\n",
       "      <td>u45</td>\n",
       "      <td>COSC 070</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>36</td>\n",
       "      <td>u46</td>\n",
       "      <td>ECON 036</td>\n",
       "      <td>COSC 050</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>37</td>\n",
       "      <td>u47</td>\n",
       "      <td>COSC 060</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>38</td>\n",
       "      <td>u49</td>\n",
       "      <td>MATH 013</td>\n",
       "      <td>LAT 003</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>39</td>\n",
       "      <td>u50</td>\n",
       "      <td>COSC 060</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>40</td>\n",
       "      <td>u51</td>\n",
       "      <td>COSC 070</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>41</td>\n",
       "      <td>u52</td>\n",
       "      <td>BIOL 006</td>\n",
       "      <td>COSC 050</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>42</td>\n",
       "      <td>u53</td>\n",
       "      <td>COSC 089 1</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>43</td>\n",
       "      <td>u54</td>\n",
       "      <td>ENGS 025</td>\n",
       "      <td>MATH 023</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>44</td>\n",
       "      <td>u56</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>45</td>\n",
       "      <td>u57</td>\n",
       "      <td>COSC 098</td>\n",
       "      <td>COSC 060</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>46</td>\n",
       "      <td>u58</td>\n",
       "      <td>COSC 070</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>47</td>\n",
       "      <td>u59</td>\n",
       "      <td>GERM 001</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>COSC 007</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    u00  Unnamed: 1  Unnamed: 2 Unnamed: 3 Unnamed: 4\n",
       "0   u01    ENGS 069    ENGS 022   ANTH 012        NaN\n",
       "1   u02    COSC 077    COSC 098   COSC 065        NaN\n",
       "2   u03    COSC 057    COSC 065        NaN        NaN\n",
       "3   u04    COSC 065         NaN        NaN        NaN\n",
       "4   u05    COSC 050    PSYC 028   COSC 065        NaN\n",
       "5   u07    COSC 077    COSC 060   COSC 065        NaN\n",
       "6   u08    CHIN 062  COSC 089 1   COSC 065        NaN\n",
       "7   u09    ANTH 050    COSC 065   COSC 099        NaN\n",
       "8   u10    COSC 050    BIOL 004   COSC 065        NaN\n",
       "9   u12  COSC 089 1    COSC 050   TUCK 003   COSC 065\n",
       "10  u13    COSC 070    COSC 065        NaN        NaN\n",
       "11  u14    COSC 065    COSC 027   COSC 020        NaN\n",
       "12  u15    EARS 003    SPAN 003   COSC 065        NaN\n",
       "13  u16    COSC 065    COSC 027        NaN        NaN\n",
       "14  u17  COSC 089 1     MUS 016   COSC 065        NaN\n",
       "15  u18  COSC 089 1    CHIN 033   TUCK 003   COSC 065\n",
       "16  u19    COSC 050    COSC 065   FILM 051        NaN\n",
       "17  u20    COSC 070    COSC 065        NaN        NaN\n",
       "18  u22    COSC 050     NAS 035   COSC 065        NaN\n",
       "19  u23    COSC 070    COSC 065        NaN        NaN\n",
       "20  u24    M&SS 045    COSC 060   COSC 065        NaN\n",
       "21  u25     NAS 008    COSC 065   ENGL 028        NaN\n",
       "22  u27    ECON 024    JAPN 033   FILM 042   COSC 065\n",
       "23  u30     MUS 003    COSC 050   COSC 065        NaN\n",
       "24  u31     MUS 003    COSC 077   COSC 065        NaN\n",
       "25  u32    MATH 023    COSC 069   COSC 065        NaN\n",
       "26  u33    ENGS 025    ENGS 069   ENGS 093        NaN\n",
       "27  u34    COSC 070    COSC 065        NaN        NaN\n",
       "28  u35    COSC 070    COSC 065        NaN        NaN\n",
       "29  u36         NaN         NaN        NaN        NaN\n",
       "30  u39         NaN         NaN        NaN        NaN\n",
       "31  u41    COSC 050    SPAN 002   COSC 065        NaN\n",
       "32  u42    COSC 070    COSC 065        NaN        NaN\n",
       "33  u43    ENGS 031    COSC 065        NaN        NaN\n",
       "34  u44    COSC 060    COSC 065        NaN        NaN\n",
       "35  u45    COSC 070    COSC 065        NaN        NaN\n",
       "36  u46    ECON 036    COSC 050   COSC 065        NaN\n",
       "37  u47    COSC 060    COSC 065        NaN        NaN\n",
       "38  u49    MATH 013     LAT 003   COSC 065        NaN\n",
       "39  u50    COSC 060    COSC 065        NaN        NaN\n",
       "40  u51    COSC 070    COSC 065        NaN        NaN\n",
       "41  u52    BIOL 006    COSC 050   COSC 065        NaN\n",
       "42  u53  COSC 089 1    COSC 065        NaN        NaN\n",
       "43  u54    ENGS 025    MATH 023   COSC 065        NaN\n",
       "44  u56         NaN         NaN        NaN        NaN\n",
       "45  u57    COSC 098    COSC 060   COSC 065        NaN\n",
       "46  u58    COSC 070    COSC 065        NaN        NaN\n",
       "47  u59    GERM 001    COSC 065   COSC 007        NaN"
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#This data looks at classes that students are taking over the semester. It's part of the StudentLife dataset.\n",
    "class_data = pd.read_csv(\"class_data.csv\")\n",
    "class_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Num Students in class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>ANTH 012</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>ANTH 050</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>BIOL 004</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>BIOL 006</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>CHIN 033</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>CHIN 062</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>COSC 007</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>COSC 020</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>COSC 027</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>COSC 050</td>\n",
       "      <td>9.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>COSC 057</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>COSC 060</td>\n",
       "      <td>6.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>COSC 065</td>\n",
       "      <td>43.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>COSC 069</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>COSC 070</td>\n",
       "      <td>9.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>COSC 077</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>COSC 089 1</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>COSC 098</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>COSC 099</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>EARS 003</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>ECON 024</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>ECON 036</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>ENGL 028</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>ENGS 022</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>ENGS 025</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>ENGS 031</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>ENGS 069</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>ENGS 093</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>FILM 042</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>FILM 051</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>GERM 001</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>JAPN 033</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>LAT 003</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>M&amp;SS 045</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>MATH 013</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>MATH 023</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>MUS 003</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>MUS 016</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>NAS 008</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>NAS 035</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>PSYC 028</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>SPAN 002</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>SPAN 003</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>TUCK 003</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            Num Students in class\n",
       "ANTH 012                      1.0\n",
       "ANTH 050                      1.0\n",
       "BIOL 004                      1.0\n",
       "BIOL 006                      1.0\n",
       "CHIN 033                      1.0\n",
       "CHIN 062                      1.0\n",
       "COSC 007                      1.0\n",
       "COSC 020                      1.0\n",
       "COSC 027                      2.0\n",
       "COSC 050                      9.0\n",
       "COSC 057                      1.0\n",
       "COSC 060                      6.0\n",
       "COSC 065                     43.0\n",
       "COSC 069                      1.0\n",
       "COSC 070                      9.0\n",
       "COSC 077                      3.0\n",
       "COSC 089 1                    5.0\n",
       "COSC 098                      2.0\n",
       "COSC 099                      1.0\n",
       "EARS 003                      1.0\n",
       "ECON 024                      1.0\n",
       "ECON 036                      1.0\n",
       "ENGL 028                      1.0\n",
       "ENGS 022                      1.0\n",
       "ENGS 025                      2.0\n",
       "ENGS 031                      1.0\n",
       "ENGS 069                      2.0\n",
       "ENGS 093                      1.0\n",
       "FILM 042                      1.0\n",
       "FILM 051                      1.0\n",
       "GERM 001                      1.0\n",
       "JAPN 033                      1.0\n",
       "LAT 003                       1.0\n",
       "M&SS 045                      1.0\n",
       "MATH 013                      1.0\n",
       "MATH 023                      2.0\n",
       "MUS 003                       2.0\n",
       "MUS 016                       1.0\n",
       "NAS 008                       1.0\n",
       "NAS 035                       1.0\n",
       "PSYC 028                      1.0\n",
       "SPAN 002                      1.0\n",
       "SPAN 003                      1.0\n",
       "TUCK 003                      2.0"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#This data looks at classes that students are taking over the semester.\n",
    "class_data = pd.read_csv(\"class_data.csv\")\n",
    "\n",
    "#To find the number of students in each class, I used the pd.value_counts function to do so.\n",
    "def num_students_each_class(data):\n",
    "    data = data.apply(pd.value_counts)\n",
    "    data = data.head(44)\n",
    "    #I created a column of the number of students in every class with the rows representing a class.\n",
    "    data['Num Students in class'] = data.sum(axis=1)\n",
    "    return data\n",
    "class_data = num_students_each_class(class_data)\n",
    "\n",
    "#The class_data dataframe had some irrelevant columns, which I deleted with the following code.\n",
    "del class_data['u00'], class_data['Unnamed: 1'], class_data['Unnamed: 2'], class_data['Unnamed: 3'], class_data['Unnamed: 4']\n",
    "\n",
    "#This dataframe has the number of students in each class\n",
    "class_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "#With this code, I created a new dataframe from the class_info dataframe.\n",
    "#This dataframe has the classes and periods as columns instead of rows. \n",
    "\n",
    "def get_class_list_prelim_periods(data):\n",
    "    list_classes = list(data.columns)\n",
    "    periods_data = list(data.iloc[1])\n",
    "    dataframe = pd.DataFrame({'Class': list_classes, 'periods': periods_data})\n",
    "    return dataframe\n",
    "class_info_df = get_class_list_prelim_periods(class_info)\n",
    "\n",
    "#Since some classes appeared in the class dataframe while others did not appear in the class info dataframe, I found the common classes amongst both dataframes using this function.\n",
    "def get_final_class_list(data1, data2):\n",
    "    list_classes_prelim = list(data1.index)\n",
    "    class_list = list(set(list_classes_prelim).intersection(list(data2.Class)))\n",
    "    return class_list\n",
    "class_list = get_final_class_list(class_data, class_info_df)\n",
    "periods_data = list(class_info.iloc[1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/anaconda3/lib/python3.7/site-packages/ipykernel_launcher.py:6: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  \n",
      "/opt/anaconda3/lib/python3.7/site-packages/ipykernel_launcher.py:7: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  import sys\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Num Students in class</th>\n",
       "      <th>periods</th>\n",
       "      <th>Class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>ANTH 012</td>\n",
       "      <td>1.0</td>\n",
       "      <td>[{'day': 1, 'end': '12:20', 'start': '11:15'},...</td>\n",
       "      <td>ANTH 012</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>BIOL 004</td>\n",
       "      <td>1.0</td>\n",
       "      <td>[{'day': 2, 'end': '11:50', 'start': '10:00'},...</td>\n",
       "      <td>BIOL 004</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>BIOL 006</td>\n",
       "      <td>1.0</td>\n",
       "      <td>[{'day': 2, 'end': '11:50', 'start': '10:00'},...</td>\n",
       "      <td>BIOL 006</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>CHIN 033</td>\n",
       "      <td>1.0</td>\n",
       "      <td>[{'day': 1, 'end': '09:50', 'start': '09:00'},...</td>\n",
       "      <td>CHIN 033</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>CHIN 062</td>\n",
       "      <td>1.0</td>\n",
       "      <td>[{'day': 2, 'end': '17:50', 'start': '16:00'},...</td>\n",
       "      <td>CHIN 062</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>COSC 007</td>\n",
       "      <td>1.0</td>\n",
       "      <td>[{'day': 1, 'end': '14:50', 'start': '13:45'},...</td>\n",
       "      <td>COSC 007</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>COSC 020</td>\n",
       "      <td>1.0</td>\n",
       "      <td>[{'day': 2, 'end': '15:50', 'start': '14:00'},...</td>\n",
       "      <td>COSC 020</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>COSC 027</td>\n",
       "      <td>2.0</td>\n",
       "      <td>[{'day': 1, 'end': '16:50', 'start': '15:00'},...</td>\n",
       "      <td>COSC 027</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>COSC 050</td>\n",
       "      <td>9.0</td>\n",
       "      <td>[{'day': 1, 'end': '13:35', 'start': '12:30'},...</td>\n",
       "      <td>COSC 050</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>COSC 057</td>\n",
       "      <td>1.0</td>\n",
       "      <td>[{'day': 1, 'end': '14:50', 'start': '13:45'},...</td>\n",
       "      <td>COSC 057</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>COSC 060</td>\n",
       "      <td>6.0</td>\n",
       "      <td>[{'day': 1, 'end': '14:50', 'start': '13:45'},...</td>\n",
       "      <td>COSC 060</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>COSC 065</td>\n",
       "      <td>43.0</td>\n",
       "      <td>[{'day': 1, 'end': '11:05', 'start': '10:00'},...</td>\n",
       "      <td>COSC 065</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>COSC 069</td>\n",
       "      <td>1.0</td>\n",
       "      <td>[{'day': 2, 'end': '17:50', 'start': '16:00'},...</td>\n",
       "      <td>COSC 069</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>COSC 070</td>\n",
       "      <td>9.0</td>\n",
       "      <td>[{'day': 2, 'end': '15:50', 'start': '14:00'},...</td>\n",
       "      <td>COSC 070</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>COSC 077</td>\n",
       "      <td>3.0</td>\n",
       "      <td>[{'day': 2, 'end': '15:50', 'start': '14:00'},...</td>\n",
       "      <td>COSC 077</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>COSC 089 1</td>\n",
       "      <td>5.0</td>\n",
       "      <td>[{'day': 1, 'end': '12:20', 'start': '11:15'},...</td>\n",
       "      <td>COSC 089 1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>EARS 003</td>\n",
       "      <td>1.0</td>\n",
       "      <td>[{'day': 1, 'end': '12:20', 'start': '11:15'},...</td>\n",
       "      <td>EARS 003</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>ECON 024</td>\n",
       "      <td>1.0</td>\n",
       "      <td>[{'day': 1, 'end': '09:50', 'start': '08:45'},...</td>\n",
       "      <td>ECON 024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>ECON 036</td>\n",
       "      <td>1.0</td>\n",
       "      <td>[{'day': 1, 'end': '09:50', 'start': '08:45'},...</td>\n",
       "      <td>ECON 036</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>ENGL 028</td>\n",
       "      <td>1.0</td>\n",
       "      <td>[{'day': 1, 'end': '09:50', 'start': '08:45'},...</td>\n",
       "      <td>ENGL 028</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>ENGS 022</td>\n",
       "      <td>1.0</td>\n",
       "      <td>[{'day': 1, 'end': '09:50', 'start': '08:45'},...</td>\n",
       "      <td>ENGS 022</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>ENGS 025</td>\n",
       "      <td>2.0</td>\n",
       "      <td>[{'day': 1, 'end': '14:50', 'start': '13:45'},...</td>\n",
       "      <td>ENGS 025</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>ENGS 069</td>\n",
       "      <td>2.0</td>\n",
       "      <td>[{'day': 1, 'end': '11:05', 'start': '10:00'},...</td>\n",
       "      <td>ENGS 069</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>ENGS 093</td>\n",
       "      <td>1.0</td>\n",
       "      <td>[{'day': 1, 'end': '13:35', 'start': '12:30'},...</td>\n",
       "      <td>ENGS 093</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>FILM 051</td>\n",
       "      <td>1.0</td>\n",
       "      <td>[{'day': 2, 'end': '11:50', 'start': '10:00'},...</td>\n",
       "      <td>FILM 051</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>GERM 001</td>\n",
       "      <td>1.0</td>\n",
       "      <td>[{'day': 1, 'end': '09:50', 'start': '09:00'},...</td>\n",
       "      <td>GERM 001</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>JAPN 033</td>\n",
       "      <td>1.0</td>\n",
       "      <td>[{'day': 1, 'end': '12:20', 'start': '11:15'},...</td>\n",
       "      <td>JAPN 033</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>LAT 003</td>\n",
       "      <td>1.0</td>\n",
       "      <td>[{'day': 1, 'end': '09:50', 'start': '08:45'},...</td>\n",
       "      <td>LAT 003</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>M&amp;SS 045</td>\n",
       "      <td>1.0</td>\n",
       "      <td>[{'day': 1, 'end': '09:50', 'start': '08:45'},...</td>\n",
       "      <td>M&amp;SS 045</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>MATH 013</td>\n",
       "      <td>1.0</td>\n",
       "      <td>[{'day': 1, 'end': '13:35', 'start': '12:30'},...</td>\n",
       "      <td>MATH 013</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>MATH 023</td>\n",
       "      <td>2.0</td>\n",
       "      <td>[{'day': 1, 'end': '12:20', 'start': '11:15'},...</td>\n",
       "      <td>MATH 023</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>MUS 003</td>\n",
       "      <td>2.0</td>\n",
       "      <td>[{'day': 2, 'end': '15:50', 'start': '14:00'},...</td>\n",
       "      <td>MUS 003</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>MUS 016</td>\n",
       "      <td>1.0</td>\n",
       "      <td>[{'day': 2, 'end': '11:50', 'start': '10:00'},...</td>\n",
       "      <td>MUS 016</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>NAS 008</td>\n",
       "      <td>1.0</td>\n",
       "      <td>[{'day': 1, 'end': '13:35', 'start': '12:30'},...</td>\n",
       "      <td>NAS 008</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>NAS 035</td>\n",
       "      <td>1.0</td>\n",
       "      <td>[{'day': 1, 'end': '12:20', 'start': '11:15'},...</td>\n",
       "      <td>NAS 035</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>PSYC 028</td>\n",
       "      <td>1.0</td>\n",
       "      <td>[{'day': 1, 'end': '14:50', 'start': '13:45'},...</td>\n",
       "      <td>PSYC 028</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>SPAN 003</td>\n",
       "      <td>1.0</td>\n",
       "      <td>[{'day': 1, 'end': '09:50', 'start': '09:00'},...</td>\n",
       "      <td>SPAN 003</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>TUCK 003</td>\n",
       "      <td>2.0</td>\n",
       "      <td>[{'day': 2, 'end': '15:50', 'start': '14:00'},...</td>\n",
       "      <td>TUCK 003</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            Num Students in class  \\\n",
       "ANTH 012                      1.0   \n",
       "BIOL 004                      1.0   \n",
       "BIOL 006                      1.0   \n",
       "CHIN 033                      1.0   \n",
       "CHIN 062                      1.0   \n",
       "COSC 007                      1.0   \n",
       "COSC 020                      1.0   \n",
       "COSC 027                      2.0   \n",
       "COSC 050                      9.0   \n",
       "COSC 057                      1.0   \n",
       "COSC 060                      6.0   \n",
       "COSC 065                     43.0   \n",
       "COSC 069                      1.0   \n",
       "COSC 070                      9.0   \n",
       "COSC 077                      3.0   \n",
       "COSC 089 1                    5.0   \n",
       "EARS 003                      1.0   \n",
       "ECON 024                      1.0   \n",
       "ECON 036                      1.0   \n",
       "ENGL 028                      1.0   \n",
       "ENGS 022                      1.0   \n",
       "ENGS 025                      2.0   \n",
       "ENGS 069                      2.0   \n",
       "ENGS 093                      1.0   \n",
       "FILM 051                      1.0   \n",
       "GERM 001                      1.0   \n",
       "JAPN 033                      1.0   \n",
       "LAT 003                       1.0   \n",
       "M&SS 045                      1.0   \n",
       "MATH 013                      1.0   \n",
       "MATH 023                      2.0   \n",
       "MUS 003                       2.0   \n",
       "MUS 016                       1.0   \n",
       "NAS 008                       1.0   \n",
       "NAS 035                       1.0   \n",
       "PSYC 028                      1.0   \n",
       "SPAN 003                      1.0   \n",
       "TUCK 003                      2.0   \n",
       "\n",
       "                                                      periods       Class  \n",
       "ANTH 012    [{'day': 1, 'end': '12:20', 'start': '11:15'},...    ANTH 012  \n",
       "BIOL 004    [{'day': 2, 'end': '11:50', 'start': '10:00'},...    BIOL 004  \n",
       "BIOL 006    [{'day': 2, 'end': '11:50', 'start': '10:00'},...    BIOL 006  \n",
       "CHIN 033    [{'day': 1, 'end': '09:50', 'start': '09:00'},...    CHIN 033  \n",
       "CHIN 062    [{'day': 2, 'end': '17:50', 'start': '16:00'},...    CHIN 062  \n",
       "COSC 007    [{'day': 1, 'end': '14:50', 'start': '13:45'},...    COSC 007  \n",
       "COSC 020    [{'day': 2, 'end': '15:50', 'start': '14:00'},...    COSC 020  \n",
       "COSC 027    [{'day': 1, 'end': '16:50', 'start': '15:00'},...    COSC 027  \n",
       "COSC 050    [{'day': 1, 'end': '13:35', 'start': '12:30'},...    COSC 050  \n",
       "COSC 057    [{'day': 1, 'end': '14:50', 'start': '13:45'},...    COSC 057  \n",
       "COSC 060    [{'day': 1, 'end': '14:50', 'start': '13:45'},...    COSC 060  \n",
       "COSC 065    [{'day': 1, 'end': '11:05', 'start': '10:00'},...    COSC 065  \n",
       "COSC 069    [{'day': 2, 'end': '17:50', 'start': '16:00'},...    COSC 069  \n",
       "COSC 070    [{'day': 2, 'end': '15:50', 'start': '14:00'},...    COSC 070  \n",
       "COSC 077    [{'day': 2, 'end': '15:50', 'start': '14:00'},...    COSC 077  \n",
       "COSC 089 1  [{'day': 1, 'end': '12:20', 'start': '11:15'},...  COSC 089 1  \n",
       "EARS 003    [{'day': 1, 'end': '12:20', 'start': '11:15'},...    EARS 003  \n",
       "ECON 024    [{'day': 1, 'end': '09:50', 'start': '08:45'},...    ECON 024  \n",
       "ECON 036    [{'day': 1, 'end': '09:50', 'start': '08:45'},...    ECON 036  \n",
       "ENGL 028    [{'day': 1, 'end': '09:50', 'start': '08:45'},...    ENGL 028  \n",
       "ENGS 022    [{'day': 1, 'end': '09:50', 'start': '08:45'},...    ENGS 022  \n",
       "ENGS 025    [{'day': 1, 'end': '14:50', 'start': '13:45'},...    ENGS 025  \n",
       "ENGS 069    [{'day': 1, 'end': '11:05', 'start': '10:00'},...    ENGS 069  \n",
       "ENGS 093    [{'day': 1, 'end': '13:35', 'start': '12:30'},...    ENGS 093  \n",
       "FILM 051    [{'day': 2, 'end': '11:50', 'start': '10:00'},...    FILM 051  \n",
       "GERM 001    [{'day': 1, 'end': '09:50', 'start': '09:00'},...    GERM 001  \n",
       "JAPN 033    [{'day': 1, 'end': '12:20', 'start': '11:15'},...    JAPN 033  \n",
       "LAT 003     [{'day': 1, 'end': '09:50', 'start': '08:45'},...     LAT 003  \n",
       "M&SS 045    [{'day': 1, 'end': '09:50', 'start': '08:45'},...    M&SS 045  \n",
       "MATH 013    [{'day': 1, 'end': '13:35', 'start': '12:30'},...    MATH 013  \n",
       "MATH 023    [{'day': 1, 'end': '12:20', 'start': '11:15'},...    MATH 023  \n",
       "MUS 003     [{'day': 2, 'end': '15:50', 'start': '14:00'},...     MUS 003  \n",
       "MUS 016     [{'day': 2, 'end': '11:50', 'start': '10:00'},...     MUS 016  \n",
       "NAS 008     [{'day': 1, 'end': '13:35', 'start': '12:30'},...     NAS 008  \n",
       "NAS 035     [{'day': 1, 'end': '12:20', 'start': '11:15'},...     NAS 035  \n",
       "PSYC 028    [{'day': 1, 'end': '14:50', 'start': '13:45'},...    PSYC 028  \n",
       "SPAN 003    [{'day': 1, 'end': '09:50', 'start': '09:00'},...    SPAN 003  \n",
       "TUCK 003    [{'day': 2, 'end': '15:50', 'start': '14:00'},...    TUCK 003  "
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Once I had the common classes shared between both dataframes, I merged them together.\n",
    "def get_final_class_data(data1, data2, list1):\n",
    "    data2 = data2[data2.Class.isin(list1)]\n",
    "    data1 = data1[data1.index.isin(list1)]\n",
    "    periods = list(data2.periods)\n",
    "    data1['periods'] = periods\n",
    "    data1[\"Class\"] = data1.index\n",
    "    return data1\n",
    "class_data = get_final_class_data(class_data, class_info_df, class_list)\n",
    "\n",
    "#This final dataframe has information for class type, number of students in each class, and when the class occurs\n",
    "class_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "#This function returns a dataframe with two columns: Days, representing weekdays from 1-5 (Mon-Fri) and Class.\n",
    "#The dataframe that is returned has the days in which each class takes place. \n",
    "#However, each individual day has its own row, with the corresponding class.\n",
    "def get_class_days(data):\n",
    "    lst_days = []\n",
    "    lst_class = []\n",
    "    for i in list(range(0, len(data['periods']))):\n",
    "        for j in data['periods'][i]:\n",
    "            try:\n",
    "                lst_days.append(j['day'])\n",
    "                lst_class.append(data.Class[i])\n",
    "            except:\n",
    "                lst_days.append('Error')\n",
    "    df_class_days = pd.DataFrame({'Days': lst_days, 'Class': lst_class})\n",
    "    return df_class_days\n",
    "df_class_days = get_class_days(class_data)\n",
    "#df_class_days"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Days</th>\n",
       "      <th>Class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>ANTH 012</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>ANTH 012</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>5</td>\n",
       "      <td>ANTH 012</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>ANTH 012</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>BIOL 004</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>138</td>\n",
       "      <td>5</td>\n",
       "      <td>SPAN 003</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>139</td>\n",
       "      <td>3</td>\n",
       "      <td>SPAN 003</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>140</td>\n",
       "      <td>2</td>\n",
       "      <td>TUCK 003</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>141</td>\n",
       "      <td>4</td>\n",
       "      <td>TUCK 003</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>142</td>\n",
       "      <td>3</td>\n",
       "      <td>TUCK 003</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>143 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     Days     Class\n",
       "0       1  ANTH 012\n",
       "1       3  ANTH 012\n",
       "2       5  ANTH 012\n",
       "3       2  ANTH 012\n",
       "4       2  BIOL 004\n",
       "..    ...       ...\n",
       "138     5  SPAN 003\n",
       "139     3  SPAN 003\n",
       "140     2  TUCK 003\n",
       "141     4  TUCK 003\n",
       "142     3  TUCK 003\n",
       "\n",
       "[143 rows x 2 columns]"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_class_days"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we can observe the dataframe: if a class is on MWF, it will appear three times in the dataframe with the \"Days\" as 1, 3 and 5 since that is how MWF is represented. However, this was suboptimal, so we optimized it show that each row contains the days (as an array) that each class occurs on."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>index</th>\n",
       "      <th>Days</th>\n",
       "      <th>Class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>ANTH 012</td>\n",
       "      <td>[1, 3, 5, 2]</td>\n",
       "      <td>ANTH 012</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>BIOL 004</td>\n",
       "      <td>[2, 4, 3]</td>\n",
       "      <td>BIOL 004</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>BIOL 006</td>\n",
       "      <td>[2, 4, 3]</td>\n",
       "      <td>BIOL 006</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>CHIN 033</td>\n",
       "      <td>[1, 2, 4, 5, 3]</td>\n",
       "      <td>CHIN 033</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>CHIN 062</td>\n",
       "      <td>[2, 4, 1]</td>\n",
       "      <td>CHIN 062</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      index             Days     Class\n",
       "0  ANTH 012     [1, 3, 5, 2]  ANTH 012\n",
       "1  BIOL 004        [2, 4, 3]  BIOL 004\n",
       "2  BIOL 006        [2, 4, 3]  BIOL 006\n",
       "3  CHIN 033  [1, 2, 4, 5, 3]  CHIN 033\n",
       "4  CHIN 062        [2, 4, 1]  CHIN 062"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#This dataframe improves upon the previous dataframe by allocating all of the days in which each class ocurrs into a single row.\n",
    "def get_class_days_array(data):\n",
    "    group = data.groupby('Class')\n",
    "    df2 = group.apply(lambda x: x['Days'].unique())\n",
    "    df3 = pd.DataFrame({'Days': list(df2), 'Class': class_data.Class})\n",
    "    df_class_days_final = df3.reset_index()\n",
    "    return df_class_days_final\n",
    "df_class_days_final = get_class_days_array(df_class_days)\n",
    "#This dataframe has a column of arrays of numbers from 1-5 representing the days of Mon-Fri.\n",
    "#Each array represents the days of the week in which the corresponding class takes place.\n",
    "df_class_days_final.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ahh, this is much better! Let's move on. For our next step, we have to read the class_data file from the dataset once again."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "class_2 = pd.read_csv('class_data.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "#This function will be used to make a new column in the class_2 dataframe. \n",
    "#This new column will have arrays of the classes that each student is taking over the semester.\n",
    "def class_p(class_series): \n",
    "    class_series = class_series.dropna()\n",
    "    class_list = []\n",
    "    for class_4 in class_series.values: \n",
    "        if class_4[0] == 'u': \n",
    "            continue\n",
    "        else: \n",
    "            class_list.append(class_4)\n",
    "    return class_list\n",
    "class_2['total'] = class_2.apply(func = class_p, axis = 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>u00</th>\n",
       "      <th>Unnamed: 1</th>\n",
       "      <th>Unnamed: 2</th>\n",
       "      <th>Unnamed: 3</th>\n",
       "      <th>Unnamed: 4</th>\n",
       "      <th>total</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>u01</td>\n",
       "      <td>ENGS 069</td>\n",
       "      <td>ENGS 022</td>\n",
       "      <td>ANTH 012</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[ENGS 069, ENGS 022, ANTH 012]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>u02</td>\n",
       "      <td>COSC 077</td>\n",
       "      <td>COSC 098</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 077, COSC 098, COSC 065]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>u03</td>\n",
       "      <td>COSC 057</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 057, COSC 065]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>u04</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 065]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>u05</td>\n",
       "      <td>COSC 050</td>\n",
       "      <td>PSYC 028</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 050, PSYC 028, COSC 065]</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   u00 Unnamed: 1 Unnamed: 2 Unnamed: 3 Unnamed: 4  \\\n",
       "0  u01   ENGS 069   ENGS 022   ANTH 012        NaN   \n",
       "1  u02   COSC 077   COSC 098   COSC 065        NaN   \n",
       "2  u03   COSC 057   COSC 065        NaN        NaN   \n",
       "3  u04   COSC 065        NaN        NaN        NaN   \n",
       "4  u05   COSC 050   PSYC 028   COSC 065        NaN   \n",
       "\n",
       "                            total  \n",
       "0  [ENGS 069, ENGS 022, ANTH 012]  \n",
       "1  [COSC 077, COSC 098, COSC 065]  \n",
       "2            [COSC 057, COSC 065]  \n",
       "3                      [COSC 065]  \n",
       "4  [COSC 050, PSYC 028, COSC 065]  "
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "class_2.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see, the function from before added a new column that has every class that each student is taking as an array. Let's also add a column for the days on which these classes occur. We already have this data from before, so we just have to add it in:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>u00</th>\n",
       "      <th>Unnamed: 1</th>\n",
       "      <th>Unnamed: 2</th>\n",
       "      <th>Unnamed: 3</th>\n",
       "      <th>Unnamed: 4</th>\n",
       "      <th>total</th>\n",
       "      <th>days</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>u01</td>\n",
       "      <td>ENGS 069</td>\n",
       "      <td>ENGS 022</td>\n",
       "      <td>ANTH 012</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[ENGS 069, ENGS 022, ANTH 012]</td>\n",
       "      <td>[[1, 3, 5, 4], [1, 3, 5, 4], [1, 3, 5, 2]]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>u02</td>\n",
       "      <td>COSC 077</td>\n",
       "      <td>COSC 098</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 077, COSC 098, COSC 065]</td>\n",
       "      <td>[[2, 4, 3], [1, 3, 5, 4]]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>u03</td>\n",
       "      <td>COSC 057</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 057, COSC 065]</td>\n",
       "      <td>[[1, 3, 5, 4], [1, 3, 5, 4]]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>u04</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 065]</td>\n",
       "      <td>[[1, 3, 5, 4]]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>u05</td>\n",
       "      <td>COSC 050</td>\n",
       "      <td>PSYC 028</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 050, PSYC 028, COSC 065]</td>\n",
       "      <td>[[1, 3, 5, 2], [1, 3, 5, 4], [1, 3, 5, 4]]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>5</td>\n",
       "      <td>u07</td>\n",
       "      <td>COSC 077</td>\n",
       "      <td>COSC 060</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 077, COSC 060, COSC 065]</td>\n",
       "      <td>[[2, 4, 3], [1, 3, 5, 4], [1, 3, 5, 4]]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>6</td>\n",
       "      <td>u08</td>\n",
       "      <td>CHIN 062</td>\n",
       "      <td>COSC 089 1</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[CHIN 062, COSC 089 1, COSC 065]</td>\n",
       "      <td>[[2, 4, 1], [1, 3, 5, 2], [1, 3, 5, 4]]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>7</td>\n",
       "      <td>u09</td>\n",
       "      <td>ANTH 050</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>COSC 099</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[ANTH 050, COSC 065, COSC 099]</td>\n",
       "      <td>[[1, 3, 5, 4]]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>8</td>\n",
       "      <td>u10</td>\n",
       "      <td>COSC 050</td>\n",
       "      <td>BIOL 004</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 050, BIOL 004, COSC 065]</td>\n",
       "      <td>[[1, 3, 5, 2], [2, 4, 3], [1, 3, 5, 4]]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>9</td>\n",
       "      <td>u12</td>\n",
       "      <td>COSC 089 1</td>\n",
       "      <td>COSC 050</td>\n",
       "      <td>TUCK 003</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>[COSC 089 1, COSC 050, TUCK 003, COSC 065]</td>\n",
       "      <td>[[1, 3, 5, 2], [1, 3, 5, 2], [2, 4, 3], [1, 3,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>10</td>\n",
       "      <td>u13</td>\n",
       "      <td>COSC 070</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 070, COSC 065]</td>\n",
       "      <td>[[2, 4, 3], [1, 3, 5, 4]]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>11</td>\n",
       "      <td>u14</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>COSC 027</td>\n",
       "      <td>COSC 020</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 065, COSC 027, COSC 020]</td>\n",
       "      <td>[[1, 3, 5, 4], [1, 4], [2, 4, 3]]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>12</td>\n",
       "      <td>u15</td>\n",
       "      <td>EARS 003</td>\n",
       "      <td>SPAN 003</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[EARS 003, SPAN 003, COSC 065]</td>\n",
       "      <td>[[1, 3, 5, 2], [1, 2, 4, 5, 3], [1, 3, 5, 4]]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>13</td>\n",
       "      <td>u16</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>COSC 027</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 065, COSC 027]</td>\n",
       "      <td>[[1, 3, 5, 4], [1, 4]]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>14</td>\n",
       "      <td>u17</td>\n",
       "      <td>COSC 089 1</td>\n",
       "      <td>MUS 016</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 089 1, MUS 016, COSC 065]</td>\n",
       "      <td>[[1, 3, 5, 2], [2, 4, 3], [1, 3, 5, 4]]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>15</td>\n",
       "      <td>u18</td>\n",
       "      <td>COSC 089 1</td>\n",
       "      <td>CHIN 033</td>\n",
       "      <td>TUCK 003</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>[COSC 089 1, CHIN 033, TUCK 003, COSC 065]</td>\n",
       "      <td>[[1, 3, 5, 2], [1, 2, 4, 5, 3], [2, 4, 3], [1,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>16</td>\n",
       "      <td>u19</td>\n",
       "      <td>COSC 050</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>FILM 051</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 050, COSC 065, FILM 051]</td>\n",
       "      <td>[[1, 3, 5, 2], [1, 3, 5, 4], [2, 4, 3]]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>17</td>\n",
       "      <td>u20</td>\n",
       "      <td>COSC 070</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 070, COSC 065]</td>\n",
       "      <td>[[2, 4, 3], [1, 3, 5, 4]]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>18</td>\n",
       "      <td>u22</td>\n",
       "      <td>COSC 050</td>\n",
       "      <td>NAS 035</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 050, NAS 035, COSC 065]</td>\n",
       "      <td>[[1, 3, 5, 2], [1, 3, 5, 2], [1, 3, 5, 4]]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>19</td>\n",
       "      <td>u23</td>\n",
       "      <td>COSC 070</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 070, COSC 065]</td>\n",
       "      <td>[[2, 4, 3], [1, 3, 5, 4]]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>20</td>\n",
       "      <td>u24</td>\n",
       "      <td>M&amp;SS 045</td>\n",
       "      <td>COSC 060</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[M&amp;SS 045, COSC 060, COSC 065]</td>\n",
       "      <td>[[1, 3, 5, 4], [1, 3, 5, 4], [1, 3, 5, 4]]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>21</td>\n",
       "      <td>u25</td>\n",
       "      <td>NAS 008</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>ENGL 028</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[NAS 008, COSC 065, ENGL 028]</td>\n",
       "      <td>[[1, 3, 5, 2], [1, 3, 5, 4], [1, 3, 5, 4]]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>22</td>\n",
       "      <td>u27</td>\n",
       "      <td>ECON 024</td>\n",
       "      <td>JAPN 033</td>\n",
       "      <td>FILM 042</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>[ECON 024, JAPN 033, FILM 042, COSC 065]</td>\n",
       "      <td>[[1, 3, 5, 4], [1, 3, 5, 2], [1, 3, 5, 4]]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>23</td>\n",
       "      <td>u30</td>\n",
       "      <td>MUS 003</td>\n",
       "      <td>COSC 050</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[MUS 003, COSC 050, COSC 065]</td>\n",
       "      <td>[[2, 4, 3], [1, 3, 5, 2], [1, 3, 5, 4]]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>24</td>\n",
       "      <td>u31</td>\n",
       "      <td>MUS 003</td>\n",
       "      <td>COSC 077</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[MUS 003, COSC 077, COSC 065]</td>\n",
       "      <td>[[2, 4, 3], [2, 4, 3], [1, 3, 5, 4]]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>25</td>\n",
       "      <td>u32</td>\n",
       "      <td>MATH 023</td>\n",
       "      <td>COSC 069</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[MATH 023, COSC 069, COSC 065]</td>\n",
       "      <td>[[1, 3, 5, 2], [2, 4, 1], [1, 3, 5, 4]]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>26</td>\n",
       "      <td>u33</td>\n",
       "      <td>ENGS 025</td>\n",
       "      <td>ENGS 069</td>\n",
       "      <td>ENGS 093</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[ENGS 025, ENGS 069, ENGS 093]</td>\n",
       "      <td>[[1, 3, 5, 4], [1, 3, 5, 4], [1, 3, 5, 2]]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>27</td>\n",
       "      <td>u34</td>\n",
       "      <td>COSC 070</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 070, COSC 065]</td>\n",
       "      <td>[[2, 4, 3], [1, 3, 5, 4]]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>28</td>\n",
       "      <td>u35</td>\n",
       "      <td>COSC 070</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 070, COSC 065]</td>\n",
       "      <td>[[2, 4, 3], [1, 3, 5, 4]]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>29</td>\n",
       "      <td>u36</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[]</td>\n",
       "      <td>[]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>30</td>\n",
       "      <td>u39</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[]</td>\n",
       "      <td>[]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>31</td>\n",
       "      <td>u41</td>\n",
       "      <td>COSC 050</td>\n",
       "      <td>SPAN 002</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 050, SPAN 002, COSC 065]</td>\n",
       "      <td>[[1, 3, 5, 2], [1, 3, 5, 4]]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>32</td>\n",
       "      <td>u42</td>\n",
       "      <td>COSC 070</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 070, COSC 065]</td>\n",
       "      <td>[[2, 4, 3], [1, 3, 5, 4]]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>33</td>\n",
       "      <td>u43</td>\n",
       "      <td>ENGS 031</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[ENGS 031, COSC 065]</td>\n",
       "      <td>[[1, 3, 5, 4]]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>34</td>\n",
       "      <td>u44</td>\n",
       "      <td>COSC 060</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 060, COSC 065]</td>\n",
       "      <td>[[1, 3, 5, 4], [1, 3, 5, 4]]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>35</td>\n",
       "      <td>u45</td>\n",
       "      <td>COSC 070</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 070, COSC 065]</td>\n",
       "      <td>[[2, 4, 3], [1, 3, 5, 4]]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>36</td>\n",
       "      <td>u46</td>\n",
       "      <td>ECON 036</td>\n",
       "      <td>COSC 050</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[ECON 036, COSC 050, COSC 065]</td>\n",
       "      <td>[[1, 3, 5, 4], [1, 3, 5, 2], [1, 3, 5, 4]]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>37</td>\n",
       "      <td>u47</td>\n",
       "      <td>COSC 060</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 060, COSC 065]</td>\n",
       "      <td>[[1, 3, 5, 4], [1, 3, 5, 4]]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>38</td>\n",
       "      <td>u49</td>\n",
       "      <td>MATH 013</td>\n",
       "      <td>LAT 003</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[MATH 013, LAT 003, COSC 065]</td>\n",
       "      <td>[[1, 3, 5, 2], [1, 3, 5, 4], [1, 3, 5, 4]]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>39</td>\n",
       "      <td>u50</td>\n",
       "      <td>COSC 060</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 060, COSC 065]</td>\n",
       "      <td>[[1, 3, 5, 4], [1, 3, 5, 4]]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>40</td>\n",
       "      <td>u51</td>\n",
       "      <td>COSC 070</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 070, COSC 065]</td>\n",
       "      <td>[[2, 4, 3], [1, 3, 5, 4]]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>41</td>\n",
       "      <td>u52</td>\n",
       "      <td>BIOL 006</td>\n",
       "      <td>COSC 050</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[BIOL 006, COSC 050, COSC 065]</td>\n",
       "      <td>[[2, 4, 3], [1, 3, 5, 2], [1, 3, 5, 4]]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>42</td>\n",
       "      <td>u53</td>\n",
       "      <td>COSC 089 1</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 089 1, COSC 065]</td>\n",
       "      <td>[[1, 3, 5, 2], [1, 3, 5, 4]]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>43</td>\n",
       "      <td>u54</td>\n",
       "      <td>ENGS 025</td>\n",
       "      <td>MATH 023</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[ENGS 025, MATH 023, COSC 065]</td>\n",
       "      <td>[[1, 3, 5, 4], [1, 3, 5, 2], [1, 3, 5, 4]]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>44</td>\n",
       "      <td>u56</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[]</td>\n",
       "      <td>[]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>45</td>\n",
       "      <td>u57</td>\n",
       "      <td>COSC 098</td>\n",
       "      <td>COSC 060</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 098, COSC 060, COSC 065]</td>\n",
       "      <td>[[1, 3, 5, 4], [1, 3, 5, 4]]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>46</td>\n",
       "      <td>u58</td>\n",
       "      <td>COSC 070</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 070, COSC 065]</td>\n",
       "      <td>[[2, 4, 3], [1, 3, 5, 4]]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>47</td>\n",
       "      <td>u59</td>\n",
       "      <td>GERM 001</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>COSC 007</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[GERM 001, COSC 065, COSC 007]</td>\n",
       "      <td>[[1, 2, 4, 5, 3], [1, 3, 5, 4], [1, 3, 5, 4]]</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    u00  Unnamed: 1  Unnamed: 2 Unnamed: 3 Unnamed: 4  \\\n",
       "0   u01    ENGS 069    ENGS 022   ANTH 012        NaN   \n",
       "1   u02    COSC 077    COSC 098   COSC 065        NaN   \n",
       "2   u03    COSC 057    COSC 065        NaN        NaN   \n",
       "3   u04    COSC 065         NaN        NaN        NaN   \n",
       "4   u05    COSC 050    PSYC 028   COSC 065        NaN   \n",
       "5   u07    COSC 077    COSC 060   COSC 065        NaN   \n",
       "6   u08    CHIN 062  COSC 089 1   COSC 065        NaN   \n",
       "7   u09    ANTH 050    COSC 065   COSC 099        NaN   \n",
       "8   u10    COSC 050    BIOL 004   COSC 065        NaN   \n",
       "9   u12  COSC 089 1    COSC 050   TUCK 003   COSC 065   \n",
       "10  u13    COSC 070    COSC 065        NaN        NaN   \n",
       "11  u14    COSC 065    COSC 027   COSC 020        NaN   \n",
       "12  u15    EARS 003    SPAN 003   COSC 065        NaN   \n",
       "13  u16    COSC 065    COSC 027        NaN        NaN   \n",
       "14  u17  COSC 089 1     MUS 016   COSC 065        NaN   \n",
       "15  u18  COSC 089 1    CHIN 033   TUCK 003   COSC 065   \n",
       "16  u19    COSC 050    COSC 065   FILM 051        NaN   \n",
       "17  u20    COSC 070    COSC 065        NaN        NaN   \n",
       "18  u22    COSC 050     NAS 035   COSC 065        NaN   \n",
       "19  u23    COSC 070    COSC 065        NaN        NaN   \n",
       "20  u24    M&SS 045    COSC 060   COSC 065        NaN   \n",
       "21  u25     NAS 008    COSC 065   ENGL 028        NaN   \n",
       "22  u27    ECON 024    JAPN 033   FILM 042   COSC 065   \n",
       "23  u30     MUS 003    COSC 050   COSC 065        NaN   \n",
       "24  u31     MUS 003    COSC 077   COSC 065        NaN   \n",
       "25  u32    MATH 023    COSC 069   COSC 065        NaN   \n",
       "26  u33    ENGS 025    ENGS 069   ENGS 093        NaN   \n",
       "27  u34    COSC 070    COSC 065        NaN        NaN   \n",
       "28  u35    COSC 070    COSC 065        NaN        NaN   \n",
       "29  u36         NaN         NaN        NaN        NaN   \n",
       "30  u39         NaN         NaN        NaN        NaN   \n",
       "31  u41    COSC 050    SPAN 002   COSC 065        NaN   \n",
       "32  u42    COSC 070    COSC 065        NaN        NaN   \n",
       "33  u43    ENGS 031    COSC 065        NaN        NaN   \n",
       "34  u44    COSC 060    COSC 065        NaN        NaN   \n",
       "35  u45    COSC 070    COSC 065        NaN        NaN   \n",
       "36  u46    ECON 036    COSC 050   COSC 065        NaN   \n",
       "37  u47    COSC 060    COSC 065        NaN        NaN   \n",
       "38  u49    MATH 013     LAT 003   COSC 065        NaN   \n",
       "39  u50    COSC 060    COSC 065        NaN        NaN   \n",
       "40  u51    COSC 070    COSC 065        NaN        NaN   \n",
       "41  u52    BIOL 006    COSC 050   COSC 065        NaN   \n",
       "42  u53  COSC 089 1    COSC 065        NaN        NaN   \n",
       "43  u54    ENGS 025    MATH 023   COSC 065        NaN   \n",
       "44  u56         NaN         NaN        NaN        NaN   \n",
       "45  u57    COSC 098    COSC 060   COSC 065        NaN   \n",
       "46  u58    COSC 070    COSC 065        NaN        NaN   \n",
       "47  u59    GERM 001    COSC 065   COSC 007        NaN   \n",
       "\n",
       "                                         total  \\\n",
       "0               [ENGS 069, ENGS 022, ANTH 012]   \n",
       "1               [COSC 077, COSC 098, COSC 065]   \n",
       "2                         [COSC 057, COSC 065]   \n",
       "3                                   [COSC 065]   \n",
       "4               [COSC 050, PSYC 028, COSC 065]   \n",
       "5               [COSC 077, COSC 060, COSC 065]   \n",
       "6             [CHIN 062, COSC 089 1, COSC 065]   \n",
       "7               [ANTH 050, COSC 065, COSC 099]   \n",
       "8               [COSC 050, BIOL 004, COSC 065]   \n",
       "9   [COSC 089 1, COSC 050, TUCK 003, COSC 065]   \n",
       "10                        [COSC 070, COSC 065]   \n",
       "11              [COSC 065, COSC 027, COSC 020]   \n",
       "12              [EARS 003, SPAN 003, COSC 065]   \n",
       "13                        [COSC 065, COSC 027]   \n",
       "14             [COSC 089 1, MUS 016, COSC 065]   \n",
       "15  [COSC 089 1, CHIN 033, TUCK 003, COSC 065]   \n",
       "16              [COSC 050, COSC 065, FILM 051]   \n",
       "17                        [COSC 070, COSC 065]   \n",
       "18               [COSC 050, NAS 035, COSC 065]   \n",
       "19                        [COSC 070, COSC 065]   \n",
       "20              [M&SS 045, COSC 060, COSC 065]   \n",
       "21               [NAS 008, COSC 065, ENGL 028]   \n",
       "22    [ECON 024, JAPN 033, FILM 042, COSC 065]   \n",
       "23               [MUS 003, COSC 050, COSC 065]   \n",
       "24               [MUS 003, COSC 077, COSC 065]   \n",
       "25              [MATH 023, COSC 069, COSC 065]   \n",
       "26              [ENGS 025, ENGS 069, ENGS 093]   \n",
       "27                        [COSC 070, COSC 065]   \n",
       "28                        [COSC 070, COSC 065]   \n",
       "29                                          []   \n",
       "30                                          []   \n",
       "31              [COSC 050, SPAN 002, COSC 065]   \n",
       "32                        [COSC 070, COSC 065]   \n",
       "33                        [ENGS 031, COSC 065]   \n",
       "34                        [COSC 060, COSC 065]   \n",
       "35                        [COSC 070, COSC 065]   \n",
       "36              [ECON 036, COSC 050, COSC 065]   \n",
       "37                        [COSC 060, COSC 065]   \n",
       "38               [MATH 013, LAT 003, COSC 065]   \n",
       "39                        [COSC 060, COSC 065]   \n",
       "40                        [COSC 070, COSC 065]   \n",
       "41              [BIOL 006, COSC 050, COSC 065]   \n",
       "42                      [COSC 089 1, COSC 065]   \n",
       "43              [ENGS 025, MATH 023, COSC 065]   \n",
       "44                                          []   \n",
       "45              [COSC 098, COSC 060, COSC 065]   \n",
       "46                        [COSC 070, COSC 065]   \n",
       "47              [GERM 001, COSC 065, COSC 007]   \n",
       "\n",
       "                                                 days  \n",
       "0          [[1, 3, 5, 4], [1, 3, 5, 4], [1, 3, 5, 2]]  \n",
       "1                           [[2, 4, 3], [1, 3, 5, 4]]  \n",
       "2                        [[1, 3, 5, 4], [1, 3, 5, 4]]  \n",
       "3                                      [[1, 3, 5, 4]]  \n",
       "4          [[1, 3, 5, 2], [1, 3, 5, 4], [1, 3, 5, 4]]  \n",
       "5             [[2, 4, 3], [1, 3, 5, 4], [1, 3, 5, 4]]  \n",
       "6             [[2, 4, 1], [1, 3, 5, 2], [1, 3, 5, 4]]  \n",
       "7                                      [[1, 3, 5, 4]]  \n",
       "8             [[1, 3, 5, 2], [2, 4, 3], [1, 3, 5, 4]]  \n",
       "9   [[1, 3, 5, 2], [1, 3, 5, 2], [2, 4, 3], [1, 3,...  \n",
       "10                          [[2, 4, 3], [1, 3, 5, 4]]  \n",
       "11                  [[1, 3, 5, 4], [1, 4], [2, 4, 3]]  \n",
       "12      [[1, 3, 5, 2], [1, 2, 4, 5, 3], [1, 3, 5, 4]]  \n",
       "13                             [[1, 3, 5, 4], [1, 4]]  \n",
       "14            [[1, 3, 5, 2], [2, 4, 3], [1, 3, 5, 4]]  \n",
       "15  [[1, 3, 5, 2], [1, 2, 4, 5, 3], [2, 4, 3], [1,...  \n",
       "16            [[1, 3, 5, 2], [1, 3, 5, 4], [2, 4, 3]]  \n",
       "17                          [[2, 4, 3], [1, 3, 5, 4]]  \n",
       "18         [[1, 3, 5, 2], [1, 3, 5, 2], [1, 3, 5, 4]]  \n",
       "19                          [[2, 4, 3], [1, 3, 5, 4]]  \n",
       "20         [[1, 3, 5, 4], [1, 3, 5, 4], [1, 3, 5, 4]]  \n",
       "21         [[1, 3, 5, 2], [1, 3, 5, 4], [1, 3, 5, 4]]  \n",
       "22         [[1, 3, 5, 4], [1, 3, 5, 2], [1, 3, 5, 4]]  \n",
       "23            [[2, 4, 3], [1, 3, 5, 2], [1, 3, 5, 4]]  \n",
       "24               [[2, 4, 3], [2, 4, 3], [1, 3, 5, 4]]  \n",
       "25            [[1, 3, 5, 2], [2, 4, 1], [1, 3, 5, 4]]  \n",
       "26         [[1, 3, 5, 4], [1, 3, 5, 4], [1, 3, 5, 2]]  \n",
       "27                          [[2, 4, 3], [1, 3, 5, 4]]  \n",
       "28                          [[2, 4, 3], [1, 3, 5, 4]]  \n",
       "29                                                 []  \n",
       "30                                                 []  \n",
       "31                       [[1, 3, 5, 2], [1, 3, 5, 4]]  \n",
       "32                          [[2, 4, 3], [1, 3, 5, 4]]  \n",
       "33                                     [[1, 3, 5, 4]]  \n",
       "34                       [[1, 3, 5, 4], [1, 3, 5, 4]]  \n",
       "35                          [[2, 4, 3], [1, 3, 5, 4]]  \n",
       "36         [[1, 3, 5, 4], [1, 3, 5, 2], [1, 3, 5, 4]]  \n",
       "37                       [[1, 3, 5, 4], [1, 3, 5, 4]]  \n",
       "38         [[1, 3, 5, 2], [1, 3, 5, 4], [1, 3, 5, 4]]  \n",
       "39                       [[1, 3, 5, 4], [1, 3, 5, 4]]  \n",
       "40                          [[2, 4, 3], [1, 3, 5, 4]]  \n",
       "41            [[2, 4, 3], [1, 3, 5, 2], [1, 3, 5, 4]]  \n",
       "42                       [[1, 3, 5, 2], [1, 3, 5, 4]]  \n",
       "43         [[1, 3, 5, 4], [1, 3, 5, 2], [1, 3, 5, 4]]  \n",
       "44                                                 []  \n",
       "45                       [[1, 3, 5, 4], [1, 3, 5, 4]]  \n",
       "46                          [[2, 4, 3], [1, 3, 5, 4]]  \n",
       "47      [[1, 2, 4, 5, 3], [1, 3, 5, 4], [1, 3, 5, 4]]  "
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#This dataframe adds on a new column of the days for each class.\n",
    "def match_class_days(class_list):\n",
    "    days = []\n",
    "    for class_4 in class_list: \n",
    "        class_days = df_class_days_final[df_class_days_final.Class == class_4]['Days']       \n",
    "        days.extend(class_days)\n",
    "    return days\n",
    "class_2['days'] = class_2['total'].apply(match_class_days)\n",
    "class_2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The function below adds on more columns: it finds the number of classes each student has on a given day: basically, it goes through the last column on the dataframe above and finds the number of times a particular day appears. So, from looking at the last column for u59 from above, we expect to see three classes on Monday since 1 appears three times in the array of arrays. This function does this for all students:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "#This function returns five new columns and adds them to the class_2 dataframe from before. \n",
    "#Each of the five columns represent the number of classes (and therefore number of attendances) each student has (should have) per day depending on the day of the week.\n",
    "def get_classes_per_day(data):\n",
    "    mon=[]\n",
    "    tues=[]\n",
    "    wed=[]\n",
    "    thurs=[]\n",
    "    fri=[]\n",
    "    user_list = []\n",
    "    list_arrays = []\n",
    "    counter = 0\n",
    "    for i in list(range(0, 48)):\n",
    "    #print(i)\n",
    "        for j in data.days[i]:\n",
    "            mon.append(np.count_nonzero(j == 1))\n",
    "            tues.append(np.count_nonzero(j == 2))\n",
    "            wed.append(np.count_nonzero(j == 3))\n",
    "            thurs.append(np.count_nonzero(j == 4))\n",
    "            fri.append(np.count_nonzero(j == 5))\n",
    "            user_list.append(data.u00[i])\n",
    "\n",
    "    data = data[data['u00'] != 'u56']\n",
    "    data = data[data['u00'] != 'u36']\n",
    "    data = data[data['u00'] != 'u39']\n",
    "    days_classes = pd.DataFrame({'uid': user_list, 'Monday': mon, 'Tuesday': tues, 'Wednesday': wed, 'Thursday': thurs, 'Friday': fri})\n",
    "    num_classes_monday = (days_classes.groupby('uid').Monday.sum())\n",
    "    num_classes_tuesday = (days_classes.groupby('uid').Tuesday.sum())\n",
    "    num_classes_wednesday = (days_classes.groupby('uid').Wednesday.sum())\n",
    "    num_classes_thursday = (days_classes.groupby('uid').Thursday.sum())\n",
    "    num_classes_friday = (days_classes.groupby('uid').Friday.sum())\n",
    "    data['Monday'] = list(num_classes_monday)\n",
    "    data['Tuesday'] = list(num_classes_tuesday)\n",
    "    data['Wednesday'] = list(num_classes_wednesday)\n",
    "    data['Thursday'] = list(num_classes_thursday)\n",
    "    data['Friday'] = list(num_classes_friday)\n",
    "    return data\n",
    "class_2 = get_classes_per_day(class_2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>u00</th>\n",
       "      <th>Unnamed: 1</th>\n",
       "      <th>Unnamed: 2</th>\n",
       "      <th>Unnamed: 3</th>\n",
       "      <th>Unnamed: 4</th>\n",
       "      <th>total</th>\n",
       "      <th>days</th>\n",
       "      <th>Monday</th>\n",
       "      <th>Tuesday</th>\n",
       "      <th>Wednesday</th>\n",
       "      <th>Thursday</th>\n",
       "      <th>Friday</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>u01</td>\n",
       "      <td>ENGS 069</td>\n",
       "      <td>ENGS 022</td>\n",
       "      <td>ANTH 012</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[ENGS 069, ENGS 022, ANTH 012]</td>\n",
       "      <td>[[1, 3, 5, 4], [1, 3, 5, 4], [1, 3, 5, 2]]</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>u02</td>\n",
       "      <td>COSC 077</td>\n",
       "      <td>COSC 098</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 077, COSC 098, COSC 065]</td>\n",
       "      <td>[[2, 4, 3], [1, 3, 5, 4]]</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>u03</td>\n",
       "      <td>COSC 057</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 057, COSC 065]</td>\n",
       "      <td>[[1, 3, 5, 4], [1, 3, 5, 4]]</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>u04</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 065]</td>\n",
       "      <td>[[1, 3, 5, 4]]</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>u05</td>\n",
       "      <td>COSC 050</td>\n",
       "      <td>PSYC 028</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 050, PSYC 028, COSC 065]</td>\n",
       "      <td>[[1, 3, 5, 2], [1, 3, 5, 4], [1, 3, 5, 4]]</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>5</td>\n",
       "      <td>u07</td>\n",
       "      <td>COSC 077</td>\n",
       "      <td>COSC 060</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 077, COSC 060, COSC 065]</td>\n",
       "      <td>[[2, 4, 3], [1, 3, 5, 4], [1, 3, 5, 4]]</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>6</td>\n",
       "      <td>u08</td>\n",
       "      <td>CHIN 062</td>\n",
       "      <td>COSC 089 1</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[CHIN 062, COSC 089 1, COSC 065]</td>\n",
       "      <td>[[2, 4, 1], [1, 3, 5, 2], [1, 3, 5, 4]]</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>7</td>\n",
       "      <td>u09</td>\n",
       "      <td>ANTH 050</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>COSC 099</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[ANTH 050, COSC 065, COSC 099]</td>\n",
       "      <td>[[1, 3, 5, 4]]</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>8</td>\n",
       "      <td>u10</td>\n",
       "      <td>COSC 050</td>\n",
       "      <td>BIOL 004</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 050, BIOL 004, COSC 065]</td>\n",
       "      <td>[[1, 3, 5, 2], [2, 4, 3], [1, 3, 5, 4]]</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>9</td>\n",
       "      <td>u12</td>\n",
       "      <td>COSC 089 1</td>\n",
       "      <td>COSC 050</td>\n",
       "      <td>TUCK 003</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>[COSC 089 1, COSC 050, TUCK 003, COSC 065]</td>\n",
       "      <td>[[1, 3, 5, 2], [1, 3, 5, 2], [2, 4, 3], [1, 3,...</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>10</td>\n",
       "      <td>u13</td>\n",
       "      <td>COSC 070</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 070, COSC 065]</td>\n",
       "      <td>[[2, 4, 3], [1, 3, 5, 4]]</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>11</td>\n",
       "      <td>u14</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>COSC 027</td>\n",
       "      <td>COSC 020</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 065, COSC 027, COSC 020]</td>\n",
       "      <td>[[1, 3, 5, 4], [1, 4], [2, 4, 3]]</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>12</td>\n",
       "      <td>u15</td>\n",
       "      <td>EARS 003</td>\n",
       "      <td>SPAN 003</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[EARS 003, SPAN 003, COSC 065]</td>\n",
       "      <td>[[1, 3, 5, 2], [1, 2, 4, 5, 3], [1, 3, 5, 4]]</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>13</td>\n",
       "      <td>u16</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>COSC 027</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 065, COSC 027]</td>\n",
       "      <td>[[1, 3, 5, 4], [1, 4]]</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>14</td>\n",
       "      <td>u17</td>\n",
       "      <td>COSC 089 1</td>\n",
       "      <td>MUS 016</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 089 1, MUS 016, COSC 065]</td>\n",
       "      <td>[[1, 3, 5, 2], [2, 4, 3], [1, 3, 5, 4]]</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>15</td>\n",
       "      <td>u18</td>\n",
       "      <td>COSC 089 1</td>\n",
       "      <td>CHIN 033</td>\n",
       "      <td>TUCK 003</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>[COSC 089 1, CHIN 033, TUCK 003, COSC 065]</td>\n",
       "      <td>[[1, 3, 5, 2], [1, 2, 4, 5, 3], [2, 4, 3], [1,...</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>16</td>\n",
       "      <td>u19</td>\n",
       "      <td>COSC 050</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>FILM 051</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 050, COSC 065, FILM 051]</td>\n",
       "      <td>[[1, 3, 5, 2], [1, 3, 5, 4], [2, 4, 3]]</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>17</td>\n",
       "      <td>u20</td>\n",
       "      <td>COSC 070</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 070, COSC 065]</td>\n",
       "      <td>[[2, 4, 3], [1, 3, 5, 4]]</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>18</td>\n",
       "      <td>u22</td>\n",
       "      <td>COSC 050</td>\n",
       "      <td>NAS 035</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 050, NAS 035, COSC 065]</td>\n",
       "      <td>[[1, 3, 5, 2], [1, 3, 5, 2], [1, 3, 5, 4]]</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>19</td>\n",
       "      <td>u23</td>\n",
       "      <td>COSC 070</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 070, COSC 065]</td>\n",
       "      <td>[[2, 4, 3], [1, 3, 5, 4]]</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>20</td>\n",
       "      <td>u24</td>\n",
       "      <td>M&amp;SS 045</td>\n",
       "      <td>COSC 060</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[M&amp;SS 045, COSC 060, COSC 065]</td>\n",
       "      <td>[[1, 3, 5, 4], [1, 3, 5, 4], [1, 3, 5, 4]]</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>21</td>\n",
       "      <td>u25</td>\n",
       "      <td>NAS 008</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>ENGL 028</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[NAS 008, COSC 065, ENGL 028]</td>\n",
       "      <td>[[1, 3, 5, 2], [1, 3, 5, 4], [1, 3, 5, 4]]</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>22</td>\n",
       "      <td>u27</td>\n",
       "      <td>ECON 024</td>\n",
       "      <td>JAPN 033</td>\n",
       "      <td>FILM 042</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>[ECON 024, JAPN 033, FILM 042, COSC 065]</td>\n",
       "      <td>[[1, 3, 5, 4], [1, 3, 5, 2], [1, 3, 5, 4]]</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>23</td>\n",
       "      <td>u30</td>\n",
       "      <td>MUS 003</td>\n",
       "      <td>COSC 050</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[MUS 003, COSC 050, COSC 065]</td>\n",
       "      <td>[[2, 4, 3], [1, 3, 5, 2], [1, 3, 5, 4]]</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>24</td>\n",
       "      <td>u31</td>\n",
       "      <td>MUS 003</td>\n",
       "      <td>COSC 077</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[MUS 003, COSC 077, COSC 065]</td>\n",
       "      <td>[[2, 4, 3], [2, 4, 3], [1, 3, 5, 4]]</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>25</td>\n",
       "      <td>u32</td>\n",
       "      <td>MATH 023</td>\n",
       "      <td>COSC 069</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[MATH 023, COSC 069, COSC 065]</td>\n",
       "      <td>[[1, 3, 5, 2], [2, 4, 1], [1, 3, 5, 4]]</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>26</td>\n",
       "      <td>u33</td>\n",
       "      <td>ENGS 025</td>\n",
       "      <td>ENGS 069</td>\n",
       "      <td>ENGS 093</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[ENGS 025, ENGS 069, ENGS 093]</td>\n",
       "      <td>[[1, 3, 5, 4], [1, 3, 5, 4], [1, 3, 5, 2]]</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>27</td>\n",
       "      <td>u34</td>\n",
       "      <td>COSC 070</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 070, COSC 065]</td>\n",
       "      <td>[[2, 4, 3], [1, 3, 5, 4]]</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>28</td>\n",
       "      <td>u35</td>\n",
       "      <td>COSC 070</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 070, COSC 065]</td>\n",
       "      <td>[[2, 4, 3], [1, 3, 5, 4]]</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>31</td>\n",
       "      <td>u41</td>\n",
       "      <td>COSC 050</td>\n",
       "      <td>SPAN 002</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 050, SPAN 002, COSC 065]</td>\n",
       "      <td>[[1, 3, 5, 2], [1, 3, 5, 4]]</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>32</td>\n",
       "      <td>u42</td>\n",
       "      <td>COSC 070</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 070, COSC 065]</td>\n",
       "      <td>[[2, 4, 3], [1, 3, 5, 4]]</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>33</td>\n",
       "      <td>u43</td>\n",
       "      <td>ENGS 031</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[ENGS 031, COSC 065]</td>\n",
       "      <td>[[1, 3, 5, 4]]</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>34</td>\n",
       "      <td>u44</td>\n",
       "      <td>COSC 060</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 060, COSC 065]</td>\n",
       "      <td>[[1, 3, 5, 4], [1, 3, 5, 4]]</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>35</td>\n",
       "      <td>u45</td>\n",
       "      <td>COSC 070</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 070, COSC 065]</td>\n",
       "      <td>[[2, 4, 3], [1, 3, 5, 4]]</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>36</td>\n",
       "      <td>u46</td>\n",
       "      <td>ECON 036</td>\n",
       "      <td>COSC 050</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[ECON 036, COSC 050, COSC 065]</td>\n",
       "      <td>[[1, 3, 5, 4], [1, 3, 5, 2], [1, 3, 5, 4]]</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>37</td>\n",
       "      <td>u47</td>\n",
       "      <td>COSC 060</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 060, COSC 065]</td>\n",
       "      <td>[[1, 3, 5, 4], [1, 3, 5, 4]]</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>38</td>\n",
       "      <td>u49</td>\n",
       "      <td>MATH 013</td>\n",
       "      <td>LAT 003</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[MATH 013, LAT 003, COSC 065]</td>\n",
       "      <td>[[1, 3, 5, 2], [1, 3, 5, 4], [1, 3, 5, 4]]</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>39</td>\n",
       "      <td>u50</td>\n",
       "      <td>COSC 060</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 060, COSC 065]</td>\n",
       "      <td>[[1, 3, 5, 4], [1, 3, 5, 4]]</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>40</td>\n",
       "      <td>u51</td>\n",
       "      <td>COSC 070</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 070, COSC 065]</td>\n",
       "      <td>[[2, 4, 3], [1, 3, 5, 4]]</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>41</td>\n",
       "      <td>u52</td>\n",
       "      <td>BIOL 006</td>\n",
       "      <td>COSC 050</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[BIOL 006, COSC 050, COSC 065]</td>\n",
       "      <td>[[2, 4, 3], [1, 3, 5, 2], [1, 3, 5, 4]]</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>42</td>\n",
       "      <td>u53</td>\n",
       "      <td>COSC 089 1</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 089 1, COSC 065]</td>\n",
       "      <td>[[1, 3, 5, 2], [1, 3, 5, 4]]</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>43</td>\n",
       "      <td>u54</td>\n",
       "      <td>ENGS 025</td>\n",
       "      <td>MATH 023</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[ENGS 025, MATH 023, COSC 065]</td>\n",
       "      <td>[[1, 3, 5, 4], [1, 3, 5, 2], [1, 3, 5, 4]]</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>45</td>\n",
       "      <td>u57</td>\n",
       "      <td>COSC 098</td>\n",
       "      <td>COSC 060</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 098, COSC 060, COSC 065]</td>\n",
       "      <td>[[1, 3, 5, 4], [1, 3, 5, 4]]</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>46</td>\n",
       "      <td>u58</td>\n",
       "      <td>COSC 070</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[COSC 070, COSC 065]</td>\n",
       "      <td>[[2, 4, 3], [1, 3, 5, 4]]</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>47</td>\n",
       "      <td>u59</td>\n",
       "      <td>GERM 001</td>\n",
       "      <td>COSC 065</td>\n",
       "      <td>COSC 007</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[GERM 001, COSC 065, COSC 007]</td>\n",
       "      <td>[[1, 2, 4, 5, 3], [1, 3, 5, 4], [1, 3, 5, 4]]</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    u00  Unnamed: 1  Unnamed: 2 Unnamed: 3 Unnamed: 4  \\\n",
       "0   u01    ENGS 069    ENGS 022   ANTH 012        NaN   \n",
       "1   u02    COSC 077    COSC 098   COSC 065        NaN   \n",
       "2   u03    COSC 057    COSC 065        NaN        NaN   \n",
       "3   u04    COSC 065         NaN        NaN        NaN   \n",
       "4   u05    COSC 050    PSYC 028   COSC 065        NaN   \n",
       "5   u07    COSC 077    COSC 060   COSC 065        NaN   \n",
       "6   u08    CHIN 062  COSC 089 1   COSC 065        NaN   \n",
       "7   u09    ANTH 050    COSC 065   COSC 099        NaN   \n",
       "8   u10    COSC 050    BIOL 004   COSC 065        NaN   \n",
       "9   u12  COSC 089 1    COSC 050   TUCK 003   COSC 065   \n",
       "10  u13    COSC 070    COSC 065        NaN        NaN   \n",
       "11  u14    COSC 065    COSC 027   COSC 020        NaN   \n",
       "12  u15    EARS 003    SPAN 003   COSC 065        NaN   \n",
       "13  u16    COSC 065    COSC 027        NaN        NaN   \n",
       "14  u17  COSC 089 1     MUS 016   COSC 065        NaN   \n",
       "15  u18  COSC 089 1    CHIN 033   TUCK 003   COSC 065   \n",
       "16  u19    COSC 050    COSC 065   FILM 051        NaN   \n",
       "17  u20    COSC 070    COSC 065        NaN        NaN   \n",
       "18  u22    COSC 050     NAS 035   COSC 065        NaN   \n",
       "19  u23    COSC 070    COSC 065        NaN        NaN   \n",
       "20  u24    M&SS 045    COSC 060   COSC 065        NaN   \n",
       "21  u25     NAS 008    COSC 065   ENGL 028        NaN   \n",
       "22  u27    ECON 024    JAPN 033   FILM 042   COSC 065   \n",
       "23  u30     MUS 003    COSC 050   COSC 065        NaN   \n",
       "24  u31     MUS 003    COSC 077   COSC 065        NaN   \n",
       "25  u32    MATH 023    COSC 069   COSC 065        NaN   \n",
       "26  u33    ENGS 025    ENGS 069   ENGS 093        NaN   \n",
       "27  u34    COSC 070    COSC 065        NaN        NaN   \n",
       "28  u35    COSC 070    COSC 065        NaN        NaN   \n",
       "31  u41    COSC 050    SPAN 002   COSC 065        NaN   \n",
       "32  u42    COSC 070    COSC 065        NaN        NaN   \n",
       "33  u43    ENGS 031    COSC 065        NaN        NaN   \n",
       "34  u44    COSC 060    COSC 065        NaN        NaN   \n",
       "35  u45    COSC 070    COSC 065        NaN        NaN   \n",
       "36  u46    ECON 036    COSC 050   COSC 065        NaN   \n",
       "37  u47    COSC 060    COSC 065        NaN        NaN   \n",
       "38  u49    MATH 013     LAT 003   COSC 065        NaN   \n",
       "39  u50    COSC 060    COSC 065        NaN        NaN   \n",
       "40  u51    COSC 070    COSC 065        NaN        NaN   \n",
       "41  u52    BIOL 006    COSC 050   COSC 065        NaN   \n",
       "42  u53  COSC 089 1    COSC 065        NaN        NaN   \n",
       "43  u54    ENGS 025    MATH 023   COSC 065        NaN   \n",
       "45  u57    COSC 098    COSC 060   COSC 065        NaN   \n",
       "46  u58    COSC 070    COSC 065        NaN        NaN   \n",
       "47  u59    GERM 001    COSC 065   COSC 007        NaN   \n",
       "\n",
       "                                         total  \\\n",
       "0               [ENGS 069, ENGS 022, ANTH 012]   \n",
       "1               [COSC 077, COSC 098, COSC 065]   \n",
       "2                         [COSC 057, COSC 065]   \n",
       "3                                   [COSC 065]   \n",
       "4               [COSC 050, PSYC 028, COSC 065]   \n",
       "5               [COSC 077, COSC 060, COSC 065]   \n",
       "6             [CHIN 062, COSC 089 1, COSC 065]   \n",
       "7               [ANTH 050, COSC 065, COSC 099]   \n",
       "8               [COSC 050, BIOL 004, COSC 065]   \n",
       "9   [COSC 089 1, COSC 050, TUCK 003, COSC 065]   \n",
       "10                        [COSC 070, COSC 065]   \n",
       "11              [COSC 065, COSC 027, COSC 020]   \n",
       "12              [EARS 003, SPAN 003, COSC 065]   \n",
       "13                        [COSC 065, COSC 027]   \n",
       "14             [COSC 089 1, MUS 016, COSC 065]   \n",
       "15  [COSC 089 1, CHIN 033, TUCK 003, COSC 065]   \n",
       "16              [COSC 050, COSC 065, FILM 051]   \n",
       "17                        [COSC 070, COSC 065]   \n",
       "18               [COSC 050, NAS 035, COSC 065]   \n",
       "19                        [COSC 070, COSC 065]   \n",
       "20              [M&SS 045, COSC 060, COSC 065]   \n",
       "21               [NAS 008, COSC 065, ENGL 028]   \n",
       "22    [ECON 024, JAPN 033, FILM 042, COSC 065]   \n",
       "23               [MUS 003, COSC 050, COSC 065]   \n",
       "24               [MUS 003, COSC 077, COSC 065]   \n",
       "25              [MATH 023, COSC 069, COSC 065]   \n",
       "26              [ENGS 025, ENGS 069, ENGS 093]   \n",
       "27                        [COSC 070, COSC 065]   \n",
       "28                        [COSC 070, COSC 065]   \n",
       "31              [COSC 050, SPAN 002, COSC 065]   \n",
       "32                        [COSC 070, COSC 065]   \n",
       "33                        [ENGS 031, COSC 065]   \n",
       "34                        [COSC 060, COSC 065]   \n",
       "35                        [COSC 070, COSC 065]   \n",
       "36              [ECON 036, COSC 050, COSC 065]   \n",
       "37                        [COSC 060, COSC 065]   \n",
       "38               [MATH 013, LAT 003, COSC 065]   \n",
       "39                        [COSC 060, COSC 065]   \n",
       "40                        [COSC 070, COSC 065]   \n",
       "41              [BIOL 006, COSC 050, COSC 065]   \n",
       "42                      [COSC 089 1, COSC 065]   \n",
       "43              [ENGS 025, MATH 023, COSC 065]   \n",
       "45              [COSC 098, COSC 060, COSC 065]   \n",
       "46                        [COSC 070, COSC 065]   \n",
       "47              [GERM 001, COSC 065, COSC 007]   \n",
       "\n",
       "                                                 days  Monday  Tuesday  \\\n",
       "0          [[1, 3, 5, 4], [1, 3, 5, 4], [1, 3, 5, 2]]       3        1   \n",
       "1                           [[2, 4, 3], [1, 3, 5, 4]]       1        1   \n",
       "2                        [[1, 3, 5, 4], [1, 3, 5, 4]]       2        0   \n",
       "3                                      [[1, 3, 5, 4]]       1        0   \n",
       "4          [[1, 3, 5, 2], [1, 3, 5, 4], [1, 3, 5, 4]]       3        1   \n",
       "5             [[2, 4, 3], [1, 3, 5, 4], [1, 3, 5, 4]]       2        1   \n",
       "6             [[2, 4, 1], [1, 3, 5, 2], [1, 3, 5, 4]]       3        2   \n",
       "7                                      [[1, 3, 5, 4]]       1        0   \n",
       "8             [[1, 3, 5, 2], [2, 4, 3], [1, 3, 5, 4]]       2        2   \n",
       "9   [[1, 3, 5, 2], [1, 3, 5, 2], [2, 4, 3], [1, 3,...       3        3   \n",
       "10                          [[2, 4, 3], [1, 3, 5, 4]]       1        1   \n",
       "11                  [[1, 3, 5, 4], [1, 4], [2, 4, 3]]       2        1   \n",
       "12      [[1, 3, 5, 2], [1, 2, 4, 5, 3], [1, 3, 5, 4]]       3        2   \n",
       "13                             [[1, 3, 5, 4], [1, 4]]       2        0   \n",
       "14            [[1, 3, 5, 2], [2, 4, 3], [1, 3, 5, 4]]       2        2   \n",
       "15  [[1, 3, 5, 2], [1, 2, 4, 5, 3], [2, 4, 3], [1,...       3        3   \n",
       "16            [[1, 3, 5, 2], [1, 3, 5, 4], [2, 4, 3]]       2        2   \n",
       "17                          [[2, 4, 3], [1, 3, 5, 4]]       1        1   \n",
       "18         [[1, 3, 5, 2], [1, 3, 5, 2], [1, 3, 5, 4]]       3        2   \n",
       "19                          [[2, 4, 3], [1, 3, 5, 4]]       1        1   \n",
       "20         [[1, 3, 5, 4], [1, 3, 5, 4], [1, 3, 5, 4]]       3        0   \n",
       "21         [[1, 3, 5, 2], [1, 3, 5, 4], [1, 3, 5, 4]]       3        1   \n",
       "22         [[1, 3, 5, 4], [1, 3, 5, 2], [1, 3, 5, 4]]       3        1   \n",
       "23            [[2, 4, 3], [1, 3, 5, 2], [1, 3, 5, 4]]       2        2   \n",
       "24               [[2, 4, 3], [2, 4, 3], [1, 3, 5, 4]]       1        2   \n",
       "25            [[1, 3, 5, 2], [2, 4, 1], [1, 3, 5, 4]]       3        2   \n",
       "26         [[1, 3, 5, 4], [1, 3, 5, 4], [1, 3, 5, 2]]       3        1   \n",
       "27                          [[2, 4, 3], [1, 3, 5, 4]]       1        1   \n",
       "28                          [[2, 4, 3], [1, 3, 5, 4]]       1        1   \n",
       "31                       [[1, 3, 5, 2], [1, 3, 5, 4]]       2        1   \n",
       "32                          [[2, 4, 3], [1, 3, 5, 4]]       1        1   \n",
       "33                                     [[1, 3, 5, 4]]       1        0   \n",
       "34                       [[1, 3, 5, 4], [1, 3, 5, 4]]       2        0   \n",
       "35                          [[2, 4, 3], [1, 3, 5, 4]]       1        1   \n",
       "36         [[1, 3, 5, 4], [1, 3, 5, 2], [1, 3, 5, 4]]       3        1   \n",
       "37                       [[1, 3, 5, 4], [1, 3, 5, 4]]       2        0   \n",
       "38         [[1, 3, 5, 2], [1, 3, 5, 4], [1, 3, 5, 4]]       3        1   \n",
       "39                       [[1, 3, 5, 4], [1, 3, 5, 4]]       2        0   \n",
       "40                          [[2, 4, 3], [1, 3, 5, 4]]       1        1   \n",
       "41            [[2, 4, 3], [1, 3, 5, 2], [1, 3, 5, 4]]       2        2   \n",
       "42                       [[1, 3, 5, 2], [1, 3, 5, 4]]       2        1   \n",
       "43         [[1, 3, 5, 4], [1, 3, 5, 2], [1, 3, 5, 4]]       3        1   \n",
       "45                       [[1, 3, 5, 4], [1, 3, 5, 4]]       2        0   \n",
       "46                          [[2, 4, 3], [1, 3, 5, 4]]       1        1   \n",
       "47      [[1, 2, 4, 5, 3], [1, 3, 5, 4], [1, 3, 5, 4]]       3        1   \n",
       "\n",
       "    Wednesday  Thursday  Friday  \n",
       "0           3         2       3  \n",
       "1           2         2       1  \n",
       "2           2         2       2  \n",
       "3           1         1       1  \n",
       "4           3         2       3  \n",
       "5           3         3       2  \n",
       "6           2         2       2  \n",
       "7           1         1       1  \n",
       "8           3         2       2  \n",
       "9           4         2       3  \n",
       "10          2         2       1  \n",
       "11          2         3       1  \n",
       "12          3         2       3  \n",
       "13          1         2       1  \n",
       "14          3         2       2  \n",
       "15          4         3       3  \n",
       "16          3         2       2  \n",
       "17          2         2       1  \n",
       "18          3         1       3  \n",
       "19          2         2       1  \n",
       "20          3         3       3  \n",
       "21          3         2       3  \n",
       "22          3         2       3  \n",
       "23          3         2       2  \n",
       "24          3         3       1  \n",
       "25          2         2       2  \n",
       "26          3         2       3  \n",
       "27          2         2       1  \n",
       "28          2         2       1  \n",
       "31          2         1       2  \n",
       "32          2         2       1  \n",
       "33          1         1       1  \n",
       "34          2         2       2  \n",
       "35          2         2       1  \n",
       "36          3         2       3  \n",
       "37          2         2       2  \n",
       "38          3         2       3  \n",
       "39          2         2       2  \n",
       "40          2         2       1  \n",
       "41          3         2       2  \n",
       "42          2         1       2  \n",
       "43          3         2       3  \n",
       "45          2         2       2  \n",
       "46          2         2       1  \n",
       "47          3         3       3  "
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "class_2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another thing we need to find is the class start/end times. Our class_data file in the dataset has this information, and we'll extract it here in the cell below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Start</th>\n",
       "      <th>End</th>\n",
       "      <th>Start_low</th>\n",
       "      <th>End_high</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>2021-01-11 11:15:00</td>\n",
       "      <td>2021-01-11 12:20:00</td>\n",
       "      <td>2021-01-11 10:55:00</td>\n",
       "      <td>2021-01-11 12:40:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>2021-01-11 12:00:00</td>\n",
       "      <td>2021-01-11 12:50:00</td>\n",
       "      <td>2021-01-11 11:40:00</td>\n",
       "      <td>2021-01-11 13:10:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>2021-01-11 10:00:00</td>\n",
       "      <td>2021-01-11 11:50:00</td>\n",
       "      <td>2021-01-11 09:40:00</td>\n",
       "      <td>2021-01-11 12:10:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>6</td>\n",
       "      <td>2021-01-11 15:00:00</td>\n",
       "      <td>2021-01-11 15:50:00</td>\n",
       "      <td>2021-01-11 14:40:00</td>\n",
       "      <td>2021-01-11 16:10:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>10</td>\n",
       "      <td>2021-01-11 09:00:00</td>\n",
       "      <td>2021-01-11 09:50:00</td>\n",
       "      <td>2021-01-11 08:40:00</td>\n",
       "      <td>2021-01-11 10:10:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>15</td>\n",
       "      <td>2021-01-11 16:00:00</td>\n",
       "      <td>2021-01-11 17:50:00</td>\n",
       "      <td>2021-01-11 15:40:00</td>\n",
       "      <td>2021-01-11 18:10:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>17</td>\n",
       "      <td>2021-01-11 17:00:00</td>\n",
       "      <td>2021-01-11 17:50:00</td>\n",
       "      <td>2021-01-11 16:40:00</td>\n",
       "      <td>2021-01-11 18:10:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>18</td>\n",
       "      <td>2021-01-11 13:45:00</td>\n",
       "      <td>2021-01-11 14:50:00</td>\n",
       "      <td>2021-01-11 13:25:00</td>\n",
       "      <td>2021-01-11 15:10:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>21</td>\n",
       "      <td>2021-01-11 13:00:00</td>\n",
       "      <td>2021-01-11 13:50:00</td>\n",
       "      <td>2021-01-11 12:40:00</td>\n",
       "      <td>2021-01-11 14:10:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>22</td>\n",
       "      <td>2021-01-11 14:00:00</td>\n",
       "      <td>2021-01-11 15:50:00</td>\n",
       "      <td>2021-01-11 13:40:00</td>\n",
       "      <td>2021-01-11 16:10:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>24</td>\n",
       "      <td>2021-01-11 16:15:00</td>\n",
       "      <td>2021-01-11 17:05:00</td>\n",
       "      <td>2021-01-11 15:55:00</td>\n",
       "      <td>2021-01-11 17:25:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>25</td>\n",
       "      <td>2021-01-11 15:00:00</td>\n",
       "      <td>2021-01-11 16:50:00</td>\n",
       "      <td>2021-01-11 14:40:00</td>\n",
       "      <td>2021-01-11 17:10:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>28</td>\n",
       "      <td>2021-01-11 12:30:00</td>\n",
       "      <td>2021-01-11 13:35:00</td>\n",
       "      <td>2021-01-11 12:10:00</td>\n",
       "      <td>2021-01-11 13:55:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>40</td>\n",
       "      <td>2021-01-11 10:00:00</td>\n",
       "      <td>2021-01-11 11:05:00</td>\n",
       "      <td>2021-01-11 09:40:00</td>\n",
       "      <td>2021-01-11 11:25:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>61</td>\n",
       "      <td>2021-01-11 08:45:00</td>\n",
       "      <td>2021-01-11 09:50:00</td>\n",
       "      <td>2021-01-11 08:25:00</td>\n",
       "      <td>2021-01-11 10:10:00</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 Start                 End           Start_low  \\\n",
       "0  2021-01-11 11:15:00 2021-01-11 12:20:00 2021-01-11 10:55:00   \n",
       "3  2021-01-11 12:00:00 2021-01-11 12:50:00 2021-01-11 11:40:00   \n",
       "4  2021-01-11 10:00:00 2021-01-11 11:50:00 2021-01-11 09:40:00   \n",
       "6  2021-01-11 15:00:00 2021-01-11 15:50:00 2021-01-11 14:40:00   \n",
       "10 2021-01-11 09:00:00 2021-01-11 09:50:00 2021-01-11 08:40:00   \n",
       "15 2021-01-11 16:00:00 2021-01-11 17:50:00 2021-01-11 15:40:00   \n",
       "17 2021-01-11 17:00:00 2021-01-11 17:50:00 2021-01-11 16:40:00   \n",
       "18 2021-01-11 13:45:00 2021-01-11 14:50:00 2021-01-11 13:25:00   \n",
       "21 2021-01-11 13:00:00 2021-01-11 13:50:00 2021-01-11 12:40:00   \n",
       "22 2021-01-11 14:00:00 2021-01-11 15:50:00 2021-01-11 13:40:00   \n",
       "24 2021-01-11 16:15:00 2021-01-11 17:05:00 2021-01-11 15:55:00   \n",
       "25 2021-01-11 15:00:00 2021-01-11 16:50:00 2021-01-11 14:40:00   \n",
       "28 2021-01-11 12:30:00 2021-01-11 13:35:00 2021-01-11 12:10:00   \n",
       "40 2021-01-11 10:00:00 2021-01-11 11:05:00 2021-01-11 09:40:00   \n",
       "61 2021-01-11 08:45:00 2021-01-11 09:50:00 2021-01-11 08:25:00   \n",
       "\n",
       "              End_high  \n",
       "0  2021-01-11 12:40:00  \n",
       "3  2021-01-11 13:10:00  \n",
       "4  2021-01-11 12:10:00  \n",
       "6  2021-01-11 16:10:00  \n",
       "10 2021-01-11 10:10:00  \n",
       "15 2021-01-11 18:10:00  \n",
       "17 2021-01-11 18:10:00  \n",
       "18 2021-01-11 15:10:00  \n",
       "21 2021-01-11 14:10:00  \n",
       "22 2021-01-11 16:10:00  \n",
       "24 2021-01-11 17:25:00  \n",
       "25 2021-01-11 17:10:00  \n",
       "28 2021-01-11 13:55:00  \n",
       "40 2021-01-11 11:25:00  \n",
       "61 2021-01-11 10:10:00  "
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#This data frame has the start and end times for each class\n",
    "#I also added two new columns, Start_low and End_high since students arrive early to class, but can also leave late.\n",
    "#This creates a range for start and end times.\n",
    "def get_start_end(data): #data is class_data\n",
    "    lst_start_time = [] \n",
    "    lst_end_time = []\n",
    "    lst_days = []\n",
    "    lst_class = []\n",
    "    for i in list(range(0, len(data['periods']))):\n",
    "        for j in data['periods'][i]:\n",
    "            try:\n",
    "                lst_start_time.append(j['start'])\n",
    "                lst_end_time.append(j['end'])\n",
    "                lst_days.append(j['day'])\n",
    "                lst_class.append(data.Class[i])\n",
    "            except:\n",
    "                lst_days.append('Error')\n",
    "    df_class_times = pd.DataFrame({'Start': lst_start_time, 'End': lst_end_time,}) #'Days': lst_days, 'Class': lst_class})\n",
    "    df_class_times = df_class_times.drop_duplicates()\n",
    "    df_class_times[\"Start\"] = pd.to_datetime(df_class_times['Start'])\n",
    "    df_class_times['Start_low'] = df_class_times['Start'] - datetime.timedelta(minutes=20)\n",
    "    df_class_times['End'] = pd.to_datetime(df_class_times['End'])\n",
    "    df_class_times['End_high'] = df_class_times['End'] + datetime.timedelta(minutes=20)\n",
    "    return df_class_times\n",
    "df_class_times = get_start_end(class_data)\n",
    "#This dataframe has start and end times, as well as low and high estimates, respectively.\n",
    "df_class_times"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Although this dataframe is fine, there's an issue: the year is shown as 2021 in the times, which is not correct. Let's edit this using the below function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Since the columns of the previous dataframe also had the date (which is the day when you run the code), and since we want just the hours, minutes and seconds, this code extracts that info.\n",
    "df_class_times['Start']\n",
    "df_class_times['End']\n",
    "def get_time_format(times1, times2, times3, times4, data):\n",
    "    start_times = []\n",
    "    start_times_low = []\n",
    "    end_times_high = []\n",
    "    end_times = []\n",
    "    for i in times1:\n",
    "        start_times.append(i.strftime(\"%H:%M:%S\"))\n",
    "    for i in times2:\n",
    "        start_times_low.append(i.strftime(\"%H:%M:%S\"))\n",
    "    for i in times3:\n",
    "        end_times.append(i.strftime(\"%H:%M:%S\"))\n",
    "    for i in times4:\n",
    "        end_times_high.append(i.strftime(\"%H:%M:%S\"))\n",
    "    data['Start'] = start_times\n",
    "    data['End'] = end_times\n",
    "    data['Start_low'] = start_times_low\n",
    "    data['End_high'] = end_times_high\n",
    "    data['Start Range'] = list(zip(data['Start_low'], data['Start']))\n",
    "    data['End Range'] = list(zip(data['End'], data['End_high']))\n",
    "    return data\n",
    "#This dataframe has the class start, start_low, end, end_high, as well as ranges of times.\n",
    "#The ranges of times are tuples of (start_low, start) and (end, end_high)\n",
    "df_class_times_final = get_time_format(df_class_times['Start'], df_class_times['Start_low'], df_class_times['End'], df_class_times['End_high'], df_class_times)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Start</th>\n",
       "      <th>End</th>\n",
       "      <th>Start_low</th>\n",
       "      <th>End_high</th>\n",
       "      <th>Start Range</th>\n",
       "      <th>End Range</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>11:15:00</td>\n",
       "      <td>12:20:00</td>\n",
       "      <td>10:55:00</td>\n",
       "      <td>12:40:00</td>\n",
       "      <td>(10:55:00, 11:15:00)</td>\n",
       "      <td>(12:20:00, 12:40:00)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>12:00:00</td>\n",
       "      <td>12:50:00</td>\n",
       "      <td>11:40:00</td>\n",
       "      <td>13:10:00</td>\n",
       "      <td>(11:40:00, 12:00:00)</td>\n",
       "      <td>(12:50:00, 13:10:00)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>10:00:00</td>\n",
       "      <td>11:50:00</td>\n",
       "      <td>09:40:00</td>\n",
       "      <td>12:10:00</td>\n",
       "      <td>(09:40:00, 10:00:00)</td>\n",
       "      <td>(11:50:00, 12:10:00)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>6</td>\n",
       "      <td>15:00:00</td>\n",
       "      <td>15:50:00</td>\n",
       "      <td>14:40:00</td>\n",
       "      <td>16:10:00</td>\n",
       "      <td>(14:40:00, 15:00:00)</td>\n",
       "      <td>(15:50:00, 16:10:00)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>10</td>\n",
       "      <td>09:00:00</td>\n",
       "      <td>09:50:00</td>\n",
       "      <td>08:40:00</td>\n",
       "      <td>10:10:00</td>\n",
       "      <td>(08:40:00, 09:00:00)</td>\n",
       "      <td>(09:50:00, 10:10:00)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>15</td>\n",
       "      <td>16:00:00</td>\n",
       "      <td>17:50:00</td>\n",
       "      <td>15:40:00</td>\n",
       "      <td>18:10:00</td>\n",
       "      <td>(15:40:00, 16:00:00)</td>\n",
       "      <td>(17:50:00, 18:10:00)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>17</td>\n",
       "      <td>17:00:00</td>\n",
       "      <td>17:50:00</td>\n",
       "      <td>16:40:00</td>\n",
       "      <td>18:10:00</td>\n",
       "      <td>(16:40:00, 17:00:00)</td>\n",
       "      <td>(17:50:00, 18:10:00)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>18</td>\n",
       "      <td>13:45:00</td>\n",
       "      <td>14:50:00</td>\n",
       "      <td>13:25:00</td>\n",
       "      <td>15:10:00</td>\n",
       "      <td>(13:25:00, 13:45:00)</td>\n",
       "      <td>(14:50:00, 15:10:00)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>21</td>\n",
       "      <td>13:00:00</td>\n",
       "      <td>13:50:00</td>\n",
       "      <td>12:40:00</td>\n",
       "      <td>14:10:00</td>\n",
       "      <td>(12:40:00, 13:00:00)</td>\n",
       "      <td>(13:50:00, 14:10:00)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>22</td>\n",
       "      <td>14:00:00</td>\n",
       "      <td>15:50:00</td>\n",
       "      <td>13:40:00</td>\n",
       "      <td>16:10:00</td>\n",
       "      <td>(13:40:00, 14:00:00)</td>\n",
       "      <td>(15:50:00, 16:10:00)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>24</td>\n",
       "      <td>16:15:00</td>\n",
       "      <td>17:05:00</td>\n",
       "      <td>15:55:00</td>\n",
       "      <td>17:25:00</td>\n",
       "      <td>(15:55:00, 16:15:00)</td>\n",
       "      <td>(17:05:00, 17:25:00)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>25</td>\n",
       "      <td>15:00:00</td>\n",
       "      <td>16:50:00</td>\n",
       "      <td>14:40:00</td>\n",
       "      <td>17:10:00</td>\n",
       "      <td>(14:40:00, 15:00:00)</td>\n",
       "      <td>(16:50:00, 17:10:00)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>28</td>\n",
       "      <td>12:30:00</td>\n",
       "      <td>13:35:00</td>\n",
       "      <td>12:10:00</td>\n",
       "      <td>13:55:00</td>\n",
       "      <td>(12:10:00, 12:30:00)</td>\n",
       "      <td>(13:35:00, 13:55:00)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>40</td>\n",
       "      <td>10:00:00</td>\n",
       "      <td>11:05:00</td>\n",
       "      <td>09:40:00</td>\n",
       "      <td>11:25:00</td>\n",
       "      <td>(09:40:00, 10:00:00)</td>\n",
       "      <td>(11:05:00, 11:25:00)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>61</td>\n",
       "      <td>08:45:00</td>\n",
       "      <td>09:50:00</td>\n",
       "      <td>08:25:00</td>\n",
       "      <td>10:10:00</td>\n",
       "      <td>(08:25:00, 08:45:00)</td>\n",
       "      <td>(09:50:00, 10:10:00)</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       Start       End Start_low  End_high           Start Range  \\\n",
       "0   11:15:00  12:20:00  10:55:00  12:40:00  (10:55:00, 11:15:00)   \n",
       "3   12:00:00  12:50:00  11:40:00  13:10:00  (11:40:00, 12:00:00)   \n",
       "4   10:00:00  11:50:00  09:40:00  12:10:00  (09:40:00, 10:00:00)   \n",
       "6   15:00:00  15:50:00  14:40:00  16:10:00  (14:40:00, 15:00:00)   \n",
       "10  09:00:00  09:50:00  08:40:00  10:10:00  (08:40:00, 09:00:00)   \n",
       "15  16:00:00  17:50:00  15:40:00  18:10:00  (15:40:00, 16:00:00)   \n",
       "17  17:00:00  17:50:00  16:40:00  18:10:00  (16:40:00, 17:00:00)   \n",
       "18  13:45:00  14:50:00  13:25:00  15:10:00  (13:25:00, 13:45:00)   \n",
       "21  13:00:00  13:50:00  12:40:00  14:10:00  (12:40:00, 13:00:00)   \n",
       "22  14:00:00  15:50:00  13:40:00  16:10:00  (13:40:00, 14:00:00)   \n",
       "24  16:15:00  17:05:00  15:55:00  17:25:00  (15:55:00, 16:15:00)   \n",
       "25  15:00:00  16:50:00  14:40:00  17:10:00  (14:40:00, 15:00:00)   \n",
       "28  12:30:00  13:35:00  12:10:00  13:55:00  (12:10:00, 12:30:00)   \n",
       "40  10:00:00  11:05:00  09:40:00  11:25:00  (09:40:00, 10:00:00)   \n",
       "61  08:45:00  09:50:00  08:25:00  10:10:00  (08:25:00, 08:45:00)   \n",
       "\n",
       "               End Range  \n",
       "0   (12:20:00, 12:40:00)  \n",
       "3   (12:50:00, 13:10:00)  \n",
       "4   (11:50:00, 12:10:00)  \n",
       "6   (15:50:00, 16:10:00)  \n",
       "10  (09:50:00, 10:10:00)  \n",
       "15  (17:50:00, 18:10:00)  \n",
       "17  (17:50:00, 18:10:00)  \n",
       "18  (14:50:00, 15:10:00)  \n",
       "21  (13:50:00, 14:10:00)  \n",
       "22  (15:50:00, 16:10:00)  \n",
       "24  (17:05:00, 17:25:00)  \n",
       "25  (16:50:00, 17:10:00)  \n",
       "28  (13:35:00, 13:55:00)  \n",
       "40  (11:05:00, 11:25:00)  \n",
       "61  (09:50:00, 10:10:00)  "
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#df_class_times_final['Start Range'] = list(zip(df_class_times['Start_low'], df_class_times['Start']))\n",
    "#df_class_times_final['End Range'] = list(zip(df_class_times['End'], df_class_times['End_high']))\n",
    "df_class_times_final"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Moving on, the function below finds the actual amount of time spent by each student in a particular location:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/anaconda3/lib/python3.7/site-packages/ipykernel_launcher.py:8: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  \n",
      "/opt/anaconda3/lib/python3.7/site-packages/ipykernel_launcher.py:9: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  if __name__ == '__main__':\n",
      "/opt/anaconda3/lib/python3.7/site-packages/ipykernel_launcher.py:10: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  # Remove the CWD from sys.path while we load stuff.\n",
      "/opt/anaconda3/lib/python3.7/site-packages/ipykernel_launcher.py:11: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  # This is added back by InteractiveShellApp.init_path()\n",
      "/opt/anaconda3/lib/python3.7/site-packages/ipykernel_launcher.py:12: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  if sys.path[0] == '':\n",
      "/opt/anaconda3/lib/python3.7/site-packages/ipykernel_launcher.py:13: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  del sys.path[0]\n",
      "/opt/anaconda3/lib/python3.7/site-packages/ipykernel_launcher.py:18: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n"
     ]
    }
   ],
   "source": [
    "#This function returns the start and end time for each span of time a student was in a particular location\n",
    "#The start and end times are also represented in hour form too.\n",
    "def get_days(data):\n",
    "    days_of_classes = list(range(0,5))\n",
    "    data['Day'] = data['start time in regular notation'].dt.dayofyear\n",
    "    data['Day of Week'] = data['start time in regular notation'].dt.dayofweek\n",
    "    data = data[data['Day of Week'].isin(days_of_classes)]\n",
    "    data['minutes_spent'] = data['Time Delta (in minutes)'].astype(int)\n",
    "    data['seconds_spent'] = data['Time Delta (in seconds)'].astype(int)\n",
    "    data['end time'] = data['start time'] + data['seconds_spent']\n",
    "    data['start_hour'] = data['start time in regular notation'].dt.hour\n",
    "    data['end time in regular notation'] = pd.to_datetime(data['end time'], unit = 's')\n",
    "    data['end_hour'] = data['end time in regular notation'].dt.hour\n",
    "    \n",
    "    return data\n",
    "df_new = get_days(df)\n",
    "#The following code creates tuples of the stat and end times for further analysis\n",
    "df_new['Start and End Times'] = list(zip(df_new['start time in regular notation'], df_new['end time in regular notation']))\n",
    "#df_new"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Time Delta</th>\n",
       "      <th>uid</th>\n",
       "      <th>start time</th>\n",
       "      <th>location</th>\n",
       "      <th>Time Delta (in seconds)</th>\n",
       "      <th>Time Delta (in minutes)</th>\n",
       "      <th>start time in regular notation</th>\n",
       "      <th>Day</th>\n",
       "      <th>Day of Week</th>\n",
       "      <th>minutes_spent</th>\n",
       "      <th>seconds_spent</th>\n",
       "      <th>end time</th>\n",
       "      <th>start_hour</th>\n",
       "      <th>end time in regular notation</th>\n",
       "      <th>end_hour</th>\n",
       "      <th>Start and End Times</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>00:23:36</td>\n",
       "      <td>u45</td>\n",
       "      <td>1364397661</td>\n",
       "      <td>in[sudikoff]</td>\n",
       "      <td>1416.0</td>\n",
       "      <td>23.600000</td>\n",
       "      <td>2013-03-27 15:21:01</td>\n",
       "      <td>86</td>\n",
       "      <td>2</td>\n",
       "      <td>23</td>\n",
       "      <td>1416</td>\n",
       "      <td>1364399077</td>\n",
       "      <td>15</td>\n",
       "      <td>2013-03-27 15:44:37</td>\n",
       "      <td>15</td>\n",
       "      <td>(2013-03-27 15:21:01, 2013-03-27 15:44:37)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>00:20:58</td>\n",
       "      <td>u45</td>\n",
       "      <td>1364839542</td>\n",
       "      <td>in[sudikoff]</td>\n",
       "      <td>1258.0</td>\n",
       "      <td>20.966667</td>\n",
       "      <td>2013-04-01 18:05:42</td>\n",
       "      <td>91</td>\n",
       "      <td>0</td>\n",
       "      <td>20</td>\n",
       "      <td>1258</td>\n",
       "      <td>1364840800</td>\n",
       "      <td>18</td>\n",
       "      <td>2013-04-01 18:26:40</td>\n",
       "      <td>18</td>\n",
       "      <td>(2013-04-01 18:05:42, 2013-04-01 18:26:40)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>00:03:44</td>\n",
       "      <td>u45</td>\n",
       "      <td>1364842439</td>\n",
       "      <td>in[lsb]</td>\n",
       "      <td>224.0</td>\n",
       "      <td>3.733333</td>\n",
       "      <td>2013-04-01 18:53:59</td>\n",
       "      <td>91</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>224</td>\n",
       "      <td>1364842663</td>\n",
       "      <td>18</td>\n",
       "      <td>2013-04-01 18:57:43</td>\n",
       "      <td>18</td>\n",
       "      <td>(2013-04-01 18:53:59, 2013-04-01 18:57:43)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>00:00:13</td>\n",
       "      <td>u45</td>\n",
       "      <td>1364848920</td>\n",
       "      <td>in[baker-berry]</td>\n",
       "      <td>13.0</td>\n",
       "      <td>0.216667</td>\n",
       "      <td>2013-04-01 20:42:00</td>\n",
       "      <td>91</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13</td>\n",
       "      <td>1364848933</td>\n",
       "      <td>20</td>\n",
       "      <td>2013-04-01 20:42:13</td>\n",
       "      <td>20</td>\n",
       "      <td>(2013-04-01 20:42:00, 2013-04-01 20:42:13)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>00:00:40</td>\n",
       "      <td>u45</td>\n",
       "      <td>1364848920</td>\n",
       "      <td>in[baker-berry]</td>\n",
       "      <td>40.0</td>\n",
       "      <td>0.666667</td>\n",
       "      <td>2013-04-01 20:42:00</td>\n",
       "      <td>91</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>1364848960</td>\n",
       "      <td>20</td>\n",
       "      <td>2013-04-01 20:42:40</td>\n",
       "      <td>20</td>\n",
       "      <td>(2013-04-01 20:42:00, 2013-04-01 20:42:40)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>11881</td>\n",
       "      <td>00:01:39</td>\n",
       "      <td>u49</td>\n",
       "      <td>1366374288</td>\n",
       "      <td>in[baker-berry]</td>\n",
       "      <td>99.0</td>\n",
       "      <td>1.650000</td>\n",
       "      <td>2013-04-19 12:24:48</td>\n",
       "      <td>109</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>99</td>\n",
       "      <td>1366374387</td>\n",
       "      <td>12</td>\n",
       "      <td>2013-04-19 12:26:27</td>\n",
       "      <td>12</td>\n",
       "      <td>(2013-04-19 12:24:48, 2013-04-19 12:26:27)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>11882</td>\n",
       "      <td>00:04:48</td>\n",
       "      <td>u49</td>\n",
       "      <td>1366383400</td>\n",
       "      <td>in[kemeny]</td>\n",
       "      <td>288.0</td>\n",
       "      <td>4.800000</td>\n",
       "      <td>2013-04-19 14:56:40</td>\n",
       "      <td>109</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>288</td>\n",
       "      <td>1366383688</td>\n",
       "      <td>14</td>\n",
       "      <td>2013-04-19 15:01:28</td>\n",
       "      <td>15</td>\n",
       "      <td>(2013-04-19 14:56:40, 2013-04-19 15:01:28)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>11883</td>\n",
       "      <td>01:17:51</td>\n",
       "      <td>u49</td>\n",
       "      <td>1366384317</td>\n",
       "      <td>in[cummings]</td>\n",
       "      <td>4671.0</td>\n",
       "      <td>77.850000</td>\n",
       "      <td>2013-04-19 15:11:57</td>\n",
       "      <td>109</td>\n",
       "      <td>4</td>\n",
       "      <td>77</td>\n",
       "      <td>4671</td>\n",
       "      <td>1366388988</td>\n",
       "      <td>15</td>\n",
       "      <td>2013-04-19 16:29:48</td>\n",
       "      <td>16</td>\n",
       "      <td>(2013-04-19 15:11:57, 2013-04-19 16:29:48)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>11884</td>\n",
       "      <td>08:49:00</td>\n",
       "      <td>u49</td>\n",
       "      <td>1366393411</td>\n",
       "      <td>in[sudikoff]</td>\n",
       "      <td>31740.0</td>\n",
       "      <td>529.000000</td>\n",
       "      <td>2013-04-19 17:43:31</td>\n",
       "      <td>109</td>\n",
       "      <td>4</td>\n",
       "      <td>529</td>\n",
       "      <td>31740</td>\n",
       "      <td>1366425151</td>\n",
       "      <td>17</td>\n",
       "      <td>2013-04-20 02:32:31</td>\n",
       "      <td>2</td>\n",
       "      <td>(2013-04-19 17:43:31, 2013-04-20 02:32:31)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>11885</td>\n",
       "      <td>00:32:09</td>\n",
       "      <td>u49</td>\n",
       "      <td>1366415357</td>\n",
       "      <td>in[hopkins]</td>\n",
       "      <td>1929.0</td>\n",
       "      <td>32.150000</td>\n",
       "      <td>2013-04-19 23:49:17</td>\n",
       "      <td>109</td>\n",
       "      <td>4</td>\n",
       "      <td>32</td>\n",
       "      <td>1929</td>\n",
       "      <td>1366417286</td>\n",
       "      <td>23</td>\n",
       "      <td>2013-04-20 00:21:26</td>\n",
       "      <td>0</td>\n",
       "      <td>(2013-04-19 23:49:17, 2013-04-20 00:21:26)</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>10262 rows × 16 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      Time Delta  uid  start time         location  Time Delta (in seconds)  \\\n",
       "0       00:23:36  u45  1364397661     in[sudikoff]                   1416.0   \n",
       "1       00:20:58  u45  1364839542     in[sudikoff]                   1258.0   \n",
       "2       00:03:44  u45  1364842439          in[lsb]                    224.0   \n",
       "3       00:00:13  u45  1364848920  in[baker-berry]                     13.0   \n",
       "4       00:00:40  u45  1364848920  in[baker-berry]                     40.0   \n",
       "...          ...  ...         ...              ...                      ...   \n",
       "11881   00:01:39  u49  1366374288  in[baker-berry]                     99.0   \n",
       "11882   00:04:48  u49  1366383400       in[kemeny]                    288.0   \n",
       "11883   01:17:51  u49  1366384317     in[cummings]                   4671.0   \n",
       "11884   08:49:00  u49  1366393411     in[sudikoff]                  31740.0   \n",
       "11885   00:32:09  u49  1366415357      in[hopkins]                   1929.0   \n",
       "\n",
       "       Time Delta (in minutes) start time in regular notation  Day  \\\n",
       "0                    23.600000            2013-03-27 15:21:01   86   \n",
       "1                    20.966667            2013-04-01 18:05:42   91   \n",
       "2                     3.733333            2013-04-01 18:53:59   91   \n",
       "3                     0.216667            2013-04-01 20:42:00   91   \n",
       "4                     0.666667            2013-04-01 20:42:00   91   \n",
       "...                        ...                            ...  ...   \n",
       "11881                 1.650000            2013-04-19 12:24:48  109   \n",
       "11882                 4.800000            2013-04-19 14:56:40  109   \n",
       "11883                77.850000            2013-04-19 15:11:57  109   \n",
       "11884               529.000000            2013-04-19 17:43:31  109   \n",
       "11885                32.150000            2013-04-19 23:49:17  109   \n",
       "\n",
       "       Day of Week  minutes_spent  seconds_spent    end time  start_hour  \\\n",
       "0                2             23           1416  1364399077          15   \n",
       "1                0             20           1258  1364840800          18   \n",
       "2                0              3            224  1364842663          18   \n",
       "3                0              0             13  1364848933          20   \n",
       "4                0              0             40  1364848960          20   \n",
       "...            ...            ...            ...         ...         ...   \n",
       "11881            4              1             99  1366374387          12   \n",
       "11882            4              4            288  1366383688          14   \n",
       "11883            4             77           4671  1366388988          15   \n",
       "11884            4            529          31740  1366425151          17   \n",
       "11885            4             32           1929  1366417286          23   \n",
       "\n",
       "      end time in regular notation  end_hour  \\\n",
       "0              2013-03-27 15:44:37        15   \n",
       "1              2013-04-01 18:26:40        18   \n",
       "2              2013-04-01 18:57:43        18   \n",
       "3              2013-04-01 20:42:13        20   \n",
       "4              2013-04-01 20:42:40        20   \n",
       "...                            ...       ...   \n",
       "11881          2013-04-19 12:26:27        12   \n",
       "11882          2013-04-19 15:01:28        15   \n",
       "11883          2013-04-19 16:29:48        16   \n",
       "11884          2013-04-20 02:32:31         2   \n",
       "11885          2013-04-20 00:21:26         0   \n",
       "\n",
       "                              Start and End Times  \n",
       "0      (2013-03-27 15:21:01, 2013-03-27 15:44:37)  \n",
       "1      (2013-04-01 18:05:42, 2013-04-01 18:26:40)  \n",
       "2      (2013-04-01 18:53:59, 2013-04-01 18:57:43)  \n",
       "3      (2013-04-01 20:42:00, 2013-04-01 20:42:13)  \n",
       "4      (2013-04-01 20:42:00, 2013-04-01 20:42:40)  \n",
       "...                                           ...  \n",
       "11881  (2013-04-19 12:24:48, 2013-04-19 12:26:27)  \n",
       "11882  (2013-04-19 14:56:40, 2013-04-19 15:01:28)  \n",
       "11883  (2013-04-19 15:11:57, 2013-04-19 16:29:48)  \n",
       "11884  (2013-04-19 17:43:31, 2013-04-20 02:32:31)  \n",
       "11885  (2013-04-19 23:49:17, 2013-04-20 00:21:26)  \n",
       "\n",
       "[10262 rows x 16 columns]"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_new"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From the SmartGPA paper, only times certain times spent in class locations counted towards class attendance. The times were atleast 90% of Dartmouth class times. I also added a buffer of 5 min for students that stayed after. The times were: 45-55 min, 58-70 min, and 99-120 min.\n",
    "\n",
    "This function searched through the previous dataframe to find only the instances where time spent in a class location matched these specifications."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [],
   "source": [
    "def eligible_times(data):\n",
    "    list_eligible_times = []\n",
    "    list_eligible_uids = []\n",
    "    list_eligible_locations = []\n",
    "    list_eligible_days = []\n",
    "    list_eligible_weekdays = []\n",
    "    for i in data.index:\n",
    "        minutes_spent = data.loc[i]['minutes_spent']\n",
    "        if 45 <= minutes_spent <= 60:\n",
    "            list_eligible_times.append(minutes_spent)\n",
    "            list_eligible_uids.append(df.loc[i]['uid'])\n",
    "            list_eligible_locations.append(df.loc[i]['location'])\n",
    "            list_eligible_days.append(df.loc[i]['Day'])\n",
    "            list_eligible_weekdays.append(df.loc[i]['Day of Week'])\n",
    "        elif 58 <= minutes_spent <= 75:\n",
    "            list_eligible_times.append(minutes_spent)\n",
    "            list_eligible_uids.append(df.loc[i]['uid'])\n",
    "            list_eligible_locations.append(df.loc[i]['location'])\n",
    "            list_eligible_days.append(df.loc[i]['Day'])\n",
    "            list_eligible_weekdays.append(df.loc[i]['Day of Week'])\n",
    "        elif 99 <= minutes_spent <= 120:\n",
    "            list_eligible_times.append(minutes_spent)\n",
    "            list_eligible_uids.append(df.loc[i]['uid'])\n",
    "            list_eligible_locations.append(df.loc[i]['location'])\n",
    "            list_eligible_days.append(df.loc[i]['Day'])\n",
    "            list_eligible_weekdays.append(df.loc[i]['Day of Week'])\n",
    "    df_new = pd.DataFrame({'uid': list_eligible_uids, 'minutes spent': list_eligible_times, 'location': list_eligible_locations, 'Day': list_eligible_days, 'Day of Week': list_eligible_weekdays})\n",
    "    return df_new\n",
    "#This new dataframe contains only the time deltas that met the SmartGPA paper's certifications.\n",
    "df_final_processing = eligible_times(df_new)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>uid</th>\n",
       "      <th>minutes spent</th>\n",
       "      <th>location</th>\n",
       "      <th>Day</th>\n",
       "      <th>Day of Week</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>787</td>\n",
       "      <td>u16</td>\n",
       "      <td>104</td>\n",
       "      <td>in[baker-berry]</td>\n",
       "      <td>148</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>788</td>\n",
       "      <td>u15</td>\n",
       "      <td>47</td>\n",
       "      <td>in[lsb]</td>\n",
       "      <td>86</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>789</td>\n",
       "      <td>u15</td>\n",
       "      <td>53</td>\n",
       "      <td>in[baker-berry]</td>\n",
       "      <td>87</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>790</td>\n",
       "      <td>u15</td>\n",
       "      <td>54</td>\n",
       "      <td>in[lsb]</td>\n",
       "      <td>88</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>791</td>\n",
       "      <td>u15</td>\n",
       "      <td>54</td>\n",
       "      <td>in[lsb]</td>\n",
       "      <td>98</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>792</td>\n",
       "      <td>u15</td>\n",
       "      <td>62</td>\n",
       "      <td>in[lsb]</td>\n",
       "      <td>120</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>793</td>\n",
       "      <td>u15</td>\n",
       "      <td>53</td>\n",
       "      <td>in[lsb]</td>\n",
       "      <td>130</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>794</td>\n",
       "      <td>u15</td>\n",
       "      <td>50</td>\n",
       "      <td>in[hopkins]</td>\n",
       "      <td>133</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>795</td>\n",
       "      <td>u15</td>\n",
       "      <td>73</td>\n",
       "      <td>in[sudikoff]</td>\n",
       "      <td>147</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>796</td>\n",
       "      <td>u15</td>\n",
       "      <td>45</td>\n",
       "      <td>in[lsb]</td>\n",
       "      <td>149</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>797</td>\n",
       "      <td>u49</td>\n",
       "      <td>72</td>\n",
       "      <td>in[reed]</td>\n",
       "      <td>88</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>798</td>\n",
       "      <td>u49</td>\n",
       "      <td>64</td>\n",
       "      <td>in[lsb]</td>\n",
       "      <td>93</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>799</td>\n",
       "      <td>u49</td>\n",
       "      <td>110</td>\n",
       "      <td>in[lsb]</td>\n",
       "      <td>95</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>800</td>\n",
       "      <td>u49</td>\n",
       "      <td>120</td>\n",
       "      <td>in[baker-berry]</td>\n",
       "      <td>95</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>801</td>\n",
       "      <td>u49</td>\n",
       "      <td>100</td>\n",
       "      <td>in[hopkins]</td>\n",
       "      <td>95</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>802</td>\n",
       "      <td>u49</td>\n",
       "      <td>65</td>\n",
       "      <td>in[hopkins]</td>\n",
       "      <td>95</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>803</td>\n",
       "      <td>u49</td>\n",
       "      <td>110</td>\n",
       "      <td>in[baker-berry]</td>\n",
       "      <td>100</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>804</td>\n",
       "      <td>u49</td>\n",
       "      <td>53</td>\n",
       "      <td>in[sudikoff]</td>\n",
       "      <td>100</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>805</td>\n",
       "      <td>u49</td>\n",
       "      <td>60</td>\n",
       "      <td>in[baker-berry]</td>\n",
       "      <td>101</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>806</td>\n",
       "      <td>u49</td>\n",
       "      <td>51</td>\n",
       "      <td>in[kemeny]</td>\n",
       "      <td>101</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>807</td>\n",
       "      <td>u49</td>\n",
       "      <td>68</td>\n",
       "      <td>in[reed]</td>\n",
       "      <td>101</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>808</td>\n",
       "      <td>u49</td>\n",
       "      <td>68</td>\n",
       "      <td>in[hopkins]</td>\n",
       "      <td>101</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>809</td>\n",
       "      <td>u49</td>\n",
       "      <td>64</td>\n",
       "      <td>in[baker-berry]</td>\n",
       "      <td>102</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>810</td>\n",
       "      <td>u49</td>\n",
       "      <td>56</td>\n",
       "      <td>in[baker-berry]</td>\n",
       "      <td>102</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>811</td>\n",
       "      <td>u49</td>\n",
       "      <td>62</td>\n",
       "      <td>in[hopkins]</td>\n",
       "      <td>105</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     uid  minutes spent         location  Day  Day of Week\n",
       "787  u16            104  in[baker-berry]  148            1\n",
       "788  u15             47          in[lsb]   86            2\n",
       "789  u15             53  in[baker-berry]   87            3\n",
       "790  u15             54          in[lsb]   88            4\n",
       "791  u15             54          in[lsb]   98            0\n",
       "792  u15             62          in[lsb]  120            1\n",
       "793  u15             53          in[lsb]  130            4\n",
       "794  u15             50      in[hopkins]  133            0\n",
       "795  u15             73     in[sudikoff]  147            0\n",
       "796  u15             45          in[lsb]  149            2\n",
       "797  u49             72         in[reed]   88            4\n",
       "798  u49             64          in[lsb]   93            2\n",
       "799  u49            110          in[lsb]   95            4\n",
       "800  u49            120  in[baker-berry]   95            4\n",
       "801  u49            100      in[hopkins]   95            4\n",
       "802  u49             65      in[hopkins]   95            4\n",
       "803  u49            110  in[baker-berry]  100            2\n",
       "804  u49             53     in[sudikoff]  100            2\n",
       "805  u49             60  in[baker-berry]  101            3\n",
       "806  u49             51       in[kemeny]  101            3\n",
       "807  u49             68         in[reed]  101            3\n",
       "808  u49             68      in[hopkins]  101            3\n",
       "809  u49             64  in[baker-berry]  102            4\n",
       "810  u49             56  in[baker-berry]  102            4\n",
       "811  u49             62      in[hopkins]  105            0"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_final_processing.tail(25)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since we have a dataframe of time spans in class locations within the periods in which classes take place, we can infer class attendance rates: the number of rows for each student for which the day is the same is equivalent to the number of class attendances that student had for that particular day. We can then divide this number by the number of classes a student had on that particular day (depending on the day of the week). Let's do that below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [],
   "source": [
    "def attendance_counter(data):\n",
    "    attendance_num = 1\n",
    "    list_attendance_counter = []\n",
    "    list_id = []\n",
    "    for i in range(0, len(data)):\n",
    "        try: \n",
    "            if data.Day[i] == data.Day[i+1]:\n",
    "                attendance_num += 1\n",
    "            else:\n",
    "                list_attendance_counter.append(attendance_num)\n",
    "                attendance_num = 1\n",
    "        except:\n",
    "            list_attendance_counter.append(attendance_num)\n",
    "            attendance_num = 1\n",
    "    return list_attendance_counter\n",
    "\n",
    "#This list has all of the attendances per day (# of consecutive rows where day is the same)\n",
    "list_attendances = attendance_counter(df_final_processing)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [],
   "source": [
    "#To be able to track user id, day and day of week alongside the number of class attendances, I had to use this function:\n",
    "#This function is based off a formula I found that can find the index for the following data:\n",
    "    #user id, day and day of week\n",
    "def get_sums(list1):\n",
    "    list_sums_b = []\n",
    "    for i in range(0,len(list1)+1):\n",
    "        new_list = list1[:i]\n",
    "        sums = sum(new_list)\n",
    "        list_sums_b.append(sums)\n",
    "    return list_sums_b\n",
    "list_attendance_shifts = get_sums(list_attendances)\n",
    "list_new = list_attendance_shifts\n",
    "\n",
    "def get_index_list(list1):\n",
    "    list_finale = []\n",
    "    for i in list1:\n",
    "        if i <= 811:\n",
    "            list_finale.append(i)\n",
    "    return list_finale\n",
    "list_finale = get_index_list(list_new)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [],
   "source": [
    "#This function creates a new dataframe that consists of, for everyday, the user id, day and day of week parameters.\n",
    "#I was able to create a new column in this dataframe that consists of the number of class attendances per student per day.\n",
    "def get_days(data, list1):\n",
    "    days = []\n",
    "    uid = []\n",
    "    weekday = []\n",
    "    for i in range(0, len(list1)):\n",
    "        weekday.append(data['Day of Week'][list1[i]])\n",
    "        uid.append(data.uid[list1[i]])\n",
    "        days.append(data.Day[list1[i]])\n",
    "        df1 = pd.DataFrame({'uid': uid, 'Day': days, 'Weekday': weekday})#'Num Attendances': list_attendances})\n",
    "    return df1\n",
    "df_finale = get_days(df_final_processing, list_finale)\n",
    "df_finale['Num Attendances'] = list_attendances"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>uid</th>\n",
       "      <th>Day</th>\n",
       "      <th>Weekday</th>\n",
       "      <th>Num Attendances</th>\n",
       "      <th>Name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>u45</td>\n",
       "      <td>92</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Tuesday</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>u45</td>\n",
       "      <td>98</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>Monday</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>u45</td>\n",
       "      <td>99</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>Tuesday</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>u45</td>\n",
       "      <td>102</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>Friday</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>u45</td>\n",
       "      <td>105</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>Monday</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>529</td>\n",
       "      <td>u49</td>\n",
       "      <td>95</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>Friday</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>530</td>\n",
       "      <td>u49</td>\n",
       "      <td>100</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>Wednesday</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>531</td>\n",
       "      <td>u49</td>\n",
       "      <td>101</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>Thursday</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>532</td>\n",
       "      <td>u49</td>\n",
       "      <td>102</td>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>Friday</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>533</td>\n",
       "      <td>u49</td>\n",
       "      <td>105</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>Monday</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>534 rows × 5 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     uid  Day  Weekday  Num Attendances       Name\n",
       "0    u45   92        1                1    Tuesday\n",
       "1    u45   98        0                2     Monday\n",
       "2    u45   99        1                2    Tuesday\n",
       "3    u45  102        4                1     Friday\n",
       "4    u45  105        0                1     Monday\n",
       "..   ...  ...      ...              ...        ...\n",
       "529  u49   95        4                4     Friday\n",
       "530  u49  100        2                2  Wednesday\n",
       "531  u49  101        3                4   Thursday\n",
       "532  u49  102        4                2     Friday\n",
       "533  u49  105        0                1     Monday\n",
       "\n",
       "[534 rows x 5 columns]"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#This function assigns the name of the day for each day of week parameter in the previous dataframe.\n",
    "def get_days(data):\n",
    "    list_day_names = []\n",
    "    for i in data.Weekday:\n",
    "        if i == 0:\n",
    "            list_day_names.append('Monday')\n",
    "        elif i == 1:\n",
    "            list_day_names.append('Tuesday')\n",
    "        elif i == 2:\n",
    "            list_day_names.append('Wednesday')\n",
    "        elif i == 3:\n",
    "            list_day_names.append('Thursday')\n",
    "        else:\n",
    "            list_day_names.append('Friday')\n",
    "    data['Name'] = list_day_names\n",
    "    return data\n",
    "df_finale = get_days(df_finale)\n",
    "df_finale"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [],
   "source": [
    "class_2 = class_2.reset_index()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>index</th>\n",
       "      <th>uid</th>\n",
       "      <th>Day</th>\n",
       "      <th>Weekday</th>\n",
       "      <th>Num Attendances</th>\n",
       "      <th>Name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>u45</td>\n",
       "      <td>92</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Tuesday</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>u45</td>\n",
       "      <td>98</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>Monday</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>u45</td>\n",
       "      <td>99</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>Tuesday</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>u45</td>\n",
       "      <td>102</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>Friday</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>u45</td>\n",
       "      <td>105</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>Monday</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>478</td>\n",
       "      <td>529</td>\n",
       "      <td>u49</td>\n",
       "      <td>95</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>Friday</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>479</td>\n",
       "      <td>530</td>\n",
       "      <td>u49</td>\n",
       "      <td>100</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>Wednesday</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>480</td>\n",
       "      <td>531</td>\n",
       "      <td>u49</td>\n",
       "      <td>101</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>Thursday</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>481</td>\n",
       "      <td>532</td>\n",
       "      <td>u49</td>\n",
       "      <td>102</td>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>Friday</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>482</td>\n",
       "      <td>533</td>\n",
       "      <td>u49</td>\n",
       "      <td>105</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>Monday</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>483 rows × 6 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     index  uid  Day  Weekday  Num Attendances       Name\n",
       "0        0  u45   92        1                1    Tuesday\n",
       "1        1  u45   98        0                2     Monday\n",
       "2        2  u45   99        1                2    Tuesday\n",
       "3        3  u45  102        4                1     Friday\n",
       "4        4  u45  105        0                1     Monday\n",
       "..     ...  ...  ...      ...              ...        ...\n",
       "478    529  u49   95        4                4     Friday\n",
       "479    530  u49  100        2                2  Wednesday\n",
       "480    531  u49  101        3                4   Thursday\n",
       "481    532  u49  102        4                2     Friday\n",
       "482    533  u49  105        0                1     Monday\n",
       "\n",
       "[483 rows x 6 columns]"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#This function removes student data for which there is no class data.\n",
    "def remove_students_no_data(data):\n",
    "    data = data[data['uid'] != 'u56']\n",
    "    data = data[data['uid'] != 'u36']\n",
    "    data = data[data['uid'] != 'u39']\n",
    "    data = data[data['uid'] != 'u00']\n",
    "    return data\n",
    "df_finale = remove_students_no_data(df_finale)\n",
    "df_finale = df_finale.reset_index()\n",
    "df_finale"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For each student in the dataframe, there is data on the number of class attendances per day. There isn't data on number of classes each student had that day, but we already found this previously. Let's add that data as a column to our dataframe using the following function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [],
   "source": [
    "def match_classes(data):\n",
    "    num_classes = []\n",
    "    for i in range(0,len(data)):\n",
    "        if data.Name[i] == 'Monday':\n",
    "            for j in range(0, len(class_2)):\n",
    "                if class_2.u00[j] == data.uid[i]:\n",
    "                    num_classes.append(class_2['Monday'][j])\n",
    "        elif data.Name[i] == 'Tuesday':\n",
    "            for j in range(0, len(class_2)):\n",
    "                if class_2.u00[j] == data.uid[i]:\n",
    "                    num_classes.append(class_2['Tuesday'][j])\n",
    "        elif data.Name[i] == 'Wednesday':\n",
    "            for j in range(0, len(class_2)):\n",
    "                if class_2.u00[j] == data.uid[i]:\n",
    "                    num_classes.append(class_2['Wednesday'][j])\n",
    "        elif data.Name[i] == 'Thursday':\n",
    "            for j in range(0, len(class_2)):\n",
    "                if class_2.u00[j] == data.uid[i]:\n",
    "                    num_classes.append(class_2['Thursday'][j])\n",
    "        else:\n",
    "            for j in range(0, len(class_2)):\n",
    "                if class_2.u00[j] == data.uid[i]:\n",
    "                    num_classes.append(class_2['Friday'][j])\n",
    "        #data['Num Classes on Day'] = num_classes\n",
    "    return num_classes\n",
    "num_classes_list = match_classes(df_finale)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This function adds the data on the number of classes each student had for a particular data point in the dataframe to a new column in the dataframe.\n",
    "\n",
    "So now we have data on number of class attendances and number of classes per day.\n",
    "\n",
    "The final step is to find the class attendance rates, which is simply a ratio of the two parametersL"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_class_attend_rates(data, list1):\n",
    "    data['Num Classes on Day'] = list1 \n",
    "    data['Attendance Rate'] = data['Num Attendances']/data['Num Classes on Day']\n",
    "    return data\n",
    "\n",
    "#This dataframe has class attendance rate information\n",
    "df_finale = get_class_attend_rates(df_finale, num_classes_list)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There's one slight issue: the dataframe has class attendance rates over 1.0 for some days. However, this is fine as some students have extracurriculars, events or study sessions in class location. We can thus simply reset rates above 1.0 to 1.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_actual_rates(data):\n",
    "    list_rates = []\n",
    "    for i in data['Attendance Rate']:\n",
    "        if i <= 1.0:\n",
    "            list_rates.append(i)\n",
    "        else:\n",
    "            list_rates.append(1.0)\n",
    "    data['Actual Rates'] = list_rates\n",
    "    return data\n",
    "\n",
    "#This is the final dataframe with the final class attendance rate data\n",
    "df_class_attendance_rates = get_actual_rates(df_finale)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>index</th>\n",
       "      <th>uid</th>\n",
       "      <th>Day</th>\n",
       "      <th>Weekday</th>\n",
       "      <th>Num Attendances</th>\n",
       "      <th>Name</th>\n",
       "      <th>Num Classes on Day</th>\n",
       "      <th>Attendance Rate</th>\n",
       "      <th>Actual Rates</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>u45</td>\n",
       "      <td>92</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Tuesday</td>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>u45</td>\n",
       "      <td>98</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>Monday</td>\n",
       "      <td>1</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>u45</td>\n",
       "      <td>99</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>Tuesday</td>\n",
       "      <td>1</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>u45</td>\n",
       "      <td>102</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>Friday</td>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>u45</td>\n",
       "      <td>105</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>Monday</td>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>478</td>\n",
       "      <td>529</td>\n",
       "      <td>u49</td>\n",
       "      <td>95</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>Friday</td>\n",
       "      <td>3</td>\n",
       "      <td>1.333333</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>479</td>\n",
       "      <td>530</td>\n",
       "      <td>u49</td>\n",
       "      <td>100</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>Wednesday</td>\n",
       "      <td>3</td>\n",
       "      <td>0.666667</td>\n",
       "      <td>0.666667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>480</td>\n",
       "      <td>531</td>\n",
       "      <td>u49</td>\n",
       "      <td>101</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>Thursday</td>\n",
       "      <td>2</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>481</td>\n",
       "      <td>532</td>\n",
       "      <td>u49</td>\n",
       "      <td>102</td>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>Friday</td>\n",
       "      <td>3</td>\n",
       "      <td>0.666667</td>\n",
       "      <td>0.666667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>482</td>\n",
       "      <td>533</td>\n",
       "      <td>u49</td>\n",
       "      <td>105</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>Monday</td>\n",
       "      <td>3</td>\n",
       "      <td>0.333333</td>\n",
       "      <td>0.333333</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>483 rows × 9 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     index  uid  Day  Weekday  Num Attendances       Name  Num Classes on Day  \\\n",
       "0        0  u45   92        1                1    Tuesday                   1   \n",
       "1        1  u45   98        0                2     Monday                   1   \n",
       "2        2  u45   99        1                2    Tuesday                   1   \n",
       "3        3  u45  102        4                1     Friday                   1   \n",
       "4        4  u45  105        0                1     Monday                   1   \n",
       "..     ...  ...  ...      ...              ...        ...                 ...   \n",
       "478    529  u49   95        4                4     Friday                   3   \n",
       "479    530  u49  100        2                2  Wednesday                   3   \n",
       "480    531  u49  101        3                4   Thursday                   2   \n",
       "481    532  u49  102        4                2     Friday                   3   \n",
       "482    533  u49  105        0                1     Monday                   3   \n",
       "\n",
       "     Attendance Rate  Actual Rates  \n",
       "0           1.000000      1.000000  \n",
       "1           2.000000      1.000000  \n",
       "2           2.000000      1.000000  \n",
       "3           1.000000      1.000000  \n",
       "4           1.000000      1.000000  \n",
       "..               ...           ...  \n",
       "478         1.333333      1.000000  \n",
       "479         0.666667      0.666667  \n",
       "480         2.000000      1.000000  \n",
       "481         0.666667      0.666667  \n",
       "482         0.333333      0.333333  \n",
       "\n",
       "[483 rows x 9 columns]"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_class_attendance_rates"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alright! We've finally derived class attendance rates. Now this process was pretty complex as a whole, so, once again, don't fret if you don't quite understand everything yet. However, try to understand the code and, most importantly, the thought process and methodology used to find the class attendance rates. In essence, we used the following steps:\n",
    "- using \"shift\" numbers, find the amount of time each student spent at class locations\n",
    "- find when classes occurred and when students had class as well has how many classes each student had on each day of the week\n",
    "- finally, look for instances of time spent at class locations that match the specifications of the SmartGPA paper (i.e. attending 90% of class). Then, count the number of valid class attendances in each day for each student and find the ratio of the number of attendances they had and number of classes they had for that particular day (found before)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With your own data exploration, feel free to use the functions/thought processes used here if helpful! As before, if you have any questions, don't hesitate to ask!"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}