hourly_echo_collect.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Hourly Echo\n",
    "\n",
    "This notebook downloads Quotes data and create a database."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Quotes Dataset\n",
    "\n",
    "We use Quotes Dataset from [ShivaliGoel/Quotes-500K](https://github.com/ShivaliGoel/Quotes-500K)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Download Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import gdown"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "# !gdown --fuzzy https://drive.google.com/file/d/1dPCpNIUxqhe2GccLF0tmAapnwgA5Olr2/view"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load csv data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "quotes = open('quotes_dataset.csv').read().splitlines()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total quotes in dataset: 499996\n"
     ]
    }
   ],
   "source": [
    "print(\"Total quotes in dataset:\", len(quotes))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\"I'm selfish, impatient and a little insecure. I make mistakes, I am out of control and at times hard to handle. But if you can't handle me at my worst, then you sure as hell don't deserve me at my best.\",Marilyn Monroe,\"attributed-no-source, best, life, love, mistakes, out-of-control, truth, worst\",,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,\n"
     ]
    }
   ],
   "source": [
    "print(quotes[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Clean Data\n",
    "\n",
    "This Dataset doesn't have cleaned data. So, we have to clean it and extract data as per our need"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "quote_list = []\n",
    "\n",
    "for quote_data in quotes:\n",
    "    temp_quote_data = quote_data\n",
    "    if quote_data.strip() != '':\n",
    "        try:\n",
    "            if quote_data.startswith('\"'):\n",
    "                quote = quote_data.split('\",')[0].replace('\"', '')\n",
    "                quote_data = quote_data.split('\",')[1]\n",
    "            elif quote_data.startswith(\"'\"):\n",
    "                quote = quote_data.split(\"',\")[0].replace(\"'\", '')\n",
    "                quote_data = quote_data.split(\"',\")[1]\n",
    "            else:\n",
    "                quote = quote_data.split(',')[0]\n",
    "                quote_data = quote_data.split(',')[1]\n",
    "        except:\n",
    "            # print(\"Error => \", temp_quote_data)\n",
    "            continue\n",
    "\n",
    "        try:\n",
    "            if quote_data.startswith('\"'):\n",
    "                author = quote_data.split('\",')[0].replace('\"', '')\n",
    "            elif quote_data.startswith(\"'\"):\n",
    "                author = quote_data.split(\"',\")[0].replace(\"'\", '')\n",
    "            else:\n",
    "                author = quote_data.split(',')[0]\n",
    "            author = author.split(\",\")[0].strip()\n",
    "        except:\n",
    "            # print(\"Error => \", temp_quote_data)\n",
    "            continue\n",
    "\n",
    "        if quote.strip() != '' and quote.strip().endswith(\".\") and author.strip() != '':\n",
    "            quote_list.append([quote, author])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total Quotes after cleaning: 418368\n"
     ]
    }
   ],
   "source": [
    "print(\"Total Quotes after cleaning:\", len(quote_list))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[\"I'm selfish, impatient and a little insecure. I make mistakes, I am out of control and at times hard to handle. But if you can't handle me at my worst, then you sure as hell don't deserve me at my best.\", 'Marilyn Monroe']\n"
     ]
    }
   ],
   "source": [
    "print(quote_list[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Select Quotes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([2.50977e+05, 1.04897e+05, 3.14640e+04, 1.35760e+04, 6.89500e+03,\n",
       "        3.79400e+03, 2.31300e+03, 1.38700e+03, 8.99000e+02, 5.53000e+02,\n",
       "        3.79000e+02, 2.53000e+02, 2.21000e+02, 1.55000e+02, 1.24000e+02,\n",
       "        9.70000e+01, 7.50000e+01, 6.50000e+01, 5.50000e+01, 4.70000e+01,\n",
       "        3.80000e+01, 3.30000e+01, 2.50000e+01, 2.20000e+01, 2.40000e+01]),\n",
       " array([   4.  ,  163.28,  322.56,  481.84,  641.12,  800.4 ,  959.68,\n",
       "        1118.96, 1278.24, 1437.52, 1596.8 , 1756.08, 1915.36, 2074.64,\n",
       "        2233.92, 2393.2 , 2552.48, 2711.76, 2871.04, 3030.32, 3189.6 ,\n",
       "        3348.88, 3508.16, 3667.44, 3826.72, 3986.  ]),\n",
       " <BarContainer object of 25 artists>)"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkIAAAGdCAYAAAD+JxxnAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAsqUlEQVR4nO3dfXBUVZ7/8U8S6CQI3QFCnoYAQRwQeRoChHaU0SJFoxlXRqYKkHIiIhRsYglRnpQJ6G+q4mLNKo4IO2utcWtFkC1BBY1mgoRVAkgkQhCywsQNLnRAMGmIkEByfn9YuUvzHAyE5LxfVbcqfc+3T59vbpj+TPe91xBjjBEAAICFQlt6AQAAAC2FIAQAAKxFEAIAANYiCAEAAGsRhAAAgLUIQgAAwFoEIQAAYC2CEAAAsFa7ll7AzayhoUGHDh1Sp06dFBIS0tLLAQAAV8EYoxMnTighIUGhoZf/zIcgdBmHDh1SYmJiSy8DAABcg4MHD6p79+6XrSEIXUanTp0k/fSLdLvdLbwaAABwNQKBgBITE5338cshCF1G49dhbrebIAQAQCtzNae1cLI0AACwFkEIAABYiyAEAACsRRACAADWIggBAABrEYQAAIC1CEIAAMBaBCEAAGAtghAAALAWQQgAAFiLIAQAAKzVpCCUk5Oj4cOHq1OnToqJidG4ceNUVlYWVHPPPfcoJCQkaJsxY0ZQTUVFhdLS0tShQwfFxMRozpw5Onv2bFDNpk2bNHToUIWHh6tPnz7Kzc29YD3Lli1Tr169FBERoZSUFG3fvj1o/PTp08rIyFDXrl3VsWNHjR8/XpWVlU1pGQAAtGFNCkKFhYXKyMjQ1q1blZ+frzNnzmjMmDGqqakJqps2bZoOHz7sbEuWLHHG6uvrlZaWprq6Om3ZskVvvvmmcnNzlZ2d7dSUl5crLS1N9957r0pKSjRr1iw9/vjj+vjjj52a1atXKysrS4sWLdKXX36pwYMHy+fz6ciRI07N7Nmz9cEHH2jNmjUqLCzUoUOH9NBDDzX5lwQAANoo8zMcOXLESDKFhYXOvt/85jfmySefvORzPvzwQxMaGmr8fr+zb/ny5cbtdpva2lpjjDFz5841d9xxR9DzJkyYYHw+n/N4xIgRJiMjw3lcX19vEhISTE5OjjHGmKqqKtO+fXuzZs0ap2bv3r1GkikqKrqq/qqrq40kU11dfVX1AACg5TXl/bvdzwlR1dXVkqQuXboE7X/rrbf0H//xH4qLi9MDDzygP/7xj+rQoYMkqaioSAMHDlRsbKxT7/P5NHPmTO3Zs0e/+tWvVFRUpNTU1KA5fT6fZs2aJUmqq6tTcXGxFixY4IyHhoYqNTVVRUVFkqTi4mKdOXMmaJ5+/fqpR48eKioq0siRI39O682i1/wNzTLPty+kNcs8AADY5pqDUENDg2bNmqVf//rXGjBggLP/4YcfVs+ePZWQkKBdu3Zp3rx5Kisr07vvvitJ8vv9QSFIkvPY7/dftiYQCOjUqVP64YcfVF9ff9Gaffv2OXO4XC5FRUVdUNP4Ouerra1VbW2t8zgQCFztrwMAALRC1xyEMjIyVFpaqs8++yxo//Tp052fBw4cqPj4eI0ePVoHDhzQrbfeeu0rvQFycnL03HPPtfQyAADADXJNl89nZmZq/fr1+vTTT9W9e/fL1qakpEiS9u/fL0mKi4u74MqtxsdxcXGXrXG73YqMjFR0dLTCwsIuWnPuHHV1daqqqrpkzfkWLFig6upqZzt48OBlewMAAK1bk4KQMUaZmZlau3atNm7cqKSkpCs+p6SkRJIUHx8vSfJ6vdq9e3fQ1V35+flyu93q37+/U1NQUBA0T35+vrxeryTJ5XIpOTk5qKahoUEFBQVOTXJystq3bx9UU1ZWpoqKCqfmfOHh4XK73UEbAABou5r01VhGRoZWrlyp9957T506dXLOtfF4PIqMjNSBAwe0cuVK3X///eratat27dql2bNna9SoURo0aJAkacyYMerfv78eeeQRLVmyRH6/XwsXLlRGRobCw8MlSTNmzNCrr76quXPn6rHHHtPGjRv1zjvvaMOG/zu5OCsrS+np6Ro2bJhGjBihl19+WTU1NZoyZYqzpqlTpyorK0tdunSR2+3WE088Ia/Xe1OcKA0AAFpek4LQ8uXLJf1008RzvfHGG3r00Uflcrn0t7/9zQkliYmJGj9+vBYuXOjUhoWFaf369Zo5c6a8Xq9uueUWpaen6/nnn3dqkpKStGHDBs2ePVtLly5V9+7d9frrr8vn8zk1EyZM0NGjR5WdnS2/368hQ4YoLy8v6ATql156SaGhoRo/frxqa2vl8/n02muvNekXBAAA2q4QY4xp6UXcrAKBgDwej6qrq6/L12RcPg8AQPNryvs3/60xAABgLYIQAACwFkEIAABYiyAEAACsRRACAADWIggBAABrEYQAAIC1CEIAAMBaBCEAAGAtghAAALAWQQgAAFiLIAQAAKxFEAIAANYiCAEAAGsRhAAAgLUIQgAAwFoEIQAAYC2CEAAAsBZBCAAAWIsgBAAArEUQAgAA1iIIAQAAaxGEAACAtQhCAADAWgQhAABgLYIQAACwFkEIAABYiyAEAACsRRACAADWIggBAABrEYQAAIC1CEIAAMBaBCEAAGAtghAAALAWQQgAAFiLIAQAAKxFEAIAANYiCAEAAGsRhAAAgLUIQgAAwFoEIQAAYC2CEAAAsBZBCAAAWIsgBAAArEUQAgAA1iIIAQAAaxGEAACAtQhCAADAWgQhAABgLYIQAACwFkEIAABYiyAEAACsRRACAADWIggBAABrEYQAAIC1CEIAAMBaBCEAAGAtghAAALAWQQgAAFiLIAQAAKzVpCCUk5Oj4cOHq1OnToqJidG4ceNUVlYWVHP69GllZGSoa9eu6tixo8aPH6/KysqgmoqKCqWlpalDhw6KiYnRnDlzdPbs2aCaTZs2aejQoQoPD1efPn2Um5t7wXqWLVumXr16KSIiQikpKdq+fXuT1wIAAOzVpCBUWFiojIwMbd26Vfn5+Tpz5ozGjBmjmpoap2b27Nn64IMPtGbNGhUWFurQoUN66KGHnPH6+nqlpaWprq5OW7Zs0Ztvvqnc3FxlZ2c7NeXl5UpLS9O9996rkpISzZo1S48//rg+/vhjp2b16tXKysrSokWL9OWXX2rw4MHy+Xw6cuTIVa8FAADYLcQYY671yUePHlVMTIwKCws1atQoVVdXq1u3blq5cqV+//vfS5L27dun22+/XUVFRRo5cqQ++ugj/fa3v9WhQ4cUGxsrSVqxYoXmzZuno0ePyuVyad68edqwYYNKS0ud15o4caKqqqqUl5cnSUpJSdHw4cP16quvSpIaGhqUmJioJ554QvPnz7+qtVxJIBCQx+NRdXW13G73tf6aLqnX/A3NMs+3L6Q1yzwAALQFTXn//lnnCFVXV0uSunTpIkkqLi7WmTNnlJqa6tT069dPPXr0UFFRkSSpqKhIAwcOdEKQJPl8PgUCAe3Zs8epOXeOxprGOerq6lRcXBxUExoaqtTUVKfmatZyvtraWgUCgaANAAC0XdcchBoaGjRr1iz9+te/1oABAyRJfr9fLpdLUVFRQbWxsbHy+/1OzbkhqHG8cexyNYFAQKdOndL333+v+vr6i9acO8eV1nK+nJwceTweZ0tMTLzK3wYAAGiNrjkIZWRkqLS0VKtWrWrO9bSoBQsWqLq62tkOHjzY0ksCAADXUbtreVJmZqbWr1+vzZs3q3v37s7+uLg41dXVqaqqKuiTmMrKSsXFxTk151/d1Xgl17k151/dVVlZKbfbrcjISIWFhSksLOyiNefOcaW1nC88PFzh4eFN+E0AAIDWrEmfCBljlJmZqbVr12rjxo1KSkoKGk9OTlb79u1VUFDg7CsrK1NFRYW8Xq8kyev1avfu3UFXd+Xn58vtdqt///5OzblzNNY0zuFyuZScnBxU09DQoIKCAqfmatYCAADs1qRPhDIyMrRy5Uq999576tSpk3OujcfjUWRkpDwej6ZOnaqsrCx16dJFbrdbTzzxhLxer3OV1pgxY9S/f3898sgjWrJkifx+vxYuXKiMjAzn05gZM2bo1Vdf1dy5c/XYY49p48aNeuedd7Rhw/9dZZWVlaX09HQNGzZMI0aM0Msvv6yamhpNmTLFWdOV1gIAAOzWpCC0fPlySdI999wTtP+NN97Qo48+Kkl66aWXFBoaqvHjx6u2tlY+n0+vvfaaUxsWFqb169dr5syZ8nq9uuWWW5Senq7nn3/eqUlKStKGDRs0e/ZsLV26VN27d9frr78un8/n1EyYMEFHjx5Vdna2/H6/hgwZory8vKATqK+0FgAAYLefdR+hto77CAEA0PrcsPsIAQAAtGYEIQAAYC2CEAAAsBZBCAAAWIsgBAAArEUQAgAA1iIIAQAAaxGEAACAtQhCAADAWgQhAABgLYIQAACwFkEIAABYiyAEAACsRRACAADWIggBAABrEYQAAIC1CEIAAMBaBCEAAGAtghAAALAWQQgAAFiLIAQAAKxFEAIAANYiCAEAAGsRhAAAgLUIQgAAwFoEIQAAYC2CEAAAsBZBCAAAWIsgBAAArEUQAgAA1iIIAQAAaxGEAACAtQhCAADAWgQhAABgLYIQAACwFkEIAABYiyAEAACsRRACAADWIggBAABrEYQAAIC1CEIAAMBaBCEAAGAtghAAALAWQQgAAFiLIAQAAKxFEAIAANYiCAEAAGsRhAAAgLUIQgAAwFoEIQAAYC2CEAAAsBZBCAAAWIsgBAAArEUQAgAA1iIIAQAAaxGEAACAtQhCAADAWgQhAABgLYIQAACwVpOD0ObNm/XAAw8oISFBISEhWrduXdD4o48+qpCQkKBt7NixQTXHjx/X5MmT5Xa7FRUVpalTp+rkyZNBNbt27dLdd9+tiIgIJSYmasmSJResZc2aNerXr58iIiI0cOBAffjhh0HjxhhlZ2crPj5ekZGRSk1N1TfffNPUlgEAQBvV5CBUU1OjwYMHa9myZZesGTt2rA4fPuxsb7/9dtD45MmTtWfPHuXn52v9+vXavHmzpk+f7owHAgGNGTNGPXv2VHFxsV588UUtXrxYf/3rX52aLVu2aNKkSZo6dap27typcePGady4cSotLXVqlixZoldeeUUrVqzQtm3bdMstt8jn8+n06dNNbRsAALRBIcYYc81PDgnR2rVrNW7cOGffo48+qqqqqgs+KWq0d+9e9e/fX1988YWGDRsmScrLy9P999+v7777TgkJCVq+fLmeffZZ+f1+uVwuSdL8+fO1bt067du3T5I0YcIE1dTUaP369c7cI0eO1JAhQ7RixQoZY5SQkKCnnnpKTz/9tCSpurpasbGxys3N1cSJE6/YXyAQkMfjUXV1tdxu97X8ii6r1/wNzTLPty+kNcs8AAC0BU15/74u5wht2rRJMTEx6tu3r2bOnKljx445Y0VFRYqKinJCkCSlpqYqNDRU27Ztc2pGjRrlhCBJ8vl8Kisr0w8//ODUpKamBr2uz+dTUVGRJKm8vFx+vz+oxuPxKCUlxak5X21trQKBQNAGAADarmYPQmPHjtW///u/q6CgQP/0T/+kwsJC3Xfffaqvr5ck+f1+xcTEBD2nXbt26tKli/x+v1MTGxsbVNP4+Eo1546f+7yL1ZwvJydHHo/H2RITE5vcPwAAaD3aNfeE537lNHDgQA0aNEi33nqrNm3apNGjRzf3yzWrBQsWKCsry3kcCAQIQwAAtGHX/fL53r17Kzo6Wvv375ckxcXF6ciRI0E1Z8+e1fHjxxUXF+fUVFZWBtU0Pr5Szbnj5z7vYjXnCw8Pl9vtDtoAAEDbdd2D0Hfffadjx44pPj5ekuT1elVVVaXi4mKnZuPGjWpoaFBKSopTs3nzZp05c8apyc/PV9++fdW5c2enpqCgIOi18vPz5fV6JUlJSUmKi4sLqgkEAtq2bZtTAwAA7NbkIHTy5EmVlJSopKRE0k8nJZeUlKiiokInT57UnDlztHXrVn377bcqKCjQgw8+qD59+sjn80mSbr/9do0dO1bTpk3T9u3b9fnnnyszM1MTJ05UQkKCJOnhhx+Wy+XS1KlTtWfPHq1evVpLly4N+trqySefVF5env785z9r3759Wrx4sXbs2KHMzExJP13RNmvWLP3pT3/S+++/r927d+sPf/iDEhISgq5yAwAA9mryOUI7duzQvffe6zxuDCfp6elavny5du3apTfffFNVVVVKSEjQmDFj9P/+3/9TeHi485y33npLmZmZGj16tEJDQzV+/Hi98sorzrjH49Enn3yijIwMJScnKzo6WtnZ2UH3Grrzzju1cuVKLVy4UM8884xuu+02rVu3TgMGDHBq5s6dq5qaGk2fPl1VVVW66667lJeXp4iIiKa2DQAA2qCfdR+hto77CAEA0Pq0+H2EAAAAWgOCEAAAsBZBCAAAWIsgBAAArEUQAgAA1iIIAQAAaxGEAACAtQhCAADAWgQhAABgLYIQAACwFkEIAABYiyAEAACsRRACAADWIggBAABrEYQAAIC1CEIAAMBaBCEAAGAtghAAALAWQQgAAFiLIAQAAKxFEAIAANYiCAEAAGsRhAAAgLUIQgAAwFoEIQAAYC2CEAAAsBZBCAAAWIsgBAAArEUQAgAA1iIIAQAAaxGEAACAtQhCAADAWgQhAABgLYIQAACwFkEIAABYiyAEAACsRRACAADWIggBAABrEYQAAIC1CEIAAMBaBCEAAGAtghAAALAWQQgAAFiLIAQAAKxFEAIAANYiCAEAAGsRhAAAgLUIQgAAwFoEIQAAYC2CEAAAsBZBCAAAWIsgBAAArEUQAgAA1iIIAQAAaxGEAACAtQhCAADAWu1aegH4+XrN39As83z7QlqzzAMAQGvBJ0IAAMBaBCEAAGCtJgehzZs364EHHlBCQoJCQkK0bt26oHFjjLKzsxUfH6/IyEilpqbqm2++Cao5fvy4Jk+eLLfbraioKE2dOlUnT54Mqtm1a5fuvvtuRUREKDExUUuWLLlgLWvWrFG/fv0UERGhgQMH6sMPP2zyWgAAgL2aHIRqamo0ePBgLVu27KLjS5Ys0SuvvKIVK1Zo27ZtuuWWW+Tz+XT69GmnZvLkydqzZ4/y8/O1fv16bd68WdOnT3fGA4GAxowZo549e6q4uFgvvviiFi9erL/+9a9OzZYtWzRp0iRNnTpVO3fu1Lhx4zRu3DiVlpY2aS0AAMBeIcYYc81PDgnR2rVrNW7cOEk/fQKTkJCgp556Sk8//bQkqbq6WrGxscrNzdXEiRO1d+9e9e/fX1988YWGDRsmScrLy9P999+v7777TgkJCVq+fLmeffZZ+f1+uVwuSdL8+fO1bt067du3T5I0YcIE1dTUaP369c56Ro4cqSFDhmjFihVXtZYrCQQC8ng8qq6ultvtvtZf0yU110nOzYWTpQEAbUFT3r+b9Ryh8vJy+f1+paamOvs8Ho9SUlJUVFQkSSoqKlJUVJQTgiQpNTVVoaGh2rZtm1MzatQoJwRJks/nU1lZmX744Qen5tzXaaxpfJ2rWcv5amtrFQgEgjYAANB2NWsQ8vv9kqTY2Nig/bGxsc6Y3+9XTExM0Hi7du3UpUuXoJqLzXHua1yq5tzxK63lfDk5OfJ4PM6WmJh4FV0DAIDWiqvGzrFgwQJVV1c728GDB1t6SQAA4Dpq1iAUFxcnSaqsrAzaX1lZ6YzFxcXpyJEjQeNnz57V8ePHg2ouNse5r3GpmnPHr7SW84WHh8vtdgdtAACg7WrWIJSUlKS4uDgVFBQ4+wKBgLZt2yav1ytJ8nq9qqqqUnFxsVOzceNGNTQ0KCUlxanZvHmzzpw549Tk5+erb9++6ty5s1Nz7us01jS+ztWsBQAA2K3JQejkyZMqKSlRSUmJpJ9OSi4pKVFFRYVCQkI0a9Ys/elPf9L777+v3bt36w9/+IMSEhKcK8tuv/12jR07VtOmTdP27dv1+eefKzMzUxMnTlRCQoIk6eGHH5bL5dLUqVO1Z88erV69WkuXLlVWVpazjieffFJ5eXn685//rH379mnx4sXasWOHMjMzJemq1gIAAOzW5P/W2I4dO3Tvvfc6jxvDSXp6unJzczV37lzV1NRo+vTpqqqq0l133aW8vDxFREQ4z3nrrbeUmZmp0aNHKzQ0VOPHj9crr7zijHs8Hn3yySfKyMhQcnKyoqOjlZ2dHXSvoTvvvFMrV67UwoUL9cwzz+i2227TunXrNGDAAKfmatYCAADs9bPuI9TWcR8hAABanxa7jxAAAEBrQhACAADWIggBAABrEYQAAIC1CEIAAMBaBCEAAGAtghAAALAWQQgAAFiLIAQAAKxFEAIAANYiCAEAAGsRhAAAgLUIQgAAwFoEIQAAYC2CEAAAsBZBCAAAWIsgBAAArEUQAgAA1iIIAQAAaxGEAACAtQhCAADAWgQhAABgLYIQAACwFkEIAABYiyAEAACsRRACAADWIggBAABrEYQAAIC1CEIAAMBaBCEAAGAtghAAALAWQQgAAFiLIAQAAKxFEAIAANYiCAEAAGsRhAAAgLUIQgAAwFoEIQAAYC2CEAAAsBZBCAAAWIsgBAAArEUQAgAA1iIIAQAAaxGEAACAtQhCAADAWgQhAABgLYIQAACwFkEIAABYiyAEAACsRRACAADWIggBAABrEYQAAIC1CEIAAMBaBCEAAGAtghAAALAWQQgAAFiLIAQAAKxFEAIAANYiCAEAAGs1exBavHixQkJCgrZ+/fo546dPn1ZGRoa6du2qjh07avz48aqsrAyao6KiQmlpaerQoYNiYmI0Z84cnT17Nqhm06ZNGjp0qMLDw9WnTx/l5uZesJZly5apV69eioiIUEpKirZv397c7QIAgFbsunwidMcdd+jw4cPO9tlnnzljs2fP1gcffKA1a9aosLBQhw4d0kMPPeSM19fXKy0tTXV1ddqyZYvefPNN5ebmKjs726kpLy9XWlqa7r33XpWUlGjWrFl6/PHH9fHHHzs1q1evVlZWlhYtWqQvv/xSgwcPls/n05EjR65HywAAoBUKMcaY5pxw8eLFWrdunUpKSi4Yq66uVrdu3bRy5Ur9/ve/lyTt27dPt99+u4qKijRy5Eh99NFH+u1vf6tDhw4pNjZWkrRixQrNmzdPR48elcvl0rx587RhwwaVlpY6c0+cOFFVVVXKy8uTJKWkpGj48OF69dVXJUkNDQ1KTEzUE088ofnz519VL4FAQB6PR9XV1XK73T/n13JRveZvaPY5f45vX0hr6SUAAPCzNeX9+7p8IvTNN98oISFBvXv31uTJk1VRUSFJKi4u1pkzZ5SamurU9uvXTz169FBRUZEkqaioSAMHDnRCkCT5fD4FAgHt2bPHqTl3jsaaxjnq6upUXFwcVBMaGqrU1FSn5mJqa2sVCASCNgAA0HY1exBKSUlRbm6u8vLytHz5cpWXl+vuu+/WiRMn5Pf75XK5FBUVFfSc2NhY+f1+SZLf7w8KQY3jjWOXqwkEAjp16pS+//571dfXX7SmcY6LycnJkcfjcbbExMRr+h0AAIDWoV1zT3jfffc5Pw8aNEgpKSnq2bOn3nnnHUVGRjb3yzWrBQsWKCsry3kcCAQIQwAAtGHX/fL5qKgo/fKXv9T+/fsVFxenuro6VVVVBdVUVlYqLi5OkhQXF3fBVWSNj69U43a7FRkZqejoaIWFhV20pnGOiwkPD5fb7Q7aAABA23Xdg9DJkyd14MABxcfHKzk5We3bt1dBQYEzXlZWpoqKCnm9XkmS1+vV7t27g67uys/Pl9vtVv/+/Z2ac+dorGmcw+VyKTk5OaimoaFBBQUFTg0AAECzB6Gnn35ahYWF+vbbb7Vlyxb97ne/U1hYmCZNmiSPx6OpU6cqKytLn376qYqLizVlyhR5vV6NHDlSkjRmzBj1799fjzzyiL766it9/PHHWrhwoTIyMhQeHi5JmjFjhv7+979r7ty52rdvn1577TW98847mj17trOOrKws/eu//qvefPNN7d27VzNnzlRNTY2mTJnS3C0DAIBWqtnPEfruu+80adIkHTt2TN26ddNdd92lrVu3qlu3bpKkl156SaGhoRo/frxqa2vl8/n02muvOc8PCwvT+vXrNXPmTHm9Xt1yyy1KT0/X888/79QkJSVpw4YNmj17tpYuXaru3bvr9ddfl8/nc2omTJigo0ePKjs7W36/X0OGDFFeXt4FJ1ADAAB7Nft9hNoS7iMEAEDr0+L3EQIAAGgNCEIAAMBaBCEAAGAtghAAALAWQQgAAFiLIAQAAKxFEAIAANYiCAEAAGsRhAAAgLUIQgAAwFoEIQAAYC2CEAAAsBZBCAAAWIsgBAAArEUQAgAA1iIIAQAAaxGEAACAtQhCAADAWgQhAABgrXYtvQDcPHrN39As83z7QlqzzAMAwPXGJ0IAAMBaBCEAAGAtghAAALAWQQgAAFiLIAQAAKxFEAIAANYiCAEAAGsRhAAAgLUIQgAAwFoEIQAAYC2CEAAAsBZBCAAAWIsgBAAArEUQAgAA1iIIAQAAaxGEAACAtQhCAADAWgQhAABgLYIQAACwFkEIAABYiyAEAACsRRACAADWIggBAABrEYQAAIC12rX0AtD29Jq/oVnm+faFtGaZBwCAS+ETIQAAYC2CEAAAsBZBCAAAWIsgBAAArEUQAgAA1iIIAQAAaxGEAACAtQhCAADAWgQhAABgLe4sjZsWd6gGAFxvfCIEAACsRRACAADWIggBAABrcY4Q2jzONQIAXIoVnwgtW7ZMvXr1UkREhFJSUrR9+/aWXhIAALgJtPlPhFavXq2srCytWLFCKSkpevnll+Xz+VRWVqaYmJiWXh5aET5ZAoC2J8QYY1p6EddTSkqKhg8frldffVWS1NDQoMTERD3xxBOaP3/+ZZ8bCATk8XhUXV0tt9vd7GtrrjdW2IlABQAX15T37zb9iVBdXZ2Ki4u1YMECZ19oaKhSU1NVVFR0QX1tba1qa2udx9XV1ZJ++oVeDw21P16XeWGHHrPXNMs8pc/5mmUeALhZNL5vX81nPW06CH3//feqr69XbGxs0P7Y2Fjt27fvgvqcnBw999xzF+xPTEy8bmsEWprn5ZZeAQBcHydOnJDH47lsTZsOQk21YMECZWVlOY8bGhp0/Phxde3aVSEhIc32OoFAQImJiTp48OB1+crtZkCPrV9b70+ix7agrfcntf0er0d/xhidOHFCCQkJV6xt00EoOjpaYWFhqqysDNpfWVmpuLi4C+rDw8MVHh4etC8qKuq6rc/tdrfJP+pz0WPr19b7k+ixLWjr/Ultv8fm7u9KnwQ1atOXz7tcLiUnJ6ugoMDZ19DQoIKCAnm93hZcGQAAuBm06U+EJCkrK0vp6ekaNmyYRowYoZdfflk1NTWaMmVKSy8NAAC0sDYfhCZMmKCjR48qOztbfr9fQ4YMUV5e3gUnUN9I4eHhWrRo0QVfw7Ul9Nj6tfX+JHpsC9p6f1Lb77Gl+2vz9xECAAC4lDZ9jhAAAMDlEIQAAIC1CEIAAMBaBCEAAGAtglALWLZsmXr16qWIiAilpKRo+/btLb2kq7J48WKFhIQEbf369XPGT58+rYyMDHXt2lUdO3bU+PHjL7iZZUVFhdLS0tShQwfFxMRozpw5Onv27I1uxbF582Y98MADSkhIUEhIiNatWxc0boxRdna24uPjFRkZqdTUVH3zzTdBNcePH9fkyZPldrsVFRWlqVOn6uTJk0E1u3bt0t13362IiAglJiZqyZIl17s1SVfu79FHH73gmI4dOzao5mbuLycnR8OHD1enTp0UExOjcePGqaysLKimuf4uN23apKFDhyo8PFx9+vRRbm7u9W5P0tX1eM8991xwHGfMmBFUczP3uHz5cg0aNMi5oZ7X69VHH33kjLf2Y3il/lr78buYF154QSEhIZo1a5az76Y9jgY31KpVq4zL5TL/9m//Zvbs2WOmTZtmoqKiTGVlZUsv7YoWLVpk7rjjDnP48GFnO3r0qDM+Y8YMk5iYaAoKCsyOHTvMyJEjzZ133umMnz171gwYMMCkpqaanTt3mg8//NBER0ebBQsWtEQ7xhhjPvzwQ/Pss8+ad99910gya9euDRp/4YUXjMfjMevWrTNfffWV+Yd/+AeTlJRkTp065dSMHTvWDB482GzdutX813/9l+nTp4+ZNGmSM15dXW1iY2PN5MmTTWlpqXn77bdNZGSk+Zd/+ZcW7y89Pd2MHTs26JgeP348qOZm7s/n85k33njDlJaWmpKSEnP//febHj16mJMnTzo1zfF3+fe//9106NDBZGVlma+//tr85S9/MWFhYSYvL++m6PE3v/mNmTZtWtBxrK6ubjU9vv/++2bDhg3mv//7v01ZWZl55plnTPv27U1paakxpvUfwyv119qP3/m2b99uevXqZQYNGmSefPJJZ//NehwJQjfYiBEjTEZGhvO4vr7eJCQkmJycnBZc1dVZtGiRGTx48EXHqqqqTPv27c2aNWucfXv37jWSTFFRkTHmpzfl0NBQ4/f7nZrly5cbt9ttamtrr+var8b5QaGhocHExcWZF1980dlXVVVlwsPDzdtvv22MMebrr782kswXX3zh1Hz00UcmJCTE/O///q8xxpjXXnvNdO7cOajHefPmmb59+17njoJdKgg9+OCDl3xOa+rPGGOOHDliJJnCwkJjTPP9Xc6dO9fccccdQa81YcIE4/P5rndLFzi/R2N+eiM99w3nfK2tR2OM6dy5s3n99dfb5DE05v/6M6ZtHb8TJ06Y2267zeTn5wf1dTMfR74au4Hq6upUXFys1NRUZ19oaKhSU1NVVFTUgiu7et98840SEhLUu3dvTZ48WRUVFZKk4uJinTlzJqi3fv36qUePHk5vRUVFGjhwYNDNLH0+nwKBgPbs2XNjG7kK5eXl8vv9QT15PB6lpKQE9RQVFaVhw4Y5NampqQoNDdW2bducmlGjRsnlcjk1Pp9PZWVl+uGHH25QN5e2adMmxcTEqG/fvpo5c6aOHTvmjLW2/qqrqyVJXbp0kdR8f5dFRUVBczTWtMS/2/N7bPTWW28pOjpaAwYM0IIFC/Tjjz86Y62px/r6eq1atUo1NTXyer1t7hie31+jtnL8MjIylJaWdsFabubj2ObvLH0z+f7771VfX3/BXa1jY2O1b9++FlrV1UtJSVFubq769u2rw4cP67nnntPdd9+t0tJS+f1+uVyuC/4jtbGxsfL7/ZIkv99/0d4bx242jWu62JrP7SkmJiZovF27durSpUtQTVJS0gVzNI517tz5uqz/aowdO1YPPfSQkpKSdODAAT3zzDO67777VFRUpLCwsFbVX0NDg2bNmqVf//rXGjBggPP6zfF3eamaQCCgU6dOKTIy8nq0dIGL9ShJDz/8sHr27KmEhATt2rVL8+bNU1lZmd59993Lrr9x7HI1N6rH3bt3y+v16vTp0+rYsaPWrl2r/v37q6SkpE0cw0v1J7WN4ydJq1at0pdffqkvvvjigrGb+d8iQQhX7b777nN+HjRokFJSUtSzZ0+98847N+yNAM1r4sSJzs8DBw7UoEGDdOutt2rTpk0aPXp0C66s6TIyMlRaWqrPPvuspZdy3Vyqx+nTpzs/Dxw4UPHx8Ro9erQOHDigW2+99UYv85r07dtXJSUlqq6u1n/+538qPT1dhYWFLb2sZnOp/vr3798mjt/Bgwf15JNPKj8/XxERES29nCbhq7EbKDo6WmFhYRecJV9ZWam4uLgWWtW1i4qK0i9/+Uvt379fcXFxqqurU1VVVVDNub3FxcVdtPfGsZtN45oud7zi4uJ05MiRoPGzZ8/q+PHjrbLv3r17Kzo6Wvv375fUevrLzMzU+vXr9emnn6p79+7O/ub6u7xUjdvtvmH/J+BSPV5MSkqKJAUdx5u9R5fLpT59+ig5OVk5OTkaPHiwli5d2maO4aX6u5jWePyKi4t15MgRDR06VO3atVO7du1UWFioV155Re3atVNsbOxNexwJQjeQy+VScnKyCgoKnH0NDQ0qKCgI+q64tTh58qQOHDig+Ph4JScnq3379kG9lZWVqaKiwunN6/Vq9+7dQW+s+fn5crvdzkfEN5OkpCTFxcUF9RQIBLRt27agnqqqqlRcXOzUbNy4UQ0NDc7/mHm9Xm3evFlnzpxxavLz89W3b98W/VrsYr777jsdO3ZM8fHxkm7+/owxyszM1Nq1a7Vx48YLvqJrrr9Lr9cbNEdjzY34d3ulHi+mpKREkoKO483c48U0NDSotra2TRzDi2ns72Ja4/EbPXq0du/erZKSEmcbNmyYJk+e7Px80x7Haz7NGtdk1apVJjw83OTm5pqvv/7aTJ8+3URFRQWdJX+zeuqpp8ymTZtMeXm5+fzzz01qaqqJjo42R44cMcb8dGlkjx49zMaNG82OHTuM1+s1Xq/XeX7jpZFjxowxJSUlJi8vz3Tr1q1FL58/ceKE2blzp9m5c6eRZP75n//Z7Ny50/zP//yPMeany+ejoqLMe++9Z3bt2mUefPDBi14+/6tf/cps27bNfPbZZ+a2224Lury8qqrKxMbGmkceecSUlpaaVatWmQ4dOtyQy8sv19+JEyfM008/bYqKikx5ebn529/+ZoYOHWpuu+02c/r06VbR38yZM43H4zGbNm0KuvT4xx9/dGqa4++y8ZLdOXPmmL1795ply5bdsEuTr9Tj/v37zfPPP2927NhhysvLzXvvvWd69+5tRo0a1Wp6nD9/viksLDTl5eVm165dZv78+SYkJMR88sknxpjWfwwv119bOH6Xcv7VcDfrcSQItYC//OUvpkePHsblcpkRI0aYrVu3tvSSrsqECRNMfHy8cblc5he/+IWZMGGC2b9/vzN+6tQp84//+I+mc+fOpkOHDuZ3v/udOXz4cNAc3377rbnvvvtMZGSkiY6ONk899ZQ5c+bMjW7F8emnnxpJF2zp6enGmJ8uof/jH/9oYmNjTXh4uBk9erQpKysLmuPYsWNm0qRJpmPHjsbtdpspU6aYEydOBNV89dVX5q677jLh4eHmF7/4hXnhhRdavL8ff/zRjBkzxnTr1s20b9/e9OzZ00ybNu2CUH4z93ex3iSZN954w6lprr/LTz/91AwZMsS4XC7Tu3fvoNe4nq7UY0VFhRk1apTp0qWLCQ8PN3369DFz5swJug/Nzd7jY489Znr27GlcLpfp1q2bGT16tBOCjGn9x/By/bWF43cp5wehm/U4hhhjzLV/ngQAANB6cY4QAACwFkEIAABYiyAEAACsRRACAADWIggBAABrEYQAAIC1CEIAAMBaBCEAAGAtghAAALAWQQgAAFiLIAQAAKxFEAIAANb6/4IuHRze3YhrAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.hist([len(q[0]) for q in quote_list], bins=25)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this project, we will use Quotes with length smaller than 80 characters."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create Database"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "!mkdir -p db"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sqlite3\n",
    "\n",
    "conn = sqlite3.connect('db/quotes.db')\n",
    "cur = conn.cursor()\n",
    "\n",
    "cur.execute(\n",
    "    \"\"\"CREATE TABLE IF NOT EXISTS quotes (\n",
    "        quote TEXT UNIQUE, \n",
    "        author TEXT,\n",
    "        tweeted INTEGER DEFAULT 0\n",
    "    )\"\"\")\n",
    "conn.commit()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Insert Quotes into Database"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "for q in quote_list:\n",
    "    if len(q[0]) < 80:\n",
    "        cur.execute(\n",
    "            \"INSERT OR IGNORE INTO quotes (quote, author) VALUES (?, ?)\", (q[0], q[1]))\n",
    "conn.commit()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total quotes in database: 107099\n"
     ]
    }
   ],
   "source": [
    "cur.execute(\"SELECT COUNT(*) FROM quotes\")\n",
    "print(\"Total quotes in database:\", cur.fetchone()[0])\n",
    "conn.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook saves 107099 Quotes in the database.\n",
    "\n",
    "If we tweet 1 quote per hour, it will take 107099/(24*365)= 12.225913242 years to tweet all quotes."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p align=\"center\">Made with 💖 by Codinasion</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.10.6 ('.venv': venv)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.6"
  },
  "orig_nbformat": 4,
  "vscode": {
   "interpreter": {
    "hash": "5b94c429e1aee99f8230f19ffb7ae8c7beee29b996ce0b142e33b9f3536e8c14"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}