docs/textgenrnn-train-context-wordlevel.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# textgenrnn 1.1 Training with Context (Word Level)\n",
    "\n",
    "by [Max Woolf](http://minimaxir.com)\n",
    "\n",
    "*Max's open-source projects are supported by his [Patreon](https://www.patreon.com/minimaxir). If you found this project helpful, any monetary contributions to the Patreon are appreciated and will be put to good creative use.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "from textgenrnn import textgenrnn\n",
    "\n",
    "textgen = textgenrnn()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train on New Text\n",
    "\n",
    "Here's textgenrnn trained on a dataset of the Top 1000 submissions to /r/politics and /r/rarepuppers during 2017: two subreddits with drastically different styles! How well can a new model learn these styles?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2,000 texts collected.\n",
      "Training new model w/ 2-layer, 128-cell LSTMs\n",
      "Training on 25,792 word sequences.\n",
      "Epoch 1/20\n",
      "201/201 [==============================] - 21s 102ms/step - loss: 6.4697\n",
      "Epoch 2/20\n",
      "201/201 [==============================] - 18s 90ms/step - loss: 5.4531\n",
      "Epoch 3/20\n",
      "201/201 [==============================] - 18s 87ms/step - loss: 4.5949\n",
      "Epoch 4/20\n",
      "201/201 [==============================] - 18s 88ms/step - loss: 3.6975\n",
      "Epoch 5/20\n",
      "201/201 [==============================] - 18s 88ms/step - loss: 2.8250\n",
      "####################\n",
      "Temperature: 0.2\n",
      "####################\n",
      "trump ’ s ‘ travel , ’ re a president since trump ’ s power\n",
      "\n",
      "trump ’ s ‘ travel , ’ re ‘ ban ’ re sentencing to be replaced than any president in the history\n",
      "\n",
      "trump ’ s ‘ travel , ’ re ‘ ban ’ re sentencing to be as the worst than popular president is ' s twitter\n",
      "\n",
      "####################\n",
      "Temperature: 0.5\n",
      "####################\n",
      "trump ’ s approval rating in interview military\n",
      "\n",
      "trump called obama for 122 '\n",
      "\n",
      "trump loses speech , one comey to testify under oath\n",
      "\n",
      "####################\n",
      "Temperature: 1.0\n",
      "####################\n",
      "doin a heckin showing g o o t e is a heckin good jam with with unsubstantiated time period\n",
      "\n",
      "corgo is a new of hard : )\n",
      "\n",
      "report : how yates a paying to republican back .\n",
      "\n",
      "Epoch 6/20\n",
      "201/201 [==============================] - 18s 87ms/step - loss: 2.1161\n",
      "Epoch 7/20\n",
      "201/201 [==============================] - 17s 87ms/step - loss: 1.6342\n",
      "Epoch 8/20\n",
      "201/201 [==============================] - 18s 91ms/step - loss: 1.3264\n",
      "Epoch 9/20\n",
      "201/201 [==============================] - 18s 90ms/step - loss: 1.1307\n",
      "Epoch 10/20\n",
      "201/201 [==============================] - 18s 88ms/step - loss: 0.9997\n",
      "####################\n",
      "Temperature: 0.2\n",
      "####################\n",
      "trump ’ s pullout from paris climate agreement is biggest failure of leadership in american history\n",
      "\n",
      "trump ' s motorcade greeted with chants of ' lock him up ' in nyc\n",
      "\n",
      "trump ’ s pullout from paris climate agreement is biggest failure of leadership in american history\n",
      "\n",
      "####################\n",
      "Temperature: 0.5\n",
      "####################\n",
      "trump ’ s pullout from paris climate agreement is biggest failure of leadership in american history\n",
      "\n",
      "top epa official resigns over direction of agency under trump\n",
      "\n",
      "trump ' exhausted ' three days into first trip abroad as president : reports\n",
      "\n",
      "####################\n",
      "Temperature: 1.0\n",
      "####################\n",
      "running the president fires through back by the pupper , md that presidents businesses was being a dismissed\n",
      "\n",
      "lord of the bamboozle\n",
      "\n",
      "brave smol pupper does pupper shopping\n",
      "\n",
      "Epoch 11/20\n",
      "201/201 [==============================] - 17s 86ms/step - loss: 0.9139\n",
      "Epoch 12/20\n",
      "201/201 [==============================] - 18s 88ms/step - loss: 0.8533\n",
      "Epoch 13/20\n",
      "201/201 [==============================] - 17s 87ms/step - loss: 0.8103\n",
      "Epoch 14/20\n",
      "201/201 [==============================] - 18s 88ms/step - loss: 0.7788\n",
      "Epoch 15/20\n",
      "201/201 [==============================] - 18s 88ms/step - loss: 0.7512\n",
      "####################\n",
      "Temperature: 0.2\n",
      "####################\n",
      "trump ’ s pullout from paris climate agreement is biggest failure of leadership in american history\n",
      "\n",
      "trump ’ s chief strategist says news media should ‘ keep its mouth shut ’\n",
      "\n",
      "trump ’ s inauguration pledged millions to charity . to date : zero given .\n",
      "\n",
      "####################\n",
      "Temperature: 0.5\n",
      "####################\n",
      "trump says he never promised to quickly repeal obamacare . here ’ s a bunch of times he promised exactly that .\n",
      "\n",
      "trump ’ s inauguration pledged millions to charity . to date : zero given .\n",
      "\n",
      "c o m p y b o y e\n",
      "\n",
      "####################\n",
      "Temperature: 1.0\n",
      "####################\n",
      "new jersey ' s newly elected governor wants to legalize pot\n",
      "\n",
      "megathread : flynn offers to testify for immunity deal .\n",
      "\n",
      "donald trump to completely turn off epa ' s data service , one of government ' s most important websites\n",
      "\n",
      "Epoch 16/20\n",
      "201/201 [==============================] - 18s 89ms/step - loss: 0.7322\n",
      "Epoch 17/20\n",
      "201/201 [==============================] - 17s 87ms/step - loss: 0.7128\n",
      "Epoch 18/20\n",
      "201/201 [==============================] - 18s 89ms/step - loss: 0.6940\n",
      "Epoch 19/20\n",
      "201/201 [==============================] - 19s 95ms/step - loss: 0.6790\n",
      "Epoch 20/20\n",
      "201/201 [==============================] - 18s 92ms/step - loss: 0.6628\n",
      "####################\n",
      "Temperature: 0.2\n",
      "####################\n",
      "trump ’ s lies are not the problem . it ’ s the millions who swallow them who really matter\n",
      "\n",
      "trump ’ s inauguration pledged millions to charity . to date : zero given .\n",
      "\n",
      "trump ’ s inauguration pledged millions to charity . to date : zero given .\n",
      "\n",
      "####################\n",
      "Temperature: 0.5\n",
      "####################\n",
      "heckin big doggo or heckin big bamboozle ?\n",
      "\n",
      "heckin good bamboozler\n",
      "\n",
      "trump ’ s old tweets are quickly becoming a minefield of hypocrisy\n",
      "\n",
      "####################\n",
      "Temperature: 1.0\n",
      "####################\n",
      "hawaii rep . beth fukumoto leaving the republican party\n",
      "\n",
      "lil ' pupper awoos !\n",
      "\n",
      "frens , i am doing a bamboozle 2 these days after just an years\n",
      "\n"
     ]
    }
   ],
   "source": [
    "file_path = \"../datasets/reddit_rarepuppers_politics_2000.txt\"\n",
    "\n",
    "textgen.reset()\n",
    "textgen.train_from_file(file_path,\n",
    "                        new_model=True,\n",
    "                        max_length=5,\n",
    "                        num_epochs=20,\n",
    "                        gen_epochs=5,\n",
    "                        word_level=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2,000 texts collected.\n",
      "Training new model w/ 2-layer, 128-cell LSTMs\n",
      "Training on 31,427 word sequences.\n",
      "Epoch 1/20\n",
      "245/245 [==============================] - 21s 87ms/step - loss: 6.3976\n",
      "Epoch 2/20\n",
      "245/245 [==============================] - 19s 79ms/step - loss: 5.2978\n",
      "Epoch 3/20\n",
      "245/245 [==============================] - 19s 78ms/step - loss: 4.5012\n",
      "Epoch 4/20\n",
      "245/245 [==============================] - 20s 81ms/step - loss: 3.7612\n",
      "Epoch 5/20\n",
      "245/245 [==============================] - 19s 79ms/step - loss: 3.1187\n",
      "####################\n",
      "Temperature: 0.2\n",
      "####################\n",
      "apple is a significant battery draw\n",
      "\n",
      "apple releases ios 11 is already destroying google ’ s new mac os x command line\n",
      "\n",
      "apple is a bit apps\n",
      "\n",
      "####################\n",
      "Temperature: 0.5\n",
      "####################\n",
      "apple releases ios 10 . 3 - inch model\n",
      "\n",
      "google pixel 2\n",
      "\n",
      "apple releases ios 11\n",
      "\n",
      "####################\n",
      "Temperature: 1.0\n",
      "####################\n",
      "boost for reddit getting also in summer\n",
      "\n",
      "should anyone be including a free api to use the macos window app on android phones\n",
      "\n",
      "new report suggests\n",
      "\n",
      "Epoch 6/20\n",
      "245/245 [==============================] - 19s 78ms/step - loss: 2.6197\n",
      "Epoch 7/20\n",
      "245/245 [==============================] - 19s 78ms/step - loss: 2.2651\n",
      "Epoch 8/20\n",
      "245/245 [==============================] - 19s 79ms/step - loss: 2.0169\n",
      "Epoch 9/20\n",
      "245/245 [==============================] - 19s 79ms/step - loss: 1.8421\n",
      "Epoch 10/20\n",
      "245/245 [==============================] - 19s 78ms/step - loss: 1.7100\n",
      "####################\n",
      "Temperature: 0.2\n",
      "####################\n",
      "apple releases ios 10 . 5 - inch ipad pro 10 . 3 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 .\n",
      "\n",
      "apple ’ s new ipad pro 10 . 3 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 . 1 .\n",
      "\n",
      "apple music\n",
      "\n",
      "####################\n",
      "Temperature: 0.5\n",
      "####################\n",
      "apple music there is no whatsapp\n",
      "\n",
      "google photos rolling out\n",
      "\n",
      "samsung galaxy s8 and s8 +\n",
      "\n",
      "####################\n",
      "Temperature: 1.0\n",
      "####################\n",
      "hangouts is the most natural sounding and human voice of apple a11 chip for iphone\n",
      "\n",
      "google git has you already use ios 11 public ipads and macos desktop wallpapers by luke chesser\n",
      "\n",
      "how is apple revealed my app stolen by reverse engineering\n",
      "\n",
      "Epoch 11/20\n",
      "245/245 [==============================] - 19s 79ms/step - loss: 1.6085\n",
      "Epoch 12/20\n",
      "245/245 [==============================] - 19s 79ms/step - loss: 1.5258\n",
      "Epoch 13/20\n",
      "245/245 [==============================] - 19s 79ms/step - loss: 1.4574\n",
      "Epoch 14/20\n",
      "245/245 [==============================] - 19s 79ms/step - loss: 1.3989\n",
      "Epoch 15/20\n",
      "245/245 [==============================] - 20s 83ms/step - loss: 1.3434\n",
      "####################\n",
      "Temperature: 0.2\n",
      "####################\n",
      "apple ' s new campus auditorium ' steve jobs theater '\n",
      "\n",
      "apple ' s new campus auditorium ' steve jobs created the iphone 8 '\n",
      "\n",
      "apple ' s new campus auditorium ' steve jobs introduce the iphone 8 '\n",
      "\n",
      "####################\n",
      "Temperature: 0.5\n",
      "####################\n",
      "google maps & amp ; performance\n",
      "\n",
      "til you can remap the bixby button on the galaxy s8 and s8 plus is only 7 % slower than carriers . . . . . . i ' ve got some exciting news ! ( razer bought nextbit )\n",
      "\n",
      "google maps introduces location sharing\n",
      "\n",
      "####################\n",
      "Temperature: 1.0\n",
      "####################\n",
      "beta sign up available for display , taking up to ios 9 . 0\n",
      "\n",
      "qualcomm snapdragon 835 performance preview\n",
      "\n",
      "oneplus 5 gets twrp and root\n",
      "\n",
      "Epoch 16/20\n",
      "245/245 [==============================] - 20s 80ms/step - loss: 1.2978\n",
      "Epoch 17/20\n",
      "245/245 [==============================] - 20s 80ms/step - loss: 1.2520\n",
      "Epoch 18/20\n",
      "245/245 [==============================] - 19s 79ms/step - loss: 1.2130\n",
      "Epoch 19/20\n",
      "245/245 [==============================] - 19s 79ms/step - loss: 1.1709\n",
      "Epoch 20/20\n",
      "245/245 [==============================] - 19s 78ms/step - loss: 1.1305\n",
      "####################\n",
      "Temperature: 0.2\n",
      "####################\n",
      "apple ' s new campus , connected to apple tv this summer\n",
      "\n",
      "samsung galaxy s8 and s8 plus most clear look !\n",
      "\n",
      "apple ' s new apple watch\n",
      "\n",
      "####################\n",
      "Temperature: 0.5\n",
      "####################\n",
      "apple : i went to a new imac pro specs configured on a screen protector\n",
      "\n",
      "i wish i could get a similar early look at nokia 6 , 5 . 0 ( android 7 . 1 . 2\n",
      "\n",
      "a ' brand new ' google earth will arrive april 18th\n",
      "\n",
      "####################\n",
      "Temperature: 1.0\n",
      "####################\n",
      "\n",
      "\n",
      "there is no whatsapp ' backdoor '\n",
      "\n",
      "steve jobs on my ipad\n",
      "\n"
     ]
    }
   ],
   "source": [
    "file_path = \"../datasets/reddit_apple_android_2000.txt\"\n",
    "\n",
    "textgen.reset()\n",
    "textgen.train_from_file(file_path,\n",
    "                        new_model=True,\n",
    "                        max_length=2,\n",
    "                        num_epochs=20,\n",
    "                        gen_epochs=5,\n",
    "                        word_level=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And here's the same dataset, trained with the context labels indicating the source subreddit. If using `train_from_file`, the input file must be a 2-column CSV, with the first column containing the texts and the second containing the labels.\n",
    "\n",
    "The `output_loss` is the loss along the text-only path. In this example, the context-label model has a slightly lower loss than the text-only training. (the more disparate the texts, the more helpful providing context will help)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2,000 texts collected.\n",
      "Training new model w/ 2-layer, 128-cell LSTMs\n",
      "Training on 25,796 word sequences.\n",
      "Epoch 1/20\n",
      "201/201 [==============================] - 26s 131ms/step - loss: 6.1417 - context_output_loss: 6.1367 - output_loss: 6.1616\n",
      "Epoch 2/20\n",
      "201/201 [==============================] - 25s 124ms/step - loss: 5.0178 - context_output_loss: 4.9999 - output_loss: 5.0894\n",
      "Epoch 3/20\n",
      "201/201 [==============================] - 25s 126ms/step - loss: 4.1797 - context_output_loss: 4.1570 - output_loss: 4.2707\n",
      "Epoch 4/20\n",
      "201/201 [==============================] - 25s 126ms/step - loss: 3.4161 - context_output_loss: 3.3908 - output_loss: 3.5171\n",
      "Epoch 5/20\n",
      "201/201 [==============================] - 25s 126ms/step - loss: 2.7769 - context_output_loss: 2.7516 - output_loss: 2.8783\n",
      "####################\n",
      "Temperature: 0.2\n",
      "####################\n",
      "trump ' s approval rating sinks to take congress to fight gerrymandering\n",
      "\n",
      "trump ' s approval rating sinks to take congress to fight gerrymandering\n",
      "\n",
      "trump ' s approval rating sinks to take him to the vet . send thoughts and other tv news networks that criticise him\n",
      "\n",
      "####################\n",
      "Temperature: 0.5\n",
      "####################\n",
      "trump says he ' s approval rating sinks to take a quick guide to trump\n",
      "\n",
      "mueller obtains \" tens of giving evidence of trump ’ s mar - a - lago trips cost a far\n",
      "\n",
      "b a l b o y e\n",
      "\n",
      "####################\n",
      "Temperature: 1.0\n",
      "####################\n",
      "b a l i t s all funding for npr and sleeping boye\n",
      "\n",
      "tonight this guy salesman\n",
      "\n",
      "v v good b u l f s s i paid sure schwarzenegger ' s puppet yourself ' gear to uk downgraded ' of the boy\n",
      "\n",
      "Epoch 6/20\n",
      "201/201 [==============================] - 25s 126ms/step - loss: 2.2963 - context_output_loss: 2.2731 - output_loss: 2.3892\n",
      "Epoch 7/20\n",
      "201/201 [==============================] - 25s 126ms/step - loss: 1.9835 - context_output_loss: 1.9636 - output_loss: 2.0630\n",
      "Epoch 8/20\n",
      "201/201 [==============================] - 25s 126ms/step - loss: 1.7752 - context_output_loss: 1.7578 - output_loss: 1.8452\n",
      "Epoch 9/20\n",
      "201/201 [==============================] - 26s 127ms/step - loss: 1.6401 - context_output_loss: 1.6242 - output_loss: 1.7037\n",
      "Epoch 10/20\n",
      "201/201 [==============================] - 25s 125ms/step - loss: 1.5396 - context_output_loss: 1.5248 - output_loss: 1.5986\n",
      "####################\n",
      "Temperature: 0.2\n",
      "####################\n",
      "trump ’ s lies represent “ who we are ' grateful for this boy\n",
      "\n",
      "trump ’ s mar - a - lago for third straight weekend , obliterating campaign promise since “ read my lips . ”\n",
      "\n",
      "trump ’ s lies represent “ who we are '\n",
      "\n",
      "####################\n",
      "Temperature: 0.5\n",
      "####################\n",
      "trump ’ s attacks on press are\n",
      "\n",
      "trump ’ s firing hours after trump ’ s business sought deal on russia\n",
      "\n",
      "donald trump ' s impeachment\n",
      "\n",
      "####################\n",
      "Temperature: 1.0\n",
      "####################\n",
      "h a s t e b o y e\n",
      "\n",
      "just a doggo hanging\n",
      "\n",
      "disgraced michael flynn failed to know why a major step toward destroying the internet ’ s still ' s manufacturing council\n",
      "\n",
      "Epoch 11/20\n",
      "201/201 [==============================] - 25s 125ms/step - loss: 1.4667 - context_output_loss: 1.4525 - output_loss: 1.5236\n",
      "Epoch 12/20\n",
      "201/201 [==============================] - 26s 128ms/step - loss: 1.4041 - context_output_loss: 1.3904 - output_loss: 1.4590\n",
      "Epoch 13/20\n",
      "201/201 [==============================] - 25s 126ms/step - loss: 1.3523 - context_output_loss: 1.3388 - output_loss: 1.4065\n",
      "Epoch 14/20\n",
      "201/201 [==============================] - 25s 123ms/step - loss: 1.3102 - context_output_loss: 1.2970 - output_loss: 1.3631\n",
      "Epoch 15/20\n",
      "201/201 [==============================] - 25s 125ms/step - loss: 1.2678 - context_output_loss: 1.2548 - output_loss: 1.3201\n",
      "####################\n",
      "Temperature: 0.2\n",
      "####################\n",
      "trump ' s russia investigation\n",
      "\n",
      "trump ’ s russia investigation\n",
      "\n",
      "trump ’ s russia investigation\n",
      "\n",
      "####################\n",
      "Temperature: 0.5\n",
      "####################\n",
      "trump ’ s list of false and misleading claims tops 1 , 070 , 594 . 19 for luxury travel\n",
      "\n",
      "one boye ' s russia investigation\n",
      "\n",
      "donald trump ' s tax returns\n",
      "\n",
      "####################\n",
      "Temperature: 1.0\n",
      "####################\n",
      "happy husko boye\n",
      "\n",
      "democrats warn : ' he knew about foreign tax bill hope analysis '\n",
      "\n",
      "a l a v e r c e r l e b o y e does a deejay\n",
      "\n",
      "Epoch 16/20\n",
      "201/201 [==============================] - 27s 132ms/step - loss: 1.2314 - context_output_loss: 1.2186 - output_loss: 1.2830\n",
      "Epoch 17/20\n",
      "201/201 [==============================] - 25s 126ms/step - loss: 1.1992 - context_output_loss: 1.1864 - output_loss: 1.2503\n",
      "Epoch 18/20\n",
      "201/201 [==============================] - 26s 130ms/step - loss: 1.1645 - context_output_loss: 1.1518 - output_loss: 1.2154\n",
      "Epoch 19/20\n",
      "201/201 [==============================] - 26s 127ms/step - loss: 1.1316 - context_output_loss: 1.1190 - output_loss: 1.1820\n",
      "Epoch 20/20\n",
      "201/201 [==============================] - 26s 128ms/step - loss: 1.1007 - context_output_loss: 1.0882 - output_loss: 1.1508\n",
      "####################\n",
      "Temperature: 0.2\n",
      "####################\n",
      "trump ’ s tax returns\n",
      "\n",
      "donald trump ' s impeachment is most popular things in america who continues to be a president to take him out despite just going . turned out we had a gas leak in our apartment and he just wanted us to get down stairs .\n",
      "\n",
      "trump ' s a f e b o y e\n",
      "\n",
      "####################\n",
      "Temperature: 0.5\n",
      "####################\n",
      "trump ’ s official twitter account is spreading falsehoods about the flag . .\n",
      "\n",
      "trump ’ s tax returns\n",
      "\n",
      "trump associate boasted that moscow deal ‘ will get donald elected ’\n",
      "\n",
      "####################\n",
      "Temperature: 1.0\n",
      "####################\n",
      "pupper tries to save fish ; (\n",
      "\n",
      "us government like his info again\n",
      "\n",
      "pence lied : led the flynn scandal\n",
      "\n"
     ]
    }
   ],
   "source": [
    "file_path = \"../datasets/reddit_rarepuppers_politics_2000_context.csv\"\n",
    "\n",
    "textgen.reset()\n",
    "textgen.train_from_file(file_path,\n",
    "                        new_model=True,\n",
    "                        max_length=2,\n",
    "                        num_epochs=20,\n",
    "                        gen_epochs=5,\n",
    "                        word_level=True,\n",
    "                        context=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# LICENSE\n",
    "\n",
    "MIT License\n",
    "\n",
    "Copyright (c) 2018 Max Woolf\n",
    "\n",
    "Permission is hereby granted, free of charge, to any person obtaining a copy\n",
    "of this software and associated documentation files (the \"Software\"), to deal\n",
    "in the Software without restriction, including without limitation the rights\n",
    "to use, copy, modify, merge, publish, distribute, sublicense, and/or sell\n",
    "copies of the Software, and to permit persons to whom the Software is\n",
    "furnished to do so, subject to the following conditions:\n",
    "\n",
    "The above copyright notice and this permission notice shall be included in all\n",
    "copies or substantial portions of the Software.\n",
    "\n",
    "THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\n",
    "IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\n",
    "FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\n",
    "AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\n",
    "LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\n",
    "OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\n",
    "SOFTWARE."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}