bert-huggingface.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qY1lXPCAJg2E"
      },
      "source": [
        "# BERT tutorial using Hugging Face\n",
        "## 教學目標\n",
        "利用 Hugging Face 套件快速使用 BERT 模型來進行下游任務訓練\n",
        "- 單一句型分類任務 (single-sentence text classification)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SNg9uH2-Jg2H"
      },
      "source": [
        "## 適用對象\n",
        "已經有基本的機器學習知識，且擁有 Python、`numpy`、`pandas`、`scikit-learn` 以及 `PyTorch` 基礎的學生。\n",
        "\n",
        "若沒有先學過 Python，請參考 [python-入門語法](./python-入門語法.ipynb) 教學。\n",
        "\n",
        "若沒有先學過 `pandas`，請參考 [pandas-基本功能](./pandas-基本功能.ipynb) 教學。\n",
        "\n",
        "若沒有先學過 `numpy`，請參考 [numpy-基本功能](./numpy-基本功能.ipynb) 教學。\n",
        "\n",
        "若沒有先學過 `scikit-learn`，請參考 [scikit-learn-基本功能](./scikit-learn-基本功能.ipynb) 教學。\n",
        "\n",
        "若沒有先學過  `PyTorch` ，請參考 [PyTorch-基本功能](./PyTorch-基本功能.ipynb) 教學。\n",
        "\n",
        "若沒有先學過如何使用 `PyTorch` 建立自然語言處理序列模型，請參考 [NN-中文文本分類](./NN-中文文本分類.ipynb) 教學。"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wYSdGFduJg2H"
      },
      "source": [
        "## BERT 簡易介紹\n",
        "### Word embeddings 的問題\n",
        "![Imgur](https://i.imgur.com/h6U5k41.png)\n",
        "- 每個單詞的意思在不同的場合下應該有不同的意義表達\n",
        "- 我們可以利用 RNN 作為語言模型，透過語言模型的輸入與輸出的處理來產生能夠理解上下文語意的 contextual embeddings\n",
        "    - Language model: 語言模型，藉由估計(或最佳化)一整個序列的生成機率來輸出字詞的模型\n",
        "        - 可以參考 [language model 的詳細教學](https://youtu.be/LheoxKjeop8?t=50)\n",
        "- 藉由此種做法，我們可以將單詞語意的 word embeddings 轉換為具有上下文語意的 contextual embeddings"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7n0nQj0qJg2I"
      },
      "source": [
        "## 所以什麼是 BERT?\n",
        "- 請參考理論層面的詳細教學 ([影片連結](https://www.youtube.com/watch?v=gh0hewYkjgo))\n",
        "- 想進行 PyTorch 的 BERT 實作來獲得深入理解可以參考 ([網誌連結](https://leemeng.tw/attack_on_bert_transfer_learning_in_nlp.html))\n",
        "- 也可以參考 Jay Alammar 的 The Illustrated BERT ([網誌連結](https://jalammar.github.io/illustrated-bert/))\n",
        "- 也可以參考原始論文 ([論文連結](https://www.aclweb.org/anthology/N19-1423/))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "p2Je-RIxJg2I"
      },
      "source": [
        "### BERT 的 Pre-training 和 Fine-tuning 與先前方法比較\n",
        "![Imgur](https://i.imgur.com/qfLhUaG.png)\n",
        "- Pre-training 已經是 NLP 領域中不可或缺的方法\n",
        "- 像 BERT 這類基於 Transformers 的[模型非常多](http://speech.ee.ntu.edu.tw/~tlkagk/courses/DLHLP20/BERT%20train%20(v8).pdf)，可以前往 [Hugging Face models](https://huggingface.co/models) 一覽究竟"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OT5oltZrJg2J"
      },
      "source": [
        "## Hugging Face 介紹\n",
        "- 🤗 Hugging Face 是專門提供自然語言處理領域的函式庫\n",
        "- 其函式庫支援 PyTorch 和 TensorFlow\n",
        "- 🤗 Hugging Face 的主要套件為:\n",
        "    1. Transformers ([連結](https://huggingface.co/transformers/index.html))\n",
        "    - 提供了現今最強大的自然語言處理模型，使用上非常彈性且方便\n",
        "    2. Tokenizers ([連結](https://huggingface.co/docs/tokenizers/python/latest/))\n",
        "    - 讓你可以快速做好 BERT 系列模型 tokenization\n",
        "    3. Datasets ([連結](https://huggingface.co/docs/datasets/))\n",
        "    - 提供多種自然語言處理任務的資料集"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "TUjyDQ-qJg2J",
        "outputId": "acce619e-2d84-41bf-89ea-c015a54e03a3"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
            "Requirement already satisfied: transformers==4.28.0 in /usr/local/lib/python3.10/dist-packages (4.28.0)\n",
            "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from transformers==4.28.0) (3.12.0)\n",
            "Requirement already satisfied: huggingface-hub<1.0,>=0.11.0 in /usr/local/lib/python3.10/dist-packages (from transformers==4.28.0) (0.14.1)\n",
            "Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from transformers==4.28.0) (1.22.4)\n",
            "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from transformers==4.28.0) (23.1)\n",
            "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from transformers==4.28.0) (6.0)\n",
            "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers==4.28.0) (2022.10.31)\n",
            "Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from transformers==4.28.0) (2.27.1)\n",
            "Requirement already satisfied: tokenizers!=0.11.3,<0.14,>=0.11.1 in /usr/local/lib/python3.10/dist-packages (from transformers==4.28.0) (0.13.3)\n",
            "Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.10/dist-packages (from transformers==4.28.0) (4.65.0)\n",
            "Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0,>=0.11.0->transformers==4.28.0) (2023.4.0)\n",
            "Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0,>=0.11.0->transformers==4.28.0) (4.5.0)\n",
            "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.28.0) (1.26.15)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.28.0) (2022.12.7)\n",
            "Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.28.0) (2.0.12)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.28.0) (3.4)\n",
            "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
            "Requirement already satisfied: datasets in /usr/local/lib/python3.10/dist-packages (2.12.0)\n",
            "Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from datasets) (1.22.4)\n",
            "Requirement already satisfied: pyarrow>=8.0.0 in /usr/local/lib/python3.10/dist-packages (from datasets) (9.0.0)\n",
            "Requirement already satisfied: dill<0.3.7,>=0.3.0 in /usr/local/lib/python3.10/dist-packages (from datasets) (0.3.6)\n",
            "Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from datasets) (1.5.3)\n",
            "Requirement already satisfied: requests>=2.19.0 in /usr/local/lib/python3.10/dist-packages (from datasets) (2.27.1)\n",
            "Requirement already satisfied: tqdm>=4.62.1 in /usr/local/lib/python3.10/dist-packages (from datasets) (4.65.0)\n",
            "Requirement already satisfied: xxhash in /usr/local/lib/python3.10/dist-packages (from datasets) (3.2.0)\n",
            "Requirement already satisfied: multiprocess in /usr/local/lib/python3.10/dist-packages (from datasets) (0.70.14)\n",
            "Requirement already satisfied: fsspec[http]>=2021.11.1 in /usr/local/lib/python3.10/dist-packages (from datasets) (2023.4.0)\n",
            "Requirement already satisfied: aiohttp in /usr/local/lib/python3.10/dist-packages (from datasets) (3.8.4)\n",
            "Requirement already satisfied: huggingface-hub<1.0.0,>=0.11.0 in /usr/local/lib/python3.10/dist-packages (from datasets) (0.14.1)\n",
            "Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from datasets) (23.1)\n",
            "Requirement already satisfied: responses<0.19 in /usr/local/lib/python3.10/dist-packages (from datasets) (0.18.0)\n",
            "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from datasets) (6.0)\n",
            "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (23.1.0)\n",
            "Requirement already satisfied: charset-normalizer<4.0,>=2.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (2.0.12)\n",
            "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (6.0.4)\n",
            "Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (4.0.2)\n",
            "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (1.9.2)\n",
            "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (1.3.3)\n",
            "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (1.3.1)\n",
            "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0.0,>=0.11.0->datasets) (3.12.0)\n",
            "Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0.0,>=0.11.0->datasets) (4.5.0)\n",
            "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->datasets) (1.26.15)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->datasets) (2022.12.7)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->datasets) (3.4)\n",
            "Requirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets) (2.8.2)\n",
            "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets) (2022.7.1)\n",
            "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.8.1->pandas->datasets) (1.16.0)\n"
          ]
        }
      ],
      "source": [
        "# 若沒有安裝 transformers 和 datasets 套件，請取消以下註解並執行\n",
        "!pip install transformers==4.28.0\n",
        "!pip install datasets"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Z3tiYaLoJg2K",
        "outputId": "914bac40-931c-4cda-a5c0-5616bf9ccded"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "PyTorch 的版本為: 2.0.0+cu118\n",
            "Hugging Face Transformers 的版本為: 4.28.0\n",
            "Hugging Face Datasets 的版本為: 2.12.0\n"
          ]
        }
      ],
      "source": [
        "# 1. 確認所需套件的版本\n",
        "import torch\n",
        "print(\"PyTorch 的版本為: {}\".format(torch.__version__))\n",
        "\n",
        "import transformers\n",
        "print(\"Hugging Face Transformers 的版本為: {}\".format(transformers.__version__))\n",
        "\n",
        "import datasets\n",
        "print(\"Hugging Face Datasets 的版本為: {}\".format(datasets.__version__))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "s-JMEX4oJg2L"
      },
      "outputs": [],
      "source": [
        "# 2. 載入其他所需套件\n",
        "\n",
        "import os\n",
        "import json\n",
        "import numpy as np\n",
        "from pathlib import Path # (Python3.4+)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZpZYuEYpJg2M"
      },
      "source": [
        "# 單一句型分類任務 (single-sentence text classification)\n",
        "## 準備資料集 (需先下載)\n",
        "我們使用 IMDb reviews 資料集作為範例"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "xyjqPyKuJg2M",
        "outputId": "7b364ae2-2a9e-4651-dbc2-a6dd329d408e"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
            "Collecting wget\n",
            "  Downloading wget-3.2.zip (10 kB)\n",
            "  Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "Building wheels for collected packages: wget\n",
            "  Building wheel for wget (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for wget: filename=wget-3.2-py3-none-any.whl size=9657 sha256=6564e22dff2dd9814d3a59d32116fa4061b684d0d1b6858107034df0f31b937f\n",
            "  Stored in directory: /root/.cache/pip/wheels/8b/f1/7f/5c94f0a7a505ca1c81cd1d9208ae2064675d97582078e6c769\n",
            "Successfully built wget\n",
            "Installing collected packages: wget\n",
            "Successfully installed wget-3.2\n"
          ]
        }
      ],
      "source": [
        "# 若沒有安裝 wget 套件，請取消以下註解並執行\n",
        "!pip install wget"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "-NyUXdw-Jg2M"
      },
      "outputs": [],
      "source": [
        "# 3. 下載 IMDb 資料集\n",
        "import wget\n",
        "url = 'http://ai.stanford.edu/~amaas/data/sentiment/aclImdb_v1.tar.gz'\n",
        "filename = wget.download(url, out='./')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "B9KNGJhfJg2M"
      },
      "outputs": [],
      "source": [
        "# 若沒有安裝 tarfile 套件，請取消以下註解並執行\n",
        "# !pip install tarfile"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Be-MEg3mJg2M"
      },
      "outputs": [],
      "source": [
        "# 4. 解壓縮 IMDb 資料集\n",
        "\n",
        "import tarfile\n",
        "\n",
        "# 指定檔案位置，並解壓縮 .gz 結尾的壓縮檔\n",
        "tar = tarfile.open('aclImdb_v1.tar.gz', 'r:gz')\n",
        "tar.extractall()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "je_x5Wa-Jg2N"
      },
      "source": [
        "## 接下來我們要進行資料前處理\n",
        "但首先要觀察解壓縮後的資料夾結構:\n",
        "```\n",
        "aclImdb---\n",
        "        |--train\n",
        "        |    |--neg\n",
        "        |    |--pos\n",
        "        |    |--...\n",
        "        |--test\n",
        "        |    |--neg\n",
        "        |    |--pos\n",
        "        |    |--...\n",
        "        |--imdb.vocab\n",
        "        |--imdbEr.text\n",
        "        |--README\n",
        "```\n",
        "其中 train 和 test 資料夾中分別又有 neg 和 pos 兩種資料夾\n",
        "\n",
        "我們要針對這兩個目標資料夾進行處理"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "nPGum-cuJg2N"
      },
      "outputs": [],
      "source": [
        "# 5. 前處理 IMDb 資料 (定義 function)\n",
        "def read_imdb_split(split_dir):\n",
        "    \"\"\"針對 IMDb 資料集進行讀檔及正負向歸類\n",
        "    Args:\n",
        "        - split_dir: IMDb 資料集的資料夾路徑\n",
        "    Return:\n",
        "        - texts: 資料集的語句部分\n",
        "        - labels: 資料集的標籤部分\n",
        "    \"\"\"\n",
        "    split_dir = Path(split_dir)\n",
        "    texts = []\n",
        "    labels = []\n",
        "    for label_dir in [\"pos\", \"neg\"]:\n",
        "        # 利用iterdir() 來列出資料夾底下的所有檔案，此功能等同於 os.path.listdir()\n",
        "        # 使用 glob 的語法分取得副檔名為 .txt 的檔案\n",
        "        for text_file in (split_dir/label_dir).glob(\"*.txt\"):\n",
        "            # read_text() 是 Pathlib的好用功能\n",
        "            tmp_text = text_file.read_text()\n",
        "            # 將讀到的文字 append 到我們事先定義的 list 中\n",
        "            texts.append(tmp_text)\n",
        "            # 將資料夾標籤作為 label，並 append 到我們事先定義的 list 中\n",
        "            labels.append(0 if label_dir == \"neg\" else 1)\n",
        "    \n",
        "    return texts, labels"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "yaFWBwWTJg2N"
      },
      "outputs": [],
      "source": [
        "# 6. 前處理 IMDb 資料 (執行)\n",
        "\n",
        "train_texts, train_labels = read_imdb_split('aclImdb/train')\n",
        "test_texts, test_labels = read_imdb_split('aclImdb/test')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LR_fGzKyJg2N"
      },
      "source": [
        "### 切分訓練資料，來分出 validation set"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Ikvd4SfUJg2N"
      },
      "outputs": [],
      "source": [
        "# 7. 使用 train_test_split 來切出 validation set\n",
        "\n",
        "from sklearn.model_selection import train_test_split\n",
        "\n",
        "# 設立隨機種子來控制隨機過程\n",
        "random_seed = 42\n",
        "\n",
        "# 設定要分出多少比例的 validation data\n",
        "valid_ratio = 0.2\n",
        "\n",
        "# 使用 train_test_split 來切分資料\n",
        "train_texts, val_texts, train_labels, val_labels = train_test_split(\n",
        "    train_texts, \n",
        "    train_labels,\n",
        "    test_size=valid_ratio, \n",
        "    random_state=random_seed\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Vl-ti6RvJg2N"
      },
      "source": [
        "## 輸入 BERT 的前處理\n",
        "![Imgur](https://i.imgur.com/3C7xDlf.png)\n",
        "(圖片來源: BERT [原始論文](https://www.aclweb.org/anthology/N19-1423/))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "084CRnJqJg2N"
      },
      "source": [
        "### Tokenization\n",
        "- 斷字的部份以 DistilBERT (Sanh et al., 2019) 的 tokenizer 為例\n",
        "- Hugging Face 的 tokenizer 可以直接幫你自動將資料轉換成 BERT 的輸入型式 (也就是加入[CLS]和[SEP] tokens)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dW2UFRcGJg2N"
      },
      "source": [
        "## Hugging Face AutoTokenizer\n",
        "- 使用 AutoTokenizer 搭配 Hugging Face models 的名稱可以直接呼叫使用\n",
        "- 舉例:\n",
        "    - transformers.AutoTokenizer.from_pretrained('distilbert-base-uncased')\n",
        "    - 等同於 transformers.DistilBertTokenizer.from_pretrained('distilbert-base-uncased')\n",
        "- [點這裡來查看 Hugging Face models 的名稱](https://huggingface.co/transformers/pretrained_models.html)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 145,
          "referenced_widgets": [
            "ff8057af42304d299a67d96409a10bf4",
            "413adfd3ba1349ad9dc10012a9da433e",
            "b6ac5fd37186433b9d85151f747b87e8",
            "8d3ff57aee164942bb9489f7ba258338",
            "d49ac22a46da4452ba652bc21c5b680b",
            "f273cfdcbb684b91961c343ecd0df1af",
            "fcecd73666ee42e2b45d3cb7efc0661b",
            "a1b08ad5fff64b51ae3ddb2a853a2375",
            "c942c97458f84b90b86023b2f66f8fa7",
            "692be76bb4214638a2a5b760d956e59c",
            "20170a5cefba4a36a42e002e3795a693",
            "5aa00f04451348fa8bd23aba4ec60edb",
            "dc003d4121494336968b53983ea378dc",
            "f9169f6b95204aecba29c4efb2491992",
            "073924322b8d439ea5b3e983102a8737",
            "b1de218d44d54e81aa672e95ffcb3b5f",
            "e02ec078e4ed4fb9bcb16678e2f5d65f",
            "106a2ca9e5f7439aa8083ebd0a0aafe0",
            "c1cab38618ad4dadb9ca75e46b988e0e",
            "df03d99362c7417a828b95db79cfec4a",
            "9d8bba46b8f648d1b923891fecf0de3a",
            "7878fb0bdf144acbabafc0bbbacf648e",
            "ea09d00e15b9450d9a37169c83030a55",
            "5d10a50aac704c6b909f2d2c6c7a5e7e",
            "420c13174a1f416ba6418c2ec4104dd6",
            "be996fecb71448d790dff3da670951ef",
            "ed42392e07d14de4a9bf4e0f016dfb0c",
            "1e9e306861754ae7b600156ed51ecad0",
            "c114a9f7332644a69667f84545e9f12d",
            "9f402fa036cc4149b421bca2cb3138cc",
            "6f522e847a8a4f558225f3bdc68f26d3",
            "5502ee2fbcfb40cbbb73377e9d8411d3",
            "5e214fd696ea4124b314a0529e3868aa",
            "3bc185aef50b46f19355048202e84476",
            "d9245de10f0b483f8298d2f0808244ff",
            "8b2baaf2ee2e499cb8975adc6c55cf8b",
            "3771e073051f42e892675251ef500625",
            "a722404ff070497cb78eb6c7304f9437",
            "e809d68f854c4411929a5c58bff92e25",
            "79630f367d234755a2fa7c33f6a06100",
            "925f719f20a44fb1ab7c107c2f0f0fde",
            "f5315ac4a43f4dedb31e39c0856bf651",
            "7c52c683d76144a2b86d4f9b9477d52e",
            "745541e301254cddae85a585b7c2c136"
          ]
        },
        "id": "smRMfe1RJg2N",
        "outputId": "a72351a0-86be-4a1f-d630-0b15a75de51e"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "ff8057af42304d299a67d96409a10bf4",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Downloading (…)okenizer_config.json:   0%|          | 0.00/28.0 [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "5aa00f04451348fa8bd23aba4ec60edb",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Downloading (…)lve/main/config.json:   0%|          | 0.00/483 [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "ea09d00e15b9450d9a37169c83030a55",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Downloading (…)solve/main/vocab.txt:   0%|          | 0.00/232k [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "3bc185aef50b46f19355048202e84476",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Downloading (…)/main/tokenizer.json:   0%|          | 0.00/466k [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "# 8. 載入 tokenizer\n",
        "\n",
        "# 在 Hugging Face 套件中可使用 .from_pretrained() 的方法來導入預訓練模型\n",
        "tokenizer = transformers.AutoTokenizer.from_pretrained('distilbert-base-uncased')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9wB1drpFJg2N"
      },
      "outputs": [],
      "source": [
        "# 9. 分別將3種資料 (train/valid/test) 做 tokenization\n",
        "# truncation 代表依照 max_length 進行序列長度的裁切\n",
        "# max_length 可以在 tokenizer 的 parameters 中進行設定\n",
        "# 如果沒有指定 max_length，則依照所使用的模型的序列最大長度\n",
        "# padding 為 True 表示會將序列長度補齊至該 batch 的最大長度 (欲知詳情請查看 source code)\n",
        "\n",
        "train_encodings = tokenizer(train_texts, truncation=True, padding=True)\n",
        "val_encodings = tokenizer(val_texts, truncation=True, padding=True)\n",
        "test_encodings = tokenizer(test_texts, truncation=True, padding=True)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "VDGsIlStJg2O",
        "outputId": "6e422538-702a-44f6-e034-1781c9629471"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "512"
            ]
          },
          "execution_count": 13,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# 10. 查看 max_length\n",
        "\n",
        "tokenizer.model_max_length"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "76aDSEWVJg2O",
        "outputId": "8d1b1e7c-1b31-4f61-cb2c-24a0f8822e44"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "The ID of [CLS] token is 101.\n",
            "The ID of [SEP] token is 102.\n"
          ]
        }
      ],
      "source": [
        "# 11. 查看 [CLS] token 和 [SEP] token 在字典中的 ID\n",
        "\n",
        "print(\"The ID of [CLS] token is {}.\".format(tokenizer.vocab[\"[CLS]\"]))\n",
        "print(\"The ID of [SEP] token is {}.\".format(tokenizer.vocab[\"[SEP]\"]))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hfGCHkFpJg2O"
      },
      "source": [
        "### 檢查 tokenization 後的結果\n",
        "- 使用 Hugging Face tokenizer 進行 tokenization 後的結果是一個 dict\n",
        "- 這個 dict 的 keys 包含 'input_ids' 和 'attention_mask'\n",
        "- input_ids: 原本句子中的每個字詞被斷詞後轉換成字典的 ID\n",
        "    - 注意!! tokenizer 小小的動作已經幫你完成了斷詞和 word to ID 的轉換\n",
        "- attention_mask: tokenization 後句子中包含文字的部分為 1，padding 的部分為 0\n",
        "    - 可以想像成模型需要把注意力放在有文字的位置"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "9Bm0BoZIJg2O",
        "outputId": "704d8839-5ce2-410f-f943-bd6a80e40bd5"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "dict_keys(['input_ids', 'attention_mask'])\n",
            "[101, 28851, 8754, 2143, 2013, 3245, 1999, 2029, 1996, 2493, 1997, 12159, 23441, 2072, 2152, 2082, 2024, 12892, 2007, 1037, 2047, 1010, 21237, 4818, 4054, 2315, 3335, 2000, 6843, 1006, 2984, 24185, 4948, 4492, 1007, 1012, 2000, 6843, 2003, 1037, 2189, 5223, 2099, 1998, 7499, 2015, 1996, 3315, 16958, 1997, 1996, 2493, 2005, 2037, 9099, 17603, 28231, 2015, 1012, 2877, 1996, 3715, 2114, 2014, 2003, 4569, 1011, 8295, 24808, 14566, 5349, 1006, 1052, 1012, 1046, 1012, 7082, 2015, 1007, 1010, 1996, 1001, 1015, 12716, 2229, 5470, 2040, 1010, 2062, 2084, 2505, 1010, 4122, 1996, 2600, 2177, 2000, 2501, 2014, 2774, 1012, 1026, 7987, 1013, 1028, 1026, 7987, 1013, 1028, 2085, 1008, 2023, 1008, 2003, 2019, 5263, 3185, 2000, 9507, 1012, 2034, 1998, 16097, 1010, 1996, 6050, 2003, 9788, 1010, 2007, 2774, 2011, 2107, 3324, 2004, 5650, 6201, 1998, 1996, 10966, 5230, 1999, 2804, 2000, 1996, 16514, 2512, 1011, 2644, 26285, 1997, 12716, 2229, 2774, 1012, 1000, 9454, 8840, 18384, 16940, 1000, 1010, 1000, 20682, 2050, 2003, 1037, 7196, 24779, 1000, 1010, 1998, 1000, 22312, 21638, 28872, 29432, 1000, 2024, 2074, 1037, 2261, 1997, 2068, 1012, 2279, 1010, 1996, 3459, 5621, 3957, 2009, 2037, 2035, 1010, 2007, 7082, 2015, 2019, 7812, 3601, 2005, 1996, 2535, 1997, 24808, 1025, 2016, 2003, 1037, 2995, 12208, 1012, 6320, 3158, 6986, 6528, 1998, 2139, 2100, 2402, 2024, 17300, 2004, 3419, 1998, 5736, 1010, 24185, 4948, 4492, 2003, 2092, 3459, 2114, 2828, 2004, 1996, 1055, 3630, 4140, 2100, 1998, 25134, 3993, 2000, 6843, 1010, 16235, 4922, 2038, 2028, 1997, 2010, 2190, 2412, 3033, 2004, 9378, 9954, 1011, 13992, 10670, 6755, 27773, 2121, 1010, 1998, 2047, 2088, 24945, 2107, 2004, 5980, 4679, 1010, 2703, 12075, 2884, 1006, 3391, 4569, 2004, 2189, 3836, 2720, 1012, 11338, 28637, 1007, 1998, 1996, 2613, 2123, 12872, 2024, 4569, 2004, 2467, 1012, 1998, 1010, 1997, 2607, 1010, 2009, 1005, 1055, 1037, 7438, 2000, 2156, 1996, 12716, 2229, 2652, 3209, 1012, 1026, 7987, 1013, 1028, 1026, 7987, 1013, 1028, 1996, 3185, 2038, 2995, 4382, 1012, 1996, 2943, 2504, 2003, 2152, 1010, 2007, 2522, 1011, 2466, 3166, 1998, 2472, 8926, 15745, 20668, 5026, 1037, 2307, 3066, 1997, 22012, 2000, 1996, 8931, 1012, 2045, 1005, 1055, 2036, 1037, 2307, 3168, 1997, 8562, 1012, 1996, 3259, 13297, 18201, 2003, 1037, 21688, 2742, 1997, 2023, 1012, 2023, 8908, 2157, 2000, 1996, 1000, 13387, 1000, 2806, 1997, 3496, 22166, 1012, 2045, 2024, 2130, 26316, 5016, 12328, 2580, 2011, 2925, 5789, 3896, 3862, 6487, 28516, 7629, 1010, 1999, 2028, 1997, 2010, 5700, 20929, 1012, 1026, 7987, 1013, 1028, 1026, 7987, 1013, 1028, 2055, 2004, 2204, 2004, 2019, 3691, 1011, 13366, 14147, 1010, 6985, 1011, 2028, 1005, 1055, 1011, 2157, 1011, 2000, 1011, 2283, 2143, 2064, 2131, 1012, 1000, 2600, 1005, 1050, 1005, 4897, 2152, 2082, 1000, 2003, 1010, 3243, 3432, 1010, 1037, 6919, 8754, 2143, 1012, 1026, 7987, 1013, 1028, 1026, 7987, 1013, 1028, 1022, 1013, 2184, 102, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]\n",
            "[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]\n"
          ]
        }
      ],
      "source": [
        "# 12. 檢查 tokenization 後的結果\n",
        "\n",
        "print(val_encodings.keys())\n",
        "print(val_encodings.input_ids[0])\n",
        "print(val_encodings.attention_mask[0])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "hlrnqw9xJg2O"
      },
      "outputs": [],
      "source": [
        "# 13. 透過 PyTorch Dataset 來建立能夠進行方便資料存取的格式\n",
        "\n",
        "class IMDbDataset(torch.utils.data.Dataset):\n",
        "    def __init__(self, encodings, labels):\n",
        "        # Dataset class 的 parameters 放入我們 tokenization 後的資料以及資料的標籤\n",
        "        self.encodings = encodings\n",
        "        self.labels = labels\n",
        "\n",
        "    def __getitem__(self, idx):\n",
        "        # 請注意 tokenization 後的資料是一個 dict\n",
        "        # 在此步驟將資料以及標籤都轉換為 PyTorch 的 tensors\n",
        "        item = {key: torch.tensor(val[idx]) for key, val in self.encodings.items()}\n",
        "        item['labels'] = torch.tensor(self.labels[idx])\n",
        "\n",
        "        return item\n",
        "\n",
        "    def __len__(self):\n",
        "        # 回傳資料集的總數\n",
        "        return len(self.labels)\n",
        "\n",
        "train_dataset = IMDbDataset(train_encodings, train_labels)\n",
        "val_dataset = IMDbDataset(val_encodings, val_labels)\n",
        "test_dataset = IMDbDataset(test_encodings, test_labels)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rlHobatFJg2O"
      },
      "source": [
        "### 除了自己處理資料，你還可以使用 Hugging Face Datasets\n",
        "- Hugging Face Datasets 已經幫你收錄了自然語言處理領域常見的資料集\n",
        "- 直接呼叫 Datasets 並搭配下面幾個 cells 的語法，可省下不少時間\n",
        "- 但前提是你要進行的任務資料集有被收錄在 Hugging Face Datasets"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "jFAC_YrZJg2O",
        "outputId": "db4495a8-cb59-407d-88d5-ba4f7930b4a2"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "現在 Hugging Face Datasets 有 34781 個資料集可以使用\n",
            "===============================================\n"
          ]
        }
      ],
      "source": [
        "# 14. 查看 Hugging Face Datasets 的資訊\n",
        "\n",
        "datasets_list = datasets.list_datasets()\n",
        "\n",
        "print(\"現在 Hugging Face Datasets 有 {} 個資料集可以使用\".format(len(datasets_list)))\n",
        "print(\"===============================================\")\n",
        "# print(\"所有的資料集如下: \")\n",
        "# print(', '.join(dataset for dataset in datasets_list))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 217,
          "referenced_widgets": [
            "6d99c91aa598467da8c43a9c2e4f0099",
            "f353ca80976a4d5590664e7cae83efbf",
            "d08eeb45da3b4c6484234058ae3346f5",
            "2709773a6e06458b97e9b0aa95f8f854",
            "22a052aa12274235ac51079156a3503f",
            "33af93996db04e94b7c86e5ce025fb42",
            "5b8be14016fd4c18826f39054aa0959d",
            "4fa27a09549d4491ab5615fa9a2e63bf",
            "8daa2263e04d4d70bdcabb09a5e37768",
            "56db8c7ca2724d94868ad544d9aa556c",
            "9894b722016b4d2a8ed1b8b7e6ad624e",
            "fe17eb440b6543658d1f802853d71af2",
            "2cd9423342f74bd1b61afbfcfe407790",
            "994f543d720d4b09bd06b835fcf62878",
            "f54452fe5af64a1785a106b0cf061aa7",
            "b8ef6770827e4ff3bb2ea56c1077213a",
            "1906497eab894a70b1b20df3f78aec5d",
            "ce865d95997c4917b89e777aa0e9dc24",
            "b39befec3d2b44e6a52f73bff8e42afb",
            "1a99dbddcdcc42b78e2b17fb218f6419",
            "17714c7c793b4982bad7de2b8359ccbc",
            "3a86214abe0547ce9a031beb357ebe21",
            "ac9116019bc04b8fa53de6cf6235228f",
            "836e7339a12e41a6b241729f10818768",
            "87e9d3051b9d4126a89bff8811235291",
            "9cd84ccfd43b4909923eca0f45988a85",
            "e3d1092747e440069c1ce1551045a0ea",
            "4cb2669c09b342d2817a6819e1c2efef",
            "2d10eac4e6ef4d0bb486f7dc8ebc45b5",
            "a6c36498cdee43748930f71cfd164de2",
            "382a0f6ddc664e0ca211df8b44e6323c",
            "21547c68f72d4c8aa34cbba45115592d",
            "1124d2f81581483bb9d3d50108a5ff9d",
            "7e17bc4f5ab940d28ad0bcf04cac4d5c",
            "449e9ddf42c74c5c8499736395ea7404",
            "3f54dde4baf54ac58d109180a169fc77",
            "8c4c9735771a4c4e9747b0b0cc92bd8d",
            "aa6b35df259948eebf03725e060445b8",
            "0c030acc1f554cc88b9ee6e3c9cd0572",
            "724b1987359b4037ad9c128575ab26e2",
            "e62dab3f5c8d4ef48807ce2f9cb6a036",
            "e350175227a34998a42d2676ea57dc99",
            "138d2c7a03ce4e1b9e3a97f02c4f4318",
            "1c5ec57b12ed4177bd3db25243d29131",
            "cac0a0b51f174f95a8091df28a0deb16",
            "a651bd76b3424fa1875f115f657a710b",
            "3b02be847e38450fb7c93b1ead1ba198",
            "365f6942322244f8a0e9bd8dd2560041",
            "2376f500d77e4e04aecb6a17f50485ea",
            "e6a8b9fdfe5a43c9ae6ec21ad679e9c9",
            "53dd301e7bd2434283c2e74fdd7b5816",
            "5cb31f3410f5428c84ee4bbaab46ab59",
            "9213e3cff09a4f31aac733351ca4ee7d",
            "f754d050a21445e79d71698f269b17de",
            "e22f9c5120934a3a95bd285950a0b8bd",
            "4830c2a45d0947d5ad10bd5898865cdc",
            "635854ae59a048ac9e3d39d9c4c2fb29",
            "ac2a1b1a2a3945b9a0e17e25caf882b8",
            "e729af8b4de34754b93635980e1b8cee",
            "11033bfef6be4fdf96957dd05d5feed8",
            "cf6ce47a025d4b59be1e795051796f78",
            "bf9b09c4d2014c30af85a5bf3481f4ce",
            "699734b4fe4c4e2aac328ee1065e7f92",
            "c7b61791eede43c3805db8b6b7c8d876",
            "ee9cc4e598a04c86b038519e71cba43e",
            "e6b11f8be8ba4a288cc60af0625e72a5",
            "647307442cd747239d7b3afaa82504e3",
            "ba75df6ea53a4bbcbec504f92748950b",
            "899e566bed634ded81ede51ed48ccc76",
            "390b50f9ef06434fad440b565f743d6b",
            "3630bb47d9874735a7779d40a16ee7dc",
            "c18e8047c87d43f7bbbe19db0325a7ec",
            "1a84529e32ed47a6be353cd48dfea316",
            "fa625601c86941d598cb9cc0bd8293f5",
            "f8d2c921b5c846a2b2f3647f155a8f45",
            "d6532a4c49bb4e00b01025303272b81e",
            "d2c30672cc8842e0be96f57357f20e93"
          ]
        },
        "id": "o5nX6CEYJg2O",
        "outputId": "27f14221-1845-45b7-c476-787d14e42f1f"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "6d99c91aa598467da8c43a9c2e4f0099",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Downloading builder script:   0%|          | 0.00/4.31k [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "fe17eb440b6543658d1f802853d71af2",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Downloading metadata:   0%|          | 0.00/2.17k [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "ac9116019bc04b8fa53de6cf6235228f",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Downloading readme:   0%|          | 0.00/7.59k [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Downloading and preparing dataset imdb/plain_text to /root/.cache/huggingface/datasets/imdb/plain_text/1.0.0/d613c88cf8fa3bab83b4ded3713f1f74830d1100e171db75bbddb80b3345c9c0...\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "7e17bc4f5ab940d28ad0bcf04cac4d5c",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Downloading data:   0%|          | 0.00/84.1M [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "cac0a0b51f174f95a8091df28a0deb16",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Generating train split:   0%|          | 0/25000 [00:00<?, ? examples/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "4830c2a45d0947d5ad10bd5898865cdc",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Generating test split:   0%|          | 0/25000 [00:00<?, ? examples/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "647307442cd747239d7b3afaa82504e3",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Generating unsupervised split:   0%|          | 0/50000 [00:00<?, ? examples/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Dataset imdb downloaded and prepared to /root/.cache/huggingface/datasets/imdb/plain_text/1.0.0/d613c88cf8fa3bab83b4ded3713f1f74830d1100e171db75bbddb80b3345c9c0. Subsequent calls will reuse this data.\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "WARNING:datasets.builder:Found cached dataset imdb (/root/.cache/huggingface/datasets/imdb/plain_text/1.0.0/d613c88cf8fa3bab83b4ded3713f1f74830d1100e171db75bbddb80b3345c9c0)\n"
          ]
        }
      ],
      "source": [
        "# 15. 從 Hugging Face Datasets 載入資料並做資料切分\n",
        "\n",
        "# 載入 IMDb 的訓練資料集\n",
        "train = datasets.load_dataset(\"imdb\", split=\"train\")\n",
        "\n",
        "# 設立隨機種子來控制隨機過程\n",
        "random_seed = 42\n",
        "# 從 IMDb 的訓練資料集中切分出驗證資料集\n",
        "splits = train.train_test_split(\n",
        "    test_size=0.2,\n",
        "    seed=random_seed\n",
        ")\n",
        "train, valid = splits['train'], splits['test']\n",
        "\n",
        "# 載入 IMDb 的測試資料集\n",
        "test = datasets.load_dataset(\"imdb\", split=\"test\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "TU4Zkxx2Jg2O",
        "outputId": "6f9206e7-0042-4e84-dbc7-ad716980eba0"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "20000\n",
            "5000\n",
            "25000\n"
          ]
        }
      ],
      "source": [
        "print(len(train))\n",
        "print(len(valid))\n",
        "print(len(test))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 17,
          "referenced_widgets": [
            "e75a2f56cc9043edbfbbbef1fd9f6689",
            "4188a2d6ba9d42e5874df6b6f78f357c",
            "ac768998e26c4959918b8c31eaffd05a",
            "34bd27a3a4954f77ac2a32ec8219eed8",
            "334e1833625d43b487094848cf70281a",
            "f3eb71e00ee84b089161f5e7b535ebc1",
            "306aa84d2b294b8e9874ec724734e155",
            "894f10fb03784042b5d0f00f0cadcae4",
            "16f46279e7bc4ae3acafa5902ed1af5a",
            "4e6440cbc009413694c35f640de6b563",
            "99a6ef2f822d4d139eb9f74a05e0350c",
            "1b384cdabf104251b297542fba384a1f",
            "3b9b881799344bf39b05e99aabd1bac1",
            "e100ee4dc3934f5fade5795c8747d874",
            "cc9432d8dbda48979241c730d730b1e4",
            "a34fadcfc80f48efb669006d81086746",
            "f41d76da8e934e919d1313c0fb0066a7",
            "5ca8daa1cbd8410ea42f2f3e84bb0a53",
            "04872ffde4bc41efa6fcc2a215cc9b42",
            "e095c0d0e2b944598b45e9f46c4ffdd1",
            "6479d088f4bb48ce81733afaad75c230",
            "e177ccefc8be46179b445a4103bdeb90",
            "4f5a69ae0c6d4072ba036791a3600313",
            "69478e1b7c5949308967dd5d27798bda",
            "fe1d17935ef9439aa51c5eb640dfa3c2",
            "6d0f42aae0484b3b8788e46571f75145",
            "f93b1139aec9492bba54c90f594d9415",
            "50aa561c16c94a99872867bebdb1970e",
            "1070652770ff4180821ee5d73c80adda",
            "5e41ecc78a99447eadcee0ef8d43a4ae",
            "60b218637f4048d7973480d0d66bd503",
            "9e88eaf822394ac1aa498c857595f4e6",
            "0d77aa49fbc84c41a620a3e9dfc62da5"
          ]
        },
        "id": "mOT5JmqFJg2O",
        "outputId": "4bde45ab-6997-487e-a52f-3ebc30059c8b"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "e75a2f56cc9043edbfbbbef1fd9f6689",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Map:   0%|          | 0/20000 [00:00<?, ? examples/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "1b384cdabf104251b297542fba384a1f",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Map:   0%|          | 0/5000 [00:00<?, ? examples/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "4f5a69ae0c6d4072ba036791a3600313",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Map:   0%|          | 0/25000 [00:00<?, ? examples/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "# 16. 將 Hugging Face Datasets 轉為 PyTorch Dataset 的封裝\n",
        "\n",
        "def to_torch_data(hug_dataset):\n",
        "    \"\"\"將 Hugging Face Datasets 轉為 PyTorch Dataset\n",
        "    Args:\n",
        "        - hug_dataset: 從 Datasets 載入的資料集\n",
        "    Return:\n",
        "        - dataset: 已轉為 PyTorch Dataset 的資料集\n",
        "    \"\"\"\n",
        "    dataset = hug_dataset.map(\n",
        "        lambda batch: tokenizer(\n",
        "            batch[\"text\"],\n",
        "            truncation=True,\n",
        "            padding=True\n",
        "        ),\n",
        "        batched=True\n",
        "    )\n",
        "    dataset.set_format(\n",
        "        type='torch',\n",
        "        columns=[\n",
        "            'input_ids',\n",
        "            'attention_mask',\n",
        "            'label'\n",
        "        ]\n",
        "    )\n",
        "    return dataset\n",
        "\n",
        "train_dataset = to_torch_data(train)\n",
        "val_dataset = to_torch_data(valid)\n",
        "test_dataset = to_torch_data(test)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7MF4ZQrNJg2O"
      },
      "source": [
        "## 使用 Hugging Face 的模型\n",
        "- 在這個 API 盛行的世代，總是有人幫你設想周到\n",
        "- [Hugging Face 的模型頁面連結](https://huggingface.co/models)\n",
        "- 以 BERT 為例，只要透過 AutoModel.from_pretrained(\"bert-base-uncased\")，就可以直接使用 BertModel\n",
        "- 需要注意的是接下來你要做怎樣的下游任務訓練\n",
        "- 同樣以 BERT 為例，在原始論文中 BERT 進行過以下的任務:\n",
        "    - Sentence pair classification: MNLI/QQP/QNLI/MRPC/RTE/WNLI\n",
        "        - 對應 `BertForSequenceClassification`\n",
        "        - 使用雙句結合，並以分類的方式進行訓練\n",
        "    - Semantic textual similarity: STS-B\n",
        "        - `BertForSequenceClassification`\n",
        "        - 使用雙句結合，並以迴歸的方式進行訓練\n",
        "    - Single sentence classification: SST-2/CoLA\n",
        "        - 對應 `BertForSequenceClassification`\n",
        "        - 使用單句，並以迴歸的方式進行訓練\n",
        "    - Question answering: SQuAD v1.1/v2.0\n",
        "        - 對應 `BertForQuestionAnswering`\n",
        "        - 使用雙句(問題+原文)，並透過答案在原文中的位置進行訓練\n",
        "    - Named-entity recognition (slot filling): CoNLL-2003\n",
        "        - 對應 `BertForTokenClassification`\n",
        "        - 使用單句，並以分類的方式進行訓練\n",
        "- 如果要進行的下游任務訓練不在 Hugging Face 已經建好的模型範圍，那就需要自己寫一個 model class:\n",
        "    1. 繼承 torch.nn.Module\n",
        "    2. 利用 super 來繼承所有親屬類別的實體屬性\n",
        "    3. 定義欲使用的 pre-trained model\n",
        "    4. 定義會使用到的層如 linear 或 Dropout 等\n",
        "    5. 設計 forward function 並且設定下游任務的輸出"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 156,
          "referenced_widgets": [
            "9cf95f0b331d400c8f91dff614d7511f",
            "2ffc7b9056f24a5d81433bef9bd5d77f",
            "7ecd01555e2041bf856d9d0bed7b9845",
            "c534b33e525c4e58b35dfc59b1cd03ac",
            "780bf8ccfa5546f2898826bea42f6a8b",
            "06e2b9481fc449e69d8427fc2517d310",
            "b53b3e8ee90e4e7cad40e65cb5a05e4f",
            "9e0c701c2e5f474194fce65ad4bdca7c",
            "3e1ad418853d4b5a903b5499982c6d75",
            "4ab15edaf3494fefbb476aa936f3fb0e",
            "fe9453c4e06c4d92b90c930b9756defd"
          ]
        },
        "id": "XeyIxdSsJg2O",
        "outputId": "40c14831-bb82-479d-8a0e-e7d8c145617b"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "9cf95f0b331d400c8f91dff614d7511f",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Downloading pytorch_model.bin:   0%|          | 0.00/268M [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Some weights of the model checkpoint at distilbert-base-uncased were not used when initializing DistilBertForSequenceClassification: ['vocab_layer_norm.bias', 'vocab_projector.weight', 'vocab_transform.weight', 'vocab_layer_norm.weight', 'vocab_projector.bias', 'vocab_transform.bias']\n",
            "- This IS expected if you are initializing DistilBertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n",
            "- This IS NOT expected if you are initializing DistilBertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n",
            "Some weights of DistilBertForSequenceClassification were not initialized from the model checkpoint at distilbert-base-uncased and are newly initialized: ['classifier.weight', 'classifier.bias', 'pre_classifier.bias', 'pre_classifier.weight']\n",
            "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
          ]
        }
      ],
      "source": [
        "# 17.\n",
        "# 利用 AutoModel 呼叫模型\n",
        "model = transformers.AutoModelForSequenceClassification.from_pretrained(\"distilbert-base-uncased\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2sFWgVD0Jg2P"
      },
      "source": [
        "## 進行模型的訓練\n",
        "### 使用 Hugging Face Trainer ([Documentation](https://huggingface.co/transformers/main_classes/trainer.html))\n",
        "- Trainer 是 Hugging Face 中高度封裝的套件之一，負責模型訓練時期的\"流程\"\n",
        "- 過去我們自行寫訓練流程的程式碼可以交給 Trainer\n",
        "- Trainer 需要搭配使用 [TrainingArguments](https://huggingface.co/transformers/main_classes/trainer.html#transformers.TrainingArguments)\n",
        "    - TrainingArguments 是 Trainer 所需要的引數"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "vnSXEyE3Jg2P"
      },
      "outputs": [],
      "source": [
        "# print(train_dataset[0])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "KZgSNytdJg2P"
      },
      "outputs": [],
      "source": [
        "# 18. 建立自定的評估的指標 (定義 function)\n",
        "# 將作為 transformers.Trainer 的 parameters 之一\n",
        "\n",
        "# Scikit-learn 的 precision_recall_fscore_support 套件可以一次計算 F1 score, precision, 和 recall\n",
        "from sklearn.metrics import accuracy_score, precision_recall_fscore_support\n",
        "\n",
        "def compute_metrics(pred):\n",
        "    labels = pred.label_ids\n",
        "    preds = np.argmax(pred.predictions, axis=1)\n",
        "    precision, recall, f1, _ = precision_recall_fscore_support(labels, preds, average='binary')\n",
        "    acc = accuracy_score(labels, preds)\n",
        "    return {\n",
        "        'accuracy': acc,\n",
        "        'f1': f1,\n",
        "        'precision': precision,\n",
        "        'recall': recall\n",
        "    }"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "i5oaU6rHaBB7",
        "outputId": "bfb59f48-917f-4a97-ab09-8b151b3a2849"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
            "Collecting accelerate\n",
            "  Downloading accelerate-0.19.0-py3-none-any.whl (219 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m219.1/219.1 kB\u001b[0m \u001b[31m18.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from accelerate) (1.22.4)\n",
            "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from accelerate) (23.1)\n",
            "Requirement already satisfied: psutil in /usr/local/lib/python3.10/dist-packages (from accelerate) (5.9.5)\n",
            "Requirement already satisfied: pyyaml in /usr/local/lib/python3.10/dist-packages (from accelerate) (6.0)\n",
            "Requirement already satisfied: torch>=1.6.0 in /usr/local/lib/python3.10/dist-packages (from accelerate) (2.0.0+cu118)\n",
            "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch>=1.6.0->accelerate) (3.12.0)\n",
            "Requirement already satisfied: typing-extensions in /usr/local/lib/python3.10/dist-packages (from torch>=1.6.0->accelerate) (4.5.0)\n",
            "Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch>=1.6.0->accelerate) (1.11.1)\n",
            "Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch>=1.6.0->accelerate) (3.1)\n",
            "Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch>=1.6.0->accelerate) (3.1.2)\n",
            "Requirement already satisfied: triton==2.0.0 in /usr/local/lib/python3.10/dist-packages (from torch>=1.6.0->accelerate) (2.0.0)\n",
            "Requirement already satisfied: cmake in /usr/local/lib/python3.10/dist-packages (from triton==2.0.0->torch>=1.6.0->accelerate) (3.25.2)\n",
            "Requirement already satisfied: lit in /usr/local/lib/python3.10/dist-packages (from triton==2.0.0->torch>=1.6.0->accelerate) (16.0.3)\n",
            "Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch>=1.6.0->accelerate) (2.1.2)\n",
            "Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch>=1.6.0->accelerate) (1.3.0)\n",
            "Installing collected packages: accelerate\n",
            "Successfully installed accelerate-0.19.0\n"
          ]
        }
      ],
      "source": [
        "!pip install --upgrade accelerate"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "wHfKgmDjJg2U",
        "outputId": "7747ed69-87ca-4904-a3f3-e1fb5d394f1f"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/transformers/optimization.py:391: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this warning\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "data": {
            "text/html": [
              "\n",
              "    <div>\n",
              "      \n",
              "      <progress value='1480' max='3750' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
              "      [1480/3750 18:48 < 28:53, 1.31 it/s, Epoch 1.18/3]\n",
              "    </div>\n",
              "    <table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              " <tr style=\"text-align: left;\">\n",
              "      <th>Step</th>\n",
              "      <th>Training Loss</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>10</td>\n",
              "      <td>0.693400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>20</td>\n",
              "      <td>0.692700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>30</td>\n",
              "      <td>0.685500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>40</td>\n",
              "      <td>0.691800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>50</td>\n",
              "      <td>0.692100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>60</td>\n",
              "      <td>0.686200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>70</td>\n",
              "      <td>0.671900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>80</td>\n",
              "      <td>0.651900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>90</td>\n",
              "      <td>0.604700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>100</td>\n",
              "      <td>0.542800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>110</td>\n",
              "      <td>0.426500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>120</td>\n",
              "      <td>0.384500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>130</td>\n",
              "      <td>0.367400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>140</td>\n",
              "      <td>0.347700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>150</td>\n",
              "      <td>0.323500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>160</td>\n",
              "      <td>0.316500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>170</td>\n",
              "      <td>0.367800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>180</td>\n",
              "      <td>0.315600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>190</td>\n",
              "      <td>0.306800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>200</td>\n",
              "      <td>0.241300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>210</td>\n",
              "      <td>0.280300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>220</td>\n",
              "      <td>0.309200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>230</td>\n",
              "      <td>0.335700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>240</td>\n",
              "      <td>0.342500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>250</td>\n",
              "      <td>0.253300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>260</td>\n",
              "      <td>0.243800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>270</td>\n",
              "      <td>0.414700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>280</td>\n",
              "      <td>0.276200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>290</td>\n",
              "      <td>0.263400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>300</td>\n",
              "      <td>0.304300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>310</td>\n",
              "      <td>0.298400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>320</td>\n",
              "      <td>0.239100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>330</td>\n",
              "      <td>0.347700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>340</td>\n",
              "      <td>0.250400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>350</td>\n",
              "      <td>0.248700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>360</td>\n",
              "      <td>0.249300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>370</td>\n",
              "      <td>0.399100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>380</td>\n",
              "      <td>0.279100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>390</td>\n",
              "      <td>0.417000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>400</td>\n",
              "      <td>0.290400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>410</td>\n",
              "      <td>0.412200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>420</td>\n",
              "      <td>0.264300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>430</td>\n",
              "      <td>0.359900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>440</td>\n",
              "      <td>0.398500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>450</td>\n",
              "      <td>0.363100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>460</td>\n",
              "      <td>0.370900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>470</td>\n",
              "      <td>0.314900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>480</td>\n",
              "      <td>0.261200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>490</td>\n",
              "      <td>0.304300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>500</td>\n",
              "      <td>0.327300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>510</td>\n",
              "      <td>0.331700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>520</td>\n",
              "      <td>0.343500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>530</td>\n",
              "      <td>0.368600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>540</td>\n",
              "      <td>0.316800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>550</td>\n",
              "      <td>0.283200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>560</td>\n",
              "      <td>0.324900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>570</td>\n",
              "      <td>0.194200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>580</td>\n",
              "      <td>0.332800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>590</td>\n",
              "      <td>0.232500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>600</td>\n",
              "      <td>0.238500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>610</td>\n",
              "      <td>0.200400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>620</td>\n",
              "      <td>0.289400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>630</td>\n",
              "      <td>0.312600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>640</td>\n",
              "      <td>0.259400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>650</td>\n",
              "      <td>0.217300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>660</td>\n",
              "      <td>0.368600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>670</td>\n",
              "      <td>0.294200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>680</td>\n",
              "      <td>0.241800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>690</td>\n",
              "      <td>0.246500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>700</td>\n",
              "      <td>0.387400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>710</td>\n",
              "      <td>0.282400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>720</td>\n",
              "      <td>0.387000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>730</td>\n",
              "      <td>0.260300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>740</td>\n",
              "      <td>0.342500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>750</td>\n",
              "      <td>0.380600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>760</td>\n",
              "      <td>0.238200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>770</td>\n",
              "      <td>0.206300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>780</td>\n",
              "      <td>0.259200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>790</td>\n",
              "      <td>0.294600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>800</td>\n",
              "      <td>0.343100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>810</td>\n",
              "      <td>0.214800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>820</td>\n",
              "      <td>0.316300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>830</td>\n",
              "      <td>0.249700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>840</td>\n",
              "      <td>0.246600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>850</td>\n",
              "      <td>0.218000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>860</td>\n",
              "      <td>0.208200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>870</td>\n",
              "      <td>0.186600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>880</td>\n",
              "      <td>0.185800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>890</td>\n",
              "      <td>0.266000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>900</td>\n",
              "      <td>0.257500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>910</td>\n",
              "      <td>0.247500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>920</td>\n",
              "      <td>0.353000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>930</td>\n",
              "      <td>0.310600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>940</td>\n",
              "      <td>0.315000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>950</td>\n",
              "      <td>0.277100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>960</td>\n",
              "      <td>0.271800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>970</td>\n",
              "      <td>0.230400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>980</td>\n",
              "      <td>0.200500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>990</td>\n",
              "      <td>0.516700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1000</td>\n",
              "      <td>0.340200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1010</td>\n",
              "      <td>0.285400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1020</td>\n",
              "      <td>0.273400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1030</td>\n",
              "      <td>0.312800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1040</td>\n",
              "      <td>0.259500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1050</td>\n",
              "      <td>0.273200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1060</td>\n",
              "      <td>0.154100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1070</td>\n",
              "      <td>0.398600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1080</td>\n",
              "      <td>0.224900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1090</td>\n",
              "      <td>0.287400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1100</td>\n",
              "      <td>0.324500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1110</td>\n",
              "      <td>0.259100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1120</td>\n",
              "      <td>0.332800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1130</td>\n",
              "      <td>0.252300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1140</td>\n",
              "      <td>0.221400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1150</td>\n",
              "      <td>0.195200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1160</td>\n",
              "      <td>0.219100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1170</td>\n",
              "      <td>0.235800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1180</td>\n",
              "      <td>0.231700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1190</td>\n",
              "      <td>0.281500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1200</td>\n",
              "      <td>0.242200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1210</td>\n",
              "      <td>0.216000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1220</td>\n",
              "      <td>0.234600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1230</td>\n",
              "      <td>0.251800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1240</td>\n",
              "      <td>0.240200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1250</td>\n",
              "      <td>0.255600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1260</td>\n",
              "      <td>0.150600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1270</td>\n",
              "      <td>0.149700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1280</td>\n",
              "      <td>0.218800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1290</td>\n",
              "      <td>0.159300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1300</td>\n",
              "      <td>0.214900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1310</td>\n",
              "      <td>0.148700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1320</td>\n",
              "      <td>0.164300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1330</td>\n",
              "      <td>0.193700</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1340</td>\n",
              "      <td>0.235400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1350</td>\n",
              "      <td>0.135100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1360</td>\n",
              "      <td>0.176500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1370</td>\n",
              "      <td>0.173900</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1380</td>\n",
              "      <td>0.183400</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1390</td>\n",
              "      <td>0.105200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1400</td>\n",
              "      <td>0.128500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1410</td>\n",
              "      <td>0.149200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1420</td>\n",
              "      <td>0.179100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1430</td>\n",
              "      <td>0.076800</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1440</td>\n",
              "      <td>0.217300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1450</td>\n",
              "      <td>0.152000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1460</td>\n",
              "      <td>0.150000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1470</td>\n",
              "      <td>0.145100</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table><p>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "# 19. 訓練模型\n",
        "\n",
        "# 設定 TrainingArguments\n",
        "training_args = transformers.TrainingArguments(\n",
        "    output_dir='./results',          # 輸出的資料夾\n",
        "    num_train_epochs=3,              # 總共訓練的 epoch 數目\n",
        "    learning_rate=2e-5,              # 學習率\n",
        "    per_device_train_batch_size=16,  # 訓練模型時每個裝置的 batch size\n",
        "    per_device_eval_batch_size=64,   # 驗證模型時每個裝置的 batch size\n",
        "    gradient_accumulation_steps=2,   # 梯度累積的步數\n",
        "    warmup_steps=500,                # learning rate scheduler 的參數\n",
        "    weight_decay=0.01,               # 最佳化演算法 (optimizer) 中的權重衰退率\n",
        "    evaluation_strategy='steps',     # 設定驗證的時機\n",
        "    save_strategy='steps',           # 設定儲存的時機\n",
        "    save_steps=500,                  # 設定多少步驟儲存一次模型\n",
        "    eval_steps=500,                  # 設定多少步驟驗證一次模型\n",
        "    load_best_model_at_end=True,     # 是否在訓練結束後載入最好的模型\n",
        "    metric_for_best_model='eval_loss',      # 設定最好的模型的指標\n",
        "    report_to='tensorboard',         # 是否將訓練結果儲存到 TensorBoard\n",
        "    save_total_limit=10,              # 最多儲存幾個模型\n",
        "    logging_dir='./logs',            # 存放 log 的資料夾\n",
        "    logging_steps=10,\n",
        "    seed=random_seed\n",
        ")\n",
        "\n",
        "trainer = transformers.Trainer(\n",
        "    model=model,                         # 🤗 的模型\n",
        "    args=training_args,                  # Trainer 所需要的引數\n",
        "    train_dataset=train_dataset,         # 訓練集 (注意是 PyTorch Dataset)\n",
        "    eval_dataset=val_dataset,            # 驗證集 (注意是 PyTorch Dataset)，可使 Trainer 在進行訓練時也進行驗證\n",
        "    compute_metrics=compute_metrics      # 自定的評估的指標\n",
        ")\n",
        "\n",
        "# 指定使用 1 個 GPU 進行訓練\n",
        "trainer.args._n_gpu=1\n",
        "\n",
        "# 開始進行模型訓練\n",
        "trainer.train()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 201
        },
        "id": "c_kNcENTJg2U",
        "outputId": "9fa60303-2de8-47ea-9d28-820e930498d3"
      },
      "outputs": [
        {
          "ename": "NameError",
          "evalue": "ignored",
          "output_type": "error",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-1-6dc0be8eaf93>\u001b[0m in \u001b[0;36m<cell line: 3>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;31m# 21. 測試模型\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      2\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 3\u001b[0;31m \u001b[0mtrainer\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpredict\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtest_dataset\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
            "\u001b[0;31mNameError\u001b[0m: name 'trainer' is not defined"
          ]
        }
      ],
      "source": [
        "# 21. 測試模型\n",
        "\n",
        "trainer.predict(test_dataset)"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "gpuType": "T4",
      "provenance": []
    },
    "gpuClass": "standard",
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.8.16"
    },
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "04872ffde4bc41efa6fcc2a215cc9b42": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "06e2b9481fc449e69d8427fc2517d310": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "073924322b8d439ea5b3e983102a8737": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_9d8bba46b8f648d1b923891fecf0de3a",
            "placeholder": "​",
            "style": "IPY_MODEL_7878fb0bdf144acbabafc0bbbacf648e",
            "value": " 483/483 [00:00&lt;00:00, 25.1kB/s]"
          }
        },
        "0c030acc1f554cc88b9ee6e3c9cd0572": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0d77aa49fbc84c41a620a3e9dfc62da5": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "106a2ca9e5f7439aa8083ebd0a0aafe0": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "1070652770ff4180821ee5d73c80adda": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "11033bfef6be4fdf96957dd05d5feed8": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": "hidden",
            "width": null
          }
        },
        "1124d2f81581483bb9d3d50108a5ff9d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "138d2c7a03ce4e1b9e3a97f02c4f4318": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "16f46279e7bc4ae3acafa5902ed1af5a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "17714c7c793b4982bad7de2b8359ccbc": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1906497eab894a70b1b20df3f78aec5d": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1a84529e32ed47a6be353cd48dfea316": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "1a99dbddcdcc42b78e2b17fb218f6419": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "1b384cdabf104251b297542fba384a1f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_3b9b881799344bf39b05e99aabd1bac1",
              "IPY_MODEL_e100ee4dc3934f5fade5795c8747d874",
              "IPY_MODEL_cc9432d8dbda48979241c730d730b1e4"
            ],
            "layout": "IPY_MODEL_a34fadcfc80f48efb669006d81086746"
          }
        },
        "1c5ec57b12ed4177bd3db25243d29131": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "1e9e306861754ae7b600156ed51ecad0": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "20170a5cefba4a36a42e002e3795a693": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "21547c68f72d4c8aa34cbba45115592d": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "22a052aa12274235ac51079156a3503f": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2376f500d77e4e04aecb6a17f50485ea": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": "hidden",
            "width": null
          }
        },
        "2709773a6e06458b97e9b0aa95f8f854": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_56db8c7ca2724d94868ad544d9aa556c",
            "placeholder": "​",
            "style": "IPY_MODEL_9894b722016b4d2a8ed1b8b7e6ad624e",
            "value": " 4.31k/4.31k [00:00&lt;00:00, 208kB/s]"
          }
        },
        "2cd9423342f74bd1b61afbfcfe407790": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1906497eab894a70b1b20df3f78aec5d",
            "placeholder": "​",
            "style": "IPY_MODEL_ce865d95997c4917b89e777aa0e9dc24",
            "value": "Downloading metadata: 100%"
          }
        },
        "2d10eac4e6ef4d0bb486f7dc8ebc45b5": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "2ffc7b9056f24a5d81433bef9bd5d77f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_06e2b9481fc449e69d8427fc2517d310",
            "placeholder": "​",
            "style": "IPY_MODEL_b53b3e8ee90e4e7cad40e65cb5a05e4f",
            "value": "Downloading pytorch_model.bin: 100%"
          }
        },
        "306aa84d2b294b8e9874ec724734e155": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "334e1833625d43b487094848cf70281a": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": "hidden",
            "width": null
          }
        },
        "33af93996db04e94b7c86e5ce025fb42": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "34bd27a3a4954f77ac2a32ec8219eed8": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_4e6440cbc009413694c35f640de6b563",
            "placeholder": "​",
            "style": "IPY_MODEL_99a6ef2f822d4d139eb9f74a05e0350c",
            "value": " 20000/20000 [00:20&lt;00:00, 863.70 examples/s]"
          }
        },
        "3630bb47d9874735a7779d40a16ee7dc": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": "hidden",
            "width": null
          }
        },
        "365f6942322244f8a0e9bd8dd2560041": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_f754d050a21445e79d71698f269b17de",
            "placeholder": "​",
            "style": "IPY_MODEL_e22f9c5120934a3a95bd285950a0b8bd",
            "value": " 24920/25000 [00:05&lt;00:00, 5879.08 examples/s]"
          }
        },
        "3771e073051f42e892675251ef500625": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_7c52c683d76144a2b86d4f9b9477d52e",
            "placeholder": "​",
            "style": "IPY_MODEL_745541e301254cddae85a585b7c2c136",
            "value": " 466k/466k [00:00&lt;00:00, 749kB/s]"
          }
        },
        "382a0f6ddc664e0ca211df8b44e6323c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "390b50f9ef06434fad440b565f743d6b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d6532a4c49bb4e00b01025303272b81e",
            "placeholder": "​",
            "style": "IPY_MODEL_d2c30672cc8842e0be96f57357f20e93",
            "value": " 50000/50000 [00:11&lt;00:00, 9160.77 examples/s]"
          }
        },
        "3a86214abe0547ce9a031beb357ebe21": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "3b02be847e38450fb7c93b1ead1ba198": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_5cb31f3410f5428c84ee4bbaab46ab59",
            "max": 25000,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_9213e3cff09a4f31aac733351ca4ee7d",
            "value": 25000
          }
        },
        "3b9b881799344bf39b05e99aabd1bac1": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_f41d76da8e934e919d1313c0fb0066a7",
            "placeholder": "​",
            "style": "IPY_MODEL_5ca8daa1cbd8410ea42f2f3e84bb0a53",
            "value": "Map: 100%"
          }
        },
        "3bc185aef50b46f19355048202e84476": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_d9245de10f0b483f8298d2f0808244ff",
              "IPY_MODEL_8b2baaf2ee2e499cb8975adc6c55cf8b",
              "IPY_MODEL_3771e073051f42e892675251ef500625"
            ],
            "layout": "IPY_MODEL_a722404ff070497cb78eb6c7304f9437"
          }
        },
        "3e1ad418853d4b5a903b5499982c6d75": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "3f54dde4baf54ac58d109180a169fc77": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e62dab3f5c8d4ef48807ce2f9cb6a036",
            "max": 84125825,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_e350175227a34998a42d2676ea57dc99",
            "value": 84125825
          }
        },
        "413adfd3ba1349ad9dc10012a9da433e": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_f273cfdcbb684b91961c343ecd0df1af",
            "placeholder": "​",
            "style": "IPY_MODEL_fcecd73666ee42e2b45d3cb7efc0661b",
            "value": "Downloading (…)okenizer_config.json: 100%"
          }
        },
        "4188a2d6ba9d42e5874df6b6f78f357c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_f3eb71e00ee84b089161f5e7b535ebc1",
            "placeholder": "​",
            "style": "IPY_MODEL_306aa84d2b294b8e9874ec724734e155",
            "value": "Map: 100%"
          }
        },
        "420c13174a1f416ba6418c2ec4104dd6": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_9f402fa036cc4149b421bca2cb3138cc",
            "max": 231508,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_6f522e847a8a4f558225f3bdc68f26d3",
            "value": 231508
          }
        },
        "449e9ddf42c74c5c8499736395ea7404": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_0c030acc1f554cc88b9ee6e3c9cd0572",
            "placeholder": "​",
            "style": "IPY_MODEL_724b1987359b4037ad9c128575ab26e2",
            "value": "Downloading data: 100%"
          }
        },
        "4830c2a45d0947d5ad10bd5898865cdc": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_635854ae59a048ac9e3d39d9c4c2fb29",
              "IPY_MODEL_ac2a1b1a2a3945b9a0e17e25caf882b8",
              "IPY_MODEL_e729af8b4de34754b93635980e1b8cee"
            ],
            "layout": "IPY_MODEL_11033bfef6be4fdf96957dd05d5feed8"
          }
        },
        "4ab15edaf3494fefbb476aa936f3fb0e": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4cb2669c09b342d2817a6819e1c2efef": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4e6440cbc009413694c35f640de6b563": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4f5a69ae0c6d4072ba036791a3600313": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_69478e1b7c5949308967dd5d27798bda",
              "IPY_MODEL_fe1d17935ef9439aa51c5eb640dfa3c2",
              "IPY_MODEL_6d0f42aae0484b3b8788e46571f75145"
            ],
            "layout": "IPY_MODEL_f93b1139aec9492bba54c90f594d9415"
          }
        },
        "4fa27a09549d4491ab5615fa9a2e63bf": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "50aa561c16c94a99872867bebdb1970e": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "53dd301e7bd2434283c2e74fdd7b5816": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "5502ee2fbcfb40cbbb73377e9d8411d3": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "56db8c7ca2724d94868ad544d9aa556c": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5aa00f04451348fa8bd23aba4ec60edb": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_dc003d4121494336968b53983ea378dc",
              "IPY_MODEL_f9169f6b95204aecba29c4efb2491992",
              "IPY_MODEL_073924322b8d439ea5b3e983102a8737"
            ],
            "layout": "IPY_MODEL_b1de218d44d54e81aa672e95ffcb3b5f"
          }
        },
        "5b8be14016fd4c18826f39054aa0959d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "5ca8daa1cbd8410ea42f2f3e84bb0a53": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "5cb31f3410f5428c84ee4bbaab46ab59": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5d10a50aac704c6b909f2d2c6c7a5e7e": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1e9e306861754ae7b600156ed51ecad0",
            "placeholder": "​",
            "style": "IPY_MODEL_c114a9f7332644a69667f84545e9f12d",
            "value": "Downloading (…)solve/main/vocab.txt: 100%"
          }
        },
        "5e214fd696ea4124b314a0529e3868aa": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "5e41ecc78a99447eadcee0ef8d43a4ae": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "60b218637f4048d7973480d0d66bd503": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "635854ae59a048ac9e3d39d9c4c2fb29": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_cf6ce47a025d4b59be1e795051796f78",
            "placeholder": "​",
            "style": "IPY_MODEL_bf9b09c4d2014c30af85a5bf3481f4ce",
            "value": "Generating test split: 100%"
          }
        },
        "647307442cd747239d7b3afaa82504e3": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_ba75df6ea53a4bbcbec504f92748950b",
              "IPY_MODEL_899e566bed634ded81ede51ed48ccc76",
              "IPY_MODEL_390b50f9ef06434fad440b565f743d6b"
            ],
            "layout": "IPY_MODEL_3630bb47d9874735a7779d40a16ee7dc"
          }
        },
        "6479d088f4bb48ce81733afaad75c230": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "692be76bb4214638a2a5b760d956e59c": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "69478e1b7c5949308967dd5d27798bda": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_50aa561c16c94a99872867bebdb1970e",
            "placeholder": "​",
            "style": "IPY_MODEL_1070652770ff4180821ee5d73c80adda",
            "value": "Map: 100%"
          }
        },
        "699734b4fe4c4e2aac328ee1065e7f92": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6d0f42aae0484b3b8788e46571f75145": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_9e88eaf822394ac1aa498c857595f4e6",
            "placeholder": "​",
            "style": "IPY_MODEL_0d77aa49fbc84c41a620a3e9dfc62da5",
            "value": " 25000/25000 [00:22&lt;00:00, 1258.54 examples/s]"
          }
        },
        "6d99c91aa598467da8c43a9c2e4f0099": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_f353ca80976a4d5590664e7cae83efbf",
              "IPY_MODEL_d08eeb45da3b4c6484234058ae3346f5",
              "IPY_MODEL_2709773a6e06458b97e9b0aa95f8f854"
            ],
            "layout": "IPY_MODEL_22a052aa12274235ac51079156a3503f"
          }
        },
        "6f522e847a8a4f558225f3bdc68f26d3": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "724b1987359b4037ad9c128575ab26e2": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "745541e301254cddae85a585b7c2c136": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "780bf8ccfa5546f2898826bea42f6a8b": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7878fb0bdf144acbabafc0bbbacf648e": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "79630f367d234755a2fa7c33f6a06100": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "7c52c683d76144a2b86d4f9b9477d52e": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7e17bc4f5ab940d28ad0bcf04cac4d5c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_449e9ddf42c74c5c8499736395ea7404",
              "IPY_MODEL_3f54dde4baf54ac58d109180a169fc77",
              "IPY_MODEL_8c4c9735771a4c4e9747b0b0cc92bd8d"
            ],
            "layout": "IPY_MODEL_aa6b35df259948eebf03725e060445b8"
          }
        },
        "7ecd01555e2041bf856d9d0bed7b9845": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_9e0c701c2e5f474194fce65ad4bdca7c",
            "max": 267967963,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_3e1ad418853d4b5a903b5499982c6d75",
            "value": 267967963
          }
        },
        "836e7339a12e41a6b241729f10818768": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_4cb2669c09b342d2817a6819e1c2efef",
            "placeholder": "​",
            "style": "IPY_MODEL_2d10eac4e6ef4d0bb486f7dc8ebc45b5",
            "value": "Downloading readme: 100%"
          }
        },
        "87e9d3051b9d4126a89bff8811235291": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a6c36498cdee43748930f71cfd164de2",
            "max": 7590,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_382a0f6ddc664e0ca211df8b44e6323c",
            "value": 7590
          }
        },
        "894f10fb03784042b5d0f00f0cadcae4": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "899e566bed634ded81ede51ed48ccc76": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_fa625601c86941d598cb9cc0bd8293f5",
            "max": 50000,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_f8d2c921b5c846a2b2f3647f155a8f45",
            "value": 50000
          }
        },
        "8b2baaf2ee2e499cb8975adc6c55cf8b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_925f719f20a44fb1ab7c107c2f0f0fde",
            "max": 466062,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_f5315ac4a43f4dedb31e39c0856bf651",
            "value": 466062
          }
        },
        "8c4c9735771a4c4e9747b0b0cc92bd8d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_138d2c7a03ce4e1b9e3a97f02c4f4318",
            "placeholder": "​",
            "style": "IPY_MODEL_1c5ec57b12ed4177bd3db25243d29131",
            "value": " 84.1M/84.1M [00:10&lt;00:00, 14.0MB/s]"
          }
        },
        "8d3ff57aee164942bb9489f7ba258338": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_692be76bb4214638a2a5b760d956e59c",
            "placeholder": "​",
            "style": "IPY_MODEL_20170a5cefba4a36a42e002e3795a693",
            "value": " 28.0/28.0 [00:00&lt;00:00, 1.43kB/s]"
          }
        },
        "8daa2263e04d4d70bdcabb09a5e37768": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "9213e3cff09a4f31aac733351ca4ee7d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "925f719f20a44fb1ab7c107c2f0f0fde": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9894b722016b4d2a8ed1b8b7e6ad624e": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "994f543d720d4b09bd06b835fcf62878": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_b39befec3d2b44e6a52f73bff8e42afb",
            "max": 2166,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_1a99dbddcdcc42b78e2b17fb218f6419",
            "value": 2166
          }
        },
        "99a6ef2f822d4d139eb9f74a05e0350c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "9cd84ccfd43b4909923eca0f45988a85": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_21547c68f72d4c8aa34cbba45115592d",
            "placeholder": "​",
            "style": "IPY_MODEL_1124d2f81581483bb9d3d50108a5ff9d",
            "value": " 7.59k/7.59k [00:00&lt;00:00, 401kB/s]"
          }
        },
        "9cf95f0b331d400c8f91dff614d7511f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_2ffc7b9056f24a5d81433bef9bd5d77f",
              "IPY_MODEL_7ecd01555e2041bf856d9d0bed7b9845",
              "IPY_MODEL_c534b33e525c4e58b35dfc59b1cd03ac"
            ],
            "layout": "IPY_MODEL_780bf8ccfa5546f2898826bea42f6a8b"
          }
        },
        "9d8bba46b8f648d1b923891fecf0de3a": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9e0c701c2e5f474194fce65ad4bdca7c": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9e88eaf822394ac1aa498c857595f4e6": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9f402fa036cc4149b421bca2cb3138cc": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a1b08ad5fff64b51ae3ddb2a853a2375": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a34fadcfc80f48efb669006d81086746": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": "hidden",
            "width": null
          }
        },
        "a651bd76b3424fa1875f115f657a710b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e6a8b9fdfe5a43c9ae6ec21ad679e9c9",
            "placeholder": "​",
            "style": "IPY_MODEL_53dd301e7bd2434283c2e74fdd7b5816",
            "value": "Generating train split: 100%"
          }
        },
        "a6c36498cdee43748930f71cfd164de2": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a722404ff070497cb78eb6c7304f9437": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "aa6b35df259948eebf03725e060445b8": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ac2a1b1a2a3945b9a0e17e25caf882b8": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_699734b4fe4c4e2aac328ee1065e7f92",
            "max": 25000,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_c7b61791eede43c3805db8b6b7c8d876",
            "value": 25000
          }
        },
        "ac768998e26c4959918b8c31eaffd05a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_894f10fb03784042b5d0f00f0cadcae4",
            "max": 20000,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_16f46279e7bc4ae3acafa5902ed1af5a",
            "value": 20000
          }
        },
        "ac9116019bc04b8fa53de6cf6235228f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_836e7339a12e41a6b241729f10818768",
              "IPY_MODEL_87e9d3051b9d4126a89bff8811235291",
              "IPY_MODEL_9cd84ccfd43b4909923eca0f45988a85"
            ],
            "layout": "IPY_MODEL_e3d1092747e440069c1ce1551045a0ea"
          }
        },
        "b1de218d44d54e81aa672e95ffcb3b5f": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b39befec3d2b44e6a52f73bff8e42afb": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b53b3e8ee90e4e7cad40e65cb5a05e4f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "b6ac5fd37186433b9d85151f747b87e8": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a1b08ad5fff64b51ae3ddb2a853a2375",
            "max": 28,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_c942c97458f84b90b86023b2f66f8fa7",
            "value": 28
          }
        },
        "b8ef6770827e4ff3bb2ea56c1077213a": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ba75df6ea53a4bbcbec504f92748950b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c18e8047c87d43f7bbbe19db0325a7ec",
            "placeholder": "​",
            "style": "IPY_MODEL_1a84529e32ed47a6be353cd48dfea316",
            "value": "Generating unsupervised split: 100%"
          }
        },
        "be996fecb71448d790dff3da670951ef": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_5502ee2fbcfb40cbbb73377e9d8411d3",
            "placeholder": "​",
            "style": "IPY_MODEL_5e214fd696ea4124b314a0529e3868aa",
            "value": " 232k/232k [00:00&lt;00:00, 1.10MB/s]"
          }
        },
        "bf9b09c4d2014c30af85a5bf3481f4ce": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "c114a9f7332644a69667f84545e9f12d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "c18e8047c87d43f7bbbe19db0325a7ec": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c1cab38618ad4dadb9ca75e46b988e0e": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c534b33e525c4e58b35dfc59b1cd03ac": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_4ab15edaf3494fefbb476aa936f3fb0e",
            "placeholder": "​",
            "style": "IPY_MODEL_fe9453c4e06c4d92b90c930b9756defd",
            "value": " 268M/268M [00:02&lt;00:00, 142MB/s]"
          }
        },
        "c7b61791eede43c3805db8b6b7c8d876": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "c942c97458f84b90b86023b2f66f8fa7": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "cac0a0b51f174f95a8091df28a0deb16": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_a651bd76b3424fa1875f115f657a710b",
              "IPY_MODEL_3b02be847e38450fb7c93b1ead1ba198",
              "IPY_MODEL_365f6942322244f8a0e9bd8dd2560041"
            ],
            "layout": "IPY_MODEL_2376f500d77e4e04aecb6a17f50485ea"
          }
        },
        "cc9432d8dbda48979241c730d730b1e4": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6479d088f4bb48ce81733afaad75c230",
            "placeholder": "​",
            "style": "IPY_MODEL_e177ccefc8be46179b445a4103bdeb90",
            "value": " 5000/5000 [00:05&lt;00:00, 1048.22 examples/s]"
          }
        },
        "ce865d95997c4917b89e777aa0e9dc24": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "cf6ce47a025d4b59be1e795051796f78": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d08eeb45da3b4c6484234058ae3346f5": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_4fa27a09549d4491ab5615fa9a2e63bf",
            "max": 4314,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_8daa2263e04d4d70bdcabb09a5e37768",
            "value": 4314
          }
        },
        "d2c30672cc8842e0be96f57357f20e93": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "d49ac22a46da4452ba652bc21c5b680b": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d6532a4c49bb4e00b01025303272b81e": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d9245de10f0b483f8298d2f0808244ff": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e809d68f854c4411929a5c58bff92e25",
            "placeholder": "​",
            "style": "IPY_MODEL_79630f367d234755a2fa7c33f6a06100",
            "value": "Downloading (…)/main/tokenizer.json: 100%"
          }
        },
        "dc003d4121494336968b53983ea378dc": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e02ec078e4ed4fb9bcb16678e2f5d65f",
            "placeholder": "​",
            "style": "IPY_MODEL_106a2ca9e5f7439aa8083ebd0a0aafe0",
            "value": "Downloading (…)lve/main/config.json: 100%"
          }
        },
        "df03d99362c7417a828b95db79cfec4a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "e02ec078e4ed4fb9bcb16678e2f5d65f": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e095c0d0e2b944598b45e9f46c4ffdd1": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "e100ee4dc3934f5fade5795c8747d874": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_04872ffde4bc41efa6fcc2a215cc9b42",
            "max": 5000,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_e095c0d0e2b944598b45e9f46c4ffdd1",
            "value": 5000
          }
        },
        "e177ccefc8be46179b445a4103bdeb90": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "e22f9c5120934a3a95bd285950a0b8bd": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "e350175227a34998a42d2676ea57dc99": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "e3d1092747e440069c1ce1551045a0ea": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e62dab3f5c8d4ef48807ce2f9cb6a036": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e6a8b9fdfe5a43c9ae6ec21ad679e9c9": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e6b11f8be8ba4a288cc60af0625e72a5": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "e729af8b4de34754b93635980e1b8cee": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_ee9cc4e598a04c86b038519e71cba43e",
            "placeholder": "​",
            "style": "IPY_MODEL_e6b11f8be8ba4a288cc60af0625e72a5",
            "value": " 24964/25000 [00:03&lt;00:00, 9419.66 examples/s]"
          }
        },
        "e75a2f56cc9043edbfbbbef1fd9f6689": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_4188a2d6ba9d42e5874df6b6f78f357c",
              "IPY_MODEL_ac768998e26c4959918b8c31eaffd05a",
              "IPY_MODEL_34bd27a3a4954f77ac2a32ec8219eed8"
            ],
            "layout": "IPY_MODEL_334e1833625d43b487094848cf70281a"
          }
        },
        "e809d68f854c4411929a5c58bff92e25": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ea09d00e15b9450d9a37169c83030a55": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_5d10a50aac704c6b909f2d2c6c7a5e7e",
              "IPY_MODEL_420c13174a1f416ba6418c2ec4104dd6",
              "IPY_MODEL_be996fecb71448d790dff3da670951ef"
            ],
            "layout": "IPY_MODEL_ed42392e07d14de4a9bf4e0f016dfb0c"
          }
        },
        "ed42392e07d14de4a9bf4e0f016dfb0c": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ee9cc4e598a04c86b038519e71cba43e": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f273cfdcbb684b91961c343ecd0df1af": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f353ca80976a4d5590664e7cae83efbf": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_33af93996db04e94b7c86e5ce025fb42",
            "placeholder": "​",
            "style": "IPY_MODEL_5b8be14016fd4c18826f39054aa0959d",
            "value": "Downloading builder script: 100%"
          }
        },
        "f3eb71e00ee84b089161f5e7b535ebc1": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f41d76da8e934e919d1313c0fb0066a7": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f5315ac4a43f4dedb31e39c0856bf651": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "f54452fe5af64a1785a106b0cf061aa7": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_17714c7c793b4982bad7de2b8359ccbc",
            "placeholder": "​",
            "style": "IPY_MODEL_3a86214abe0547ce9a031beb357ebe21",
            "value": " 2.17k/2.17k [00:00&lt;00:00, 143kB/s]"
          }
        },
        "f754d050a21445e79d71698f269b17de": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f8d2c921b5c846a2b2f3647f155a8f45": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "f9169f6b95204aecba29c4efb2491992": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c1cab38618ad4dadb9ca75e46b988e0e",
            "max": 483,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_df03d99362c7417a828b95db79cfec4a",
            "value": 483
          }
        },
        "f93b1139aec9492bba54c90f594d9415": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": "hidden",
            "width": null
          }
        },
        "fa625601c86941d598cb9cc0bd8293f5": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "fcecd73666ee42e2b45d3cb7efc0661b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "fe17eb440b6543658d1f802853d71af2": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_2cd9423342f74bd1b61afbfcfe407790",
              "IPY_MODEL_994f543d720d4b09bd06b835fcf62878",
              "IPY_MODEL_f54452fe5af64a1785a106b0cf061aa7"
            ],
            "layout": "IPY_MODEL_b8ef6770827e4ff3bb2ea56c1077213a"
          }
        },
        "fe1d17935ef9439aa51c5eb640dfa3c2": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_5e41ecc78a99447eadcee0ef8d43a4ae",
            "max": 25000,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_60b218637f4048d7973480d0d66bd503",
            "value": 25000
          }
        },
        "fe9453c4e06c4d92b90c930b9756defd": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "ff8057af42304d299a67d96409a10bf4": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_413adfd3ba1349ad9dc10012a9da433e",
              "IPY_MODEL_b6ac5fd37186433b9d85151f747b87e8",
              "IPY_MODEL_8d3ff57aee164942bb9489f7ba258338"
            ],
            "layout": "IPY_MODEL_d49ac22a46da4452ba652bc21c5b680b"
          }
        }
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}