Pandas1.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "anaconda-cloud": {},
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.7"
    },
    "colab": {
      "name": "Pandas1.ipynb",
      "provenance": [],
      "include_colab_link": true
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/nekrut/BMMB554/blob/master/Pandas1.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xzoyapwo5n7k"
      },
      "source": [
        "# Introduction to Pandas 1 | Basics\n",
        "\n",
        "-------\n",
        "\n",
        "\n",
        "> This is a reworked version of tutorial by [Justin Bois](http://justinbois.github.io/bootcamp/2020/index.html). It contains modified training datasets and adopts content to Colab environment. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jIdrp1IK5n7l"
      },
      "source": [
        "import numpy as np\n",
        "\n",
        "# Pandas, conventionally imported as pd\n",
        "import pandas as pd"
      ],
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VVLFmrUh5n7m"
      },
      "source": [
        "Throughout your research career, you will undoubtedly need to handle data, possibly lots of data. The data comes in lots of formats, and you will spend much of your time **wrangling** the data to get it into a usable form.\n",
        "\n",
        "Pandas is the primary tool in the Python ecosystem for handling data. Its primary object, the `DataFrame` is extremely useful in wrangling data. We will explore some of that functionality here, and will put it to use in the next lesson."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dfjlZ0cY5n7m"
      },
      "source": [
        "## The data set\n",
        "\n",
        "The dataset we will be using is a list of all SARS-CoV-2 datasets in [Sequence Read Archive](https://www.ncbi.nlm.nih.gov/sra) as of January 20, 2021.  \n",
        "\n",
        "It is obtained by going to https://www.ncbi.nlm.nih.gov/sra and performing a query with the following search terms: `txid2697049[Organism:noexp]`.\n",
        "\n",
        "Results are downloaded using `Send to:` menu selecting `File` and then `RunInfo`. Let's get these results into this notebook:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_0bzdrCS0h7L",
        "outputId": "d9087ea1-bb3c-4e70-c6b0-73a9514c0bec"
      },
      "source": [
        "!wget https://github.com/nekrut/BMMB554/raw/master/2021/data/sra_ncov_bmmb554.csv.gz"
      ],
      "execution_count": 27,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "--2021-01-26 13:48:12--  https://github.com/nekrut/BMMB554/raw/master/2021/data/sra_ncov_bmmb554.csv.gz\n",
            "Resolving github.com (github.com)... 192.30.255.113\n",
            "Connecting to github.com (github.com)|192.30.255.113|:443... connected.\n",
            "HTTP request sent, awaiting response... 302 Found\n",
            "Location: https://raw.githubusercontent.com/nekrut/BMMB554/master/2021/data/sra_ncov_bmmb554.csv.gz [following]\n",
            "--2021-01-26 13:48:12--  https://raw.githubusercontent.com/nekrut/BMMB554/master/2021/data/sra_ncov_bmmb554.csv.gz\n",
            "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.0.133, 151.101.64.133, 151.101.128.133, ...\n",
            "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.0.133|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 1831491 (1.7M) [application/octet-stream]\n",
            "Saving to: ‘sra_ncov_bmmb554.csv.gz.1’\n",
            "\n",
            "\r          sra_ncov_   0%[                    ]       0  --.-KB/s               \rsra_ncov_bmmb554.cs 100%[===================>]   1.75M  --.-KB/s    in 0.06s   \n",
            "\n",
            "2021-01-26 13:48:12 (29.8 MB/s) - ‘sra_ncov_bmmb554.csv.gz.1’ saved [1831491/1831491]\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2Qo40hYu5n7m",
        "outputId": "237acf34-1ef5-4d5b-dc85-6412f137be90"
      },
      "source": [
        "!gunzip -c  sra_ncov_bmmb554.csv.gz | head"
      ],
      "execution_count": 28,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Run,ReleaseDate,size_MB,Experiment,LibraryStrategy,LibrarySelection,LibrarySource,LibraryLayout,Platform,Model,SRAStudy,BioProject\n",
            "ERR5063394,2021-01-18 12:14:33,0,ERX4869505,Targeted-Capture,RANDOM,TRANSCRIPTOMIC,SINGLE,ILLUMINA,unspecified,ERP121228,PRJEB37886\n",
            "ERR5063392,2021-01-18 12:14:33,0,ERX4869504,AMPLICON,PCR,VIRAL RNA,SINGLE,ILLUMINA,Illumina MiSeq,ERP121228,PRJEB37886\n",
            "ERR5063395,2021-01-18 12:14:33,0,ERX4869506,Targeted-Capture,RANDOM,TRANSCRIPTOMIC,SINGLE,ILLUMINA,unspecified,ERP121228,PRJEB37886\n",
            "ERR5063397,2021-01-18 12:14:33,0,ERX4869510,AMPLICON,PCR,VIRAL RNA,SINGLE,ILLUMINA,NextSeq 550,ERP121228,PRJEB37886\n",
            "ERR5063396,2021-01-18 12:14:33,0,ERX4869507,AMPLICON,PCR,VIRAL RNA,SINGLE,ILLUMINA,NextSeq 550,ERP121228,PRJEB37886\n",
            "ERR5063399,2021-01-18 12:14:33,0,ERX4869508,Targeted-Capture,RANDOM,TRANSCRIPTOMIC,SINGLE,ILLUMINA,unspecified,ERP121228,PRJEB37886\n",
            "ERR5063400,2021-01-18 12:14:33,0,ERX4869509,AMPLICON,PCR,VIRAL RNA,SINGLE,ILLUMINA,NextSeq 550,ERP121228,PRJEB37886\n",
            "ERR5063398,2021-01-18 12:14:33,0,ERX4869511,Targeted-Capture,RANDOM,TRANSCRIPTOMIC,SINGLE,ILLUMINA,unspecified,ERP121228,PRJEB37886\n",
            "ERR5063401,2021-01-18 12:14:33,0,ERX4869512,Targeted-Capture,RANDOM,TRANSCRIPTOMIC,SINGLE,ILLUMINA,unspecified,ERP121228,PRJEB37886\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Vmp0aGqC5n7n"
      },
      "source": [
        "The first line contains the **headers** for each column. The data follow. Given the file I/O skills you recently learned, you could write some functions to parse this file and extract the data you want. You can imagine that this might be kind of painful. However, if the file format is nice and clean, like we more or less have here, we can use pre-built tools. Pandas has a very powerful function, `pd.read_csv()` that can read in a CSV file and store the contents in a convenient data structure called a **data frame**. In Pandas, the data type for a data frame is `DataFrame`, and we will use \"data frame\" and \"`DataFrame`\" interchangeably."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VVLKebpt5n7n"
      },
      "source": [
        "## Reading in data\n",
        "\n",
        "Let's first look at the doc string of `pd.read_csv()`."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JNxuzB895n7n"
      },
      "source": [
        "pd.read_csv?"
      ],
      "execution_count": 29,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EtN9o7wi5n7n"
      },
      "source": [
        "Holy cow! There are so many options we can specify for reading in a CSV file. You will likely find reasons to use many of these throughout your research. For now, however, we do not need most of them. So, let's load in the data set. Note that despite the fact that the datset is comprssed with `gzip` we do not need to do anything additional - pandas magically understands and uncompresses the data while loading it into the dataframe. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "eZl3SqVi5n7o",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "4d4155fa-3b60-40d4-a37a-bf81cb621765"
      },
      "source": [
        "df = pd.read_csv('sra_ncov_bmmb554.csv.gz')\n",
        "\n",
        "# Check the type\n",
        "type(df)"
      ],
      "execution_count": 30,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "pandas.core.frame.DataFrame"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 30
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CgiFpXXN5n7o"
      },
      "source": [
        "We now have the data stored in a data frame. We can look at it in the Jupyter notebook, since Jupyter will display it in a well-organized, pretty way. Note that because our dataframe is big, we only display first five rows using `head()` function:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qPLQR_Al5n7o",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 289
        },
        "outputId": "ad1c4f5e-737a-4273-8611-eecba19a4983"
      },
      "source": [
        "df.head()"
      ],
      "execution_count": 32,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Run</th>\n",
              "      <th>ReleaseDate</th>\n",
              "      <th>size_MB</th>\n",
              "      <th>Experiment</th>\n",
              "      <th>LibraryStrategy</th>\n",
              "      <th>LibrarySelection</th>\n",
              "      <th>LibrarySource</th>\n",
              "      <th>LibraryLayout</th>\n",
              "      <th>Platform</th>\n",
              "      <th>Model</th>\n",
              "      <th>SRAStudy</th>\n",
              "      <th>BioProject</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>ERR5063394</td>\n",
              "      <td>2021-01-18 12:14:33</td>\n",
              "      <td>0</td>\n",
              "      <td>ERX4869505</td>\n",
              "      <td>Targeted-Capture</td>\n",
              "      <td>RANDOM</td>\n",
              "      <td>TRANSCRIPTOMIC</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>ILLUMINA</td>\n",
              "      <td>unspecified</td>\n",
              "      <td>ERP121228</td>\n",
              "      <td>PRJEB37886</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>ERR5063392</td>\n",
              "      <td>2021-01-18 12:14:33</td>\n",
              "      <td>0</td>\n",
              "      <td>ERX4869504</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>ILLUMINA</td>\n",
              "      <td>Illumina MiSeq</td>\n",
              "      <td>ERP121228</td>\n",
              "      <td>PRJEB37886</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>ERR5063395</td>\n",
              "      <td>2021-01-18 12:14:33</td>\n",
              "      <td>0</td>\n",
              "      <td>ERX4869506</td>\n",
              "      <td>Targeted-Capture</td>\n",
              "      <td>RANDOM</td>\n",
              "      <td>TRANSCRIPTOMIC</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>ILLUMINA</td>\n",
              "      <td>unspecified</td>\n",
              "      <td>ERP121228</td>\n",
              "      <td>PRJEB37886</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>ERR5063397</td>\n",
              "      <td>2021-01-18 12:14:33</td>\n",
              "      <td>0</td>\n",
              "      <td>ERX4869510</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>ILLUMINA</td>\n",
              "      <td>NextSeq 550</td>\n",
              "      <td>ERP121228</td>\n",
              "      <td>PRJEB37886</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>ERR5063396</td>\n",
              "      <td>2021-01-18 12:14:33</td>\n",
              "      <td>0</td>\n",
              "      <td>ERX4869507</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>ILLUMINA</td>\n",
              "      <td>NextSeq 550</td>\n",
              "      <td>ERP121228</td>\n",
              "      <td>PRJEB37886</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "          Run          ReleaseDate  ...   SRAStudy  BioProject\n",
              "0  ERR5063394  2021-01-18 12:14:33  ...  ERP121228  PRJEB37886\n",
              "1  ERR5063392  2021-01-18 12:14:33  ...  ERP121228  PRJEB37886\n",
              "2  ERR5063395  2021-01-18 12:14:33  ...  ERP121228  PRJEB37886\n",
              "3  ERR5063397  2021-01-18 12:14:33  ...  ERP121228  PRJEB37886\n",
              "4  ERR5063396  2021-01-18 12:14:33  ...  ERP121228  PRJEB37886\n",
              "\n",
              "[5 rows x 12 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 32
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Qa1CtdAd5n7p"
      },
      "source": [
        "## Indexing data frames\n",
        "\n",
        "The data frame is a convenient data structure for many reasons that will become clear as we start exploring. Let's start by looking at how data frames are indexed. Let's try to look at the first row."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qiAaOBcW5n7p",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 561
        },
        "outputId": "e4747870-2730-41b7-eafc-f954715a006e"
      },
      "source": [
        "df[0]"
      ],
      "execution_count": 33,
      "outputs": [
        {
          "output_type": "error",
          "ename": "KeyError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mKeyError\u001b[0m                                  Traceback (most recent call last)",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/pandas/core/indexes/base.py\u001b[0m in \u001b[0;36mget_loc\u001b[0;34m(self, key, method, tolerance)\u001b[0m\n\u001b[1;32m   2897\u001b[0m             \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2898\u001b[0;31m                 \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_engine\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_loc\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcasted_key\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   2899\u001b[0m             \u001b[0;32mexcept\u001b[0m \u001b[0mKeyError\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0merr\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32mpandas/_libs/index.pyx\u001b[0m in \u001b[0;36mpandas._libs.index.IndexEngine.get_loc\u001b[0;34m()\u001b[0m\n",
            "\u001b[0;32mpandas/_libs/index.pyx\u001b[0m in \u001b[0;36mpandas._libs.index.IndexEngine.get_loc\u001b[0;34m()\u001b[0m\n",
            "\u001b[0;32mpandas/_libs/hashtable_class_helper.pxi\u001b[0m in \u001b[0;36mpandas._libs.hashtable.PyObjectHashTable.get_item\u001b[0;34m()\u001b[0m\n",
            "\u001b[0;32mpandas/_libs/hashtable_class_helper.pxi\u001b[0m in \u001b[0;36mpandas._libs.hashtable.PyObjectHashTable.get_item\u001b[0;34m()\u001b[0m\n",
            "\u001b[0;31mKeyError\u001b[0m: 0",
            "\nThe above exception was the direct cause of the following exception:\n",
            "\u001b[0;31mKeyError\u001b[0m                                  Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-33-ad11118bc8f3>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mdf\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/pandas/core/frame.py\u001b[0m in \u001b[0;36m__getitem__\u001b[0;34m(self, key)\u001b[0m\n\u001b[1;32m   2904\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcolumns\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnlevels\u001b[0m \u001b[0;34m>\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2905\u001b[0m                 \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_getitem_multilevel\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2906\u001b[0;31m             \u001b[0mindexer\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcolumns\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_loc\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   2907\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0mis_integer\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mindexer\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2908\u001b[0m                 \u001b[0mindexer\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mindexer\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/pandas/core/indexes/base.py\u001b[0m in \u001b[0;36mget_loc\u001b[0;34m(self, key, method, tolerance)\u001b[0m\n\u001b[1;32m   2898\u001b[0m                 \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_engine\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_loc\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcasted_key\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2899\u001b[0m             \u001b[0;32mexcept\u001b[0m \u001b[0mKeyError\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0merr\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2900\u001b[0;31m                 \u001b[0;32mraise\u001b[0m \u001b[0mKeyError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0merr\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   2901\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2902\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mtolerance\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mKeyError\u001b[0m: 0"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3jiSZLBW5n7p"
      },
      "source": [
        "Yikes! Lots of errors. The problem is that we tried to index numerically by row. **We index DataFrames, by columns.** And there is no column that has the name `0` in this data frame, though there could be. Instead, a might want to look at the column with the percentage of correct face matching tasks."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iIvadvCT5n7p",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "78212de0-cbe4-43f9-edbb-8df25705b857"
      },
      "source": [
        "df['Run'].head()"
      ],
      "execution_count": 34,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0    ERR5063394\n",
              "1    ERR5063392\n",
              "2    ERR5063395\n",
              "3    ERR5063397\n",
              "4    ERR5063396\n",
              "Name: Run, dtype: object"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 34
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WbXYbNbH5n7p"
      },
      "source": [
        "This gave us the numbers we were after. Notice that when it was printed, the index of the rows came along with it. If we wanted to pull out a single percentage correct, say corresponding to index `4`, we can do that."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SUICG2015n7q",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "1f75c912-ba4a-4b82-d813-a1686c984f4e"
      },
      "source": [
        "df['Run'][4]"
      ],
      "execution_count": 35,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'ERR5063396'"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 35
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UOFDREGY5n7q"
      },
      "source": [
        "However, this is **not** the preferred way to do this. It is better to use `.loc`. This give the location in the data frame we want."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "e4BskRPu5n7q",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "4e5a5b8b-b9bc-4611-8d7f-dfc47802a708"
      },
      "source": [
        "df.loc[4, 'Run']"
      ],
      "execution_count": 36,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'ERR5063396'"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 36
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YJcwBZ775n7q"
      },
      "source": [
        "It is also important to note that **row indices need not be integers**. And you should not count on them being integers. In practice you will almost never use row indices, but rather use **Boolean indexing**."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "H4fF9_no5n7q"
      },
      "source": [
        "## Boolean indexing of data frames\n",
        "\n",
        "Let's say I wanted to pull out accession numbers of runs produced by Pacfic Biosciences machines (in this table such datasets are labeled as `PACBIO_SMRT`. I can use Boolean indexing to specify the row. Specifically, I want the row for which `df['Platform'] == 'PACBIO_SMRT'`. You can essentially plop this syntax directly when using `.loc`."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "oLmabpsE5n7q",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "7789b105-c239-482f-ab7b-ca2593822f20"
      },
      "source": [
        "df.loc[df['Platform'] == 'PACBIO_SMRT', 'Run']"
      ],
      "execution_count": 37,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "49441     SRR13144531\n",
              "49442     SRR13144533\n",
              "49443     SRR13144530\n",
              "49444     SRR13144529\n",
              "49445     SRR13144528\n",
              "49446     SRR13144534\n",
              "49447     SRR13144527\n",
              "49448     SRR13144526\n",
              "49449     SRR13144525\n",
              "49450     SRR13144524\n",
              "49451     SRR13144523\n",
              "49452     SRR13144532\n",
              "173270    SRR12038589\n",
              "173271    SRR12038590\n",
              "Name: Run, dtype: object"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 37
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ADV84bwO5n7r"
      },
      "source": [
        "If I want to pull the whole record for that participant, I can use `:` for the column index."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "g2ASIc1x5n7r",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 741
        },
        "outputId": "faf84a63-1d3d-49cc-e646-57ed09014785"
      },
      "source": [
        "df.loc[df['Platform'] == 'PACBIO_SMRT', :]"
      ],
      "execution_count": 38,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Run</th>\n",
              "      <th>ReleaseDate</th>\n",
              "      <th>size_MB</th>\n",
              "      <th>Experiment</th>\n",
              "      <th>LibraryStrategy</th>\n",
              "      <th>LibrarySelection</th>\n",
              "      <th>LibrarySource</th>\n",
              "      <th>LibraryLayout</th>\n",
              "      <th>Platform</th>\n",
              "      <th>Model</th>\n",
              "      <th>SRAStudy</th>\n",
              "      <th>BioProject</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>49441</th>\n",
              "      <td>SRR13144531</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>1137</td>\n",
              "      <td>SRX9584893</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49442</th>\n",
              "      <td>SRR13144533</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>1999</td>\n",
              "      <td>SRX9584891</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49443</th>\n",
              "      <td>SRR13144530</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>81</td>\n",
              "      <td>SRX9584894</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49444</th>\n",
              "      <td>SRR13144529</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>1311</td>\n",
              "      <td>SRX9584895</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49445</th>\n",
              "      <td>SRR13144528</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>327</td>\n",
              "      <td>SRX9584896</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49446</th>\n",
              "      <td>SRR13144534</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>2292</td>\n",
              "      <td>SRX9584890</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49447</th>\n",
              "      <td>SRR13144527</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>3377</td>\n",
              "      <td>SRX9584897</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49448</th>\n",
              "      <td>SRR13144526</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>285</td>\n",
              "      <td>SRX9584898</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49449</th>\n",
              "      <td>SRR13144525</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>406</td>\n",
              "      <td>SRX9584899</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49450</th>\n",
              "      <td>SRR13144524</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>1507</td>\n",
              "      <td>SRX9584900</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49451</th>\n",
              "      <td>SRR13144523</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>226</td>\n",
              "      <td>SRX9584901</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49452</th>\n",
              "      <td>SRR13144532</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>261</td>\n",
              "      <td>SRX9584892</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>173270</th>\n",
              "      <td>SRR12038589</td>\n",
              "      <td>2020-06-17 16:16:17</td>\n",
              "      <td>207</td>\n",
              "      <td>SRX8568231</td>\n",
              "      <td>Synthetic-Long-Read</td>\n",
              "      <td>Restriction Digest</td>\n",
              "      <td>SYNTHETIC</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel</td>\n",
              "      <td>SRP267723</td>\n",
              "      <td>PRJNA639956</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>173271</th>\n",
              "      <td>SRR12038590</td>\n",
              "      <td>2020-06-17 16:16:17</td>\n",
              "      <td>146</td>\n",
              "      <td>SRX8568230</td>\n",
              "      <td>Synthetic-Long-Read</td>\n",
              "      <td>Restriction Digest</td>\n",
              "      <td>SYNTHETIC</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel</td>\n",
              "      <td>SRP267723</td>\n",
              "      <td>PRJNA639956</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                Run          ReleaseDate  ...   SRAStudy   BioProject\n",
              "49441   SRR13144531  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "49442   SRR13144533  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "49443   SRR13144530  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "49444   SRR13144529  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "49445   SRR13144528  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "49446   SRR13144534  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "49447   SRR13144527  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "49448   SRR13144526  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "49449   SRR13144525  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "49450   SRR13144524  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "49451   SRR13144523  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "49452   SRR13144532  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "173270  SRR12038589  2020-06-17 16:16:17  ...  SRP267723  PRJNA639956\n",
              "173271  SRR12038590  2020-06-17 16:16:17  ...  SRP267723  PRJNA639956\n",
              "\n",
              "[14 rows x 12 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 38
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wPCZ_gXF5n7r"
      },
      "source": [
        "Notice that indexes (the first column), come along for the ride, but we do not need them.\n",
        "\n",
        "Now, let's pull out all PacBio records that were obtained from Amplicon sequencing. We can again use Boolean indexing, but we need to use an `&` operator. We did not cover this bitwise operator before, but the syntax is self-explanatory in the example below. Note that it is important that each Boolean operation you are doing is in parentheses because of the precedence of the operators involved."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "O97EwdRJ5n7r",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 645
        },
        "outputId": "61cf2aba-2529-4f51-95c8-346029995c22"
      },
      "source": [
        "df.loc[(df['Platform'] == 'PACBIO_SMRT') & (df['LibraryStrategy'] == 'AMPLICON'), :]"
      ],
      "execution_count": 39,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Run</th>\n",
              "      <th>ReleaseDate</th>\n",
              "      <th>size_MB</th>\n",
              "      <th>Experiment</th>\n",
              "      <th>LibraryStrategy</th>\n",
              "      <th>LibrarySelection</th>\n",
              "      <th>LibrarySource</th>\n",
              "      <th>LibraryLayout</th>\n",
              "      <th>Platform</th>\n",
              "      <th>Model</th>\n",
              "      <th>SRAStudy</th>\n",
              "      <th>BioProject</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>49441</th>\n",
              "      <td>SRR13144531</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>1137</td>\n",
              "      <td>SRX9584893</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49442</th>\n",
              "      <td>SRR13144533</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>1999</td>\n",
              "      <td>SRX9584891</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49443</th>\n",
              "      <td>SRR13144530</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>81</td>\n",
              "      <td>SRX9584894</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49444</th>\n",
              "      <td>SRR13144529</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>1311</td>\n",
              "      <td>SRX9584895</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49445</th>\n",
              "      <td>SRR13144528</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>327</td>\n",
              "      <td>SRX9584896</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49446</th>\n",
              "      <td>SRR13144534</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>2292</td>\n",
              "      <td>SRX9584890</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49447</th>\n",
              "      <td>SRR13144527</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>3377</td>\n",
              "      <td>SRX9584897</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49448</th>\n",
              "      <td>SRR13144526</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>285</td>\n",
              "      <td>SRX9584898</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49449</th>\n",
              "      <td>SRR13144525</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>406</td>\n",
              "      <td>SRX9584899</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49450</th>\n",
              "      <td>SRR13144524</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>1507</td>\n",
              "      <td>SRX9584900</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49451</th>\n",
              "      <td>SRR13144523</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>226</td>\n",
              "      <td>SRX9584901</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49452</th>\n",
              "      <td>SRR13144532</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>261</td>\n",
              "      <td>SRX9584892</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "               Run          ReleaseDate  ...   SRAStudy   BioProject\n",
              "49441  SRR13144531  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "49442  SRR13144533  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "49443  SRR13144530  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "49444  SRR13144529  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "49445  SRR13144528  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "49446  SRR13144534  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "49447  SRR13144527  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "49448  SRR13144526  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "49449  SRR13144525  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "49450  SRR13144524  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "49451  SRR13144523  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "49452  SRR13144532  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "\n",
              "[12 rows x 12 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 39
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2wR_02ag5n7r"
      },
      "source": [
        "What is going on will be clearer if we set up our Boolean indexing first, as follows."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PJhS5B365n7r",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "66b266ea-ae10-4425-98ba-40d0662ffb50"
      },
      "source": [
        "inds = (df['Platform'] == 'PACBIO_SMRT') & (df['LibraryStrategy'] == 'AMPLICON')\n",
        "\n",
        "# Take a look\n",
        "inds"
      ],
      "execution_count": 40,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0         False\n",
              "1         False\n",
              "2         False\n",
              "3         False\n",
              "4         False\n",
              "          ...  \n",
              "190351    False\n",
              "190352    False\n",
              "190353    False\n",
              "190354    False\n",
              "190355    False\n",
              "Length: 190356, dtype: bool"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 40
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UknkTwTB5n7s"
      },
      "source": [
        "Notice that `inds` is an array (actually a Pandas `Series`, essentially a `DataFrame` with one column) of `True`s and `False`s. We can apply `unique` function from [NumPy](https://numpy.org/) (we will talk about numpy next week) to see how many `True` and `False` rows we have:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "UFPwWb7R5-Zd",
        "outputId": "ad6c44e6-198d-4f67-985a-e467202d5edd"
      },
      "source": [
        "import numpy as np\n",
        "np.unique(inds, return_counts=True)"
      ],
      "execution_count": 41,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(array([False,  True]), array([190344,     12]))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 41
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YWKLbpuF6jeU"
      },
      "source": [
        "When we index with it using .loc, we get back rows where inds is True."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "d-_-_1OV5n7s",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 645
        },
        "outputId": "ac2b818f-5e30-4480-bea3-4c5633830678"
      },
      "source": [
        "df.loc[inds, :]"
      ],
      "execution_count": 42,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Run</th>\n",
              "      <th>ReleaseDate</th>\n",
              "      <th>size_MB</th>\n",
              "      <th>Experiment</th>\n",
              "      <th>LibraryStrategy</th>\n",
              "      <th>LibrarySelection</th>\n",
              "      <th>LibrarySource</th>\n",
              "      <th>LibraryLayout</th>\n",
              "      <th>Platform</th>\n",
              "      <th>Model</th>\n",
              "      <th>SRAStudy</th>\n",
              "      <th>BioProject</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>49441</th>\n",
              "      <td>SRR13144531</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>1137</td>\n",
              "      <td>SRX9584893</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49442</th>\n",
              "      <td>SRR13144533</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>1999</td>\n",
              "      <td>SRX9584891</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49443</th>\n",
              "      <td>SRR13144530</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>81</td>\n",
              "      <td>SRX9584894</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49444</th>\n",
              "      <td>SRR13144529</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>1311</td>\n",
              "      <td>SRX9584895</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49445</th>\n",
              "      <td>SRR13144528</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>327</td>\n",
              "      <td>SRX9584896</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49446</th>\n",
              "      <td>SRR13144534</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>2292</td>\n",
              "      <td>SRX9584890</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49447</th>\n",
              "      <td>SRR13144527</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>3377</td>\n",
              "      <td>SRX9584897</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49448</th>\n",
              "      <td>SRR13144526</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>285</td>\n",
              "      <td>SRX9584898</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49449</th>\n",
              "      <td>SRR13144525</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>406</td>\n",
              "      <td>SRX9584899</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49450</th>\n",
              "      <td>SRR13144524</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>1507</td>\n",
              "      <td>SRX9584900</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49451</th>\n",
              "      <td>SRR13144523</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>226</td>\n",
              "      <td>SRX9584901</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49452</th>\n",
              "      <td>SRR13144532</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>261</td>\n",
              "      <td>SRX9584892</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "               Run          ReleaseDate  ...   SRAStudy   BioProject\n",
              "49441  SRR13144531  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "49442  SRR13144533  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "49443  SRR13144530  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "49444  SRR13144529  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "49445  SRR13144528  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "49446  SRR13144534  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "49447  SRR13144527  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "49448  SRR13144526  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "49449  SRR13144525  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "49450  SRR13144524  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "49451  SRR13144523  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "49452  SRR13144532  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "\n",
              "[12 rows x 12 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 42
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IPbRSShg5n7s"
      },
      "source": [
        "Of interest in this exercise in Boolean indexing is that we never had to write a loop. To produce our indices, we could have done the following."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "quAcyEzb5n7s",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 625
        },
        "outputId": "909401ef-ec87-41b4-cfaf-a694b153ee09"
      },
      "source": [
        "# Initialize array of Boolean indices\n",
        "inds = [False] * len(df)\n",
        "\n",
        "# Iterate over the rows of the DataFrame to check if the row should be included\n",
        "for i, r in df.iterrows():\n",
        "    if r['Platform'] == 'PACBIO_SMRT' and r['LibraryStrategy'] == 'AMPLICON':\n",
        "        inds[i] = True\n",
        "\n",
        "# Make our seleciton with Boolean indexing\n",
        "df.loc[inds, :]"
      ],
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Run</th>\n",
              "      <th>ReleaseDate</th>\n",
              "      <th>Experiment</th>\n",
              "      <th>LibraryStrategy</th>\n",
              "      <th>LibrarySelection</th>\n",
              "      <th>LibrarySource</th>\n",
              "      <th>LibraryLayout</th>\n",
              "      <th>Platform</th>\n",
              "      <th>Model</th>\n",
              "      <th>SRAStudy</th>\n",
              "      <th>BioProject</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>49441</th>\n",
              "      <td>SRR13144531</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>SRX9584893</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49442</th>\n",
              "      <td>SRR13144533</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>SRX9584891</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49443</th>\n",
              "      <td>SRR13144530</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>SRX9584894</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49444</th>\n",
              "      <td>SRR13144529</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>SRX9584895</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49445</th>\n",
              "      <td>SRR13144528</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>SRX9584896</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49446</th>\n",
              "      <td>SRR13144534</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>SRX9584890</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49447</th>\n",
              "      <td>SRR13144527</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>SRX9584897</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49448</th>\n",
              "      <td>SRR13144526</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>SRX9584898</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49449</th>\n",
              "      <td>SRR13144525</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>SRX9584899</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49450</th>\n",
              "      <td>SRR13144524</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>SRX9584900</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49451</th>\n",
              "      <td>SRR13144523</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>SRX9584901</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49452</th>\n",
              "      <td>SRR13144532</td>\n",
              "      <td>2020-11-25 21:54:15</td>\n",
              "      <td>SRX9584892</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>RT-PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>PACBIO_SMRT</td>\n",
              "      <td>Sequel II</td>\n",
              "      <td>SRP294181</td>\n",
              "      <td>PRJNA680710</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "               Run          ReleaseDate  ...   SRAStudy   BioProject\n",
              "49441  SRR13144531  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "49442  SRR13144533  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "49443  SRR13144530  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "49444  SRR13144529  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "49445  SRR13144528  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "49446  SRR13144534  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "49447  SRR13144527  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "49448  SRR13144526  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "49449  SRR13144525  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "49450  SRR13144524  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "49451  SRR13144523  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "49452  SRR13144532  2020-11-25 21:54:15  ...  SRP294181  PRJNA680710\n",
              "\n",
              "[12 rows x 11 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 21
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Nipwimae5n7s"
      },
      "source": [
        "This feature, where the looping is done automatically on Pandas objects like data frames, is very powerful and saves us writing lots of lines of code. This example also showed how to use the `iterrows()` method of a data frame to iterate over the rows of a data frame. It is actually rare that you will need to do that, as we'll show next when computing with data frames."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BPoQMM1X5n7s"
      },
      "source": [
        "## Calculating with data frames\n",
        "\n",
        "The SRA data contains (sometimes) size of the run in MB (`size_MB`). Let's suppose we want to consider only those run where `size_MB` is above 100. We might like to add a column to the data frame that specifies whether or not the corresponding run is above 100Mb. We can conveniently compute with columns. This is done elementwise."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "C2FrnPRd5n7s",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "711b0ea7-3938-470f-b877-46cf2ce9b1b0"
      },
      "source": [
        "df['size_MB'] >= 100"
      ],
      "execution_count": 43,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0         False\n",
              "1         False\n",
              "2         False\n",
              "3         False\n",
              "4         False\n",
              "          ...  \n",
              "190351    False\n",
              "190352     True\n",
              "190353    False\n",
              "190354     True\n",
              "190355     True\n",
              "Name: size_MB, Length: 190356, dtype: bool"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 43
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "l-C0rQeT5n7t"
      },
      "source": [
        "This tells us which run is abobe 100 Mb. We can simply add this back to the data frame."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JACfJG8i5n7t",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 309
        },
        "outputId": "9ee41f0f-0c44-4d44-9d4e-51ebab577a59"
      },
      "source": [
        "# Add the column to the DataFrame\n",
        "df['Over100Mb'] = df['size_MB'] >= 100\n",
        "\n",
        "# Take a look\n",
        "df.head()"
      ],
      "execution_count": 44,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Run</th>\n",
              "      <th>ReleaseDate</th>\n",
              "      <th>size_MB</th>\n",
              "      <th>Experiment</th>\n",
              "      <th>LibraryStrategy</th>\n",
              "      <th>LibrarySelection</th>\n",
              "      <th>LibrarySource</th>\n",
              "      <th>LibraryLayout</th>\n",
              "      <th>Platform</th>\n",
              "      <th>Model</th>\n",
              "      <th>SRAStudy</th>\n",
              "      <th>BioProject</th>\n",
              "      <th>Over100Mb</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>ERR5063394</td>\n",
              "      <td>2021-01-18 12:14:33</td>\n",
              "      <td>0</td>\n",
              "      <td>ERX4869505</td>\n",
              "      <td>Targeted-Capture</td>\n",
              "      <td>RANDOM</td>\n",
              "      <td>TRANSCRIPTOMIC</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>ILLUMINA</td>\n",
              "      <td>unspecified</td>\n",
              "      <td>ERP121228</td>\n",
              "      <td>PRJEB37886</td>\n",
              "      <td>False</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>ERR5063392</td>\n",
              "      <td>2021-01-18 12:14:33</td>\n",
              "      <td>0</td>\n",
              "      <td>ERX4869504</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>ILLUMINA</td>\n",
              "      <td>Illumina MiSeq</td>\n",
              "      <td>ERP121228</td>\n",
              "      <td>PRJEB37886</td>\n",
              "      <td>False</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>ERR5063395</td>\n",
              "      <td>2021-01-18 12:14:33</td>\n",
              "      <td>0</td>\n",
              "      <td>ERX4869506</td>\n",
              "      <td>Targeted-Capture</td>\n",
              "      <td>RANDOM</td>\n",
              "      <td>TRANSCRIPTOMIC</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>ILLUMINA</td>\n",
              "      <td>unspecified</td>\n",
              "      <td>ERP121228</td>\n",
              "      <td>PRJEB37886</td>\n",
              "      <td>False</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>ERR5063397</td>\n",
              "      <td>2021-01-18 12:14:33</td>\n",
              "      <td>0</td>\n",
              "      <td>ERX4869510</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>ILLUMINA</td>\n",
              "      <td>NextSeq 550</td>\n",
              "      <td>ERP121228</td>\n",
              "      <td>PRJEB37886</td>\n",
              "      <td>False</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>ERR5063396</td>\n",
              "      <td>2021-01-18 12:14:33</td>\n",
              "      <td>0</td>\n",
              "      <td>ERX4869507</td>\n",
              "      <td>AMPLICON</td>\n",
              "      <td>PCR</td>\n",
              "      <td>VIRAL RNA</td>\n",
              "      <td>SINGLE</td>\n",
              "      <td>ILLUMINA</td>\n",
              "      <td>NextSeq 550</td>\n",
              "      <td>ERP121228</td>\n",
              "      <td>PRJEB37886</td>\n",
              "      <td>False</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "          Run          ReleaseDate  size_MB  ...   SRAStudy  BioProject Over100Mb\n",
              "0  ERR5063394  2021-01-18 12:14:33        0  ...  ERP121228  PRJEB37886     False\n",
              "1  ERR5063392  2021-01-18 12:14:33        0  ...  ERP121228  PRJEB37886     False\n",
              "2  ERR5063395  2021-01-18 12:14:33        0  ...  ERP121228  PRJEB37886     False\n",
              "3  ERR5063397  2021-01-18 12:14:33        0  ...  ERP121228  PRJEB37886     False\n",
              "4  ERR5063396  2021-01-18 12:14:33        0  ...  ERP121228  PRJEB37886     False\n",
              "\n",
              "[5 rows x 13 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 44
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3iWWPneR5n7t"
      },
      "source": [
        "## A note about vectorization\n",
        "\n",
        "Notice how applying the `<=` operator to a `Series` resulted in **elementwise** application. This is called `vectorization`. It means that we do not have to write a `for` loop to do operations on the elements of a `Series` or other array-like object. Imagine if we had to do that with a `for` loop."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fbpKF1Np5n7t"
      },
      "source": [
        "big_runs = []\n",
        "for size in df['size_MB']:\n",
        "    big_runs.append(size >= 100)"
      ],
      "execution_count": 45,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JhX7WCL85n7t"
      },
      "source": [
        "This is cumbersome. The vectorization allows for much more convenient calculation. Beyond that, the vectorized code is almost always faster when using Pandas and Numpy because the looping is done with compiled code under the hood. This can be done with many operators, including those you've already seen, like `+`, `-`, `*`, `/`, `**`, etc."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OGyKCJTB5n7u"
      },
      "source": [
        "## Applying functions to Pandas objects\n",
        "\n",
        "Remember when we briefly saw the `np.mean()` function? We can compute with that as well. Let's compare the mean run size for Ilumina versus everything else."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VM9Sw67e5n7u",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "af17b840-d41d-4bfb-b788-6cb1c2b15081"
      },
      "source": [
        "print('Illumina:          ', np.mean(df.loc[df['Over100Mb'], 'size_MB']))\n",
        "print('Other platforms:   ', np.mean(df.loc[~df['Over100Mb'], 'size_MB']))"
      ],
      "execution_count": 49,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Illumina:           219.13786583360738\n",
            "Other platforms:    41.131544740810064\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iHREANJX5n7u"
      },
      "source": [
        "Notice that I used the `~` operator, which is a bit switcher. It changes all `True`s to `False`s and vice versa. In this case, it functions like NOT.\n",
        "\n",
        "We will do a lot more computing with Pandas data frames in the next lessons. For our last demonstration in this lesson, we can quickly compute summary statistics about each column of a data frame using its `describe()` method."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PfO03_uF5n7u",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 297
        },
        "outputId": "6a3bff84-c451-4b4f-cf2b-cf2a2abe866f"
      },
      "source": [
        "df.describe()"
      ],
      "execution_count": 50,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>size_MB</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>count</th>\n",
              "      <td>190356.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>mean</th>\n",
              "      <td>75.256062</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>std</th>\n",
              "      <td>241.814860</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>min</th>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25%</th>\n",
              "      <td>19.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>50%</th>\n",
              "      <td>51.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>75%</th>\n",
              "      <td>87.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>max</th>\n",
              "      <td>25911.000000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "             size_MB\n",
              "count  190356.000000\n",
              "mean       75.256062\n",
              "std       241.814860\n",
              "min         0.000000\n",
              "25%        19.000000\n",
              "50%        51.000000\n",
              "75%        87.000000\n",
              "max     25911.000000"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 50
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aeAdo_-G5n7u"
      },
      "source": [
        "This gives us a data frame with summary statistics. Note that in this data frame, the row indices are not integers, but are the names of the summary statistics. If we wanted to extract the median value of each entry, we could do that with `.loc`."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Ei-KQ9XM5n7u",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "b59f8e72-57c1-465c-9937-582d8ff5425c"
      },
      "source": [
        "df.describe().loc['50%', :]"
      ],
      "execution_count": 51,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "size_MB    51.0\n",
              "Name: 50%, dtype: float64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 51
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "H7vs0zLg5n7u"
      },
      "source": [
        "### Outputting a new CSV file\n",
        "\n",
        "Now that we added the `Over100Mb` column, we might like to save our data frame as a new CSV that we can reload later. We use `df.to_csv()` for this with the `index` kwarg to ask Pandas not to explicitly write the indices to the file."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "I3XMRD-A5n7v"
      },
      "source": [
        "df.to_csv('over100Mb_data.csv', index=False)"
      ],
      "execution_count": 52,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YHqxuBcC5n7v"
      },
      "source": [
        "Let's take a look at what this file looks like."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "93heibSY5n7v",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "e6fdf454-2a9b-4749-d978-bba9fbbfa2c6"
      },
      "source": [
        "!head over100Mb_data.csv"
      ],
      "execution_count": 53,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Run,ReleaseDate,size_MB,Experiment,LibraryStrategy,LibrarySelection,LibrarySource,LibraryLayout,Platform,Model,SRAStudy,BioProject,Over100Mb\n",
            "ERR5063394,2021-01-18 12:14:33,0,ERX4869505,Targeted-Capture,RANDOM,TRANSCRIPTOMIC,SINGLE,ILLUMINA,unspecified,ERP121228,PRJEB37886,False\n",
            "ERR5063392,2021-01-18 12:14:33,0,ERX4869504,AMPLICON,PCR,VIRAL RNA,SINGLE,ILLUMINA,Illumina MiSeq,ERP121228,PRJEB37886,False\n",
            "ERR5063395,2021-01-18 12:14:33,0,ERX4869506,Targeted-Capture,RANDOM,TRANSCRIPTOMIC,SINGLE,ILLUMINA,unspecified,ERP121228,PRJEB37886,False\n",
            "ERR5063397,2021-01-18 12:14:33,0,ERX4869510,AMPLICON,PCR,VIRAL RNA,SINGLE,ILLUMINA,NextSeq 550,ERP121228,PRJEB37886,False\n",
            "ERR5063396,2021-01-18 12:14:33,0,ERX4869507,AMPLICON,PCR,VIRAL RNA,SINGLE,ILLUMINA,NextSeq 550,ERP121228,PRJEB37886,False\n",
            "ERR5063399,2021-01-18 12:14:33,0,ERX4869508,Targeted-Capture,RANDOM,TRANSCRIPTOMIC,SINGLE,ILLUMINA,unspecified,ERP121228,PRJEB37886,False\n",
            "ERR5063400,2021-01-18 12:14:33,0,ERX4869509,AMPLICON,PCR,VIRAL RNA,SINGLE,ILLUMINA,NextSeq 550,ERP121228,PRJEB37886,False\n",
            "ERR5063398,2021-01-18 12:14:33,0,ERX4869511,Targeted-Capture,RANDOM,TRANSCRIPTOMIC,SINGLE,ILLUMINA,unspecified,ERP121228,PRJEB37886,False\n",
            "ERR5063401,2021-01-18 12:14:33,0,ERX4869512,Targeted-Capture,RANDOM,TRANSCRIPTOMIC,SINGLE,ILLUMINA,unspecified,ERP121228,PRJEB37886,False\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pbOJB_uS5n7v"
      },
      "source": [
        "Very nice. Notice that by default Pandas leaves an empty field for NaNs, and we do not need the `na_values` kwarg when we load in this CSV file."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "txJxQano5n7v"
      },
      "source": [
        "## Computing environment"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "q-FVBnrGFWFh",
        "outputId": "a93aaa3f-23f1-4b73-8ea5-07a09e4c9207"
      },
      "source": [
        "!pip install watermark"
      ],
      "execution_count": 58,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Collecting watermark\n",
            "  Downloading https://files.pythonhosted.org/packages/60/fe/3ed83b6122e70dce6fe269dfd763103c333f168bf91037add73ea4fe81c2/watermark-2.0.2-py2.py3-none-any.whl\n",
            "Requirement already satisfied: ipython in /usr/local/lib/python3.6/dist-packages (from watermark) (5.5.0)\n",
            "Requirement already satisfied: prompt-toolkit<2.0.0,>=1.0.4 in /usr/local/lib/python3.6/dist-packages (from ipython->watermark) (1.0.18)\n",
            "Requirement already satisfied: simplegeneric>0.8 in /usr/local/lib/python3.6/dist-packages (from ipython->watermark) (0.8.1)\n",
            "Requirement already satisfied: traitlets>=4.2 in /usr/local/lib/python3.6/dist-packages (from ipython->watermark) (4.3.3)\n",
            "Requirement already satisfied: setuptools>=18.5 in /usr/local/lib/python3.6/dist-packages (from ipython->watermark) (51.3.3)\n",
            "Requirement already satisfied: pygments in /usr/local/lib/python3.6/dist-packages (from ipython->watermark) (2.6.1)\n",
            "Requirement already satisfied: pexpect; sys_platform != \"win32\" in /usr/local/lib/python3.6/dist-packages (from ipython->watermark) (4.8.0)\n",
            "Requirement already satisfied: decorator in /usr/local/lib/python3.6/dist-packages (from ipython->watermark) (4.4.2)\n",
            "Requirement already satisfied: pickleshare in /usr/local/lib/python3.6/dist-packages (from ipython->watermark) (0.7.5)\n",
            "Requirement already satisfied: six>=1.9.0 in /usr/local/lib/python3.6/dist-packages (from prompt-toolkit<2.0.0,>=1.0.4->ipython->watermark) (1.15.0)\n",
            "Requirement already satisfied: wcwidth in /usr/local/lib/python3.6/dist-packages (from prompt-toolkit<2.0.0,>=1.0.4->ipython->watermark) (0.2.5)\n",
            "Requirement already satisfied: ipython-genutils in /usr/local/lib/python3.6/dist-packages (from traitlets>=4.2->ipython->watermark) (0.2.0)\n",
            "Requirement already satisfied: ptyprocess>=0.5 in /usr/local/lib/python3.6/dist-packages (from pexpect; sys_platform != \"win32\"->ipython->watermark) (0.7.0)\n",
            "Installing collected packages: watermark\n",
            "Successfully installed watermark-2.0.2\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "QxhhW93P5n7v",
        "outputId": "d2b4acc3-8780-4f01-a0fa-f17bdf334479"
      },
      "source": [
        "%load_ext watermark\n",
        "%watermark -v -p numpy,pandas"
      ],
      "execution_count": 61,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "The watermark extension is already loaded. To reload it, use:\n",
            "  %reload_ext watermark\n",
            "CPython 3.6.9\n",
            "IPython 5.5.0\n",
            "\n",
            "numpy 1.19.5\n",
            "pandas 1.1.5\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zEvEZsXdEeqU"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}