04_loops_lists.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 04_loops_lists notebook"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook will cover the basics of creating, calliing upon and manipulating lists; to help store data. Combining the use of lists with looping through data. Looping through list contents and being able to store data in lists. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Topics covered:\n",
    "\n",
    "**1. Basic of lists**\n",
    "* Creating simple lists\n",
    "* Ammending lists\n",
    "* Storing data in lists\n",
    "\n",
    "\n",
    "**2. Loops** \n",
    "* basic of looping through data\n",
    "* Loop through and store data in lists\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Looking in indexes: https://s3-eu-west-2.amazonaws.com/mirrors.notebook.uktrade.io/pypi/\n",
      "Requirement already satisfied: openpyxl in /opt/conda/lib/python3.9/site-packages (3.0.10)\n",
      "Requirement already satisfied: et-xmlfile in /opt/conda/lib/python3.9/site-packages (from openpyxl) (1.1.0)\n"
     ]
    }
   ],
   "source": [
    "# Set up\n",
    "\n",
    "# pandas and numpy are universally used in python, like tidyverse is in R. \n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "!pip install openpyxl\n",
    "\n",
    "# chnage from scientific notation \n",
    "pd.set_option('display.float_format', lambda x: '%.5f' % x)\n",
    "\n",
    "trade = pd.read_excel(\"data/trade_data.xlsx\") # upload xlsxl\n",
    "tariff = pd.read_excel(\"data/tariff_data.xlsx\")\n",
    "uk_trqs = pd.read_csv(\"data/uk_trqs.csv\",dtype={'quota__order_number': str})\n",
    "\n",
    "trade.columns = trade.columns.str.lower().str.replace(\" \",\"_\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1 Lists. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lists are a very useful tool working in Python. You cna use them to simply store values and use them as arrays. They provide an easy ability to combine multiple inputs together into a single input. These can be used to filter and manipulate dataframes. Lists can be used to store data and provide an ability to store large amounts of information. \n",
    "\n",
    "Lists are a very important feature and are incredibly useful to learn. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. basics of lists"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "# creating a basic list\n",
    "\n",
    "empty_list = [] # empty square brackets\n",
    "basic_list = [\"item1\", \"item2\",\"item3\",\"item4\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['item1', 'item2', 'item3', 'item4'] []\n"
     ]
    }
   ],
   "source": [
    "print(basic_list, empty_list)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['item1', 'item2', 'item3', 'item4', 'item5'] ['item', 'item2']\n"
     ]
    }
   ],
   "source": [
    "# add items into list\n",
    "#empty_list\n",
    "empty_list.append(\"item\")\n",
    "empty_list.append(\"item2\")\n",
    "basic_list.append(\"item5\")\n",
    "\n",
    "print(basic_list, empty_list)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[[1, 2, 3], [4, 5, 6], [7, 8, 9]]"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lists_in_list = [[1,2,3] , [4,5,6] , [7,8,9]]\n",
    "lists_in_list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1, 2, 3]"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lists_in_list[0] # 0 is the first index representing the first item. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1, 2, 3]\n",
      "[4, 5, 6]\n",
      "[7, 8, 9]\n"
     ]
    }
   ],
   "source": [
    "# iterate through and print list items:\n",
    "for i in lists_in_list:\n",
    "    print(i)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['item2', 'item3', 'item4']"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\n",
    "# select specifc items. Item2 to 4:\n",
    "basic_list[1:4]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['item1', 'item2']"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "basic_list[0:2] # first two items"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'item5'"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "basic_list[-1] # end of list"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Combine lists"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [],
   "source": [
    "list1 = [\"Apple\",\"Orange\",\"Banana\"]\n",
    "list2 = [\"Mango\", \"Pineapple\"]\n",
    "list3 = [\"Blueberry\", \"Raspberry\", \"Strawberry\", \"Blackberry\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Apple', 'Banana', 'Orange'] ['Mango', 'Pineapple'] ['Blackberry', 'Blueberry', 'Raspberry', 'Strawberry']\n"
     ]
    }
   ],
   "source": [
    "# sort lists - as not in alphabetical order:\n",
    "\n",
    "list1 = sorted(list1)\n",
    "list2 = sorted(list2)\n",
    "list3 = sorted(list3)\n",
    "\n",
    "print(list1,list2,list3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Apple',\n",
       " 'Banana',\n",
       " 'Orange',\n",
       " ['Mango', 'Pineapple'],\n",
       " ['Blackberry', 'Blueberry', 'Raspberry', 'Strawberry']]"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# combine the three lists into one single:\n",
    "# append method\n",
    "list1.append(list2)\n",
    "list1.append(list3)\n",
    "list1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This creates lists within a list. We want one single list. Here we can use extend."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Apple',\n",
       " 'Orange',\n",
       " 'Banana',\n",
       " 'Mango',\n",
       " 'Pineapple',\n",
       " 'Blueberry',\n",
       " 'Raspberry',\n",
       " 'Strawberry',\n",
       " 'Blackberry']"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list1.extend(list2)\n",
    "list1.extend(list3)\n",
    "list1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[['Apple', 'Orange', 'Banana'],\n",
       " ['Mango', 'Pineapple'],\n",
       " ['Blueberry', 'Raspberry', 'Strawberry', 'Blackberry']]"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# alternatively if we wanted to store thelists within a list:\n",
    "list0 = [\"Apple\",\"Orange\",\"Banana\"]\n",
    "list2 = [\"Mango\", \"Pineapple\"]\n",
    "list3 = [\"Blueberry\", \"Raspberry\", \"Strawberry\", \"Blackberry\"]\n",
    "list4 = [list0, list2, list3]\n",
    "list4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 3\n"
     ]
    }
   ],
   "source": [
    "# index method will produce the position of item within the list. \n",
    "print(list1.index('Apple'), list1.index(\"Mango\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#you can check if an item exists in lists\n",
    "\"Apple\" in list1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\"Papaya\" in list1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Apple\n",
      "Orange\n",
      "Banana\n",
      "Mango\n",
      "Pineapple\n",
      "Blueberry\n",
      "Raspberry\n",
      "Strawberry\n",
      "Blackberry\n"
     ]
    }
   ],
   "source": [
    "# basic loop to iterate through list item:\n",
    "for fruit in list1:\n",
    "    print(fruit)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Papaya **not in list**\n",
      "Appleis in list\n",
      "Kiwi **not in list**\n",
      "Bananais in list\n",
      "Mangois in list\n"
     ]
    }
   ],
   "source": [
    "# basic loop to check if values exist in list\n",
    "fruit_list = [\"Papaya\",\"Apple\",\"Kiwi\",\"Banana\",\"Mango\"]\n",
    "\n",
    "for fruit in fruit_list:\n",
    "    if(fruit in list1):\n",
    "        print(fruit + \"is in list\")\n",
    "    else:\n",
    "        print(fruit + \" **not in list**\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Remove items from list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[]"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# remove all list items\n",
    "ls = list1.copy()\n",
    "ls.clear()\n",
    "ls"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "['Apple',\n",
       " 'Orange',\n",
       " 'Banana',\n",
       " 'Mango',\n",
       " 'Pineapple',\n",
       " 'Raspberry',\n",
       " 'Strawberry',\n",
       " 'Blackberry']"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# remove using pop()\n",
    "# remove Blueberry from list\n",
    "position = list1.index(\"Blueberry\")\n",
    "print(position)\n",
    "list1.pop(position)\n",
    "list1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Blueberry'"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list1.pop(position)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Apple', 'Orange', 'Banana', 'Mango', 'Pineapple', 'Raspberry']"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# the default is to remove the last list item. Therefore Blackberry will be remvoed. \n",
    "list1.pop()\n",
    "list1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Apple', 'Orange', 'Banana', 'Raspberry']"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list1.pop(-2) # remove second to last item\n",
    "list1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Orange', 'Banana', 'Raspberry']"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# remove()\n",
    "list1.remove(\"Apple\")\n",
    "list1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Blueberry', 'Raspberry', 'Strawberry', 'Blackberry']\n"
     ]
    }
   ],
   "source": [
    "# identify items with string pattern recognition\n",
    "list1 = [\"Apple\",\"Orange\",\"Banana\"]\n",
    "list2 = [\"Mango\", \"Pineapple\"]\n",
    "list3 = [\"Blueberry\", \"Raspberry\", \"Strawberry\", \"Blackberry\"]\n",
    "list1.extend(list2)\n",
    "list1.extend(list3)\n",
    "\n",
    "print([fruit for fruit in list1 if fruit.endswith('y')])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Banana', 'Blueberry', 'Blackberry']\n"
     ]
    }
   ],
   "source": [
    "print([fruit for fruit in list1 if fruit.startswith('B')])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Dataframe column names as list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['year',\n",
       " 'flow',\n",
       " 'commodity_code',\n",
       " 'country_code',\n",
       " 'country_name',\n",
       " 'value_gbp',\n",
       " 'suppression_notes']"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = trade.copy()\n",
    "colnames = df.columns.to_list()\n",
    "colnames"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['year',\n",
       " 'commodity_code',\n",
       " 'country_code',\n",
       " 'country_name',\n",
       " 'value_gbp',\n",
       " 'suppression_notes']"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# remove items\n",
    "colnames.remove(\"flow\")\n",
    "colnames"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['country_name', 'value_gbp', 'suppression_notes']"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# remove n list items:\n",
    "del colnames[:4] # keep last 3 items. (4th column to 7th)\n",
    "colnames"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['year', 'flow']"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "colnames = df.columns.to_list()\n",
    "del colnames[2:] # select first two columns \n",
    "colnames"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['year', 'flow', 'country_name', 'value_gbp', 'suppression_notes']"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "colnames = df.columns.to_list()\n",
    "del colnames[2:4] # delete the 3rd column (0 index is the first) to the 4th. \n",
    "colnames"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**** "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Storing dataframes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can store dataframe objects in list too, not just values creating an array. Lists are a useful place to store dataframes to later call on later in the script. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "df1 = trade.head(10)\n",
    "df2 = trade.head(20)\n",
    "df3 = trade.head(30)\n",
    "df4 = trade.head(40)\n",
    "\n",
    "ls = []"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "ls.append(df1)\n",
    "ls.append(df2)\n",
    "ls.append(df3)\n",
    "ls.append(df4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>year</th>\n",
       "      <th>flow</th>\n",
       "      <th>commodity_code</th>\n",
       "      <th>country_code</th>\n",
       "      <th>country_name</th>\n",
       "      <th>value_gbp</th>\n",
       "      <th>suppression_notes</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2020</td>\n",
       "      <td>Exports</td>\n",
       "      <td>01012100</td>\n",
       "      <td>TW</td>\n",
       "      <td>Taiwan</td>\n",
       "      <td>892</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2020</td>\n",
       "      <td>Exports</td>\n",
       "      <td>01062000</td>\n",
       "      <td>TW</td>\n",
       "      <td>Taiwan</td>\n",
       "      <td>14101</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2020</td>\n",
       "      <td>Exports</td>\n",
       "      <td>01063100</td>\n",
       "      <td>TW</td>\n",
       "      <td>Taiwan</td>\n",
       "      <td>1750</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2020</td>\n",
       "      <td>Exports</td>\n",
       "      <td>02031913</td>\n",
       "      <td>TW</td>\n",
       "      <td>Taiwan</td>\n",
       "      <td>290818</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2020</td>\n",
       "      <td>Exports</td>\n",
       "      <td>02031990</td>\n",
       "      <td>TW</td>\n",
       "      <td>Taiwan</td>\n",
       "      <td>1140</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2020</td>\n",
       "      <td>Exports</td>\n",
       "      <td>02032219</td>\n",
       "      <td>TW</td>\n",
       "      <td>Taiwan</td>\n",
       "      <td>125461</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>2020</td>\n",
       "      <td>Exports</td>\n",
       "      <td>02032290</td>\n",
       "      <td>TW</td>\n",
       "      <td>Taiwan</td>\n",
       "      <td>70982</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>2020</td>\n",
       "      <td>Exports</td>\n",
       "      <td>02032955</td>\n",
       "      <td>TW</td>\n",
       "      <td>Taiwan</td>\n",
       "      <td>508591</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>2020</td>\n",
       "      <td>Exports</td>\n",
       "      <td>02032959</td>\n",
       "      <td>TW</td>\n",
       "      <td>Taiwan</td>\n",
       "      <td>53000</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>2020</td>\n",
       "      <td>Exports</td>\n",
       "      <td>02032990</td>\n",
       "      <td>TW</td>\n",
       "      <td>Taiwan</td>\n",
       "      <td>276857</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   year     flow commodity_code country_code country_name  value_gbp  \\\n",
       "0  2020  Exports       01012100           TW       Taiwan        892   \n",
       "1  2020  Exports       01062000           TW       Taiwan      14101   \n",
       "2  2020  Exports       01063100           TW       Taiwan       1750   \n",
       "3  2020  Exports       02031913           TW       Taiwan     290818   \n",
       "4  2020  Exports       02031990           TW       Taiwan       1140   \n",
       "5  2020  Exports       02032219           TW       Taiwan     125461   \n",
       "6  2020  Exports       02032290           TW       Taiwan      70982   \n",
       "7  2020  Exports       02032955           TW       Taiwan     508591   \n",
       "8  2020  Exports       02032959           TW       Taiwan      53000   \n",
       "9  2020  Exports       02032990           TW       Taiwan     276857   \n",
       "\n",
       "   suppression_notes  \n",
       "0                NaN  \n",
       "1                NaN  \n",
       "2                NaN  \n",
       "3                NaN  \n",
       "4                NaN  \n",
       "5                NaN  \n",
       "6                NaN  \n",
       "7                NaN  \n",
       "8                NaN  \n",
       "9                NaN  "
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ls[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "ls2 = [df1,df2,df3,df4]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>year</th>\n",
       "      <th>flow</th>\n",
       "      <th>commodity_code</th>\n",
       "      <th>country_code</th>\n",
       "      <th>country_name</th>\n",
       "      <th>value_gbp</th>\n",
       "      <th>suppression_notes</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2020</td>\n",
       "      <td>Exports</td>\n",
       "      <td>01012100</td>\n",
       "      <td>TW</td>\n",
       "      <td>Taiwan</td>\n",
       "      <td>892</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2020</td>\n",
       "      <td>Exports</td>\n",
       "      <td>01062000</td>\n",
       "      <td>TW</td>\n",
       "      <td>Taiwan</td>\n",
       "      <td>14101</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2020</td>\n",
       "      <td>Exports</td>\n",
       "      <td>01063100</td>\n",
       "      <td>TW</td>\n",
       "      <td>Taiwan</td>\n",
       "      <td>1750</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2020</td>\n",
       "      <td>Exports</td>\n",
       "      <td>02031913</td>\n",
       "      <td>TW</td>\n",
       "      <td>Taiwan</td>\n",
       "      <td>290818</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2020</td>\n",
       "      <td>Exports</td>\n",
       "      <td>02031990</td>\n",
       "      <td>TW</td>\n",
       "      <td>Taiwan</td>\n",
       "      <td>1140</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2020</td>\n",
       "      <td>Exports</td>\n",
       "      <td>02032219</td>\n",
       "      <td>TW</td>\n",
       "      <td>Taiwan</td>\n",
       "      <td>125461</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>2020</td>\n",
       "      <td>Exports</td>\n",
       "      <td>02032290</td>\n",
       "      <td>TW</td>\n",
       "      <td>Taiwan</td>\n",
       "      <td>70982</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>2020</td>\n",
       "      <td>Exports</td>\n",
       "      <td>02032955</td>\n",
       "      <td>TW</td>\n",
       "      <td>Taiwan</td>\n",
       "      <td>508591</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>2020</td>\n",
       "      <td>Exports</td>\n",
       "      <td>02032959</td>\n",
       "      <td>TW</td>\n",
       "      <td>Taiwan</td>\n",
       "      <td>53000</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>2020</td>\n",
       "      <td>Exports</td>\n",
       "      <td>02032990</td>\n",
       "      <td>TW</td>\n",
       "      <td>Taiwan</td>\n",
       "      <td>276857</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   year     flow commodity_code country_code country_name  value_gbp  \\\n",
       "0  2020  Exports       01012100           TW       Taiwan        892   \n",
       "1  2020  Exports       01062000           TW       Taiwan      14101   \n",
       "2  2020  Exports       01063100           TW       Taiwan       1750   \n",
       "3  2020  Exports       02031913           TW       Taiwan     290818   \n",
       "4  2020  Exports       02031990           TW       Taiwan       1140   \n",
       "5  2020  Exports       02032219           TW       Taiwan     125461   \n",
       "6  2020  Exports       02032290           TW       Taiwan      70982   \n",
       "7  2020  Exports       02032955           TW       Taiwan     508591   \n",
       "8  2020  Exports       02032959           TW       Taiwan      53000   \n",
       "9  2020  Exports       02032990           TW       Taiwan     276857   \n",
       "\n",
       "   suppression_notes  \n",
       "0                NaN  \n",
       "1                NaN  \n",
       "2                NaN  \n",
       "3                NaN  \n",
       "4                NaN  \n",
       "5                NaN  \n",
       "6                NaN  \n",
       "7                NaN  \n",
       "8                NaN  \n",
       "9                NaN  "
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ls2[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(100, 7)"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# concatonate dfs together in list\n",
    "# we can use pd.concat easily. \n",
    "\n",
    "df_combined = pd.concat(ls2)\n",
    "df_combined.shape "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "****"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Loops"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Loops are very helpful as they allow you to excute repeated code to a specified end point. This helps us iterate through dataframes applying a repeated process based on a spcific argument or condition. For example we cna loop through a large dataframe based on unique items within a group. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using the trade data example - we can loop through to filter the data based on a specific country/year/flow type. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = trade.head(10).copy()\n",
    "df2 = trade.copy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Empty DataFrame\n",
      "Columns: [year, flow, commodity_code, country_code, country_name, value_gbp, suppression_notes]\n",
      "Index: []\n",
      "   year     flow commodity_code country_code country_name  value_gbp  \\\n",
      "0  2020  Exports       01012100           TW       Taiwan        892   \n",
      "\n",
      "   suppression_notes  \n",
      "0                NaN  \n",
      "   year     flow commodity_code country_code country_name  value_gbp  \\\n",
      "0  2020  Exports       01012100           TW       Taiwan        892   \n",
      "1  2020  Exports       01062000           TW       Taiwan      14101   \n",
      "\n",
      "   suppression_notes  \n",
      "0                NaN  \n",
      "1                NaN  \n",
      "   year     flow commodity_code country_code country_name  value_gbp  \\\n",
      "0  2020  Exports       01012100           TW       Taiwan        892   \n",
      "1  2020  Exports       01062000           TW       Taiwan      14101   \n",
      "2  2020  Exports       01063100           TW       Taiwan       1750   \n",
      "\n",
      "   suppression_notes  \n",
      "0                NaN  \n",
      "1                NaN  \n",
      "2                NaN  \n",
      "   year     flow commodity_code country_code country_name  value_gbp  \\\n",
      "0  2020  Exports       01012100           TW       Taiwan        892   \n",
      "1  2020  Exports       01062000           TW       Taiwan      14101   \n",
      "2  2020  Exports       01063100           TW       Taiwan       1750   \n",
      "3  2020  Exports       02031913           TW       Taiwan     290818   \n",
      "\n",
      "   suppression_notes  \n",
      "0                NaN  \n",
      "1                NaN  \n",
      "2                NaN  \n",
      "3                NaN  \n",
      "   year     flow commodity_code country_code country_name  value_gbp  \\\n",
      "0  2020  Exports       01012100           TW       Taiwan        892   \n",
      "1  2020  Exports       01062000           TW       Taiwan      14101   \n",
      "2  2020  Exports       01063100           TW       Taiwan       1750   \n",
      "3  2020  Exports       02031913           TW       Taiwan     290818   \n",
      "4  2020  Exports       02031990           TW       Taiwan       1140   \n",
      "\n",
      "   suppression_notes  \n",
      "0                NaN  \n",
      "1                NaN  \n",
      "2                NaN  \n",
      "3                NaN  \n",
      "4                NaN  \n",
      "   year     flow commodity_code country_code country_name  value_gbp  \\\n",
      "0  2020  Exports       01012100           TW       Taiwan        892   \n",
      "1  2020  Exports       01062000           TW       Taiwan      14101   \n",
      "2  2020  Exports       01063100           TW       Taiwan       1750   \n",
      "3  2020  Exports       02031913           TW       Taiwan     290818   \n",
      "4  2020  Exports       02031990           TW       Taiwan       1140   \n",
      "5  2020  Exports       02032219           TW       Taiwan     125461   \n",
      "\n",
      "   suppression_notes  \n",
      "0                NaN  \n",
      "1                NaN  \n",
      "2                NaN  \n",
      "3                NaN  \n",
      "4                NaN  \n",
      "5                NaN  \n",
      "   year     flow commodity_code country_code country_name  value_gbp  \\\n",
      "0  2020  Exports       01012100           TW       Taiwan        892   \n",
      "1  2020  Exports       01062000           TW       Taiwan      14101   \n",
      "2  2020  Exports       01063100           TW       Taiwan       1750   \n",
      "3  2020  Exports       02031913           TW       Taiwan     290818   \n",
      "4  2020  Exports       02031990           TW       Taiwan       1140   \n",
      "5  2020  Exports       02032219           TW       Taiwan     125461   \n",
      "6  2020  Exports       02032290           TW       Taiwan      70982   \n",
      "\n",
      "   suppression_notes  \n",
      "0                NaN  \n",
      "1                NaN  \n",
      "2                NaN  \n",
      "3                NaN  \n",
      "4                NaN  \n",
      "5                NaN  \n",
      "6                NaN  \n",
      "   year     flow commodity_code country_code country_name  value_gbp  \\\n",
      "0  2020  Exports       01012100           TW       Taiwan        892   \n",
      "1  2020  Exports       01062000           TW       Taiwan      14101   \n",
      "2  2020  Exports       01063100           TW       Taiwan       1750   \n",
      "3  2020  Exports       02031913           TW       Taiwan     290818   \n",
      "4  2020  Exports       02031990           TW       Taiwan       1140   \n",
      "5  2020  Exports       02032219           TW       Taiwan     125461   \n",
      "6  2020  Exports       02032290           TW       Taiwan      70982   \n",
      "7  2020  Exports       02032955           TW       Taiwan     508591   \n",
      "\n",
      "   suppression_notes  \n",
      "0                NaN  \n",
      "1                NaN  \n",
      "2                NaN  \n",
      "3                NaN  \n",
      "4                NaN  \n",
      "5                NaN  \n",
      "6                NaN  \n",
      "7                NaN  \n",
      "   year     flow commodity_code country_code country_name  value_gbp  \\\n",
      "0  2020  Exports       01012100           TW       Taiwan        892   \n",
      "1  2020  Exports       01062000           TW       Taiwan      14101   \n",
      "2  2020  Exports       01063100           TW       Taiwan       1750   \n",
      "3  2020  Exports       02031913           TW       Taiwan     290818   \n",
      "4  2020  Exports       02031990           TW       Taiwan       1140   \n",
      "5  2020  Exports       02032219           TW       Taiwan     125461   \n",
      "6  2020  Exports       02032290           TW       Taiwan      70982   \n",
      "7  2020  Exports       02032955           TW       Taiwan     508591   \n",
      "8  2020  Exports       02032959           TW       Taiwan      53000   \n",
      "\n",
      "   suppression_notes  \n",
      "0                NaN  \n",
      "1                NaN  \n",
      "2                NaN  \n",
      "3                NaN  \n",
      "4                NaN  \n",
      "5                NaN  \n",
      "6                NaN  \n",
      "7                NaN  \n",
      "8                NaN  \n"
     ]
    }
   ],
   "source": [
    "# loop through trade data length:\n",
    "for i in range(len(df)):\n",
    "    print(df[:i]) # prints slice by nrow of dataframe:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "year\n",
      "flow\n",
      "commodity_code\n",
      "country_code\n",
      "country_name\n",
      "value_gbp\n",
      "suppression_notes\n"
     ]
    }
   ],
   "source": [
    "# loop through df column:\n",
    "for i in df:\n",
    "    print(i)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {},
   "outputs": [],
   "source": [
    "# loop through unique items within group. Such as country name. \n",
    "# first create a list of unique values. \n",
    "\n",
    "country_names = pd.unique(df2[\"country_name\"])\n",
    "# or alternative create your own list:\n",
    "country_names2 = [\"Thailand\",\"Taiwan\",\"Vietnam\",\"United States\"]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['Taiwan', 'Tajikistan', 'Tanzania (United Republic of)',\n",
       "       'Thailand', 'Timor-Leste', 'Tonga', 'Trinidad and Tobago',\n",
       "       'Tunisia', 'Turkmenistan', 'Tuvalu', 'Uganda', 'Ukraine',\n",
       "       'United States', 'United States Minor outlying islands',\n",
       "       'United States Virgin Islands', 'Uruguay', 'Uzbekistan', 'Vanuatu',\n",
       "       'Vatican City State', 'Venezuela, Bolivarian Republic of',\n",
       "       'Vietnam', 'Wallis and Futuna', 'Yemen', 'Zambia'], dtype=object)"
      ]
     },
     "execution_count": 102,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "country_names"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(5589, 7)\n",
      "(6436, 7)\n",
      "(4188, 7)\n",
      "(12891, 7)\n"
     ]
    }
   ],
   "source": [
    "# iterate through selected countries \n",
    "for country in country_names2:\n",
    "    df_filt = df2.loc[df2[\"country_name\"] == country]\n",
    "    print(df_filt.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2020\n",
      "2019\n"
     ]
    }
   ],
   "source": [
    "# you can set the unique group items within the for:\n",
    "for yr in pd.unique(df2[\"year\"]):\n",
    "    print(yr)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Practical example:** filter the trade dataset for all countrys. This data is to be later called upon so we can store each data within a list. Each iteration of the loop we stash the data in a list. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {},
   "outputs": [],
   "source": [
    "ls = [] # create empty list to stash data in\n",
    "for country in country_names:\n",
    "    df_filt = df2.loc[df2[\"country_name\"]==country]\n",
    "    ls.append(df_filt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "24"
      ]
     },
     "execution_count": 127,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(ls) # list item length"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "list"
      ]
     },
     "execution_count": 129,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(ls)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 140,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>year</th>\n",
       "      <th>flow</th>\n",
       "      <th>commodity_code</th>\n",
       "      <th>country_code</th>\n",
       "      <th>country_name</th>\n",
       "      <th>value_gbp</th>\n",
       "      <th>suppression_notes</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2020</td>\n",
       "      <td>Exports</td>\n",
       "      <td>01012100</td>\n",
       "      <td>TW</td>\n",
       "      <td>Taiwan</td>\n",
       "      <td>892</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2020</td>\n",
       "      <td>Exports</td>\n",
       "      <td>01062000</td>\n",
       "      <td>TW</td>\n",
       "      <td>Taiwan</td>\n",
       "      <td>14101</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2020</td>\n",
       "      <td>Exports</td>\n",
       "      <td>01063100</td>\n",
       "      <td>TW</td>\n",
       "      <td>Taiwan</td>\n",
       "      <td>1750</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2020</td>\n",
       "      <td>Exports</td>\n",
       "      <td>02031913</td>\n",
       "      <td>TW</td>\n",
       "      <td>Taiwan</td>\n",
       "      <td>290818</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2020</td>\n",
       "      <td>Exports</td>\n",
       "      <td>02031990</td>\n",
       "      <td>TW</td>\n",
       "      <td>Taiwan</td>\n",
       "      <td>1140</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   year     flow commodity_code country_code country_name  value_gbp  \\\n",
       "0  2020  Exports       01012100           TW       Taiwan        892   \n",
       "1  2020  Exports       01062000           TW       Taiwan      14101   \n",
       "2  2020  Exports       01063100           TW       Taiwan       1750   \n",
       "3  2020  Exports       02031913           TW       Taiwan     290818   \n",
       "4  2020  Exports       02031990           TW       Taiwan       1140   \n",
       "\n",
       "   suppression_notes  \n",
       "0                NaN  \n",
       "1                NaN  \n",
       "2                NaN  \n",
       "3                NaN  \n",
       "4                NaN  "
      ]
     },
     "execution_count": 140,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ls[0].head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 139,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>year</th>\n",
       "      <th>flow</th>\n",
       "      <th>commodity_code</th>\n",
       "      <th>country_code</th>\n",
       "      <th>country_name</th>\n",
       "      <th>value_gbp</th>\n",
       "      <th>suppression_notes</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>34986</th>\n",
       "      <td>2019</td>\n",
       "      <td>Imports</td>\n",
       "      <td>97011000</td>\n",
       "      <td>UY</td>\n",
       "      <td>Uruguay</td>\n",
       "      <td>7917</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34987</th>\n",
       "      <td>2019</td>\n",
       "      <td>Imports</td>\n",
       "      <td>97030000</td>\n",
       "      <td>UY</td>\n",
       "      <td>Uruguay</td>\n",
       "      <td>121150</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34988</th>\n",
       "      <td>2019</td>\n",
       "      <td>Imports</td>\n",
       "      <td>97050000</td>\n",
       "      <td>UY</td>\n",
       "      <td>Uruguay</td>\n",
       "      <td>3469</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34989</th>\n",
       "      <td>2019</td>\n",
       "      <td>Imports</td>\n",
       "      <td>97060000</td>\n",
       "      <td>UY</td>\n",
       "      <td>Uruguay</td>\n",
       "      <td>86020</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34990</th>\n",
       "      <td>2019</td>\n",
       "      <td>Imports</td>\n",
       "      <td>99209900</td>\n",
       "      <td>UY</td>\n",
       "      <td>Uruguay</td>\n",
       "      <td>89952</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       year     flow commodity_code country_code country_name  value_gbp  \\\n",
       "34986  2019  Imports       97011000           UY      Uruguay       7917   \n",
       "34987  2019  Imports       97030000           UY      Uruguay     121150   \n",
       "34988  2019  Imports       97050000           UY      Uruguay       3469   \n",
       "34989  2019  Imports       97060000           UY      Uruguay      86020   \n",
       "34990  2019  Imports       99209900           UY      Uruguay      89952   \n",
       "\n",
       "       suppression_notes  \n",
       "34986                NaN  \n",
       "34987                NaN  \n",
       "34988                NaN  \n",
       "34989                NaN  \n",
       "34990                NaN  "
      ]
     },
     "execution_count": 139,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ls[15].tail()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 156,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 ['Taiwan']\n",
      "1 ['Tajikistan']\n",
      "2 ['Tanzania (United Republic of)']\n",
      "3 ['Timor-Leste']\n",
      "4 ['Tonga']\n",
      "5 ['Trinidad and Tobago']\n",
      "6 ['Tunisia']\n",
      "7 ['Turkmenistan']\n",
      "8 ['Tuvalu']\n",
      "9 ['Uganda']\n",
      "10 ['Ukraine']\n",
      "11 ['United States Minor outlying islands']\n",
      "12 ['Uruguay']\n",
      "13 ['Vanuatu']\n",
      "14 ['Venezuela, Bolivarian Republic of']\n",
      "15 ['Vietnam']\n",
      "16 ['Wallis and Futuna']\n",
      "17 ['Yemen']\n",
      "18 ['Zambia']\n"
     ]
    }
   ],
   "source": [
    "# for example after you analysed the data - you notieced something with specific countries and wanted them remvoed. \n",
    "# remove United States and Thailand from list:\n",
    "# find position within list\n",
    "for i in range(len(ls)):\n",
    "        x = ls[i]\n",
    "        print(i, pd.unique(x[\"country_name\"]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 157,
   "metadata": {},
   "outputs": [],
   "source": [
    "# remove positions 12:14 and 3. \n",
    "index_pos = [3,12,13,14]\n",
    "for pos in index_pos:\n",
    "    ls.pop(pos)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 158,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "15"
      ]
     },
     "execution_count": 158,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(ls)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 159,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(17326, 7)"
      ]
     },
     "execution_count": 159,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Further we may want to concat the data back\n",
    "df_concat = pd.concat(ls)\n",
    "df_concat.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 160,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Taiwan' 'Tajikistan' 'Tanzania (United Republic of)' 'Tonga'\n",
      " 'Trinidad and Tobago' 'Tunisia' 'Turkmenistan' 'Tuvalu' 'Uganda'\n",
      " 'Ukraine' 'United States Minor outlying islands' 'Uruguay'\n",
      " 'Venezuela, Bolivarian Republic of' 'Wallis and Futuna' 'Zambia']\n"
     ]
    }
   ],
   "source": [
    "print(pd.unique(df_concat[\"country_name\"]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have successfully looped through and stored data within a list, removed items and then binded it back together again. This is a common practice I use and find helpful in my analysis. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "****"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### List comprehensions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "List comprehension offers a shorter syntax when you want to create a new list based on the values of an existing list."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 170,
   "metadata": {},
   "outputs": [],
   "source": [
    "ls2 = ls.copy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 178,
   "metadata": {},
   "outputs": [],
   "source": [
    "ls3 = [df for df in ls2 if len(df) < 100] # filter list items if the numer of rows in each df is less than 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 179,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5"
      ]
     },
     "execution_count": 179,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(ls3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 183,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Apple', 'Banana', 'Mango', 'Kiwi', 'Pineapple']"
      ]
     },
     "execution_count": 183,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fruit_list = [\"Apple\", \"Banana\", \"Mango\",\"Kiwi\",\"Strawberry\",\"Raspberry\",\"Blueberry\",\"Blackberry\",\"Pineapple\"]\n",
    "# revmoe any item with pattern within, i,e, Berry\n",
    "fruit_list2 = [fruit for fruit in fruit_list if \"berry\"  not in fruit] # alternatively chage \"not in\" to \"in\"\n",
    "fruit_list2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```python\n",
    "newlist = [expression for item in iterable if condition == True]\n",
    "```"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}