Lab10.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 实验课 10"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 实验10-1 Pandas库初步（下）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 步骤1 数据表信息查看"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "创建一个DataFrame对象后，我们可以从宏观角度查看这个DataFrame对象的相关基本信息，例如对象的形状、数据格式、某一列是否含有空值等等。\n",
    "\n",
    "例如我们加载鸢尾花数据集（非原始数据集，经过一些人工修改），创建一个DataFrame对象。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "f = open('data/iris_with_error.csv','r',encoding='utf-8')\n",
    "iris_data = pd.read_csv(f)\n",
    "f.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* 我们可以直接输出DataFrame对象的所有内容，当然对于包含很多行的DataFrame对象，只会输出前后几行内容。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "     Sepal_Length  Sepal_Width  Petal_Length  Petal_Width    Species\n",
      "0             5.1          3.5           1.4          0.2     setosa\n",
      "1             4.9          NaN           1.4          0.2     setosa\n",
      "2             4.7          3.2           1.3          0.2     setosa\n",
      "3             4.6          3.1           1.5          0.2     setosa\n",
      "4             5.0          3.6           1.4          0.2     setosa\n",
      "..            ...          ...           ...          ...        ...\n",
      "147           6.5          3.0           5.2          2.0  virginica\n",
      "148           6.2          3.4           5.4          2.3  virginica\n",
      "149           5.9          3.0           5.1          1.8  virginica\n",
      "150           5.9          3.0           5.1          1.8  virginica\n",
      "151           6.2          3.4           5.4          2.3  virginica\n",
      "\n",
      "[152 rows x 5 columns]\n"
     ]
    }
   ],
   "source": [
    "print(iris_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* 当然我们可以使用`head(n)`或`tail(n)`方法显示DataFrame对象的前n行或者最后n行内容（若不提供实参，默认显示5行）。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   Sepal_Length  Sepal_Width  Petal_Length  Petal_Width Species\n",
      "0           5.1          3.5           1.4          0.2  setosa\n",
      "1           4.9          NaN           1.4          0.2  setosa\n",
      "2           4.7          3.2           1.3          0.2  setosa\n",
      "3           4.6          3.1           1.5          0.2  setosa\n",
      "4           5.0          3.6           1.4          0.2  setosa\n"
     ]
    }
   ],
   "source": [
    "print(iris_data.head())  # 输出前5行"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "     Sepal_Length  Sepal_Width  Petal_Length  Petal_Width    Species\n",
      "146           6.3          2.5           5.0          1.9  virginica\n",
      "147           6.5          3.0           5.2          2.0  virginica\n",
      "148           6.2          3.4           5.4          2.3  virginica\n",
      "149           5.9          3.0           5.1          1.8  virginica\n",
      "150           5.9          3.0           5.1          1.8  virginica\n",
      "151           6.2          3.4           5.4          2.3  virginica\n"
     ]
    }
   ],
   "source": [
    "print(iris_data.tail(6))  # 输出最后6行"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* 我们可以通过DataFrame对象的`shape`属性查看其形状，例如这个数据集由152条数据组成，每条数据包含4个属性值以及鸢尾花的种类，所以`iris_data`对象的形状为(150,5)。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(152, 5)\n"
     ]
    }
   ],
   "source": [
    "print(iris_data.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* 通过`info()`方法可以查看DataFrame对象的基本信息，包括形状、列索引、列数据类型、对象所占内存大小等。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 152 entries, 0 to 151\n",
      "Data columns (total 5 columns):\n",
      " #   Column        Non-Null Count  Dtype  \n",
      "---  ------        --------------  -----  \n",
      " 0   Sepal_Length  151 non-null    float64\n",
      " 1   Sepal_Width   149 non-null    float64\n",
      " 2   Petal_Length  148 non-null    float64\n",
      " 3   Petal_Width   148 non-null    float64\n",
      " 4   Species       151 non-null    object \n",
      "dtypes: float64(4), object(1)\n",
      "memory usage: 6.1+ KB\n"
     ]
    }
   ],
   "source": [
    "iris_data.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* 如果DataFrame对象包含了大量列，可能寻找某一列的数据类型会变得很困难，我们可以利用Series对象的`dtype`属性获取某一列（不要忘了一列数据是一个Series对象）的数据类型；利用`dtypes`获取多列的数据类型。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "float64\n"
     ]
    }
   ],
   "source": [
    "print(iris_data['Sepal_Length'].dtype)  # 查看'Sepal_Length`列的数据类型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sepal_Length    float64\n",
      "Sepal_Width     float64\n",
      "dtype: object\n"
     ]
    }
   ],
   "source": [
    "print(iris_data.iloc[:,0:2].dtypes)  # 查看前2列的数据类型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 步骤2 数据清洗"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "由于客观因素，例如传感器故障、存储设施损坏、人工录入数据失误等等，我们获取到的数据并不一定是完美无缺的，在对获取到的数据进行进一步的分析之前，我们需要对原始的数据进行“清洗”。\n",
    "\n",
    "顾名思义，数据清洗就是对原始数据中的“脏”数据进行相关处理。对于原始数据来说，经常出现的问题如下所示：\n",
    "\n",
    "* 数据缺失\n",
    "* 数据值不符合实际\n",
    "* 数据重复\n",
    "* 索引不明确\n",
    "* ……\n",
    "\n",
    "可以说进行数据分析，大部分时间都是在对原始数据做预处理工作。由此可见数据清洗对后续数据分析的影响非常之大。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 1. 数据缺失值处理"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "数据缺失可能由于实际情况下未获取到有效数据而留空、或人工录入时疏忽遗漏。\n",
    "\n",
    "经人工修改的鸢尾花数据集的第2条数据中含有一个数据空缺项，在CSV文件中以空字符串表示：\n",
    "```\n",
    "4.9,,1.4,0.2,setosa\n",
    "```\n",
    "通过Pandas读入该CSV文件后，以`NaN`存储和表示（`NaN`全称*Not a Number*，是一种按照IEEE浮点数标准设计，在任何系统中都兼容的特殊浮点数，用以代替表示缺失值）。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sepal_Length       4.9\n",
      "Sepal_Width        NaN\n",
      "Petal_Length       1.4\n",
      "Petal_Width        0.2\n",
      "Species         setosa\n",
      "Name: 1, dtype: object\n"
     ]
    }
   ],
   "source": [
    "print(iris_data.iloc[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "为演示方便，我们先创建一个3*3的二维数组并将其转换为DataFrame对象，其中设置一些空缺值。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   ColA  ColB  ColC\n",
      "0   1.0   NaN     2\n",
      "1   2.0   3.0     5\n",
      "2   NaN   4.0     6\n"
     ]
    }
   ],
   "source": [
    "data_2 = pd.DataFrame([[1,     np.NaN,2],\n",
    "                       [2,     3     ,5],\n",
    "                       [np.NaN,4,     6]],columns=['ColA','ColB','ColC'])\n",
    "print(data_2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* 我们可以用`fillna()`方法，用某个值填充空值。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   ColA  ColB  ColC\n",
      "0   1.0   0.0     2\n",
      "1   2.0   3.0     5\n",
      "2   0.0   4.0     6\n"
     ]
    }
   ],
   "source": [
    "data_3 = data_2.fillna(0)  # 用0填充所有的空缺值\n",
    "print(data_3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   ColA  ColB  ColC\n",
      "0   1.0   3.5     2\n",
      "1   2.0   3.0     5\n",
      "2   NaN   4.0     6\n"
     ]
    }
   ],
   "source": [
    "data_4 = data_2.copy()\n",
    "data_4['ColB'] = data_4['ColB'].fillna(data_4['ColB'].mean())  # 用‘ColB’列的均值填充'ColB'列的空缺值\n",
    "# 也可以用中位数填充data_2['ColB'] = data_2['ColB'].fillna(data_2['ColB'].median())\n",
    "print(data_4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* 也可以添加`method='ffill'`或者`method='bfill'`参数用缺失值前面或后面的有效值向后（向前）填充（若缺失值前（后）面没有有效值，那么它仍然是缺失值）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   ColA  ColB  ColC\n",
      "0   1.0   NaN     2\n",
      "1   2.0   3.0     5\n",
      "2   2.0   4.0     6\n"
     ]
    }
   ],
   "source": [
    "data_5 = data_2.fillna(method='ffill')  # 从前向后填充\n",
    "print(data_5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   ColA  ColB  ColC\n",
      "0   1.0   3.0     2\n",
      "1   2.0   3.0     5\n",
      "2   NaN   4.0     6\n"
     ]
    }
   ],
   "source": [
    "data_6 = data_2.fillna(method='bfill')  # 从后向前填充\n",
    "print(data_6)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* 如果数据集中包含大量数据，剔除一条数据对整体数据分析没有过大影响的情况下，可以通过`dropna()`方法剔除数据行；\n",
    "  > 也可以添加`how='all'`参数设定只剔除全部数据项为空的数据行；\n",
    "  >\n",
    "  > 也可以添加`subset=['ColNameA','ColNameB',...]`参数设定只剔除指定列含有空缺值的数据行。\n",
    "* 通过`dropna(thresh = n)`方法可以保留完整数据项大于等于n个的数据行，其余的数据行将被剔除。\n",
    "\n",
    "我们再创建一个稍大的DataFrame对象用于演示以上方法。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   ColA  ColB  ColC  ColD  ColE\n",
      "0   1.0   2.0   3.0   4.0   5.0\n",
      "1   6.0   NaN   8.0   9.0  10.0\n",
      "2  11.0  12.0   NaN   NaN  15.0\n",
      "3  16.0   NaN   NaN  19.0  20.0\n",
      "4   NaN   NaN   NaN   NaN   NaN\n"
     ]
    }
   ],
   "source": [
    "arr = np.array([[1,2,3,4,5],\n",
    "                [6,np.NaN,8,9,10],\n",
    "                [11,12,np.NaN,np.NaN,15],\n",
    "                [16,np.NaN,np.NaN,19,20],\n",
    "                [np.NaN,np.NaN,np.NaN,np.NaN,np.NaN]])\n",
    "data_7 = pd.DataFrame(arr,columns=['ColA','ColB','ColC','ColD','ColE'])\n",
    "print(data_7)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   ColA  ColB  ColC  ColD  ColE\n",
      "0   1.0   2.0   3.0   4.0   5.0\n"
     ]
    }
   ],
   "source": [
    "data_8 = data_7.dropna()  # 只要数据行中含有空缺值，即剔除\n",
    "print(data_8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   ColA  ColB  ColC  ColD  ColE\n",
      "0   1.0   2.0   3.0   4.0   5.0\n",
      "1   6.0   NaN   8.0   9.0  10.0\n",
      "2  11.0  12.0   NaN   NaN  15.0\n",
      "3  16.0   NaN   NaN  19.0  20.0\n"
     ]
    }
   ],
   "source": [
    "data_9 = data_7.dropna(how='all')  # 剔除全空的数据行\n",
    "print(data_9)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   ColA  ColB  ColC  ColD  ColE\n",
      "0   1.0   2.0   3.0   4.0   5.0\n",
      "1   6.0   NaN   8.0   9.0  10.0\n"
     ]
    }
   ],
   "source": [
    "data_10 = data_7.dropna(subset=['ColC','ColD'])  # 剔除C列或D列中含有空缺值的数据行\n",
    "print(data_10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   ColA  ColB  ColC  ColD  ColE\n",
      "0   1.0   2.0   3.0   4.0   5.0\n",
      "1   6.0   NaN   8.0   9.0  10.0\n"
     ]
    }
   ],
   "source": [
    "data_11 = data_7.dropna(thresh=4)  # 仅保留完整数据项超过4个的数据行，其余数据行将被剔除\n",
    "print(data_11)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2. 更改数据类型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "有的时候某个数据列的数据类型并不是我们想要的类型，例如年龄，实际情况下该列的数据类型应该是整型，但种种因素可能导致年龄列的数据类型为浮点型，为了方便后续数据分析，我们应该对数据列的数据类型根据实际情景进行更改。\n",
    "\n",
    "例如我们把`data_9`的`ColA`列的数据类型更改为整型。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 4 entries, 0 to 3\n",
      "Data columns (total 5 columns):\n",
      " #   Column  Non-Null Count  Dtype  \n",
      "---  ------  --------------  -----  \n",
      " 0   ColA    4 non-null      int32  \n",
      " 1   ColB    2 non-null      float64\n",
      " 2   ColC    2 non-null      float64\n",
      " 3   ColD    3 non-null      float64\n",
      " 4   ColE    4 non-null      float64\n",
      "dtypes: float64(4), int32(1)\n",
      "memory usage: 176.0 bytes\n"
     ]
    }
   ],
   "source": [
    "data_12 = data_9.copy()\n",
    "data_12['ColA'] = data_12['ColA'].astype('int')\n",
    "data_12.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3. 更改列索引名称"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "原始数据中的列名可能使用缩写、中英文混合等方式，这对于我们处理来说并没有太大影响。不过为了了解以及选取数据列更加方便，我们可以通过`rename()`方法修改列索引。\n",
    "\n",
    "例如我们把`data_7`的前两列索引名称修改为`A`和`B`。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "      A     B  ColC  ColD  ColE\n",
      "0   1.0   2.0   3.0   4.0   5.0\n",
      "1   6.0   NaN   8.0   9.0  10.0\n",
      "2  11.0  12.0   NaN   NaN  15.0\n",
      "3  16.0   NaN   NaN  19.0  20.0\n",
      "4   NaN   NaN   NaN   NaN   NaN\n"
     ]
    }
   ],
   "source": [
    "data_13 = data_7.copy()\n",
    "data_13 = data_13.rename(columns={'ColA':'A','ColB':'B'})\n",
    "print(data_13)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 4. 剔除重复数据"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "有一种情况是由于传感器误重复返回数据或者人工录入疏忽，造成数据文件中含有相同的数据行，这会对后期对数据进行数学分析时产生影响，所以我们需要剔除重复的数据行。\n",
    "\n",
    "例如我们有如下数据，Id值为1和2的两个数据行除了Id字段其他字段值均相同，我们可以认为这是两条重复的数据。\n",
    "\n",
    "我们使用`drop_duplicates()`方法剔除重复的数据行。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   Id   Name  Age  Weight\n",
      "0   1    Tom   18    57.2\n",
      "1   2    Tom   18    57.2\n",
      "2   3  Jerry   17    42.7\n",
      "3   4   Eric   19    63.4\n"
     ]
    }
   ],
   "source": [
    "person_list = [{'Id':1,'Name':'Tom','Age':18,'Weight':57.2},\n",
    "               {'Id':2,'Name':'Tom','Age':18,'Weight':57.2},\n",
    "               {'Id':3,'Name':'Jerry','Age':17,'Weight':42.7},\n",
    "               {'Id':4,'Name':'Eric','Age':19,'Weight':63.4}]\n",
    "data_14 = pd.DataFrame(person_list)\n",
    "print(data_14)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   Id   Name  Age  Weight\n",
      "0   1    Tom   18    57.2\n",
      "2   3  Jerry   17    42.7\n",
      "3   4   Eric   19    63.4\n"
     ]
    }
   ],
   "source": [
    "data_15 = data_14.drop_duplicates(subset=['Name','Age'])  # 只要某两行数据的Name和Age字段相同，就认为它们是重复的数据行\n",
    "print(data_15)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们看到行索引也同时被剔除，产生了行索引断层，我们可以用`reset_index()`方法重置行索引。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   index  Id   Name  Age  Weight\n",
      "0      0   1    Tom   18    57.2\n",
      "1      2   3  Jerry   17    42.7\n",
      "2      3   4   Eric   19    63.4\n"
     ]
    }
   ],
   "source": [
    "data_15 = data_15.reset_index()\n",
    "print(data_15)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 步骤3 数据排序"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在Excel中，通过数据排序我们可以快速浏览数据的大致分布情况。Pandas也提供了`sort_values`方法依照某个字段对数据表进行排序。\n",
    "\n",
    "例如我们以花瓣长度（Petal Length）为关键字对鸢尾花数据进行排序。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "     Sepal_Length  Sepal_Width  Petal_Length  Petal_Width     Species\n",
      "22            4.6          3.6           1.0          0.2      setosa\n",
      "13            4.3          3.0           1.1          0.1      setosa\n",
      "14            5.8          4.0           1.2          0.2      setosa\n",
      "35            5.0          3.2           1.2          0.2      setosa\n",
      "2             4.7          3.2           1.3          0.2      setosa\n",
      "..            ...          ...           ...          ...         ...\n",
      "118           7.7          2.6           6.9          2.3   virginica\n",
      "6             4.6          3.4           NaN          0.3      setosa\n",
      "15            NaN          NaN           NaN          0.4      setosa\n",
      "62            6.0          2.2           NaN          1.0  versicolor\n",
      "137           6.4          3.1           NaN          1.8   virginica\n",
      "\n",
      "[152 rows x 5 columns]\n"
     ]
    }
   ],
   "source": [
    "data_16 = iris_data.sort_values(by=['Petal_Length'])\n",
    "print(data_16)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到默认排序按照升序排序，并且空缺值放置在最后，如果需要降序排序，需要设置`ascending`参数为`False`。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "     Sepal_Length  Sepal_Width  Petal_Length  Petal_Width     Species\n",
      "118           7.7          2.6           6.9          2.3   virginica\n",
      "117           7.7          3.8           6.7          2.2   virginica\n",
      "122           7.7          2.8           6.7          2.0   virginica\n",
      "105           7.6          3.0           6.6          2.1   virginica\n",
      "131           7.9          3.8           6.4          2.0   virginica\n",
      "..            ...          ...           ...          ...         ...\n",
      "22            4.6          3.6           1.0          0.2      setosa\n",
      "6             4.6          3.4           NaN          0.3      setosa\n",
      "15            NaN          NaN           NaN          0.4      setosa\n",
      "62            6.0          2.2           NaN          1.0  versicolor\n",
      "137           6.4          3.1           NaN          1.8   virginica\n",
      "\n",
      "[152 rows x 5 columns]\n"
     ]
    }
   ],
   "source": [
    "data_17 = iris_data.sort_values(by=['Petal_Length'],ascending=False)\n",
    "print(data_17)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们还可以根据多个关键字进行排序。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "     Sepal_Length  Sepal_Width  Petal_Length  Petal_Width    Species\n",
      "13            4.3          3.0           1.1          0.1     setosa\n",
      "38            4.4          3.0           1.3          0.2     setosa\n",
      "42            4.4          3.2           1.3          0.2     setosa\n",
      "8             4.4          2.9           1.4          0.2     setosa\n",
      "41            4.5          2.3           1.3          0.3     setosa\n",
      "..            ...          ...           ...          ...        ...\n",
      "117           7.7          3.8           6.7          2.2  virginica\n",
      "122           7.7          2.8           6.7          2.0  virginica\n",
      "118           7.7          2.6           6.9          2.3  virginica\n",
      "131           7.9          3.8           6.4          2.0  virginica\n",
      "15            NaN          NaN           NaN          0.4     setosa\n",
      "\n",
      "[152 rows x 5 columns]\n"
     ]
    }
   ],
   "source": [
    "data_18 = iris_data.sort_values(by=['Sepal_Length','Petal_Length'])\n",
    "print(data_18)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 步骤4 数据筛选"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "之前我们利用切片以及索引器对DataFrame对象进行行列的筛选，我们还可以在索引器中写入判断表达式，更可以配合与`&`、或`|`、非`~`，对数据进行筛选。\n",
    "\n",
    "例如我们对鸢尾花数据集进行筛选，首先筛选出花萼长度小于4.5的数据行："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    Sepal_Length  Sepal_Width  Petal_Length  Petal_Width Species\n",
      "8            4.4          2.9           1.4          0.2  setosa\n",
      "13           4.3          3.0           1.1          0.1  setosa\n",
      "38           4.4          3.0           1.3          0.2  setosa\n",
      "42           4.4          3.2           1.3          0.2  setosa\n"
     ]
    }
   ],
   "source": [
    "data_19 = iris_data.loc[iris_data['Sepal_Length'] < 4.5]\n",
    "print(data_19)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "利用逻辑“与”筛选出花萼长度小于4.5且花萼宽度等于3.0的数据行："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    Sepal_Length  Sepal_Width  Petal_Length  Petal_Width Species\n",
      "13           4.3          3.0           1.1          0.1  setosa\n",
      "38           4.4          3.0           1.3          0.2  setosa\n"
     ]
    }
   ],
   "source": [
    "data_20 = iris_data.loc[(iris_data['Sepal_Length'] < 4.5) & (iris_data['Sepal_Width'] == 3.0)]\n",
    "print(data_20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "利用逻辑“非”筛选出除了setosa品种的数据行："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "     Sepal_Length  Sepal_Width  Petal_Length  Petal_Width     Species\n",
      "50            7.0          3.2           4.7          1.4  versicolor\n",
      "51            6.4          3.2           4.5          1.5  versicolor\n",
      "52            6.9          3.1           4.9          1.5  versicolor\n",
      "53            5.5          2.3           4.0          1.3  versicolor\n",
      "54            6.5          2.8           4.6          1.5  versicolor\n",
      "..            ...          ...           ...          ...         ...\n",
      "147           6.5          3.0           5.2          2.0   virginica\n",
      "148           6.2          3.4           5.4          2.3   virginica\n",
      "149           5.9          3.0           5.1          1.8   virginica\n",
      "150           5.9          3.0           5.1          1.8   virginica\n",
      "151           6.2          3.4           5.4          2.3   virginica\n",
      "\n",
      "[102 rows x 5 columns]\n"
     ]
    }
   ],
   "source": [
    "data_21 = iris_data.loc[iris_data['Species'] != 'setosa']\n",
    "print(data_21)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们也可以使用`query()`方法利用查询字符串进行快速筛选（注意，逻辑与使用`and`；逻辑或使用`or`；逻辑非使用`not`）。\n",
    "\n",
    "例如筛选出花瓣长度大于6.0或花瓣宽度等于0.1的数据行："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "     Sepal_Length  Sepal_Width  Petal_Length  Petal_Width    Species\n",
      "9             4.9          3.1           1.5          0.1     setosa\n",
      "12            4.8          3.0           1.4          0.1     setosa\n",
      "13            4.3          3.0           1.1          0.1     setosa\n",
      "32            5.2          4.1           1.5          0.1     setosa\n",
      "34            4.9          3.1           1.5          0.1     setosa\n",
      "37            4.9          3.1           1.5          0.1     setosa\n",
      "105           7.6          3.0           6.6          2.1  virginica\n",
      "117           7.7          3.8           6.7          2.2  virginica\n",
      "118           7.7          2.6           6.9          2.3  virginica\n",
      "122           7.7          2.8           6.7          2.0  virginica\n"
     ]
    }
   ],
   "source": [
    "data_22 = iris_data.query('Petal_Length > 6.5 or Petal_Width == 0.1')\n",
    "print(data_22)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 步骤5 数据描述与简单分析"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在对较大的数据表进行分析时，我们可以利用简单的统计方法对数据做大体描述。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* `describe()`方法可以对DataFrame对象做一个统计意义上的大致描述，例如我们对鸢尾花DataFrame对象使用`describe()`方法："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "       Sepal_Length  Sepal_Width  Petal_Length  Petal_Width\n",
      "count    151.000000   149.000000    148.000000   148.000000\n",
      "mean       5.847020     3.048993      3.796622     1.196622\n",
      "std        0.825737     0.421166      1.758146     0.767132\n",
      "min        4.300000     2.000000      1.000000     0.100000\n",
      "25%        5.100000     2.800000      1.600000     0.300000\n",
      "50%        5.800000     3.000000      4.400000     1.300000\n",
      "75%        6.400000     3.300000      5.100000     1.800000\n",
      "max        7.900000     4.200000      6.900000     2.500000\n"
     ]
    }
   ],
   "source": [
    "print(iris_data.describe())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "其中`count`表示非空数据数目；`mean`表示该列均值；`std`表示该列标准差；`min`和`max`表示该列最小值和最大值；`25%`，`50%`，`75%`表示第一四分位数（升序排列后第25%的数值）、中位数、第三四分位数。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* 累计方法\n",
    "\n",
    "Pandas提供了一些累计方法，供快速数学计算使用，包括\n",
    "\n",
    "| 指标 | 含义 |\n",
    "| --- | --- |\n",
    "| `count()` | 计数 |\n",
    "| `first()` | 第一项 |\n",
    "| `last()` | 最后一项 |\n",
    "| `mean()` | 均值 |\n",
    "| `median()` | 中位数 |\n",
    "| `min()` | 最小值 |\n",
    "| `max()` | 最大值 |\n",
    "| `std()` | 标准差 |\n",
    "| `var()` | 方差 |\n",
    "| `mad()` | [均值绝对偏差](https://baike.baidu.com/item/%E5%B9%B3%E5%9D%87%E7%BB%9D%E5%AF%B9%E8%AF%AF%E5%B7%AE/9383373?fr=aladdin) |\n",
    "| `prod()` | 所有项累乘 |\n",
    "| `sum()` | 所有项累加 |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 步骤6 数据汇总"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "回忆一下我们在实验8-1中学习的简单SQL语句，利用`GROUP BY`关键字配合聚合函数（例如`AVG()`)就可以按某个字段进行分类汇总。\n",
    "\n",
    "Pandas也提供了`groupby()`方法实现类似的功能。需要注意的是`groupby()`方法返回的不是一个DataFrame对象，而是一个DataFrameGroupBy对象。\n",
    "\n",
    "例如我们先按鸢尾花种类求出各个指标的平均值："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "            Sepal_Length  Sepal_Width  Petal_Length  Petal_Width\n",
      "Species                                                         \n",
      "setosa          4.991837      3.40625      1.464583     0.244000\n",
      "versicolor      5.936000      2.77000      4.265306     1.317021\n",
      "virginica       6.554902      2.98200      5.532000     2.024000\n"
     ]
    }
   ],
   "source": [
    "data_23 = iris_data.groupby('Species').mean()\n",
    "print(data_23)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "按鸢尾花种类对数据行计数："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Species\n",
      "setosa        50\n",
      "versicolor    50\n",
      "virginica     51\n",
      "Name: Species, dtype: int64\n"
     ]
    }
   ],
   "source": [
    "data_24 = iris_data.groupby('Species')['Species'].count()\n",
    "print(data_24)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 实验10-2 数据科学过程示例"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1912年4月15日，在泰坦尼克号首次航行期间撞上冰山后沉没，2224名乘客和机组人员中有1502人遇难。这场轰动的悲剧震撼了国际社会，并导致了更好的船舶安全条例。\n",
    "\n",
    "![Titanic](img/Lab10-2-1.jpg)\n",
    "\n",
    "海难导致生命损失的原因之一是没有足够的救生艇给乘客和机组人员。虽然幸存下来的运气有一些因素，但一些人比其他人更有可能生存，比如妇女，儿童和上层阶级。\n",
    "\n",
    "我们以泰坦尼克号数据集作为示例，大致了解数据科学过程。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 步骤1 数据采集"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们从[Kaggle](https://www.kaggle.com/c/titanic/data?select=train.csv)上下载泰坦尼克号数据集的CSV文件并保存至本地。\n",
    "\n",
    "Kaggle是一个流行的数据科学竞赛平台，由 Goldbloom 和 Ben Hamner 创建于 2010 年，在这个平台上，企业和研究者可在其上发布数据和问题，并提供奖金给能解决问题的人。多个数据分析相关的能人志士在其上进行竞赛以产生最好的模型来解决问题。进而使数据科学成为一场运动。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 步骤2 数据预处理"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 1. 快速了解数据"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们先通过Pandas加载CSV文件，查看数据集中有哪些字段："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 891 entries, 0 to 890\n",
      "Data columns (total 12 columns):\n",
      " #   Column       Non-Null Count  Dtype  \n",
      "---  ------       --------------  -----  \n",
      " 0   PassengerId  891 non-null    int64  \n",
      " 1   Survived     891 non-null    int64  \n",
      " 2   Pclass       891 non-null    int64  \n",
      " 3   Name         891 non-null    object \n",
      " 4   Sex          891 non-null    object \n",
      " 5   Age          714 non-null    float64\n",
      " 6   SibSp        891 non-null    int64  \n",
      " 7   Parch        891 non-null    int64  \n",
      " 8   Ticket       891 non-null    object \n",
      " 9   Fare         891 non-null    float64\n",
      " 10  Cabin        204 non-null    object \n",
      " 11  Embarked     889 non-null    object \n",
      "dtypes: float64(2), int64(5), object(5)\n",
      "memory usage: 83.7+ KB\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import math\n",
    "\n",
    "data = pd.read_csv('data/titanic-train.csv')\n",
    "data.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* PassengerId：乘客的id号\n",
    "* Survived：是否存活，1表示人幸存；0表示遇难\n",
    "* Pclass：船舱等级，分为头等舱，二等舱和三等舱\n",
    "* Name：姓名\n",
    "* Sex：性别\n",
    "* Age：年龄\n",
    "* SibSp：该乘客在船上的同伴人数\n",
    "* Parch：该乘客在船上的家属人数\n",
    "* Ticket：船票编号\n",
    "* Fare：船票价格\n",
    "* Cabin：房间编号\n",
    "* Embarked：目的地码头代码"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们再看一下原始数据的简单统计信息："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>891.000000</td>\n",
       "      <td>891.000000</td>\n",
       "      <td>891.000000</td>\n",
       "      <td>714.000000</td>\n",
       "      <td>891.000000</td>\n",
       "      <td>891.000000</td>\n",
       "      <td>891.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>446.000000</td>\n",
       "      <td>0.383838</td>\n",
       "      <td>2.308642</td>\n",
       "      <td>29.699118</td>\n",
       "      <td>0.523008</td>\n",
       "      <td>0.381594</td>\n",
       "      <td>32.204208</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>257.353842</td>\n",
       "      <td>0.486592</td>\n",
       "      <td>0.836071</td>\n",
       "      <td>14.526497</td>\n",
       "      <td>1.102743</td>\n",
       "      <td>0.806057</td>\n",
       "      <td>49.693429</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.420000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>223.500000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>20.125000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>7.910400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>446.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>28.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>14.454200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>668.500000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>38.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>31.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>891.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>80.000000</td>\n",
       "      <td>8.000000</td>\n",
       "      <td>6.000000</td>\n",
       "      <td>512.329200</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       PassengerId    Survived      Pclass         Age       SibSp  \\\n",
       "count   891.000000  891.000000  891.000000  714.000000  891.000000   \n",
       "mean    446.000000    0.383838    2.308642   29.699118    0.523008   \n",
       "std     257.353842    0.486592    0.836071   14.526497    1.102743   \n",
       "min       1.000000    0.000000    1.000000    0.420000    0.000000   \n",
       "25%     223.500000    0.000000    2.000000   20.125000    0.000000   \n",
       "50%     446.000000    0.000000    3.000000   28.000000    0.000000   \n",
       "75%     668.500000    1.000000    3.000000   38.000000    1.000000   \n",
       "max     891.000000    1.000000    3.000000   80.000000    8.000000   \n",
       "\n",
       "            Parch        Fare  \n",
       "count  891.000000  891.000000  \n",
       "mean     0.381594   32.204208  \n",
       "std      0.806057   49.693429  \n",
       "min      0.000000    0.000000  \n",
       "25%      0.000000    7.910400  \n",
       "50%      0.000000   14.454200  \n",
       "75%      0.000000   31.000000  \n",
       "max      6.000000  512.329200  "
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们可以对数据有大概的认识：\n",
    "\n",
    "共有891条数据，乘客的平均年龄在30岁左右，沉船后乘客的生还率为38%。\n",
    "\n",
    "其中年龄（Age）、房间编号（Cabin）列中含有较多空缺值，并且某些字段例如姓名（Name）、船票编号（Ticket）等对数据分析并没有什么作用，我们需要对原始数据进行预处理工作。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2. 删除无关字段"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "某些字段对我们的数据分析并不会产生贡献，为了后续处理步骤的简洁性，我们对乘客编号、姓名等字段进行剔除。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(891, 12)\n"
     ]
    }
   ],
   "source": [
    "print(data.shape)  # 查看原始数据形状"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 891 entries, 0 to 890\n",
      "Data columns (total 7 columns):\n",
      " #   Column    Non-Null Count  Dtype  \n",
      "---  ------    --------------  -----  \n",
      " 0   Survived  891 non-null    int64  \n",
      " 1   Pclass    891 non-null    int64  \n",
      " 2   Sex       891 non-null    object \n",
      " 3   Age       714 non-null    float64\n",
      " 4   SibSp     891 non-null    int64  \n",
      " 5   Parch     891 non-null    int64  \n",
      " 6   Fare      891 non-null    float64\n",
      "dtypes: float64(2), int64(4), object(1)\n",
      "memory usage: 48.9+ KB\n"
     ]
    }
   ],
   "source": [
    "data = data.drop(columns=['PassengerId','Name','Ticket','Cabin','Embarked'])\n",
    "data.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3. 处理缺失值"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "刚才我们看到，年龄（Age）字段含有较多空缺值，一种解决方法是用平均年龄填充空缺值。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 891 entries, 0 to 890\n",
      "Data columns (total 7 columns):\n",
      " #   Column    Non-Null Count  Dtype  \n",
      "---  ------    --------------  -----  \n",
      " 0   Survived  891 non-null    int64  \n",
      " 1   Pclass    891 non-null    int64  \n",
      " 2   Sex       891 non-null    object \n",
      " 3   Age       891 non-null    float64\n",
      " 4   SibSp     891 non-null    int64  \n",
      " 5   Parch     891 non-null    int64  \n",
      " 6   Fare      891 non-null    float64\n",
      "dtypes: float64(2), int64(4), object(1)\n",
      "memory usage: 48.9+ KB\n"
     ]
    }
   ],
   "source": [
    "data['Age'] = data['Age'].fillna(data['Age'].mean())\n",
    "data.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 4. 检查字段取值范围"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们再检查一下其他字段的取值，是否有不符合实际情况的取值："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0 1]\n"
     ]
    }
   ],
   "source": [
    "print(data['Survived'].unique())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[3 1 2]\n"
     ]
    }
   ],
   "source": [
    "print(data['Pclass'].unique())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['male' 'female']\n"
     ]
    }
   ],
   "source": [
    "print(data['Sex'].unique())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.42\n",
      "80.0\n"
     ]
    }
   ],
   "source": [
    "print(data['Age'].min())\n",
    "print(data['Age'].max())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 步骤3 数据分析 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 1. 全体乘客幸存情况"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Survived\n",
      "0    549\n",
      "1    342\n",
      "Name: Survived, dtype: int64\n"
     ]
    }
   ],
   "source": [
    "survive = data.groupby('Survived')['Survived'].count()\n",
    "print(survive)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2. 乘客年龄分布"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在这里我们将乘客年龄分为4档：0-15岁为儿童；16-30岁为青年；31-59岁为中年；60岁及以上为老年。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'Child': 83, 'Youth': 503, 'Adult': 279, 'Elder': 26}\n"
     ]
    }
   ],
   "source": [
    "age_distributions = {'Child':0, 'Youth':0, 'Adult':0, 'Elder':0}\n",
    "for age in data['Age']:\n",
    "    if age <= 15:\n",
    "        age_distributions['Child'] += 1\n",
    "    elif age <= 30:\n",
    "        age_distributions['Youth'] += 1\n",
    "    elif age <= 59:\n",
    "        age_distributions['Adult'] += 1\n",
    "    else:\n",
    "        age_distributions['Elder'] += 1\n",
    "print(age_distributions)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3. 乘客舱位等级分布"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pclass\n",
      "1    216\n",
      "2    184\n",
      "3    491\n",
      "Name: Pclass, dtype: int64\n"
     ]
    }
   ],
   "source": [
    "cabin_level = data.groupby('Pclass')['Pclass'].count()\n",
    "print(cabin_level)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 4. 乘客舱位与存活率"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pclass\n",
      "1    136\n",
      "2     87\n",
      "3    119\n",
      "Name: Survived, dtype: int64\n"
     ]
    }
   ],
   "source": [
    "cabin_survive = data.loc[data['Survived'] == 1].groupby('Pclass')['Survived'].count()\n",
    "print(cabin_survive)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pclass\n",
      "1     80\n",
      "2     97\n",
      "3    372\n",
      "Name: Survived, dtype: int64\n"
     ]
    }
   ],
   "source": [
    "cabin_not_survive = data.loc[data['Survived'] == 0].groupby('Pclass')['Survived'].count()\n",
    "print(cabin_not_survive)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 5. 乘客性别与存活率"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sex\n",
      "female    233\n",
      "male      109\n",
      "Name: Survived, dtype: int64\n"
     ]
    }
   ],
   "source": [
    "sex_survive = data.loc[data['Survived'] == 1].groupby('Sex')['Survived'].count()\n",
    "print(sex_survive)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sex\n",
      "female     81\n",
      "male      468\n",
      "Name: Survived, dtype: int64\n"
     ]
    }
   ],
   "source": [
    "sex_not_survive = data.loc[data['Survived'] == 0].groupby('Sex')['Survived'].count()\n",
    "print(sex_not_survive)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们可以从更多角度来挖掘数据。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 步骤4 数据可视化"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "利用图表可以非常直观地显示之前我们进行数据分析的结果。\n",
    "\n",
    "* 例如我们利用饼图显示全体乘客的幸存情况："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAOcAAADnCAYAAADl9EEgAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAAZkUlEQVR4nO3deZhU1Z3/8fe3t2qBtjG0G7iUZtyXkLhvEZ2Mzi+liXGLT8C4R9Q8oibkV5NELcmMVjSJWxKjJi7gJKImYQilRBBBAi6MCogmLkgpiGsjRQO9Vp3549yObdtNb1V1zq36vp6nnq4u6tb5VtOfPvfeuuccMcaglPJPhesClFI903Aq5SkNp1Ke0nAq5SkNp1Ke0nAq5SkNp1Ke0nAq5SkNp1Ke0nAq5SkNp1Ke0nAq5SkNp1Ke0nAq5SkNp1Ke0nAq5SkNp1Ke0nAq5SkNp1Ke0nAq5SkNp1Keyms4RcSIyM+7fP99EUn0sc0pIrJvPuvoi4g8KiIj8/A6CRH5fh5KUuoz8t1ztgKnikjDALY5Bch7OEWksrd/M8Z81RizPt9tKpVP+Q5nB3AXcGX3fxCRXUXkCRFZHnzdRUSOBL4G3CQiS0Xk8922OUNEVojIMhF5KnjsXBH5ZZfnzBKRccH9jSIyRUSeBX4oIg91ed44EflLcD8tIg0i8lMRubTLcxIi8r3g/mQRWRLUe12X5/xIRF4VkbnAXkP/kSnVs6oCvOavgOUicmO3x38JTDXG3C8i5wO3GWNOEZGZwCxjzCM9vNY1wInGmHf6uRs6HFhhjLlGRKqAN0VkuDFmE/BNYHq35z8I3AL8Ovj+TODfReQEYA/gUECAmSLyZWATcBbwRezP7gXg+X7UpdSA5f2EkDFmAzAVuLzbPx0B/D64Pw04uh8vtwi4T0QuAnrdTe0iC/wxqKMDmA2cHAQ1BvxPt1pfBLYTkdEi8gXgY2PM28AJwe1FbAD3xob1GODPxpjNwfuc2Y+alBqUQvScYHujF4B7t/CcPteBMMZMFJHDsMFaKiJjsbvOXf+o1Ha532KMyXb5fjpwGbAOWGKMaeqhmUeA04EdsD0p2N7yBmPMnV2fKCJX9KdupfKhIOE0xqwLjvcuAO4JHl6M3SWcBowH/hY83gTU9fQ6IvJ5Y8yzwLMicjKwM5AGLhWRCmAMdtezN/OB3wEX8dld2k4PAncDDcCxwWN/BX4iIv9tjNkoImOAduApbE+exP7sTgbu7OE18yYaTw0HdgWiXb5GgdFAdfA0CW708rUJeDe4rQ2+rgZWAmvTyZj+wfFQoXpOgJ8D3+3y/eXAPSIyGfgQOC94/EHgbhG5HDjdGLOyyzY3icge2F+wJ4BlweOrgJeAFdgeukfGmKyIzALOBc7p5Tkvi0gd8I4x5t3gscdFZB/gaREB2AhMMMa8ICLTgaXAW8DC/vwg+iMaT40BxmGPZ6N8EsaBnPkejM3ReOpNbFD/jj2UWJhOxjIFblf1QXSVMTei8dRO2DB23j6/hacXWw5YDiwIbk+lk7FGtyWVHw1nkUTjqZ2B47C7zuOA3Z0WNDAGeIUgqMCCdDL2ntuSSp+Gs4CCQI4Pbvs7LiefDPZ4/l7gj+lkbLPbckqThjPPovHU1tjPVCdgP3qRLW8RehuAh4F708nYItfFlBINZ55E46lDgInYM9LDHJfjyuvAfcDUdDK2xnEtoafhHIJoPFULnA1cDBzkuByf5IC52Es5/6Qf1QyOhnMQovFUFXA+9vLCMY7L8d0y4Op0MvYX14WEjYZzAKLxlGCPJ6dgL+dT/fc08MN0MjbfdSFhoeHsp2g89f+A64GxjksJu7nYkC5xXYjvNJx9iMZTRwE3YM+8qvyZgd3dXeG6EF9pOHsRjaf2AW4ETnJdSwnLYUcqTdaLGj5Lw9lNcFx5FfBfQMRxOeViHXBlOhmb6roQn2g4u4jGU7sA92Mvr1PFlwIuTidj77guxAc6+14gGk99G3ux9zjHpZSzGPBy8H9R9sq+54zGU6OwYzJPc12L+pQ/AJeU89C1sg5n8PHIPdhZEJR/0sD4dDK22HUhLpRlOKPxVAS4GbjEdS2qT1ngmnQydr3rQoqt7MIZjadGYif6+rLjUtTA3IM9WdThupBiKatwBuMrHwP2c12LGpTZwBnpZGyj60KKoWzCGY2n9scGcyfXtagheR6IpZOx910XUmhl8VFKNJ4ah53tT4MZfgcBT0fjqZKfbb/kwxmNp87E7g7Vu65F5c1uwKJoPHWk60IKqaTDGY2nrsBOvamX4ZWeUcAT0XjqVNeFFErJhjMaT92E/bik1OfwKWe1wMPReOoi14UUQkmeEIrGUz8Bfuy6DlU0OeC0dDI2w3Uh+VRy4YzGU5cDt7quQxVdM/Cv6WTsadeF5EtJhTMaT43HrsWiu7LlqRE4Mp2Mvea6kHwomXBG46l/xy7JV93Xc1VJW4UNaOgHb5dEOIMLDBbTy2plquy8ABwb9iuJQn+2NhpPbQfMQoOpPvEl4JFgCtPQCnU4g0mdZ2CXylOqqxOB37ouYihCHU7sordHuC5CeeucaDz1H66LGKzQHnNG46kJ2DOz3si1bKTxsdto++htABq+OomOpkYyf/s97Y2r2eHbvyCyY89zUfe0bWTMPnw8/16a33yemu12o+Gk7wGwccU8ci1NbH3w14vzxsKtHTg8nYz1usiyr0LZcwYLz97uuo7u1j1xF7W7H8SYi37D6PNvp3rUztQ07Mq23/ghkZ23PEqtp21zrZtofefvjD7/lxiTo+3DNLn2VjatmEvdF2NFelehVw1MCw6BQiV04QymrrwHGOm4lE/JtW6mZfXLjDjwBACkspqK2hFUN+xM9agtD4bpbVsQTLYDYwymow2pqGTDc3+i7qCvIZWhPtdRbPtiZ+sPldCFEzu1yL+5LqK7jvXvUTlsaxofvYW1915O42O3kWtrGdK2FZFhDNvrSN6973Kq6rdHIsNpe/c1hu1xeIHfTUm6IhpPHee6iIEIVTij8dS/ADe5rqMnJpel7b2V1H3xq4w+7zakOsKGZx4e8rb1h53O6PNu53PHX0hm4QOMPGYCTcv+yoczkqxf/GAh31KpEeC+aDwVmqGDoQlnNJ6qxE747OXCtFV1DVTWNRAZbccAD9vrKNreX5m3bTu/r9pmDJtWzGPbU+K0f/gW7et0/uUB2AUPz1X0JjThBCYD3g6urRyxDVVbN9DeaBd0bnlrGdUNu+Rt2/ULH6D+6PGQ6wCTsw9KBaajNX9vojycHY2nQjFHcSg+SonGUwcCS4Aa17VsSdv7b9I4+zZMtoOqkTsw6qtX0Pr2ctbNuZNsc4aKyAhqttuN7b/5EzqaGmmcfRvbn3Fdr9tW1o4AYPNrT9P2wSpGHv0tAD6e9zuaV71A9XZRtj15srP3G2KNwP6+X3/rfTiDs7NL0GXdVX49kE7GznZdxJaEYbf2NDSYKv/GR+OpL7kuYku8DmdwEmiK6zpUSRI8PfPfyetwAhOAfVwXoUrW8dF4yttLrbw95ozGU9XAq9hpEJUqlFeAA9LJWM51Id353HNeiAZTFd6+wFmui+iJlz1nNJ7aCngDGO26FlUW/gHs51vv6WvPeRkaTFU8e+Nh7+ldzxmNp+qwkzSNcl2LKive9Z4+9pyT0GCq4tsb+IbrIrryKpzReKoC+I7rOlTZOt91AV15FU7geGBn10WosnViNJ7y5lyHb+E813UBqqxVAt92XUQnb8IZjae2xrN9flWWznNdQCdvwgmciacDqVVZ2dOXRXl9Cue5rgtQKuBF7+nF55zB3ECvu65DqUATsEM6Gdvssghfes5zXRegVBd1wOmui3AezuCzTW/OkCkVcL5r6zycwJfRzzaVf44NVrBzxodwnui6AKV6INiOwxkfwvkV1wUo1YtjXTbuNJzReGob7EKnSvmofMOJvZbWdQ1K9Wb/aDzlbISU62Ac77h9pbZEgGNcNe46nEc5bl+pvoxz1bCzcEbjqRHA/q7aV6qfnB13uuw5D8EO0VHKZwdG46mRLhp2Gc4jHLatVH9V4Oi402U4D3PYtlID4aQjcRnOPR22rdRAOJnc3GU49XpaFRa7umjUSTij8dTngOEu2lZqEMonnED/1mNXyg87RuOpoq+qruFUqm+Cg99ZDadS/VP0XVsNp1L9o+FUylMaTqU8peFUylNlE05d4k+FzdbFbtD1eE6lwqK62A26Cqf7aeaVGhgNp1Ke0nAq5amih7Oq2A0GNJx5Z8zU6uRTR1esOMB1JaUohzTBx0VtU8NZAqroaH+sJv7cHhVrnc6zWsoqME3FblPDGXLDaW5aELny9QbZoDMZFla22A1qOENse9Z98GTkex8Pk1adNb/w2ordoJ4QCqk9ZfWqv0UmtQ2T1r1c11ImGovdoKtwFn0XoZQcVbFixeyaeH21ZHdyXUsZKZtwvueo3dA7vXLBcw9UX797hZjPua6lzHxU7AZdhTPtqN1Qu7Lq4YU3Vd15kAjDXNdShorec7o6IZR21G5o3V592/yTK58Z57qOMlb0nlPD6bkKctk/1Vy7eGzFynGuaylzZdNzrnLUbqjU0to8NzJ5+U7ykbNl6NQ/vVXsBrXn9NRImj5eELlyTb1s1mUr/PBqsRvUcHpoF3l/zZyayW0R6dDrZP2wGVhT7EadnK1NJ2ONQNGvVQyDsfLGq0/WXFUdkY7dXdei/uk1EpmiXzjjciaEou/D++6EiiUv/rnmmh0rxWzvuhb1KUXfpQW34Uw7bNs751c+tvjO6pv3Eyn+XDWqT2UXzpcdtu2Vn1Tds+DqqmlHiFD09ThUv7ziolFXJ4QAFjls2xPGTKtOPnVM5Us6DtNvi1006jKci7GjU8RhDc7oAOnQWE0is9pFw852a4Mztk725V0bTnPTM5HvvrRHxVodIO0/Z3t4ruetLbtd2+1Z98GSyKVrG2SDDpAOh7IN51OO2y8qHSAdSmUbzrmO2y8aHSAdSk3AcleNOw1nOhlbSxl8pKIDpEPrcRIZZ7N2uO45AR53XUAh6QDpUPsfl437EM45rgsolNurb5s/qerPx4hQ6boWNWAdwCyXBbj8nLPTfGAjMMJxHXmjA6RLwkISmeJO8d6N854znYw1A9Nd15EvtbQ2L4xMen5sxUodIB1uM1wX4DycgXtcF5AP27Bh3XORy94YI42Huq5FDZnT403wJJzpZGwx8A/XdQzFLvL+mmci312/tWzWAdLht5hExvmQRi/CGfid6wIGSwdIl5y7XRcAfoVzKvYMWajoAOmSkwEecl0EeBTOdDL2AY5PXQ+UDpAuSf9NIrPZdRHgUTgDoTkxpAOkS5YXu7Tgx+ecXT0KvAvs6LqQ3ukA6RL2vyQyS10X0cmrnjOdjGWxx55eqqajbU7N5MUazJJ1q+sCuvIqnIFf42Ch0r4Mp7np6ch3V+gA6ZK1EviD6yK68i6c6WTsbeA3ruvoSgdIl4WkyxEoPfEunIH/xF5v65wOkC4Lq4H7XRfRnZfhTCdjHwK/cF2HDpAuGz8lkWl3XUR3XoYz8HMcrInYSQdIl4338PTqNG/DmU7GNgA3uGhbB0iXlWtJZFpcF9ETb8MZ+BX2eKBodIB0WXkB+K3rInrjdTjTyVgrkChGWxXksjNqrl6oS7uXlUkkMjnXRfTG63AG7qfAw8l0gHRZ+gOJzN9cF7ElYkzRlx0csGg8dRLwl0K89jZsWLcgctU7pTgOs6XD8OV7N9GahY4cnL5PFdcdV8vS97JMnNVCS4ehqgJ+HduKQ8f0vBefzRkOvnsTY+oqmPUtewj+/+e08NgbHYzdoZKp39gKgGnL2ljXbJh0eKRo728INgN7kcgUfUHcgQhDz0k6GZsF3Jvv1y31AdKRSph3znCWTRzB0ouHM3tlB8+s6eAHc1q49tgalk4cwZTjIvxgTu/nQ259to19Gj75Ncm0GBavybL8khFkjeGl97M0txvuW9bOpYeEZgzA9b4HE0ISzsAkYFW+XqwcBkiLCCNq7DpR7Tloz9pVo0RgQ6t9TqYFRtf1vJbUmg05Uq93cOGXPgldhUBb1mCMobkdqivhpsVtXH5oDdWVoViTailwo+si+iM04UwnY03A2cCQD+DLaYB0NmcY+5uNbHdTE/+2exWH7VTFLSfWMnlOCzvf3MT357Rww7/W9rjtFbNbuPErtVR0yVxdRDhtn2q+eOcmdhtZQX1EWLI2y9f3ri7SOxqSduAcHy846ElowgmQTsYWAT8dymuU2wDpygph6cQRrLmqjufWZlnxQZY7/redm0+sZfWVddx8Yi0XzGz+zHazXmtnu+HCQaM/eyz6g6MiLJ04gp+fWMvVT7YyZVyE377QxpkPb+Y/n2otxtsarCkkMs6WVxioUIUzcC3w4mA2LOcB0iNrhXG7VjH7jQ7uX9bGqfvYobxn7FvFc+989nrvRW9nmflqB9FbmjjrkWbmrepgwp8+HeIX37Xb7TmqgqnL2nnojGGs+CDL641eXT/eaRGOLmoZrNCFM52MtQMTgAFc1WHMtOobFpxdNfdYkfJZrPfDTTnWt9iz8c3thrmrOti7oYLRdRUseMsGaN6qLHuM+uyvwQ1fqWXNVXWkr6jjwdO34vjdqnjg1K0+9Zyrn2xlynER2nOQDU76Vwhs9m+ncQMwwbdRJ33xbSaEfkknY69E46k4cEtfz62mo+3RmviSclxB+t2NhnNmbCabg5yBM/er5qQ9qxlZK0ya3UJHDmqr4K6TbOjWNuW4cGYLj47v+6rFGf9o55DRlYyus8E+YqdKDrhjIwduX8EXdvDu4qoLSGTSrosYqFB8ztmTaDwl2EWQvtLbc4bT3LQgcuXrOg6zrF1PIvMj10UMRuh2azulkzGD3b3tcfJfHSCtgBRwtesiBiu0PWenaDy1H/Zgv77zsT1l9apUzQ+rdRxmWXsVOIxEJuO6kMEKbc/ZKZ2MvQycTjAhtQ6QVtgTQKeEOZhQAuEESCdjc4FLdIC0wk4OdyaJTKjX3oES2K3tylxbnxDhWtd1KGeywFkkMo+4LiQfSiqcACTq7wAmui5DFZ3BfmSS9wESrpTEbm03lwG/d12EKrqrSimYUIrhtCPbz6YAQ8yUt64jkbnFdRH5VnrhhM6AXoCdPV6Vth+TyCRcF1EIpXfM2V2i/mfA91yXofIuB1xCInOX60IKpfTDCZCon0KIrxRRn9EKjCeR+aPrQgqpPMIJkKifCNxOSC/2V//UhL3AYJ7rQgqtfMIJkKgfBzwCjHJciRqc1dhgvuC6kGIozRNCvUlk5gOHAq84rkQN3DzgoHIJJpRbOAESmTeBw4FZrktR/XYjcAKJzIeuCymm8tqt7SpRXwH8CLgGPQ711UbgvFK5HG+gyjecnRL1hwEPAP/iuhT1KUuBb5HI/N11Ia6U325td4nMs8BYPF7Qpsx0AFOAQ8s5mKA956cl6k8B7gYaHFdSrpYD55LIDGp2xVKjPWdXicwMYF/sYqr6V6t4OoD/Ag7RYH5Ce87eJOoPx64PqnMQFdYT2BEloZnsuVg0nFtiz+h+B/tXXWdXyK/XgO+TyBRk9bhSoOHsj0R9A/Bj4GKg54VFVH99jD3h86uwrFniioZzIBL1OwJxbG+qIR2YDdghfD8jkWl0XUwYaDgHQ0M6EB8ANwN3hH02vGLTcA6FDemlwIXADo6r8U0auAm4h0RmAOvaqE4aznxI1FcDpwCXAMe5LcapHHaJjLuBmSQyHY7rCTUNZ74l6vfGzv73LWBbx9UUy0pgGnAfiUyPy2OogdNwFkqivhI4BjgNOBUY7bagvFuFHdkznURmketiSpGGsxgS9YIdpnYa8DVgD7cFDUoWeBobyL+QyOiY2ALTcLqQqN8Ze2x6HHAUfoa1DTsy5BlsKB8nkVnntKIyo+H0QaJ+W+AI4ADstb37AnsBW21pszzajD1ufBl4FhvIF0lkWof6wiLyI+zxdxZ7wuhiY8yzQ3zNrwH7GmOSeahvozFmxFBfpxA0nL6ylw7uhg3q7sB2wPbB18772wIR7ACGngYxNAPrsVflrA9u67Brmq4E3gDeIJF5txBvQUSOAH4BjDPGtIpIA1BjjFnbj22rjDEFP9vrczh1BgBf2YmxVwa3fm5TXwl03rIkMm2FKa7fdgQ+Msa0AhhjPgIQkTRwsDHmIxE5GPiZMWaciCSwJ86iwEci8nngfGPMy8F287FzEB8AHIydyWIZsLsxJiciw7Drcu4O7IIduLAtds/gImPMP0RkN+xyHVXA7IL/BIZAw1lKEpksdvfRF48D14jIa8BcYLoxZkEf2xwEHG2MaRaRK4EzgWtFZEdgtDHmeRE5AMAYkxGRZcCxwJPAycBfjTHtInIXMNEY87qIHIa9dPB44FbgDmPMVBG5rADvOW90PKcqGGPMRmzYvgN8CEwXkXP72GymMaY5uP8QcEZw/0zg4R6ePx34ZnD/rKCNEcCRwMMishS4E9uLgz0B94fg/rSBvJ9i055TFZQxJgvMB+aLyEvAOdjB1Z0dQ/drkzd12fYdEWkUkQOxAby4hyZmAjeIyOewfwjmAcOB9caYsb2VNbh3U1zac6qCEZG9RKTrx0RjsSej0tgggf3sd0seBH4A1BtjXur+j0Hv/Bx2d3WWMSZrjNkArBKRM4I6RES+EGyyCNvDAowf8JsqIg2nKqQRwP0i8oqILMeeeU4A1wG3ishC+j5GfgQbpoe28JzpwITga6fxwAXBMenLwNeDxycBl4nIEqB+YG+nuPSjFKU8pT2nUp7ScCrlKQ2nUp7ScCrlKQ2nUp7ScCrlKQ2nUp7ScCrlKQ2nUp7ScCrlKQ2nUp7ScCrlKQ2nUp7ScCrlKQ2nUp7ScCrlKQ2nUp7ScCrlKQ2nUp7ScCrlKQ2nUp7ScCrlqf8D4hAO/t0n9RQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.pie(survive,labels=['Not survived','Survived'],autopct='%.1f%%')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* 利用柱状图显示乘客的年龄分布："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD4CAYAAAAXUaZHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAAP30lEQVR4nO3df6zd9V3H8eeLMoEM5mBcsNLqJUvVFbLBVtkMGtlYpBO1EMGU6OwMs1nG4kxcXJl/jP3R2MQ4p3FkaeZcl027ZohUUBx2w8kyVy6sMgpj1MGgaUPvWHS/kEn39o/z7Ty05957bu85vfTT5yO5+X6/n+/n+z3v+8m5r/O9n3O+96aqkCS15aTFLkCSNHqGuyQ1yHCXpAYZ7pLUIMNdkhp08mIXAHD22WfX5OTkYpchSceV++677xtVNTFo3wsi3CcnJ5mamlrsMiTpuJLk6zPtc1pGkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGDRXuSR5P8uUku5JMdW1nJbkryaPd8sy+/jcm2ZPkkSRXjKt4SdJg87lyf31VXVRVq7rtDcCOqloB7Oi2SbISWAtcAKwGbk6yZIQ1S5LmsJA7VNcAl3XrW4C7gXd37Vur6lngsSR7gEuALyzgsTRGkxvuWOwSFtXjm65c7BKkkRv2yr2ATye5L8n6ru3cqtoP0C3P6drPA57sO3Zv1/Y8SdYnmUoyNT09fXTVS5IGGvbK/dKq2pfkHOCuJF+ZpW8GtB3xv/yqajOwGWDVqlX+rz9JGqGhrtyral+3PADcSm+a5akkSwG65YGu+15ged/hy4B9oypYkjS3OcM9yYuTnHFoHfgl4EFgO7Cu67YOuK1b3w6sTXJKkvOBFcDOURcuSZrZMNMy5wK3JjnU/2+q6s4k9wLbklwPPAFcC1BVu5NsAx4CngNuqKqDY6lekjTQnOFeVV8DXjWg/Wng8hmO2QhsXHB1kqSj4h2qktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYNHe5JliT5UpLbu+2zktyV5NFueWZf3xuT7EnySJIrxlG4JGlm87lyfyfwcN/2BmBHVa0AdnTbJFkJrAUuAFYDNydZMppyJUnDGCrckywDrgQ+3Ne8BtjSrW8Brupr31pVz1bVY8Ae4JKRVCtJGsqwV+4fAP4Q+EFf27lVtR+gW57TtZ8HPNnXb2/X9jxJ1ieZSjI1PT0937olSbOYM9yT/ApwoKruG/KcGdBWRzRUba6qVVW1amJiYshTS5KGcfIQfS4Ffi3JLwOnAi9J8nHgqSRLq2p/kqXAga7/XmB53/HLgH2jLFqSNLs5r9yr6saqWlZVk/TeKP1MVf0WsB1Y13VbB9zWrW8H1iY5Jcn5wApg58grlyTNaJgr95lsArYluR54ArgWoKp2J9kGPAQ8B9xQVQcXXKkkaWjzCvequhu4u1t/Grh8hn4bgY0LrE2SdJS8Q1WSGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUoDnDPcmpSXYm+Y8ku5O8r2s/K8ldSR7tlmf2HXNjkj1JHklyxTi/AUnSkYa5cn8WeENVvQq4CFid5HXABmBHVa0AdnTbJFkJrAUuAFYDNydZMobaJUkzmDPcq+c73eaLuq8C1gBbuvYtwFXd+hpga1U9W1WPAXuAS0ZZtCRpdkPNuSdZkmQXcAC4q6q+CJxbVfsBuuU5XffzgCf7Dt/btR1+zvVJppJMTU9PL+BbkCQdbqhwr6qDVXURsAy4JMmFs3TPoFMMOOfmqlpVVasmJiaGKlaSNJx5fVqmqv4LuJveXPpTSZYCdMsDXbe9wPK+w5YB+xZaqCRpeMN8WmYiyUu79dOANwJfAbYD67pu64DbuvXtwNokpyQ5H1gB7Bxx3ZKkWZw8RJ+lwJbuEy8nAduq6vYkXwC2JbkeeAK4FqCqdifZBjwEPAfcUFUHx1O+JGmQOcO9qh4ALh7Q/jRw+QzHbAQ2Lrg6SdJR8Q5VSWqQ4S5JDTLcJalBw7yhKmkWkxvuWOwSFtXjm65c7BI0gFfuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDDHdJatCc4Z5keZLPJnk4ye4k7+zaz0pyV5JHu+WZfcfcmGRPkkeSXDHOb0CSdKRhrtyfA/6gql4BvA64IclKYAOwo6pWADu6bbp9a4ELgNXAzUmWjKN4SdJgc4Z7Ve2vqvu79W8DDwPnAWuALV23LcBV3foaYGtVPVtVjwF7gEtGXLckaRbzmnNPMglcDHwROLeq9kPvBQA4p+t2HvBk32F7u7bDz7U+yVSSqenp6aMoXZI0k6HDPcnpwC3A71fVt2brOqCtjmio2lxVq6pq1cTExLBlSJKGMFS4J3kRvWD/RFX9Xdf8VJKl3f6lwIGufS+wvO/wZcC+0ZQrSRrGMJ+WCfBXwMNV9f6+XduBdd36OuC2vva1SU5Jcj6wAtg5upIlSXM5eYg+lwJvBr6cZFfX9h5gE7AtyfXAE8C1AFW1O8k24CF6n7S5oaoOjrpwSdLM5gz3qrqHwfPoAJfPcMxGYOMC6pIkLYB3qEpSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaNGe4J/lIkgNJHuxrOyvJXUke7ZZn9u27McmeJI8kuWJchUuSZjbMlftHgdWHtW0AdlTVCmBHt02SlcBa4ILumJuTLBlZtZKkocwZ7lX1OeCbhzWvAbZ061uAq/rat1bVs1X1GLAHuGQ0pUqShnW0c+7nVtV+gG55Ttd+HvBkX7+9XZsk6Rga9RuqGdBWAzsm65NMJZmanp4ecRmSdGI72nB/KslSgG55oGvfCyzv67cM2DfoBFW1uapWVdWqiYmJoyxDkjTI0Yb7dmBdt74OuK2vfW2SU5KcD6wAdi6sREnSfJ08V4ckfwtcBpydZC/wXmATsC3J9cATwLUAVbU7yTbgIeA54IaqOjim2iVJM5gz3Kvquhl2XT5D/43AxoUUJUlaGO9QlaQGGe6S1KA5p2WOB5Mb7ljsEhbV45uuXOwSJL3AeOUuSQ0y3CWpQYa7JDXIcJekBhnuktQgw12SGmS4S1KDDHdJapDhLkkNMtwlqUGGuyQ1yHCXpAYZ7pLUIMNdkhpkuEtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIadPJiFyDpxDa54Y7FLmFRPb7pyrGc1yt3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1CDDXZIaNLZwT7I6ySNJ9iTZMK7HkSQdaSzhnmQJ8EHgTcBK4LokK8fxWJKkI43ryv0SYE9Vfa2qvg9sBdaM6bEkSYdJVY3+pMk1wOqqemu3/WbgtVX1jr4+64H13eZPA4+MvJBj52zgG4tdxHHM8VsYx29hjufx+8mqmhi0Y1x/WyYD2p73KlJVm4HNY3r8YyrJVFWtWuw6jleO38I4fgvT6viNa1pmL7C8b3sZsG9MjyVJOsy4wv1eYEWS85P8CLAW2D6mx5IkHWYs0zJV9VySdwD/DCwBPlJVu8fxWC8QTUwvLSLHb2Ecv4VpcvzG8oaqJGlxeYeqJDXIcJekBhnuAyT5sSRbk/xnkoeS/GOS9Ulun6H/hw/dgZvk8SRnD+hzU5J3jbv2xZSee5K8qa/tN5LcOY9zvDTJ2/u2L5tp3FuW5OokleRnZth/d5JZP77X/5xL8pYkPz6OWl9okhxMsqvva0PXPnDMurH5y2Nf6XgZ7odJEuBW4O6qenlVrQTeA5w70zFV9daqeuhY1fhCVb03cN4GvD/JqUleDGwEbpjHaV4KvH2uTieA64B76H3SbBTeApwQ4Q48U1UX9X1tGuXJkxwX/3vacD/S64H/raoPHWqoql3AvwGnJ/lUkq8k+UT3QjDbFcEfdX887V/o3YXbvKp6EPgH4N3Ae4GPA3+a5IEk/57klXDkbzJJHkwyCWwCXt5dcf1Jt3vguLcqyenApcD1dOGe5LTut8kHknwSOK2v/3f61q9J8tHDzncNsAr4RDeup3GCS/I7Sb6a5F/pjfWh9okktyS5t/u6tGu/KcnmJJ8GPrZYdc/HcfEKdIxdCNw3w76LgQvo3ZD1eXpPinsGdUzyGno/mBfTG+f7Zzlva95H7/v9Pr3x+VJVXZXkDfR+MC6a5dgNwIVVdRH0pmWYx7g34irgzqr6apJvJnk1cBnwvap6ZfcCef+wJ6uqT3UfTX5XVU2NpeIXltOS7Orb/uOq+uShjSRL6T1HXwP8N/BZ4Evd7j8H/qyq7knyE/Q+zv2Kbt9rgJ+vqmfGXP9IGO7zs7Oq9gJ0T55JZg6ZXwBurarvdf1PmJu4quq73dXld+hNL/x61/6ZJC9L8qPzPOV8xr0F1wEf6Na3dtsrgL8AqKoHkjywOKUdF545dHEwg9fSm3adBuieqz/V7XsjsLLvl8OXJDmjW99+vAQ7GO6D7AaumWHfs33rB5l7/E7kmwh+0H3N9HeGnuP504KnznKu+Y77cSvJy4A3ABcmKXo3ARa9K8uZnk/97bONo/7fTGN5EvBzh4d4F/bfHXdRo+Sc+5E+A5yS5HcPNST5WeAX53mezwFXd3OlZwC/OsIajyefA34TfjjF8o2q+hbwOPDqrv3VwPld/28DZxx+khPINcDHquonq2qyqpYDj9Gbhjk0jhcCr+w75qkkr0hyEnD1DOc90ce13xeBy7rfIl8EXNu379NA/1+vvegY1zYyzV4BHa2qqiRXAx/oPkL1P/SC6O/neZ77u1/3dgFfp/eG7InoJuCvu2mE7wHruvZbgN/uplnuBb4KUFVPJ/l8kgeBfwLuOOYVL67r6L2p3O8Weu87nNaN4y5gZ9/+DcDtwJPAg8DpA877UeBDSZ5hwJVpYw6fc7+zqn743+Cqan+Sm4AvAPvpvXAu6Xb/HvDBbpxPpndx8rZjUfSo+ecHJKlBTstIUoMMd0lqkOEuSQ0y3CWpQYa7JDXIcJekBhnuktSg/wN5pN7NWF+4awAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.bar(age_distributions.keys(),height=age_distributions.values())\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* 利用叠加条形图显示乘客舱位与存活率的关系："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD4CAYAAAAXUaZHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAAXn0lEQVR4nO3de5BV5Z3u8e9jw4ARULmoSBObMuiEiyJy8RJSHbGAGAJmBLs1Ry2xAqUY0VxGOOfEcFLBcYIZFROrxBMDVcIAIV5QMxmMJ2iMRuw27QWRAQcjHVAuFgygEMDf+WMvezbQTW/o3nbz8nyqutZa737XWr/eq/rZb6+99tqKCMzMLC3HtXQBZmbW/BzuZmYJcribmSXI4W5mliCHu5lZgtq0dAEAXbt2jbKyspYuw8zsqFJdXb05IrrV91irCPeysjKqqqpaugwzs6OKpL809JhPy5iZJcjhbmaWoILCXdK7kt6QVCOpKmvrLOkZSauz6cl5/adJWiNplaSRxSrezMzqdzjn3L8SEZvzlqcCz0bEXZKmZsu3S+oDVAJ9gdOB30k6KyL2HU5he/bsoba2ll27dh3OalYk7du3p7S0lLZt27Z0KWZWgKa8oToWKM/m5wLLgNuz9gURsRtYK2kNMAR46XA2XltbS8eOHSkrK0NSE8q0pooItmzZQm1tLb169WrpcsysAIWecw9gqaRqSROztlMjYgNANj0la+8BrMtbtzZr24+kiZKqJFVt2rTpoB3u2rWLLl26ONhbAUl06dLF/0WZHUUKHblfHBHrJZ0CPCPp7UP0rS+ND7r1ZETMBmYDDBo0qN5bUzrYWw8fC7OjS0Ej94hYn003Ao+RO83ygaTuANl0Y9a9FuiZt3opsL65CjYzs8Y1OnKXdAJwXERsz+ZHAD8ClgDXAXdl0yeyVZYA8yX9C7k3VHsDy5taaNnUp5u6if28e9fXGu0jie985zv89Kc/BeDuu+9mx44dTJ8+vcF1Hn/8cc466yz69Olz0GOrVq1i0qRJbN26ld27dzNs2DBmz559xL9Dvssuu4z58+dz0kknNWk706dPp0OHDnzve99rlrrMrGUUclrmVOCx7N/yNsD8iPitpFeARZJuAN4DxgNExApJi4C3gL3A5MO9Uqa1aNeuHY8++ijTpk2ja9euBa3z+OOPM3r06HrD/ZZbbuG2225j7NixALzxxhuHVc++ffsoKSmp97Hf/OY3h7Uts8My/cSWriBd07cVZbONnpaJiP+MiHOzn74RMSNr3xIRwyOidzb9MG+dGRFxZkScHRH/VpTKPwNt2rRh4sSJ3HPPPQc99pe//IXhw4dzzjnnMHz4cN577z1efPFFlixZwve//30GDBjAO++8s986GzZsoLS0tG65f//+AMyZM4ebb765rn306NEsW7YMgA4dOnDHHXcwdOhQ7rzzTq688sq6fsuWLePrX/86kLuFw+bNm7n99tt54IEH6vpMnz697j+PmTNnMnjwYM455xx++MMf1vWZMWMGZ599NpdeeimrVq060qfLzFoRf0K1EZMnT2bevHls27b/q+vNN9/Mtddey+uvv843v/lNbrnlFi666CLGjBnDzJkzqamp4cwzz9xvndtuu41LLrmEr371q9xzzz1s3bq10f3v3LmTfv368fLLLzNt2jT+9Kc/sXPnTgAWLlxIRUXFfv0rKytZuHBh3fKiRYsYP348S5cuZfXq1Sxfvpyamhqqq6t5/vnnqa6uZsGCBfz5z3/m0Ucf5ZVXXjnCZ8rMWhOHeyM6derEtddey6xZs/Zrf+mll7j66qsBuOaaa3jhhRca3db111/PypUrGT9+PMuWLeOCCy5g9+7dh1ynpKSEK664Asj9JzFq1CiefPJJ9u7dy9NPP113iudT5513Hhs3bmT9+vW89tprnHzyyXz+859n6dKlLF26lPPOO4+BAwfy9ttvs3r1av7whz/wjW98g8997nN06tSJMWPGHM7TY2atVKu4K2Rrd+uttzJw4ECuv/76BvsUeqng6aefzoQJE5gwYQL9+vXjzTffpE2bNnzyySd1ffKvJ2/fvv1+59krKir4+c9/TufOnRk8eDAdO3Y8aB/jxo1j8eLFvP/++1RWVgK5DyJNmzaNSZMm7df33nvv9WWOZgnyyL0AnTt35sorr+QXv/hFXdtFF13EggULAJg3bx5f+tKXAOjYsSPbt2+vdzu//e1v2bNnDwDvv/8+W7ZsoUePHpSVlVFTU8Mnn3zCunXrWL684YuLysvLefXVV3nooYcOOiXzqcrKShYsWMDixYsZN24cACNHjuThhx9mx44dAPz1r39l48aNfPnLX+axxx7j448/Zvv27Tz55JOH+eyYWWt01IzcC7l0sZi++93v8rOf/axuedasWUyYMIGZM2fSrVs3fvnLXwK5YP3Wt77FrFmzWLx48X7n3ZcuXcqUKVNo3749kHuD87TTTuPUU0+lV69e9O/fn379+jFw4MAG6ygpKWH06NHMmTOHuXPn1tunb9++bN++nR49etC9e3cARowYwcqVK7nwwguB3Bu1jzzyCAMHDqSiooIBAwZwxhlnMGzYsKY9UWbWKiii3g+HfqYGDRoUB35Zx8qVK/niF7/YQhVZfXxMjmG+FLJ4mnAppKTqiBhU32M+LWNmliCHu5lZghzuZmYJcribmSXI4W5mliCHu5lZgo6a69yb/VKsAi8/mjFjBvPnz6ekpITjjjuOBx98kKFDhzZp10uWLOGtt95i6tSpTdoO5K5X//SDSWZmnzp6wr0FvPTSSzz11FO8+uqrtGvXjs2bN/O3v/2toHX37t1Lmzb1P71jxozxPVzMrKh8WuYQNmzYQNeuXWnXrh0AXbt25fTTT6+7vS5AVVUV5eXlQO72uhMnTmTEiBFce+21DB06lBUrVtRtr7y8nOrq6rpb/G7bto2ysrK6+8p89NFH9OzZkz179vDOO+8watQozj//fIYNG8bbb+e+2XDt2rVceOGFDB48mB/84Aef4bNhZkcTh/shjBgxgnXr1nHWWWdx00038dxzzzW6TnV1NU888QTz58+nsrKSRYsWAbkXivXr13P++efX9T3xxBM599xz67b75JNPMnLkSNq2bcvEiRO5//77qa6u5u677+amm24CYMqUKdx444288sornHbaaUX4rc0sBQ73Q+jQoQPV1dXMnj2bbt26UVFRwZw5cw65zpgxYzj++OMBuPLKK/nVr34F/Pd91Q9UUVFRd//1BQsWUFFRwY4dO3jxxRcZP348AwYMYNKkSWzYsAGAP/7xj1x11VVA7lbDZmb18Tn3RpSUlFBeXk55eTn9+/dn7ty5+92iN//2vAAnnHBC3XyPHj3o0qULr7/+OgsXLuTBBx88aPtjxoxh2rRpfPjhh1RXV3PJJZewc+dOTjrpJGpqauqtybfoNbPGeOR+CKtWrWL16tV1yzU1NZxxxhmUlZVRXV0NwK9//etDbqOyspKf/OQnbNu2re5r9fJ16NCBIUOGMGXKFEaPHk1JSQmdOnWiV69edaP+iOC1114D4OKLL97vVsNmZvU5ekbuRfoS2UPZsWMH3/72t9m6dStt2rThC1/4ArNnz2blypXccMMN3HnnnY1eFjlu3DimTJlyyDc/Kyoq6r6d6VPz5s3jxhtv5Mc//jF79uyhsrKSc889l/vuu4+rr76a++67r+4bmszMDuRb/lrBfEyOYb7lb/H4lr9mZlYoh7uZWYJadbi3hlNGluNjYXZ0abXh3r59e7Zs2eJQaQUigi1bttR996uZtX6t9mqZ0tJSamtr2bRpU0uXYuRebEtLS1u6DDMrUKsN97Zt29KrV6+WLsPM7KjUak/LmJnZkXO4m5klyOFuZpYgh7uZWYIc7mZmCXK4m5klqOBwl1Qi6c+SnsqWO0t6RtLqbHpyXt9pktZIWiVpZDEKNzOzhh3OyH0KsDJveSrwbET0Bp7NlpHUB6gE+gKjgAcklTRPuWZmVoiCwl1SKfA14P/mNY8F5mbzc4HL89oXRMTuiFgLrAGGNEu1ZmZWkEJH7vcC/wh8ktd2akRsAMimp2TtPYB1ef1qs7b9SJooqUpSlW8xYGbWvBoNd0mjgY0RUV3gNuv7gs+D7v4VEbMjYlBEDOrWrVuBmzYzs0IUcm+Zi4Exki4D2gOdJD0CfCCpe0RskNQd2Jj1rwV65q1fCqxvzqLNzOzQGh25R8S0iCiNiDJyb5T+v4j4H8AS4Lqs23XAE9n8EqBSUjtJvYDewPJmr9zMzBrUlLtC3gUsknQD8B4wHiAiVkhaBLwF7AUmR8S+JldqZmYFO6xwj4hlwLJsfgswvIF+M4AZTazNzMyOkD+hamaWIIe7mVmCHO5mZglyuJuZJcjhbmaWIIe7mVmCmnKdu5kdI8p2zW/pEpL1bpG265G7mVmCHO5mZglyuJuZJcjhbmaWIIe7mVmCHO5mZglyuJuZJcjhbmaWIIe7mVmCHO5mZglyuJuZJcjhbmaWIIe7mVmCHO5mZglyuJuZJcjhbmaWIIe7mVmCHO5mZglyuJuZJcjhbmaWIIe7mVmCHO5mZglyuJuZJcjhbmaWIIe7mVmCHO5mZglqNNwltZe0XNJrklZI+j9Ze2dJz0hanU1PzltnmqQ1klZJGlnMX8DMzA5WyMh9N3BJRJwLDABGSboAmAo8GxG9gWezZST1ASqBvsAo4AFJJUWo3czMGtBouEfOjmyxbfYTwFhgbtY+F7g8mx8LLIiI3RGxFlgDDGnOos3M7NAKOucuqURSDbAReCYiXgZOjYgNANn0lKx7D2Bd3uq1WduB25woqUpS1aZNm5rwK5iZ2YEKCveI2BcRA4BSYIikfoforvo2Uc82Z0fEoIgY1K1bt4KKNTOzwhzW1TIRsRVYRu5c+geSugNk041Zt1qgZ95qpcD6phZqZmaFK+RqmW6STsrmjwcuBd4GlgDXZd2uA57I5pcAlZLaSeoF9AaWN3PdZmZ2CG0K6NMdmJtd8XIcsCginpL0ErBI0g3Ae8B4gIhYIWkR8BawF5gcEfuKU76ZmdWn0XCPiNeB8+pp3wIMb2CdGcCMJldnZmZHxJ9QNTNLkMPdzCxBDnczswQ53M3MEuRwNzNLkMPdzCxBDnczswQ53M3MEuRwNzNLkMPdzCxBDnczswQ53M3MEuRwNzNLkMPdzCxBDnczswQ53M3MEuRwNzNLkMPdzCxBDnczswQ53M3MEuRwNzNLkMPdzCxBDnczswQ53M3MEuRwNzNLkMPdzCxBDnczswQ53M3MEuRwNzNLkMPdzCxBbVq6gGYx/cSWriBd07e1dAVmdgQ8cjczS5DD3cwsQY2Gu6Sekn4vaaWkFZKmZO2dJT0jaXU2PTlvnWmS1khaJWlkMX8BMzM7WCEj973AdyPii8AFwGRJfYCpwLMR0Rt4Nlsme6wS6AuMAh6QVFKM4s3MrH6NhntEbIiIV7P57cBKoAcwFpibdZsLXJ7NjwUWRMTuiFgLrAGGNHPdZmZ2CId1zl1SGXAe8DJwakRsgNwLAHBK1q0HsC5vtdqs7cBtTZRUJalq06ZNR1C6mZk1pOBwl9QB+DVwa0T816G61tMWBzVEzI6IQRExqFu3boWWYWZmBSgo3CW1JRfs8yLi0az5A0nds8e7Axuz9lqgZ97qpcD65inXzMwKUcjVMgJ+AayMiH/Je2gJcF02fx3wRF57paR2knoBvYHlzVeymZk1ppBPqF4MXAO8Iakma/ufwF3AIkk3AO8B4wEiYoWkRcBb5K60mRwR+5q7cDMza1ij4R4RL1D/eXSA4Q2sMwOY0YS6zMysCfwJVTOzBKVx4zA7uvhGb8XjG71ZxiN3M7MEOdzNzBLkcDczS5DD3cwsQQ53M7MEOdzNzBLkcDczS5DD3cwsQQ53M7MEOdzNzBLkcDczS5DD3cwsQQ53M7MEOdzNzBLkcDczS5DD3cwsQQ53M7MEOdzNzBLkcDczS5DD3cwsQQ53M7MEOdzNzBLkcDczS5DD3cwsQQ53M7MEOdzNzBLkcDczS5DD3cwsQQ53M7MEOdzNzBLkcDczS1Cj4S7pYUkbJb2Z19ZZ0jOSVmfTk/MemyZpjaRVkkYWq3AzM2tYISP3OcCoA9qmAs9GRG/g2WwZSX2ASqBvts4DkkqarVozMytIo+EeEc8DHx7QPBaYm83PBS7Pa18QEbsjYi2wBhjSPKWamVmhjvSc+6kRsQEgm56StfcA1uX1q83azMzsM9Tcb6iqnraot6M0UVKVpKpNmzY1cxlmZse2Nke43geSukfEBkndgY1Zey3QM69fKbC+vg1ExGxgNsCgQYPqfQGwNJXtmt/SJSTr3ZYuwFqNIx25LwGuy+avA57Ia6+U1E5SL6A3sLxpJZqZ2eFqdOQu6V+BcqCrpFrgh8BdwCJJNwDvAeMBImKFpEXAW8BeYHJE7CtS7WZm1oBGwz0irmrgoeEN9J8BzGhKUWZm1jT+hKqZWYIc7mZmCXK4m5kl6EgvhWxVfGld8bzb0gWY2RHxyN3MLEEOdzOzBDnczcwS5HA3M0uQw93MLEEOdzOzBDnczcwS5HA3M0uQw93MLEEOdzOzBDnczcwS5HA3M0uQw93MLEEOdzOzBDnczcwS5HA3M0uQw93MLEEOdzOzBDnczcwS5HA3M0uQw93MLEEOdzOzBDnczcwS5HA3M0uQw93MLEEOdzOzBDnczcwS5HA3M0uQw93MLEEOdzOzBBUt3CWNkrRK0hpJU4u1HzMzO1hRwl1SCfBz4KtAH+AqSX2KsS8zMztYsUbuQ4A1EfGfEfE3YAEwtkj7MjOzA7Qp0nZ7AOvylmuBofkdJE0EJmaLOyStKlItrU1XYHNLF1Eo/XNLV9AqHDXHzMcLOIqOFzT5mJ3R0APFCnfV0xb7LUTMBmYXaf+tlqSqiBjU0nVY4XzMji4+XjnFOi1TC/TMWy4F1hdpX2ZmdoBihfsrQG9JvST9HVAJLCnSvszM7ABFOS0TEXsl3Qz8O1ACPBwRK4qxr6PQMXcqKgE+ZkcXHy9AEdF4LzMzO6r4E6pmZglyuJuZJcjh3kwk7ZNUk/dTJunFw9zGrZI+V6waj0WS/pekFZJez47L0MbXatb9l0t66rPcZ2okdcn7u3pf0l+z+a2S3mpgnR9JurSAbSd7fIp1nfux6OOIGHBA20UHdpJUEhH7GtjGrcAjwEfNW9qxSdKFwGhgYETsltQV+LsWLssOU0RsAQYASJoO7IiIuyWVAfUGc0TcUV97I39/SfHIvYgk7cim5ZJ+L2k+8IakEyQ9Lek1SW9KqpB0C3A68HtJv2/RwtPRHdgcEbsBImJzRKyXdL6k5yRVS/p3Sd0BJH1B0u+y4/KqpDOVMzM7Tm9Iqsj6lktaJmmxpLclzZOk7LFRWdsLwD+01C9/jCiR9FD239lSSccDSJojaVw2/66kO7LjMf5YOT4euTef4yXVZPNrI+IbBzw+BOgXEWslXQGsj4ivAUg6MSK2SfoO8JWIOGo+Ot3KLQXukPQfwO+AhcCLwP3A2IjYlIX1DGACMA+4KyIek9Se3ODnH8iNGs8l97H2VyQ9n23/PKAvuQ/o/RG4WFIV8BBwCbAm26cVT2/gqoj4lqRFwBXk/vs90K6I+FJ2XFdzDBwfj9ybz8cRMSD7OTDYAZZHxNps/g3gUkn/LGlYRGz7DOs8ZkTEDuB8cvcw2kTuD3kS0A94Jnsx/t9AqaSOQI+IeCxbd1dEfAR8CfjXiNgXER8AzwGDs10sj4jaiPgEqAHKgL8n9+K+OnLXGdcXNNZ81kZETTZfTe4Y1OfTED9mjo9H7p+dnZ/ORMR/SDofuAz4J0lLI+JHLVdaurLzq8uAZZLeACYDKyLiwvx+kjo1sIn67pP0qd158/v4778nf3jks3PgMTi+gX478+aPiePjkXsLkHQ68FFEPALcDQzMHtoOdGyxwhIj6WxJvfOaBgArgW7Zm61Iaiupb0T8F1Ar6fKsvV125dLzQIWkEkndgC8Dyw+x27eBXpLOzJavatZfyprqmDk+Hrm3jP7ATEmfAHuAG7P22cC/SdoQEV9pserS0QG4X9JJwF5y51gnknueZ0k6kdzfwL3ACuAa4EFJPyJ3XMYDjwEXAq+RG/H9Y0S8L+nv69thROxS7nbWT0vaDLxA7jSQtQLH0vHx7QfMzBLk0zJmZglyuJuZJcjhbmaWIIe7mVmCHO5mZglyuJuZJcjhbmaWoP8PHiDXpFLnXnkAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.bar(cabin_not_survive.index,cabin_not_survive.values,tick_label=['First','Second','Third'],label='Not Survived')\n",
    "plt.bar(cabin_survive.index,cabin_survive.values,tick_label=['First','Second','Third'],bottom=cabin_not_survive.values,label='Survived')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 实验练习10"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "数据库`dase_intro_2020`中有一张名为`SH_Grade`的数据表，该表为上海市某所初级中学7个班级学生就读4年的考试成绩，数据已做脱敏处理。\n",
    "\n",
    "数据表结构如下：\n",
    "\n",
    "* id：记录编号，无实际意义；\n",
    "* StuId：学生代号，例如\"G03\"表示7班3号学生；\n",
    "* Sex：性别，M代表男性，F代表女性；\n",
    "* 后续各列为科目，字段名称编码规则为【科目代号+年级+学期+期中/期末】：\n",
    "> 科目代号包括【CHI】语文、【MATH】数学、【ENG】英语、【PHY】物理、【CHE】化学；\n",
    ">\n",
    "> 年级由6年级至9年级；\n",
    ">\n",
    "> 学期包括第1学期和第2学期；\n",
    ">\n",
    "> 期中考试为1，期末考试为2（其中9年级第2学期没有期末考试）\n",
    "> \n",
    "> 请注意：物理课从8年级开始学习、化学课从9年级开始学习\n",
    ">\n",
    "> 例如：【MATH921】代表9年级第2学期数学科期中考试成绩\n",
    ">\n",
    "> 各列排序顺序为【年级，学期，期中/期末考试，科目（语文、数学、英语、物理、化学）】\n",
    "* 所以一条数据由【id,StuId,Sex,CHI611,MATH611,ENG611,CHI612,MATH612,...,CHI811,MATH811,ENG811,PHY811,CHI812,...,CHI921,MATH921,ENG921,PHY921,CHE921】构成。\n",
    "\n",
    "请依次完成以下任务："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. 数据获取：利用`pymysql`库从数据库中爬取所有数据行，并利用`csv`库将数据保存成`SH_Grade.csv`文件（不要忘记写入列标题作为第1行内容），由于`StuId`字段包含班级信息，请在`id`字段和`Sex`字段添加一个`Class`字段用以保存学生所在班级，取值范围为A-G。\n",
    "> 数据库地址：`cdb-r2g8flnu.bj.tencentcdb.com`\n",
    ">\n",
    "> 数据库端口：`10209`\n",
    ">\n",
    "> 数据库登录用户名：`dase2020`\n",
    ">\n",
    "> 数据库登录密码：`dase2020`\n",
    ">\n",
    "> 数据库名：`dase_intro_2020`\n",
    ">\n",
    "> 数据表名：`SH_Grade`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "2. 数据预处理 I：由于人工录入疏忽，数据表中有部分行重复，特征是除`id`字段之外其他字段值均相同。由于`StuId`可唯一确定某个学生，所以如果数据表中含有多条`StuId`相同的数据行，表示这几行内容相同。请先用`pandas`库读取任务1中导出的CSV文件，然后去除重复的数据行，仅保留第一次出现的数据行，并报告经处理前后的数据条目数量。\n",
    "> 参考答案：处理前252条，处理后243条"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3. 数据预处理 II：由于部分同学中途转入或转出该校，或由于疾病缺席某次考试，部分考次的数据会产生空缺，也有部分学生的性别为空。现在请在任务2的基础上剔除大于等于12个字段为空的数据行，并报告经处理前后的数据条目数量。\n",
    "> 参考答案：处理前243条，处理后189条"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "4. 数据预处理 III：对于仍空缺的数据项，请在任务3的基础上做如下处理：性别用上一条数据行的性别填充，成绩用该次该门考试的中位数填充。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "5. 数据预处理 IV：为了配合旧制中考（语数英满分150分，物理满分90分，化学60分）部分考次不是以百分制计分，为与之前的百分制计分保持一致，请将部分非百分制计分的列转换为百分制计分。\n",
    "> 提示1：可以通过查看每列的最大值查找非百分制计分的数据列。\n",
    ">\n",
    "> 提示2：8年级第2学期的期末考试中，语数英为120分制。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "6. 数据分析与可视化 I：请绘制各班男女人数的叠加条形图。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "7. 数据分析与可视化 II：请在一张图表中分别绘制学生代码为`A13`和`A15`的学生每次语文考试成绩走势折线图。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "8. 数据分析 III：请输出7年级第2学期期中考试中英语成绩小于60分或语文成绩小于60分的学生编号、班级、英语成绩、语文成绩。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "9. 数据分析 IV：请输出A班和C班6年级第2学期期末考试中各个科目的均值和方差，然后用文字（程序注释）简要比较两个班级各科目的表现情况。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "10. 数据存储：将任务8产生的DataFrame对象生成CSV文件，文件名为`task8.csv`。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**本次作业可提交py源代码或直接提交ipynb文件。**"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}