3.9. 多层感知机的从零开始实现.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "3.9. 多层感知机的从零开始实现.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/chongzicbo/Dive-into-Deep-Learning-tf.keras/blob/master/3.9.%20%E5%A4%9A%E5%B1%82%E6%84%9F%E7%9F%A5%E6%9C%BA%E7%9A%84%E4%BB%8E%E9%9B%B6%E5%BC%80%E5%A7%8B%E5%AE%9E%E7%8E%B0.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Bos42veOJ5_8",
        "colab_type": "text"
      },
      "source": [
        "##3.9. 多层感知机的从零开始实现\n",
        "我们已经从上一节里了解了多层感知机的原理。下面，我们一起来动手实现一个多层感知机。首先导入实现所需的包或模块。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gK33BR94-FNH",
        "colab_type": "code",
        "outputId": "82c25743-c5f8-45bf-b569-ec362bdea3ef",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 64
        }
      },
      "source": [
        "%matplotlib inline\n",
        "import tensorflow as tf\n",
        "import matplotlib.pyplot as plt\n",
        "from IPython import  display\n",
        "from tensorflow import keras\n",
        "import tensorflow.data as tfdata\n",
        "import numpy as np"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<p style=\"color: red;\">\n",
              "The default version of TensorFlow in Colab will soon switch to TensorFlow 2.x.<br>\n",
              "We recommend you <a href=\"https://www.tensorflow.org/guide/migrate\" target=\"_blank\">upgrade</a> now \n",
              "or ensure your notebook will continue to use TensorFlow 1.x via the <code>%tensorflow_version 1.x</code> magic:\n",
              "<a href=\"https://colab.research.google.com/notebooks/tensorflow_version.ipynb\" target=\"_blank\">more info</a>.</p>\n"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Fka3Bygr-fFP",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "tf.enable_eager_execution()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GdCzKWXzKBWe",
        "colab_type": "text"
      },
      "source": [
        "###3.9.1. 获取和读取数据\n",
        "这里继续使用Fashion-MNIST数据集。我们将使用多层感知机对图像进行分类。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "858QHsEi-hQV",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "batch_size=256\n",
        "buffer_size=10000\n",
        "def load_data_fashion_mnist(batch_size,buffer_size):\n",
        "  (x_train,y_train),(x_test,y_test)=keras.datasets.fashion_mnist.load_data()\n",
        "  x_train=np.reshape(x_train,[x_train.shape[0],-1])#将三维张量reshape成二维\n",
        "  x_test=np.reshape(x_test,[x_test.shape[0],-1])\n",
        "  train_iter=tfdata.Dataset.from_tensor_slices((x_train,y_train)).map(lambda x,y:(x/255,y)).shuffle(buffer_size).batch(batch_size)\n",
        "  test_iter=tfdata.Dataset.from_tensor_slices((x_test,y_test)).map(lambda x,y:(x/255,y)).batch(batch_size)\n",
        "  return train_iter,test_iter"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "oL36ni8O-jJO",
        "colab_type": "code",
        "outputId": "349e7cfe-4998-4f0f-a041-345a6e219f3b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 155
        }
      },
      "source": [
        "train_iter,test_iter=load_data_fashion_mnist(batch_size=batch_size,buffer_size=buffer_size)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/train-labels-idx1-ubyte.gz\n",
            "32768/29515 [=================================] - 0s 0us/step\n",
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/train-images-idx3-ubyte.gz\n",
            "26427392/26421880 [==============================] - 0s 0us/step\n",
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/t10k-labels-idx1-ubyte.gz\n",
            "8192/5148 [===============================================] - 0s 0us/step\n",
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/t10k-images-idx3-ubyte.gz\n",
            "4423680/4422102 [==============================] - 0s 0us/step\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HQnXPYa0KGyq",
        "colab_type": "text"
      },
      "source": [
        "###3.9.2. 定义模型参数\n",
        "我们在“softmax回归的从零开始实现”一节里已经介绍了，Fashion-MNIST数据集中图像形状为28×28，类别数为10。本节中我们依然使用长度为28×28=784的向量表示每一张图像。因此，输入个数为784，输出个数为10。实验中，我们设超参数隐藏单元个数为256。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QQSwT_m9-lM9",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "num_inputs,num_outputs,num_hiddens=784,10,256\n",
        "W1=tf.Variable(tf.random.normal(shape=(num_inputs,num_hiddens),stddev=0.01))\n",
        "b1=tf.Variable(tf.zeros(num_hiddens))\n",
        "W2=tf.Variable(tf.random.normal(shape=(num_hiddens,num_outputs),stddev=0.01))\n",
        "b2=tf.Variable(tf.zeros(num_outputs))\n",
        "params=[W1,b1,W2,b2]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xSmaLrzrKMdF",
        "colab_type": "text"
      },
      "source": [
        "###3.9.3. 定义激活函数\n",
        "这里我们使用基础的maximum函数来实现ReLU，而非直接调用relu函数。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4zoTlTma_5tL",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def relu(X):\n",
        "  return tf.maximum(X,0)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mssI2SCnKQiL",
        "colab_type": "text"
      },
      "source": [
        "###3.9.4. 定义模型\n",
        "同softmax回归一样，我们通过reshape函数将每张原始图像改成长度为num_inputs的向量。然后我们实现上一节中多层感知机的计算表达式。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wpR3HQT7ACw9",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def net(X):\n",
        "  H=relu(tf.matmul(X,W1)+b1)\n",
        "  return tf.matmul(H,W2)+b2"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-69bwVGdKVfP",
        "colab_type": "text"
      },
      "source": [
        "###3.9.5. 定义损失函数\n",
        "为了得到更好的数值稳定性，我们直接使用keras.losses模块提供的包括softmax运算和交叉熵损失计算的函数。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "k4niz3XBAVKt",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "loss=tf.keras.losses.sparse_categorical_crossentropy"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5ysWJupDKgFw",
        "colab_type": "text"
      },
      "source": [
        "###3.9.6. 训练模型\n",
        "训练多层感知机的步骤和“softmax回归的从零开始实现”一节中训练softmax回归的步骤没什么区别。我们直接使用之前定义的train_ch3函数，它的实现已经在“softmax回归的从零开始实现”一节里介绍过。我们在这里设超参数迭代周期数为5，学习率为0.5。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IlSrsX7JAgqe",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def sgd(params,loss,t,lr,batch_size):\n",
        "  for param in params:\n",
        "    dl_dp=t.gradient(loss,param) #求梯度\n",
        "    param.assign_sub(lr*dl_dp/batch_size) #更新梯度\n",
        "\n",
        "def evaluate_accuracy(data_iter,net):\n",
        "  acc_sum,n=0.0,0\n",
        "  for X,y in data_iter:\n",
        "    # y=tf.cast(y,tf.float32)\n",
        "    acc_sum+=tf.equal(tf.cast(tf.argmax(net(X),axis=1),tf.float32),tf.cast(y,tf.float32)).numpy().sum()\n",
        "    n+=tf.shape(y)[0].numpy()\n",
        "  return acc_sum/n\n",
        "\n",
        "def train_ch3(net,train_iter,test_iter,loss,num_epochs,batch_size,params=None,lr=None,trainer=None):\n",
        "  for epoch in range(num_epochs):\n",
        "    train_l_sum,train_acc_sum,n=0.0,0.0,0\n",
        "    for X,y in train_iter:\n",
        "      if trainer is None:\n",
        "        with tf.GradientTape(persistent=True) as t:\n",
        "          y_hat=net(X)\n",
        "          l=tf.reduce_sum(loss(y,y_hat,from_logits=True))\n",
        "        sgd(params,l,t,lr,batch_size)\n",
        "      else:\n",
        "        y_hat=net(X)\n",
        "        l=tf.reduce_sum(loss(y_hat,y))\n",
        "        trainer.minimize(lambda:loss(net(X),y),global_step=tf.train.get_or_create_global_step())\n",
        "      train_l_sum+=l.numpy()\n",
        "      train_acc_sum+=tf.equal(tf.cast(tf.argmax(y_hat,axis=1),tf.float32),tf.cast(y,tf.float32)).numpy().sum()\n",
        "      n+=tf.shape(y)[0]\n",
        "    test_acc=evaluate_accuracy(test_iter,net)\n",
        "    print('epoch %d,loss %.4f,train acc %.3f,test acc %.3f'%(epoch+1,train_l_sum/n,train_acc_sum/n,test_acc))  "
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "o4vX2ZAuA7y1",
        "colab_type": "code",
        "outputId": "899a114e-90bf-4ddf-bc6c-9cf1fcd36235",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 104
        }
      },
      "source": [
        "num_epochs,lr=5,0.5\n",
        "train_ch3(net,train_iter,test_iter,loss,num_epochs,batch_size,params,lr)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "epoch 1,loss 0.4435,train acc 0.835,test acc 0.841\n",
            "epoch 2,loss 0.4010,train acc 0.852,test acc 0.855\n",
            "epoch 3,loss 0.3731,train acc 0.862,test acc 0.864\n",
            "epoch 4,loss 0.3529,train acc 0.871,test acc 0.868\n",
            "epoch 5,loss 0.3365,train acc 0.876,test acc 0.869\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uxkU0cQ0BPks",
        "colab_type": "text"
      },
      "source": [
        "###3.9.7. 小结\n",
        "可以通过手动定义模型及其参数来实现简单的多层感知机。\n",
        "当多层感知机的层数较多时，本节的实现方法会显得较烦琐，例如在定义模型参数的时候。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Dt8jOsicKtbJ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}