3.7. softmax回归的简洁实现.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "3.7 softmax回归的简洁实现.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/chongzicbo/Dive-into-Deep-Learning-tf.keras/blob/master/3.7.%20softmax%E5%9B%9E%E5%BD%92%E7%9A%84%E7%AE%80%E6%B4%81%E5%AE%9E%E7%8E%B0.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5rbu8EZkHWiq",
        "colab_type": "text"
      },
      "source": [
        "##3.7. softmax回归的简洁实现\n",
        "我们在“线性回归的简洁实现”一节中已经了解了使用keras实现模型的便利。下面，让我们再次使用keras来实现一个softmax回归模型。首先导入所需的包或模块。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sw1Fk4Xm911K",
        "colab_type": "code",
        "outputId": "f8303ad0-b272-4688-ac82-a2f3089fa25f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 62
        }
      },
      "source": [
        "%matplotlib inline\n",
        "import tensorflow as tf\n",
        "import matplotlib.pyplot as plt\n",
        "from IPython import  display\n",
        "from tensorflow import keras\n",
        "import tensorflow.data as tfdata\n",
        "import numpy as np"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<p style=\"color: red;\">\n",
              "The default version of TensorFlow in Colab will soon switch to TensorFlow 2.x.<br>\n",
              "We recommend you <a href=\"https://www.tensorflow.org/guide/migrate\" target=\"_blank\">upgrade</a> now \n",
              "or ensure your notebook will continue to use TensorFlow 1.x via the <code>%tensorflow_version 1.x</code> magic:\n",
              "<a href=\"https://colab.research.google.com/notebooks/tensorflow_version.ipynb\" target=\"_blank\">more info</a>.</p>\n"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZqqSsefk-bSA",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "tf.enable_eager_execution()#开启动态图计算"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9O6n8wEpHl9F",
        "colab_type": "text"
      },
      "source": [
        "###3.7.1. 获取和读取数据\n",
        "我们仍然使用Fashion-MNIST数据集和上一节中设置的批量大小。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jsKXHme2-dfD",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "batch_size=256\n",
        "buffer_size=10000\n",
        "def load_data_fashion_mnist(batch_size,buffer_size):\n",
        "  (x_train,y_train),(x_test,y_test)=keras.datasets.fashion_mnist.load_data()\n",
        "  x_train=np.reshape(x_train,[x_train.shape[0],-1])#将三维张量reshape成二维\n",
        "  x_test=np.reshape(x_test,[x_test.shape[0],-1])\n",
        "  train_iter=tfdata.Dataset.from_tensor_slices((x_train,y_train)).map(lambda x,y:(x/255,y)).shuffle(buffer_size).batch(batch_size)\n",
        "  test_iter=tfdata.Dataset.from_tensor_slices((x_test,y_test)).map(lambda x,y:(x/255,y)).batch(batch_size)\n",
        "  return train_iter,test_iter"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SOoG2aGZ-7qF",
        "colab_type": "code",
        "outputId": "11b33555-6ec0-4165-f3ab-76e4eeef1635",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 151
        }
      },
      "source": [
        "train_iter,test_iter=load_data_fashion_mnist(batch_size=batch_size,buffer_size=buffer_size)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/train-labels-idx1-ubyte.gz\n",
            "32768/29515 [=================================] - 0s 0us/step\n",
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/train-images-idx3-ubyte.gz\n",
            "26427392/26421880 [==============================] - 0s 0us/step\n",
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/t10k-labels-idx1-ubyte.gz\n",
            "8192/5148 [===============================================] - 0s 0us/step\n",
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/t10k-images-idx3-ubyte.gz\n",
            "4423680/4422102 [==============================] - 0s 0us/step\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Tsn9Jrw8Hx5A",
        "colab_type": "text"
      },
      "source": [
        "###3.7.2. 定义和初始化模型\n",
        "在“softmax回归”一节中提到，softmax回归的输出层是一个全连接层。因此，我们添加一个输出个数为10的全连接层。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pcbPuDz3_Ecr",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "net=keras.Sequential()\n",
        "net.add(keras.layers.Dense(10))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qDi6E857IHXo",
        "colab_type": "text"
      },
      "source": [
        "###3.7.3. softmax和交叉熵损失函数\n",
        "如果做了上一节的练习，那么你可能意识到了分开定义softmax运算和交叉熵损失函数可能会造成数值不稳定。因此，keras.losses模块提供了一个包括softmax运算和交叉熵损失计算的函数。它的数值稳定性更好。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "t8FcORTj_LHE",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Yct9mb9QIRrF",
        "colab_type": "text"
      },
      "source": [
        "###3.7.4. 定义优化算法\n",
        "我们使用学习率为0.1的小批量随机梯度下降作为优化算法。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "v6WL7SGS_XaT",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "optimizer=keras.optimizers.SGD(learning_rate=0.1)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QI86yrrMIZt_",
        "colab_type": "text"
      },
      "source": [
        "###3.7.5. 训练模型\n",
        "接下来，我们使用compile来编译模型并使用fit_generator训练函数来训练模型。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "XhTkoeHh_fp7",
        "colab_type": "code",
        "outputId": "91ca2e7b-ada1-4dec-e828-53084f46f430",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 286
        }
      },
      "source": [
        "num_epochs=5\n",
        "net.compile(optimizer=optimizer,loss=loss,metrics=['accuracy'])\n",
        "net.fit_generator(train_iter,validation_data=test_iter,epochs=num_epochs)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/5\n",
            "231/235 [============================>.] - ETA: 0s - loss: 0.7975 - acc: 0.6541Epoch 1/5\n",
            "235/235 [==============================] - 8s 35ms/step - loss: 0.7939 - acc: 0.6559 - val_loss: 0.6474 - val_acc: 0.7757\n",
            "Epoch 2/5\n",
            "233/235 [============================>.] - ETA: 0s - loss: 0.5753 - acc: 0.8037Epoch 1/5\n",
            "235/235 [==============================] - 8s 33ms/step - loss: 0.5744 - acc: 0.8038 - val_loss: 0.5793 - val_acc: 0.7994\n",
            "Epoch 3/5\n",
            "231/235 [============================>.] - ETA: 0s - loss: 0.5289 - acc: 0.8203Epoch 1/5\n",
            "235/235 [==============================] - 8s 35ms/step - loss: 0.5282 - acc: 0.8204 - val_loss: 0.5487 - val_acc: 0.8108\n",
            "Epoch 4/5\n",
            "231/235 [============================>.] - ETA: 0s - loss: 0.5040 - acc: 0.8277Epoch 1/5\n",
            "235/235 [==============================] - 8s 33ms/step - loss: 0.5034 - acc: 0.8278 - val_loss: 0.5305 - val_acc: 0.8175\n",
            "Epoch 5/5\n",
            "232/235 [============================>.] - ETA: 0s - loss: 0.4875 - acc: 0.8334Epoch 1/5\n",
            "235/235 [==============================] - 8s 32ms/step - loss: 0.4872 - acc: 0.8335 - val_loss: 0.5182 - val_acc: 0.8212\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tensorflow.python.keras.callbacks.History at 0x7fd6718552b0>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zgYBT9y-_z-7",
        "colab_type": "text"
      },
      "source": [
        "###3.7.6. 小结\n",
        "* keras提供的函数往往具有更好的数值稳定性。\n",
        "* 可以使用keras更简洁地实现softmax回归。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1DopcL_CJBUv",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}