Colab/Swift for TensorFlow for using PyTorch users No1.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Swift for TensorFlow for using PyTorch No.1",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "swift",
      "display_name": "Swift"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/khirotaka/Swift4TF/blob/master/Colab/Swift%20for%20TensorFlow%20for%20using%20PyTorch%20users%20No1.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IERWyXBLsP5_",
        "colab_type": "text"
      },
      "source": [
        "# PyTorch使いの為の Swift for TensorFlow"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kZRlD4utdPuX",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import TensorFlow"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5U65UGH2qDnI",
        "colab_type": "code",
        "outputId": "3cafb3af-32ef-47e8-bd3d-f02240b872a2",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 102
        }
      },
      "source": [
        "let seed = TensorFlow.randomSeedForTensorFlow(using: TensorFlowSeed(graph: 0, op: 0))    // Seed固定\n",
        "\n",
        "var x: Tensor<Float> = _Raw.empty(shape: [5, 3])    // TensorFlowの低レベルAPIにアクセスするには _Raw を使う。(Raw は非推奨になった)\n",
        "print(x)"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[[0.0, 0.0, 0.0],\r\n",
            " [0.0, 0.0, 0.0],\r\n",
            " [0.0, 0.0, 0.0],\r\n",
            " [0.0, 0.0, 0.0],\r\n",
            " [0.0, 0.0, 0.0]]\r\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IZKyNJjpUj6p",
        "colab_type": "text"
      },
      "source": [
        "Swift for TensorFlowには `ShapedArray` というものがあり、`numpy.ndarray` に近いものになっている。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dE42HygpUI87",
        "colab_type": "code",
        "outputId": "fd20e562-fe14-49dc-bc7a-a777bf72c62d",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 119
        }
      },
      "source": [
        "var xx: ShapedArray<Float> = ShapedArray<Float>(shape: [5, 5], repeating: 0.0)\n",
        "print(\"Shape of xx \\(xx.shape)\")\n",
        "print(xx)"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Shape of xx [5, 5]\r\n",
            "[[0.0, 0.0, 0.0, 0.0, 0.0],\r\n",
            " [0.0, 0.0, 0.0, 0.0, 0.0],\r\n",
            " [0.0, 0.0, 0.0, 0.0, 0.0],\r\n",
            " [0.0, 0.0, 0.0, 0.0, 0.0],\r\n",
            " [0.0, 0.0, 0.0, 0.0, 0.0]]\r\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Q4h_wYUXU4rj",
        "colab_type": "text"
      },
      "source": [
        "Swift for TensorFlowで、\n",
        "```python\n",
        "torch.randn(5, 3)\n",
        "```\n",
        "を行いたいのなら、次のようにすればいい。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "izx9c64PqQ_P",
        "colab_type": "code",
        "outputId": "1dd3a3a5-7582-4e47-bbab-db6ab91dc713",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 102
        }
      },
      "source": [
        "var x: Tensor<Float> = Tensor<Float>(randomNormal: [5, 3], seed: seed)\n",
        "print(x)"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[[ 0.59503317,  -1.2243367,  -1.2307329],\r\n",
            " [ 0.64013857,   1.1960995,    2.365863],\r\n",
            " [ 0.54103607,   0.5393618,  0.27644297],\r\n",
            " [ -0.5943514, -0.94016635,  -0.4811743],\r\n",
            " [ 0.35051414,  -1.8140777,  0.38941053]]\r\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "38u2TIqFVNj5",
        "colab_type": "text"
      },
      "source": [
        "```python\n",
        "torch.zeros(5, 3)\n",
        "```\n",
        "や\n",
        "```python\n",
        "torch.ones(5, 3)\n",
        "```\n",
        "も以下のように書ける。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5ETf5rotqYKg",
        "colab_type": "code",
        "outputId": "9c5cebb7-6c5f-4813-c61d-96d89c144fa9",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        }
      },
      "source": [
        "var x = Tensor<Float>(ones: [5, 3])\n",
        "print(x)\n",
        "\n",
        "print(\"----------\")    // swift で \"-\"*50 が使えないのホント不便\n",
        "\n",
        "var xx = Tensor<UInt8>(zeros: [5, 3])\n",
        "print(xx)"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[[1.0, 1.0, 1.0],\r\n",
            " [1.0, 1.0, 1.0],\r\n",
            " [1.0, 1.0, 1.0],\r\n",
            " [1.0, 1.0, 1.0],\r\n",
            " [1.0, 1.0, 1.0]]\r\n",
            "----------\r\n",
            "[[0, 0, 0],\r\n",
            " [0, 0, 0],\r\n",
            " [0, 0, 0],\r\n",
            " [0, 0, 0],\r\n",
            " [0, 0, 0]]\r\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gRxK71jUWCcI",
        "colab_type": "text"
      },
      "source": [
        "`zeros_like` も `ones_like` もある。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "m6wV0dHTWNT1",
        "colab_type": "code",
        "outputId": "a13d0906-27f0-46ec-c567-5cc843d510cf",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        }
      },
      "source": [
        "var tmp = ShapedArray<Float>(repeating: 0.0, shape: [5, 5])\n",
        "\n",
        "var x = Tensor<Float>(onesLike: Tensor<Float>(tmp))\n",
        "print(x)\n",
        "\n",
        "print(\"----------\")\n",
        "\n",
        "var xx = Tensor<Float>(zerosLike: Tensor<Float>(tmp))\n",
        "print(xx)"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[[1.0, 1.0, 1.0, 1.0, 1.0],\r\n",
            " [1.0, 1.0, 1.0, 1.0, 1.0],\r\n",
            " [1.0, 1.0, 1.0, 1.0, 1.0],\r\n",
            " [1.0, 1.0, 1.0, 1.0, 1.0],\r\n",
            " [1.0, 1.0, 1.0, 1.0, 1.0]]\r\n",
            "----------\r\n",
            "[[0.0, 0.0, 0.0, 0.0, 0.0],\r\n",
            " [0.0, 0.0, 0.0, 0.0, 0.0],\r\n",
            " [0.0, 0.0, 0.0, 0.0, 0.0],\r\n",
            " [0.0, 0.0, 0.0, 0.0, 0.0],\r\n",
            " [0.0, 0.0, 0.0, 0.0, 0.0]]\r\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mj8ekf_3WyXh",
        "colab_type": "text"
      },
      "source": [
        "当然、\n",
        "```python\n",
        "torch.tensor([5.5, 3.0])\n",
        "```\n",
        "のような書き方も可能"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "oDqmv1q-qq--",
        "colab_type": "code",
        "outputId": "6baa16ad-d8d1-4388-f6c4-d77383105d9c",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "var x = Tensor<Float>([5.5, 3])\n",
        "print(x)"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[5.5, 3.0]\r\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VVqHYfl0W-K2",
        "colab_type": "text"
      },
      "source": [
        "先ほどから出ている正規分布からサンプリングした乱数以外にも"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dusBSceixEaw",
        "colab_type": "code",
        "outputId": "8c69b3d6-421d-462b-bcb0-14844463ed78",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 102
        }
      },
      "source": [
        "var y = Tensor<Float>(randomNormal: [5, 3], seed: seed)\n",
        "\n",
        "print(y)"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[[ 0.59503317,  -1.2243367,  -1.2307329],\r\n",
            " [ 0.64013857,   1.1960995,    2.365863],\r\n",
            " [ 0.54103607,   0.5393618,  0.27644297],\r\n",
            " [ -0.5943514, -0.94016635,  -0.4811743],\r\n",
            " [ 0.35051414,  -1.8140777,  0.38941053]]\r\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oA3_JO9d2Iot",
        "colab_type": "text"
      },
      "source": [
        "## Using GPU\n",
        "\n",
        "```swift\n",
        "var x = withDevice(.gpu, 0) {\n",
        "    Tensor<Float>([1, 2, 3])\n",
        "}    // これでGPUに載せる事が出来る。\n",
        "\n",
        "```"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2aFP0MEMXf51",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import Foundation    // Dateを使う為"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sMz2w3CExzUo",
        "colab_type": "code",
        "outputId": "8a7792b9-f613-46f5-b6a7-df6462840b77",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "let start = Date()\n",
        "\n",
        "var x = TensorFlow.withDevice(.gpu, 0) {\n",
        "    Tensor<Float>(randomNormal: [5000, 6000], seed: seed)\n",
        "}\n",
        "\n",
        "var y = TensorFlow.withDevice(.gpu, 0) {\n",
        "    matmul(x, x.transposed())\n",
        "}\n",
        "\n",
        "let end = Date().timeIntervalSince(start)\n",
        "print(end)"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "0.5399130582809448\r\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IHyp_gc_1x-x",
        "colab_type": "code",
        "outputId": "2ac6f64c-81a3-497c-a1ff-e53bb8440f4d",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "let start = Date()\n",
        "\n",
        "var x = TensorFlow.withDevice(.cpu, 0) {\n",
        "    Tensor<Float>(randomNormal: [5000, 6000], seed: seed)\n",
        "}\n",
        "\n",
        "var y = TensorFlow.withDevice(.cpu, 0) {\n",
        "    matmul(x, x.transposed())\n",
        "}\n",
        "\n",
        "let end = Date().timeIntervalSince(start)\n",
        "print(end)"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "5.296565055847168\r\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "urjIe2UNYxQ_",
        "colab_type": "text"
      },
      "source": [
        "## Numpyとの互換性"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FX9kHFdrY0RP",
        "colab_type": "text"
      },
      "source": [
        "### Numpy -> TensorFlow on Swift\n",
        "\n",
        "まず、Pythonと site-packages にインストールされているPythonライブラリを使う。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jmSy_3OtYKNK",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import PythonKit\n",
        "\n",
        "let np = Python.import(\"numpy\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SIlUdInDYPRH",
        "colab_type": "code",
        "outputId": "346f6f27-210f-483f-8f04-91a416d1fa11",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        }
      },
      "source": [
        "Python.version    // Python のバージョン"
      ],
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "3.6.9 (default, Nov  7 2019, 10:44:02) \n",
              "[GCC 8.3.0]\n"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "A-Eu6HbKbudE",
        "colab_type": "text"
      },
      "source": [
        "```python\n",
        "let np = Python.import(\"numpy\")\n",
        "```\n",
        "\n",
        "で`numpy` が使えるようになります。\n",
        "あとは、いつもPythonで書いてるように関数やオブジェクトにアクセスできます。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "htQIpWAhYsKl",
        "colab_type": "code",
        "outputId": "d272682a-3d57-48c0-e33e-4ee42c5261df",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "var np_a = np.array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype: np.float32)\n",
        "np_a = np_a.reshape(2, 5)\n",
        "print(np_a.shape)    // Numpy オブジェクト"
      ],
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "(2, 5)\r\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mb4lOvooZNAI",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        },
        "outputId": "bfce7fb5-05c4-4fdb-b2f3-39d000645276"
      },
      "source": [
        "var swift_a = Tensor<Float>(numpy: np_a)!    // Numpyを元に Tensor を作成。オプショナル型なので注意\n",
        "print(String(describing: type(of: swift_a)))\n",
        "print(swift_a.shape)"
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Tensor<Float>\r\n",
            "[2, 5]\r\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DbfQyddQccR3",
        "colab_type": "text"
      },
      "source": [
        "### Swift -> Numpy"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vTy22jZTZeum",
        "colab_type": "code",
        "outputId": "e3548db3-e436-4d45-ea76-33f116e59abd",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "var new_ndarray = swift_a.makeNumpyArray()    // PyTorch の numpy() と同じノリ\n",
        "\n",
        "print(new_ndarray.shape)"
      ],
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "(2, 5)\r\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "H_pIdk-dddn6",
        "colab_type": "text"
      },
      "source": [
        "Pythonに内蔵されている関数 (例えば、 `type` を使いたいのなら、以下のようにすれば行ける)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "i_B-CDX9c28f",
        "colab_type": "code",
        "outputId": "5d2b73bc-7adb-450b-9109-04c4a8b2cb7f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "Python.type(new_ndarray)"
      ],
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<class 'numpy.ndarray'>\n"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 17
        }
      ]
    }
  ]
}