Section 2 - Federated Learning.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Section 2 - Federated Learning.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "include_colab_link": true
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.1"
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/agungsantoso/private-ai/blob/master/Section%202%20-%20Federated%20Learning.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QzZfjd7S4_4S",
        "colab_type": "text"
      },
      "source": [
        "# Section: Federated Learning"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rhuPVkCt4_4U",
        "colab_type": "text"
      },
      "source": [
        "# Lesson: Introducing Federated Learning\n",
        "\n",
        "Federated Learning is a technique for training Deep Learning models on data to which you do not have access. Basically:\n",
        "\n",
        "Federated Learning: Instead of bringing all the data to one machine and training a model, we bring the model to the data, train it locally, and merely upload \"model updates\" to a central server.\n",
        "\n",
        "Use Cases:\n",
        "\n",
        "    - app company (Texting prediction app)\n",
        "    - predictive maintenance (automobiles / industrial engines)\n",
        "    - wearable medical devices\n",
        "    - ad blockers / autotomplete in browsers (Firefox/Brave)\n",
        "    \n",
        "Challenge Description: data is distributed amongst sources but we cannot aggregated it because of:\n",
        "\n",
        "    - privacy concerns: legal, user discomfort, competitive dynamics\n",
        "    - engineering: the bandwidth/storage requirements of aggregating the larger dataset"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CNLUlLn_4_4V",
        "colab_type": "text"
      },
      "source": [
        "# Lesson: Introducing / Installing PySyft\n",
        "\n",
        "In order to perform Federated Learning, we need to be able to use Deep Learning techniques on remote machines. This will require a new set of tools. Specifically, we will use an extensin of PyTorch called PySyft.\n",
        "\n",
        "### Install PySyft\n",
        "\n",
        "The easiest way to install the required libraries is with [Conda](https://docs.conda.io/projects/conda/en/latest/user-guide/overview.html). Create a new environment, then install the dependencies in that environment. In your terminal:\n",
        "\n",
        "```bash\n",
        "conda create -n pysyft python=3\n",
        "conda activate pysyft # some older version of conda require \"source activate pysyft\" instead.\n",
        "conda install jupyter notebook\n",
        "pip install syft\n",
        "pip install numpy\n",
        "```\n",
        "\n",
        "If you have any errors relating to zstd - run the following (if everything above installed fine then skip this step):\n",
        "\n",
        "```\n",
        "pip install --upgrade --force-reinstall zstd\n",
        "```\n",
        "\n",
        "and then retry installing syft (pip install syft).\n",
        "\n",
        "If you are using Windows, I suggest installing [Anaconda and using the Anaconda Prompt](https://docs.anaconda.com/anaconda/user-guide/getting-started/) to work from the command line. \n",
        "\n",
        "With this environment activated and in the repo directory, launch Jupyter Notebook:\n",
        "\n",
        "```bash\n",
        "jupyter notebook\n",
        "```\n",
        "\n",
        "If any part of this doesn't work for you (or any of the tests fail) - first check the [README](https://github.com/OpenMined/PySyft.git) for installation help and then open a Github Issue or ping the #beginner channel in our slack! [slack.openmined.org](http://slack.openmined.org/)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SxfVPaaA4_4W",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import torch as th"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HgE1tMqc4_4a",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "23bb1182-8e30-4f2b-a2d0-05ab007137ef"
      },
      "source": [
        "x = th.tensor([1,2,3,4,5])\n",
        "x"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([1, 2, 3, 4, 5])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 2
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ucDAz8N34_4g",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "y = x + x"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "p40Nowag4_4i",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "ca8b0499-18fa-41bf-ddee-e0efd7fad1c6"
      },
      "source": [
        "print(y)"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "tensor([ 2,  4,  6,  8, 10])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "42jOzpsL4_4o",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "outputId": "7bf695a2-5b85-4a92-a5ee-ecb0d00d3743"
      },
      "source": [
        "!pip install syft\n",
        "import syft as sy"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Collecting syft\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/38/2e/16bdefc78eb089e1efa9704c33b8f76f035a30dc935bedd7cbb22f6dabaa/syft-0.1.21a1-py3-none-any.whl (219kB)\n",
            "\u001b[K     |████████████████████████████████| 225kB 5.0MB/s \n",
            "\u001b[?25hCollecting lz4>=2.1.6 (from syft)\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/0a/c6/96bbb3525a63ebc53ea700cc7d37ab9045542d33b4d262d0f0408ad9bbf2/lz4-2.1.10-cp36-cp36m-manylinux1_x86_64.whl (385kB)\n",
            "\u001b[K     |████████████████████████████████| 389kB 43.3MB/s \n",
            "\u001b[?25hCollecting websocket-client>=0.56.0 (from syft)\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/29/19/44753eab1fdb50770ac69605527e8859468f3c0fd7dc5a76dd9c4dbd7906/websocket_client-0.56.0-py2.py3-none-any.whl (200kB)\n",
            "\u001b[K     |████████████████████████████████| 204kB 47.0MB/s \n",
            "\u001b[?25hRequirement already satisfied: scikit-learn>=0.21.0 in /usr/local/lib/python3.6/dist-packages (from syft) (0.21.2)\n",
            "Requirement already satisfied: torch>=1.1 in /usr/local/lib/python3.6/dist-packages (from syft) (1.1.0)\n",
            "Requirement already satisfied: torchvision>=0.3.0 in /usr/local/lib/python3.6/dist-packages (from syft) (0.3.0)\n",
            "Requirement already satisfied: Flask>=1.0.2 in /usr/local/lib/python3.6/dist-packages (from syft) (1.1.1)\n",
            "Collecting tf-encrypted>=0.5.4 (from syft)\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/07/ce/da9916e7e78f736894b15538b702c0b213fd5d60a7fd6e481d74033a90c0/tf_encrypted-0.5.6-py3-none-manylinux1_x86_64.whl (1.4MB)\n",
            "\u001b[K     |████████████████████████████████| 1.4MB 43.3MB/s \n",
            "\u001b[?25hCollecting msgpack>=0.6.1 (from syft)\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/92/7e/ae9e91c1bb8d846efafd1f353476e3fd7309778b582d2fb4cea4cc15b9a2/msgpack-0.6.1-cp36-cp36m-manylinux1_x86_64.whl (248kB)\n",
            "\u001b[K     |████████████████████████████████| 256kB 45.0MB/s \n",
            "\u001b[?25hRequirement already satisfied: numpy>=1.14.0 in /usr/local/lib/python3.6/dist-packages (from syft) (1.16.4)\n",
            "Collecting flask-socketio>=3.3.2 (from syft)\n",
            "  Downloading https://files.pythonhosted.org/packages/4b/68/fe4806d3a0a5909d274367eb9b3b87262906c1515024f46c2443a36a0c82/Flask_SocketIO-4.1.0-py2.py3-none-any.whl\n",
            "Collecting zstd>=1.4.0.0 (from syft)\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/8e/27/1ea8086d37424e83ab692015cc8dd7d5e37cf791e339633a40dc828dfb74/zstd-1.4.0.0.tar.gz (450kB)\n",
            "\u001b[K     |████████████████████████████████| 450kB 43.0MB/s \n",
            "\u001b[?25hCollecting websockets>=7.0 (from syft)\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/61/5e/2fe6afbb796c6ac5c006460b5503cd674d33706660337f2dbff10d4aa12d/websockets-8.0-cp36-cp36m-manylinux1_x86_64.whl (72kB)\n",
            "\u001b[K     |████████████████████████████████| 81kB 25.2MB/s \n",
            "\u001b[?25hRequirement already satisfied: tblib>=1.4.0 in /usr/local/lib/python3.6/dist-packages (from syft) (1.4.0)\n",
            "Requirement already satisfied: six in /usr/local/lib/python3.6/dist-packages (from websocket-client>=0.56.0->syft) (1.12.0)\n",
            "Requirement already satisfied: scipy>=0.17.0 in /usr/local/lib/python3.6/dist-packages (from scikit-learn>=0.21.0->syft) (1.3.0)\n",
            "Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.6/dist-packages (from scikit-learn>=0.21.0->syft) (0.13.2)\n",
            "Requirement already satisfied: pillow>=4.1.1 in /usr/local/lib/python3.6/dist-packages (from torchvision>=0.3.0->syft) (4.3.0)\n",
            "Requirement already satisfied: itsdangerous>=0.24 in /usr/local/lib/python3.6/dist-packages (from Flask>=1.0.2->syft) (1.1.0)\n",
            "Requirement already satisfied: Jinja2>=2.10.1 in /usr/local/lib/python3.6/dist-packages (from Flask>=1.0.2->syft) (2.10.1)\n",
            "Requirement already satisfied: click>=5.1 in /usr/local/lib/python3.6/dist-packages (from Flask>=1.0.2->syft) (7.0)\n",
            "Requirement already satisfied: Werkzeug>=0.15 in /usr/local/lib/python3.6/dist-packages (from Flask>=1.0.2->syft) (0.15.4)\n",
            "Collecting pyyaml>=5.1 (from tf-encrypted>=0.5.4->syft)\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/a3/65/837fefac7475963d1eccf4aa684c23b95aa6c1d033a2c5965ccb11e22623/PyYAML-5.1.1.tar.gz (274kB)\n",
            "\u001b[K     |████████████████████████████████| 276kB 46.2MB/s \n",
            "\u001b[?25hRequirement already satisfied: tensorflow<2,>=1.12.0 in /usr/local/lib/python3.6/dist-packages (from tf-encrypted>=0.5.4->syft) (1.14.0)\n",
            "Collecting python-socketio>=2.1.0 (from flask-socketio>=3.3.2->syft)\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/26/1b/57e860a86f2a01be86ae1dacfa0cd8c4dfbfcd4593322268b61b5a07b564/python_socketio-4.2.0-py2.py3-none-any.whl (46kB)\n",
            "\u001b[K     |████████████████████████████████| 51kB 19.5MB/s \n",
            "\u001b[?25hRequirement already satisfied: olefile in /usr/local/lib/python3.6/dist-packages (from pillow>=4.1.1->torchvision>=0.3.0->syft) (0.46)\n",
            "Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.6/dist-packages (from Jinja2>=2.10.1->Flask>=1.0.2->syft) (1.1.1)\n",
            "Requirement already satisfied: absl-py>=0.7.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow<2,>=1.12.0->tf-encrypted>=0.5.4->syft) (0.7.1)\n",
            "Requirement already satisfied: gast>=0.2.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow<2,>=1.12.0->tf-encrypted>=0.5.4->syft) (0.2.2)\n",
            "Requirement already satisfied: wrapt>=1.11.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow<2,>=1.12.0->tf-encrypted>=0.5.4->syft) (1.11.2)\n",
            "Requirement already satisfied: astor>=0.6.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow<2,>=1.12.0->tf-encrypted>=0.5.4->syft) (0.8.0)\n",
            "Requirement already satisfied: google-pasta>=0.1.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow<2,>=1.12.0->tf-encrypted>=0.5.4->syft) (0.1.7)\n",
            "Requirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow<2,>=1.12.0->tf-encrypted>=0.5.4->syft) (1.1.0)\n",
            "Requirement already satisfied: grpcio>=1.8.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow<2,>=1.12.0->tf-encrypted>=0.5.4->syft) (1.15.0)\n",
            "Requirement already satisfied: keras-applications>=1.0.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow<2,>=1.12.0->tf-encrypted>=0.5.4->syft) (1.0.8)\n",
            "Requirement already satisfied: wheel>=0.26 in /usr/local/lib/python3.6/dist-packages (from tensorflow<2,>=1.12.0->tf-encrypted>=0.5.4->syft) (0.33.4)\n",
            "Requirement already satisfied: tensorflow-estimator<1.15.0rc0,>=1.14.0rc0 in /usr/local/lib/python3.6/dist-packages (from tensorflow<2,>=1.12.0->tf-encrypted>=0.5.4->syft) (1.14.0)\n",
            "Requirement already satisfied: tensorboard<1.15.0,>=1.14.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow<2,>=1.12.0->tf-encrypted>=0.5.4->syft) (1.14.0)\n",
            "Requirement already satisfied: keras-preprocessing>=1.0.5 in /usr/local/lib/python3.6/dist-packages (from tensorflow<2,>=1.12.0->tf-encrypted>=0.5.4->syft) (1.1.0)\n",
            "Requirement already satisfied: protobuf>=3.6.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow<2,>=1.12.0->tf-encrypted>=0.5.4->syft) (3.7.1)\n",
            "Collecting python-engineio>=3.8.0 (from python-socketio>=2.1.0->flask-socketio>=3.3.2->syft)\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/bd/b8/0fc389ca5c445051b37b17802f80bbf1b51c1e3b48b772ee608efbb90583/python_engineio-3.8.2.post1-py2.py3-none-any.whl (119kB)\n",
            "\u001b[K     |████████████████████████████████| 122kB 15.8MB/s \n",
            "\u001b[?25hRequirement already satisfied: h5py in /usr/local/lib/python3.6/dist-packages (from keras-applications>=1.0.6->tensorflow<2,>=1.12.0->tf-encrypted>=0.5.4->syft) (2.8.0)\n",
            "Requirement already satisfied: setuptools>=41.0.0 in /usr/local/lib/python3.6/dist-packages (from tensorboard<1.15.0,>=1.14.0->tensorflow<2,>=1.12.0->tf-encrypted>=0.5.4->syft) (41.0.1)\n",
            "Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.6/dist-packages (from tensorboard<1.15.0,>=1.14.0->tensorflow<2,>=1.12.0->tf-encrypted>=0.5.4->syft) (3.1.1)\n",
            "Building wheels for collected packages: zstd, pyyaml\n",
            "  Building wheel for zstd (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Stored in directory: /root/.cache/pip/wheels/ad/9a/f4/3105b5209674ac77fcca7fede95184c62a95df0196888e0e76\n",
            "  Building wheel for pyyaml (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Stored in directory: /root/.cache/pip/wheels/16/27/a1/775c62ddea7bfa62324fd1f65847ed31c55dadb6051481ba3f\n",
            "Successfully built zstd pyyaml\n",
            "Installing collected packages: lz4, websocket-client, pyyaml, tf-encrypted, msgpack, python-engineio, python-socketio, flask-socketio, zstd, websockets, syft\n",
            "  Found existing installation: PyYAML 3.13\n",
            "    Uninstalling PyYAML-3.13:\n",
            "      Successfully uninstalled PyYAML-3.13\n",
            "  Found existing installation: msgpack 0.5.6\n",
            "    Uninstalling msgpack-0.5.6:\n",
            "      Successfully uninstalled msgpack-0.5.6\n",
            "Successfully installed flask-socketio-4.1.0 lz4-2.1.10 msgpack-0.6.1 python-engineio-3.8.2.post1 python-socketio-4.2.0 pyyaml-5.1.1 syft-0.1.21a1 tf-encrypted-0.5.6 websocket-client-0.56.0 websockets-8.0 zstd-1.4.0.0\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "WARNING: Logging before flag parsing goes to stderr.\n",
            "W0717 08:48:07.784892 140682828916608 secure_random.py:26] Falling back to insecure randomness since the required custom op could not be found for the installed version of TensorFlow. Fix this by compiling custom ops. Missing file was '/usr/local/lib/python3.6/dist-packages/tf_encrypted/operations/secure_random/secure_random_module_tf_1.14.0.so'\n",
            "W0717 08:48:07.803103 140682828916608 deprecation_wrapper.py:119] From /usr/local/lib/python3.6/dist-packages/tf_encrypted/session.py:26: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.\n",
            "\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wup2VMEM4_4r",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "hook = sy.TorchHook(th)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rFmUkPAF4_4t",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "8dee3c22-2a89-43d3-b5c8-c42a7c317023"
      },
      "source": [
        "th.tensor([1,2,3,4,5])"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([1, 2, 3, 4, 5])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wag45QUF4_4x",
        "colab_type": "text"
      },
      "source": [
        "# Lesson: Basic Remote Execution in PySyft"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2G5j_Qox4_4y",
        "colab_type": "text"
      },
      "source": [
        "## PySyft => Remote PyTorch\n",
        "\n",
        "The essence of Federated Learning is the ability to train models in parallel on a wide number of machines. Thus, we need the ability to tell remote machines to execute the operations required for Deep Learning.\n",
        "\n",
        "Thus, instead of using Torch tensors - we're now going to work with **pointers** to tensors. Let me show you what I mean. First, let's create a \"pretend\" machine owned by a \"pretend\" person - we'll call him Bob."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wzvLpzgn4_4y",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "bob = sy.VirtualWorker(hook, id=\"bob\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "AST4le504_41",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "7f215256-84a1-4415-b2a8-09c30eef2b7b"
      },
      "source": [
        "bob._objects"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Dr9P_gkc4_44",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "x = th.tensor([1,2,3,4,5])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fA929S984_47",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "x = x.send(bob)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "I73sL9GC4_49",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "8bc38950-4e9c-4523-9dfe-4387c9d6962b"
      },
      "source": [
        "bob._objects"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{47686733235: tensor([1, 2, 3, 4, 5])}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ADxqHY2J4_5C",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "d19e9a9c-22a2-41b5-a20e-852a3af17dcd"
      },
      "source": [
        "x.location"
      ],
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<VirtualWorker id:bob #objects:1>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QrJflWvn4_5G",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "23a29751-afac-4879-9961-eb8d2cd89d34"
      },
      "source": [
        "x.id_at_location"
      ],
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "47686733235"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 14
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mzBd9bOE4_5K",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "d225ccda-c17d-40c8-c9b8-0df4befedff6"
      },
      "source": [
        "x.id"
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "79273979718"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 15
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "z2lzuRV64_5P",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "538d6c24-290f-4118-f2c2-283e1b2394f7"
      },
      "source": [
        "x.owner"
      ],
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<VirtualWorker id:me #objects:0>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9iWsejrH4_5U",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "e856f11a-10a8-4adf-cf62-e04817229ebb"
      },
      "source": [
        "hook.local_worker"
      ],
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<VirtualWorker id:me #objects:0>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pP94S2Oo4_5X",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "0593ba33-9ad1-420b-de0a-c9db7de6fcdb"
      },
      "source": [
        "x"
      ],
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(Wrapper)>[PointerTensor | me:79273979718 -> bob:47686733235]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 18
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BvutjgM_4_5b",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "dbc7f3be-a81a-4606-c8ff-b2d6286ed86b"
      },
      "source": [
        "x = x.get()\n",
        "x"
      ],
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([1, 2, 3, 4, 5])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 19
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fXLQUqsC4_5f",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "230b8cc4-f1a0-4343-e127-d2096549d0cd"
      },
      "source": [
        "bob._objects"
      ],
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 20
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pHB4fHDl4_5i",
        "colab_type": "text"
      },
      "source": [
        "# Project: Playing with Remote Tensors\n",
        "\n",
        "In this project, I want you to .send() and .get() a tensor to TWO workers by calling .send(bob,alice). This will first require the creation of another VirtualWorker called alice."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BidUpepp4_5j",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "alice = sy.VirtualWorker(hook, id=\"alice\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Mh2Bw-Mn4_5l",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "x = th.tensor([1,2,3,4,5])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "A6mw7z5C4_5p",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "x_ptr = x.send(bob, alice)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Wi3nlReS4_5r",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "94446b9b-1a02-48f0-9ac1-f3471a15d87e"
      },
      "source": [
        "x_ptr.get()"
      ],
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[tensor([1, 2, 3, 4, 5]), tensor([1, 2, 3, 4, 5])]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 24
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Uag4ckjX4_5u",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "x = th.tensor([1,2,3,4,5]).send(bob, alice)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zzZKf4e24_5w",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "529bf3e4-662e-4c73-8744-671b23610795"
      },
      "source": [
        "x.get(sum_results=True)"
      ],
      "execution_count": 26,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([ 2,  4,  6,  8, 10])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 26
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "i6ndSeOD4_50",
        "colab_type": "text"
      },
      "source": [
        "# Lesson: Introducing Remote Arithmetic"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "f_WcAw7b4_51",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "x = th.tensor([1,2,3,4,5]).send(bob)\n",
        "y = th.tensor([1,1,1,1,1]).send(bob)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lRqRKNW54_53",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "6918166c-3c3d-42d5-fab1-365c59d7be6c"
      },
      "source": [
        "x"
      ],
      "execution_count": 28,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(Wrapper)>[PointerTensor | me:40425583006 -> bob:63519041415]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 28
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mQzXsQfO4_56",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "5fff0999-65d0-48bf-c7d5-d0ae2ae84c9f"
      },
      "source": [
        "y"
      ],
      "execution_count": 29,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(Wrapper)>[PointerTensor | me:59517228175 -> bob:18324039139]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 29
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BzXWLXCt4_59",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "z = x + y"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Rlpfk2E84_5-",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "086e771e-8bcf-49a8-bf78-ca23374dd912"
      },
      "source": [
        "z"
      ],
      "execution_count": 31,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(Wrapper)>[PointerTensor | me:94717697710 -> bob:93843476700]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 31
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lSdtX8yB4_6B",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "a9b0782c-0c70-40c6-e6bf-ba5c5d5e1b5d"
      },
      "source": [
        "z = z.get()\n",
        "z"
      ],
      "execution_count": 32,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([2, 3, 4, 5, 6])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 32
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PMXoEpBZ4_6D",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "bf2f484d-ba74-45e9-9520-dabb60382e1d"
      },
      "source": [
        "z = th.add(x,y)\n",
        "z"
      ],
      "execution_count": 33,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(Wrapper)>[PointerTensor | me:48498986313 -> bob:56750502689]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 33
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1VEgNrh34_6G",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "264168d4-7cb6-4118-b602-56677cab2dd3"
      },
      "source": [
        "z = z.get()\n",
        "z"
      ],
      "execution_count": 34,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([2, 3, 4, 5, 6])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 34
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "i8mZIDAs4_6J",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "x = th.tensor([1.,2,3,4,5], requires_grad=True).send(bob)\n",
        "y = th.tensor([1.,1,1,1,1], requires_grad=True).send(bob)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Ov92qgFM4_6K",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "z = (x + y).sum()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Lu01o66t4_6N",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "de9e4f47-e794-43dc-dc2b-7edbfd8d5261"
      },
      "source": [
        "z.backward()"
      ],
      "execution_count": 37,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(Wrapper)>[PointerTensor | me:59673631940 -> bob:17550709192]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 37
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "AZoWOwCx4_6P",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "x = x.get()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "F5jyzg4Z4_6V",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "fc099f40-807e-4792-b52a-8d56b5e54cee"
      },
      "source": [
        "x"
      ],
      "execution_count": 39,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([1., 2., 3., 4., 5.], requires_grad=True)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 39
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Sn2AFQr_4_6Y",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "2939b641-7b27-4f4e-ce3c-6d6a837a9000"
      },
      "source": [
        "x.grad"
      ],
      "execution_count": 40,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([1., 1., 1., 1., 1.])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 40
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Lcl9gHGg4_6d",
        "colab_type": "text"
      },
      "source": [
        "# Project: Learn a Simple Linear Model\n",
        "\n",
        "In this project, I'd like for you to create a simple linear model which will solve for the following dataset below. You should use only Variables and .backward() to do so (no optimizers or nn.Modules). Furthermore, you must do so with both the data and the model being located on Bob's machine."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tlt6EEb-4_6f",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "input = th.tensor([[1.,1],[0,1,],[1,0],[0,0]], requires_grad=True).send(bob)\n",
        "target = th.tensor([[1.],[1],[0],[0]], requires_grad=True).send(bob)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KiPzvRnh4_6i",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "weights = th.tensor([[0.],[0.]], requires_grad=True).send(bob)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zgPZiLY94_6k",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 193
        },
        "outputId": "4eaa0b7d-b2d0-45c1-de44-398c7867b13d"
      },
      "source": [
        "for i in range(10):\n",
        "\n",
        "    pred = input.mm(weights)\n",
        "\n",
        "    loss = ((pred - target)**2).sum()\n",
        "\n",
        "    loss.backward()\n",
        "\n",
        "    weights.data.sub_(weights.grad * 0.1)\n",
        "    weights.grad *= 0\n",
        "\n",
        "    print(loss.get().data)"
      ],
      "execution_count": 43,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "tensor(2.)\n",
            "tensor(0.5600)\n",
            "tensor(0.2432)\n",
            "tensor(0.1372)\n",
            "tensor(0.0849)\n",
            "tensor(0.0538)\n",
            "tensor(0.0344)\n",
            "tensor(0.0220)\n",
            "tensor(0.0141)\n",
            "tensor(0.0090)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_7dxJT0Z4_6n",
        "colab_type": "text"
      },
      "source": [
        "# Lesson: Garbage Collection and Common Errors\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "w3ATSP6c4_6q",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "bob = bob.clear_objects()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BqnMpm-l4_6r",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "72670d05-f63a-4738-ccd0-49bead1c38a5"
      },
      "source": [
        "bob._objects"
      ],
      "execution_count": 45,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 45
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mM3NJjS-4_6u",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "x = th.tensor([1,2,3,4,5]).send(bob)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YFT96KpQ4_6w",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "4ebf02a4-b234-4319-c79c-474d0a126a29"
      },
      "source": [
        "bob._objects"
      ],
      "execution_count": 47,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{28821090329: tensor([1, 2, 3, 4, 5])}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 47
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kA2zJWlw4_6z",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "del x"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "S3ZZo_Ju4_61",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "7ac24895-6181-44dc-ce48-b3d041ebb276"
      },
      "source": [
        "bob._objects"
      ],
      "execution_count": 49,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 49
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "51QZWPPN4_64",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "x = th.tensor([1,2,3,4,5]).send(bob)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6ASs6QRG4_67",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "3239b2a5-2d14-4767-868d-5d6834b3b92e"
      },
      "source": [
        "bob._objects"
      ],
      "execution_count": 51,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{27890796475: tensor([1, 2, 3, 4, 5])}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 51
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1oz5SyqN4_6-",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "x = \"asdf\""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gYAKzUqu4_7A",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "69e2a544-0671-4c41-8282-fa1864171373"
      },
      "source": [
        "bob._objects"
      ],
      "execution_count": 53,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 53
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BGo777bz4_7E",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "x = th.tensor([1,2,3,4,5]).send(bob)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Vx8oDgOE4_7F",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "8e7e1ed3-8a38-4f8e-d855-dd4f7824d6ff"
      },
      "source": [
        "x"
      ],
      "execution_count": 55,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(Wrapper)>[PointerTensor | me:43842354525 -> bob:79611036282]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 55
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Q6EA6HUJ4_7H",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "7d3e1a2a-e0c2-4fe6-8931-43e415bd4bc1"
      },
      "source": [
        "bob._objects"
      ],
      "execution_count": 56,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{79611036282: tensor([1, 2, 3, 4, 5])}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 56
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "c3jw2ByA4_7K",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "x = \"asdf\""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SH6yWZt54_7M",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "80b87f48-4cea-4f23-d61f-4bf947f4b49a"
      },
      "source": [
        "bob._objects"
      ],
      "execution_count": 58,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{79611036282: tensor([1, 2, 3, 4, 5])}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 58
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0AJ3WNqN4_7O",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "del x"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ry4EA02q4_7Q",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "72691a95-8d52-407a-b6e3-26e6a817d5a4"
      },
      "source": [
        "bob._objects"
      ],
      "execution_count": 60,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{79611036282: tensor([1, 2, 3, 4, 5])}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 60
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sKzXaeQY4_7T",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "d4c85078-1d57-4850-9faa-df738177bce4"
      },
      "source": [
        "bob = bob.clear_objects()\n",
        "bob._objects"
      ],
      "execution_count": 61,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 61
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xnqcmakL4_7V",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "for i in range(1000):\n",
        "    x = th.tensor([1,2,3,4,5]).send(bob)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_I_Rwo4Y4_7X",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "75ca74eb-a398-4393-90ca-edab48d6ed45"
      },
      "source": [
        "bob._objects"
      ],
      "execution_count": 64,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{37052575502: tensor([1, 2, 3, 4, 5])}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 64
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DGHXwCvv4_7a",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "x = th.tensor([1,2,3,4,5]).send(bob)\n",
        "y = th.tensor([1,1,1,1,1])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UOTPCImj4_7b",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 512
        },
        "outputId": "5abcb44b-8600-4d13-8d1e-2cc1ba11f533"
      },
      "source": [
        "z = x + y"
      ],
      "execution_count": 66,
      "outputs": [
        {
          "output_type": "error",
          "ename": "TensorsNotCollocatedException",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mPureTorchTensorFoundError\u001b[0m                 Traceback (most recent call last)",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/syft/frameworks/torch/hook/hook.py\u001b[0m in \u001b[0;36moverloaded_native_method\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m    668\u001b[0m                     new_self, new_args, new_kwargs = syft.frameworks.torch.hook_args.hook_method_args(\n\u001b[0;32m--> 669\u001b[0;31m                         \u001b[0mmethod_name\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkwargs\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    670\u001b[0m                     )\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/syft/frameworks/torch/hook/hook_args.py\u001b[0m in \u001b[0;36mhook_method_args\u001b[0;34m(attr, method_self, args, kwargs)\u001b[0m\n\u001b[1;32m    121\u001b[0m         \u001b[0;31m# Try running it\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 122\u001b[0;31m         \u001b[0mnew_self\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnew_args\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mhook_args\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmethod_self\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    123\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/syft/frameworks/torch/hook/hook_args.py\u001b[0m in \u001b[0;36m<lambda>\u001b[0;34m(x)\u001b[0m\n\u001b[1;32m    350\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 351\u001b[0;31m     \u001b[0;32mreturn\u001b[0m \u001b[0;32mlambda\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mf\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlambdas\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    352\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/syft/frameworks/torch/hook/hook_args.py\u001b[0m in \u001b[0;36mtwo_fold\u001b[0;34m(lambdas, args, **kwargs)\u001b[0m\n\u001b[1;32m    515\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mtwo_fold\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlambdas\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 516\u001b[0;31m     \u001b[0;32mreturn\u001b[0m \u001b[0mlambdas\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlambdas\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    517\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/syft/frameworks/torch/hook/hook_args.py\u001b[0m in \u001b[0;36m<lambda>\u001b[0;34m(x)\u001b[0m\n\u001b[1;32m    350\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 351\u001b[0;31m     \u001b[0;32mreturn\u001b[0m \u001b[0;32mlambda\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mf\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlambdas\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    352\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/syft/frameworks/torch/hook/hook_args.py\u001b[0m in \u001b[0;36mtuple_one_fold\u001b[0;34m(lambdas, args)\u001b[0m\n\u001b[1;32m    509\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mtuple_one_fold\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlambdas\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 510\u001b[0;31m         \u001b[0;32mreturn\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mlambdas\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    511\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/syft/frameworks/torch/hook/hook_args.py\u001b[0m in \u001b[0;36m<lambda>\u001b[0;34m(i)\u001b[0m\n\u001b[1;32m    328\u001b[0m         \u001b[0;31m# Last if not, rule is probably == 1 so use type to return the right transformation.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 329\u001b[0;31m         \u001b[0;32melse\u001b[0m \u001b[0;32mlambda\u001b[0m \u001b[0mi\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mforward_func\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mtype\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    330\u001b[0m         \u001b[0;32mfor\u001b[0m \u001b[0ma\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mr\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mzip\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mrules\u001b[0m\u001b[0;34m)\u001b[0m  \u001b[0;31m# And do this for all the args / rules provided\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/syft/frameworks/torch/hook/hook_args.py\u001b[0m in \u001b[0;36m<lambda>\u001b[0;34m(i)\u001b[0m\n\u001b[1;32m     55\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mhasattr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"child\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 56\u001b[0;31m     \u001b[0;32melse\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0m_\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0m_\u001b[0m \u001b[0;32min\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mthrow\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mPureTorchTensorFoundError\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     57\u001b[0m     \u001b[0mtorch\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnn\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mParameter\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;32mlambda\u001b[0m \u001b[0mi\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mi\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mchild\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/syft/frameworks/torch/hook/hook_args.py\u001b[0m in \u001b[0;36m<genexpr>\u001b[0;34m(.0)\u001b[0m\n\u001b[1;32m     55\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mhasattr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"child\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 56\u001b[0;31m     \u001b[0;32melse\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0m_\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0m_\u001b[0m \u001b[0;32min\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mthrow\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mPureTorchTensorFoundError\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     57\u001b[0m     \u001b[0mtorch\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnn\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mParameter\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;32mlambda\u001b[0m \u001b[0mi\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mi\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mchild\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mPureTorchTensorFoundError\u001b[0m: ",
            "\nDuring handling of the above exception, another exception occurred:\n",
            "\u001b[0;31mTensorsNotCollocatedException\u001b[0m             Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-66-8c8f78e0676d>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mz\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mx\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0my\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/syft/frameworks/torch/hook/hook.py\u001b[0m in \u001b[0;36moverloaded_native_method\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m    671\u001b[0m                 \u001b[0;32mexcept\u001b[0m \u001b[0mBaseException\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    672\u001b[0m                     \u001b[0;31m# we can make some errors more descriptive with this method\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 673\u001b[0;31m                     \u001b[0;32mraise\u001b[0m \u001b[0mroute_method_exception\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0me\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    674\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    675\u001b[0m                 \u001b[0;31m# Send the new command to the appropriate class and get the response\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mTensorsNotCollocatedException\u001b[0m: You tried to call a method involving two tensors where one tensor is actually located on another machine (is a PointerTensor). Call .get() on the PointerTensor or .send(bob) on the other tensor.\n\nTensor A: [PointerTensor | me:39772857675 -> bob:26740326081]\nTensor B: tensor([1, 1, 1, 1, 1])"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3vlIN6Gr4_7d",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "x = th.tensor([1,2,3,4,5]).send(bob)\n",
        "y = th.tensor([1,1,1,1,1]).send(alice)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "AlLHCbHt4_7f",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 336
        },
        "outputId": "c8003a7e-ddf7-416b-c430-133bf56b3c43"
      },
      "source": [
        "z = x + y"
      ],
      "execution_count": 68,
      "outputs": [
        {
          "output_type": "error",
          "ename": "TensorsNotCollocatedException",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mTensorsNotCollocatedException\u001b[0m             Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-68-8c8f78e0676d>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mz\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mx\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0my\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/syft/frameworks/torch/hook/hook.py\u001b[0m in \u001b[0;36moverloaded_native_method\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m    675\u001b[0m                 \u001b[0;31m# Send the new command to the appropriate class and get the response\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    676\u001b[0m                 \u001b[0mmethod\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mgetattr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnew_self\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmethod_name\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 677\u001b[0;31m                 \u001b[0mresponse\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmethod\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0mnew_args\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mnew_kwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    678\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    679\u001b[0m                 \u001b[0;31m# For inplace methods, just directly return self\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/syft/frameworks/torch/hook/hook.py\u001b[0m in \u001b[0;36moverloaded_pointer_method\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m    506\u001b[0m                 \u001b[0;32mif\u001b[0m \u001b[0misinstance\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mPointerTensor\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    507\u001b[0m                     \u001b[0;32mif\u001b[0m \u001b[0margs\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlocation\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mid\u001b[0m \u001b[0;34m!=\u001b[0m \u001b[0mlocation\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mid\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 508\u001b[0;31m                         \u001b[0;32mraise\u001b[0m \u001b[0mTensorsNotCollocatedException\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mpointer\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0margs\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mattr\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    509\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    510\u001b[0m             \u001b[0;31m# Send the command\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mTensorsNotCollocatedException\u001b[0m: You tried to call __add__ involving two tensors which are not on the same machine! One tensor is on <VirtualWorker id:bob #objects:2> while the other is on <VirtualWorker id:alice #objects:1>. Use a combination of .move(), .get(), and/or .send() to co-locate them to the same machine."
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EXBivNiA4_7h",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "saMV-8gA4_7i",
        "colab_type": "text"
      },
      "source": [
        "# Lesson: Toy Federated Learning\n",
        "\n",
        "Let's start by training a toy model the centralized way. This is about a simple as models get. We first need:\n",
        "\n",
        "- a toy dataset\n",
        "- a model\n",
        "- some basic training logic for training a model to fit the data."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Va-GzbeJ4_7i",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from torch import nn, optim"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "uszEnyzx4_7k",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# A Toy Dataset\n",
        "data = th.tensor([[1.,1],[0,1],[1,0],[0,0]], requires_grad=True)\n",
        "target = th.tensor([[1.],[1], [0], [0]], requires_grad=True)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Axm0rDNW4_7l",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# A Toy Model\n",
        "model = nn.Linear(2,1)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lbq6r76Q4_7o",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "opt = optim.SGD(params=model.parameters(), lr=0.1)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ekfbGy_T4_7r",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 369
        },
        "outputId": "536c97b3-df7d-4b9e-a25d-5494dd660373"
      },
      "source": [
        "def train(iterations=20):\n",
        "    for iter in range(iterations):\n",
        "        opt.zero_grad()\n",
        "\n",
        "        pred = model(data)\n",
        "\n",
        "        loss = ((pred - target)**2).sum()\n",
        "\n",
        "        loss.backward()\n",
        "\n",
        "        opt.step()\n",
        "\n",
        "        print(loss.data)\n",
        "        \n",
        "train()"
      ],
      "execution_count": 73,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "tensor(4.8018)\n",
            "tensor(0.4236)\n",
            "tensor(0.0776)\n",
            "tensor(0.0398)\n",
            "tensor(0.0283)\n",
            "tensor(0.0210)\n",
            "tensor(0.0158)\n",
            "tensor(0.0119)\n",
            "tensor(0.0090)\n",
            "tensor(0.0068)\n",
            "tensor(0.0051)\n",
            "tensor(0.0039)\n",
            "tensor(0.0029)\n",
            "tensor(0.0022)\n",
            "tensor(0.0017)\n",
            "tensor(0.0013)\n",
            "tensor(0.0010)\n",
            "tensor(0.0008)\n",
            "tensor(0.0006)\n",
            "tensor(0.0004)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PkIi0VEm4_7w",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "data_bob = data[0:2].send(bob)\n",
        "target_bob = target[0:2].send(bob)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wW7HqfMr4_7z",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "data_alice = data[2:4].send(alice)\n",
        "target_alice = target[2:4].send(alice)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QAdIZHOa4_71",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "datasets = [(data_bob, target_bob), (data_alice, target_alice)]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "91-7o2MV4_73",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def train(iterations=20):\n",
        "\n",
        "    model = nn.Linear(2,1)\n",
        "    opt = optim.SGD(params=model.parameters(), lr=0.1)\n",
        "    \n",
        "    for iter in range(iterations):\n",
        "\n",
        "        for _data, _target in datasets:\n",
        "\n",
        "            # send model to the data\n",
        "            model = model.send(_data.location)\n",
        "\n",
        "            # do normal training\n",
        "            opt.zero_grad()\n",
        "            pred = model(_data)\n",
        "            loss = ((pred - _target)**2).sum()\n",
        "            loss.backward()\n",
        "            opt.step()\n",
        "\n",
        "            # get smarter model back\n",
        "            model = model.get()\n",
        "\n",
        "            print(loss.get())"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CH9wXemN4_76",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 721
        },
        "outputId": "4f82eadd-7900-4d37-e88a-fd18b6f8281a"
      },
      "source": [
        "train()"
      ],
      "execution_count": 78,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "tensor(9.6533, requires_grad=True)\n",
            "tensor(0.4615, requires_grad=True)\n",
            "tensor(0.4610, requires_grad=True)\n",
            "tensor(0.3949, requires_grad=True)\n",
            "tensor(0.1891, requires_grad=True)\n",
            "tensor(0.2346, requires_grad=True)\n",
            "tensor(0.1067, requires_grad=True)\n",
            "tensor(0.1362, requires_grad=True)\n",
            "tensor(0.0616, requires_grad=True)\n",
            "tensor(0.0789, requires_grad=True)\n",
            "tensor(0.0356, requires_grad=True)\n",
            "tensor(0.0457, requires_grad=True)\n",
            "tensor(0.0206, requires_grad=True)\n",
            "tensor(0.0265, requires_grad=True)\n",
            "tensor(0.0119, requires_grad=True)\n",
            "tensor(0.0154, requires_grad=True)\n",
            "tensor(0.0069, requires_grad=True)\n",
            "tensor(0.0089, requires_grad=True)\n",
            "tensor(0.0040, requires_grad=True)\n",
            "tensor(0.0052, requires_grad=True)\n",
            "tensor(0.0023, requires_grad=True)\n",
            "tensor(0.0030, requires_grad=True)\n",
            "tensor(0.0013, requires_grad=True)\n",
            "tensor(0.0017, requires_grad=True)\n",
            "tensor(0.0008, requires_grad=True)\n",
            "tensor(0.0010, requires_grad=True)\n",
            "tensor(0.0004, requires_grad=True)\n",
            "tensor(0.0006, requires_grad=True)\n",
            "tensor(0.0003, requires_grad=True)\n",
            "tensor(0.0003, requires_grad=True)\n",
            "tensor(0.0001, requires_grad=True)\n",
            "tensor(0.0002, requires_grad=True)\n",
            "tensor(8.6314e-05, requires_grad=True)\n",
            "tensor(0.0001, requires_grad=True)\n",
            "tensor(4.9896e-05, requires_grad=True)\n",
            "tensor(6.7422e-05, requires_grad=True)\n",
            "tensor(2.8842e-05, requires_grad=True)\n",
            "tensor(3.9320e-05, requires_grad=True)\n",
            "tensor(1.6673e-05, requires_grad=True)\n",
            "tensor(2.2959e-05, requires_grad=True)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iVVtnNt44_77",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jKou5Wcs4_79",
        "colab_type": "text"
      },
      "source": [
        "# Lesson: Advanced Remote Execution Tools\n",
        "\n",
        "In the last section we trained a toy model using Federated Learning. We did this by calling .send() and .get() on our model, sending it to the location of training data, updating it, and then bringing it back. However, at the end of the example we realized that we needed to go a bit further to protect people privacy. Namely, we want to average the gradients BEFORE calling .get(). That way, we won't ever see anyone's exact gradient (thus better protecting their privacy!!!)\n",
        "\n",
        "But, in order to do this, we need a few more pieces:\n",
        "\n",
        "- use a pointer to send a Tensor directly to another worker\n",
        "\n",
        "And in addition, while we're here, we're going to learn about a few more advanced tensor operations as well which will help us both with this example and a few in the future!"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ihvmyRwm4_7-",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "7d728e25-9d42-4c89-8f78-18603546d7ea"
      },
      "source": [
        "bob.clear_objects()\n",
        "alice.clear_objects()"
      ],
      "execution_count": 79,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<VirtualWorker id:alice #objects:0>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 79
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3liHlw1F4_8D",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "x = th.tensor([1,2,3,4,5]).send(bob)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iObnj6-B4_8F",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "x = x.send(alice)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "eMw9c-204_8G",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "1c3a5488-4a93-4072-8604-4ec831fca5de"
      },
      "source": [
        "bob._objects"
      ],
      "execution_count": 82,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{38149554374: tensor([1, 2, 3, 4, 5])}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 82
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2qiuW7QX4_8I",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "87152d94-10d3-4836-da28-40887c565eb1"
      },
      "source": [
        "alice._objects"
      ],
      "execution_count": 83,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{49481512756: (Wrapper)>[PointerTensor | alice:49481512756 -> bob:38149554374]}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 83
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0HNZJJLX4_8J",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "y = x + x"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hQtZMh_K4_8K",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "537635c4-e1de-41e8-f26d-6253758c2af0"
      },
      "source": [
        "y"
      ],
      "execution_count": 85,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(Wrapper)>[PointerTensor | me:19694459472 -> alice:15044166621]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 85
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sVelWULd4_8O",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 52
        },
        "outputId": "cef4f626-0633-4e23-b7a5-48d9533b161f"
      },
      "source": [
        "bob._objects"
      ],
      "execution_count": 86,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{11175543829: tensor([ 2,  4,  6,  8, 10]),\n",
              " 38149554374: tensor([1, 2, 3, 4, 5])}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 86
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rKtWJEf74_8Q",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 52
        },
        "outputId": "856947f1-c055-4500-f518-b04918ecb14f"
      },
      "source": [
        "alice._objects"
      ],
      "execution_count": 87,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{15044166621: (Wrapper)>[PointerTensor | alice:15044166621 -> bob:11175543829],\n",
              " 49481512756: (Wrapper)>[PointerTensor | alice:49481512756 -> bob:38149554374]}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 87
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dQEsePA84_8T",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "jon = sy.VirtualWorker(hook, id=\"jon\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "m3zKLr3s4_8U",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "bob.clear_objects()\n",
        "alice.clear_objects()\n",
        "\n",
        "x = th.tensor([1,2,3,4,5]).send(bob).send(alice)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6za4vucf4_8V",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "e2c7d711-9a27-471d-ff0d-7fc821e1fc7c"
      },
      "source": [
        "bob._objects"
      ],
      "execution_count": 90,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{28829788838: tensor([1, 2, 3, 4, 5])}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 90
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zWJk1Jdj4_8X",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "6a6c5220-50b8-4bb7-a0da-f223161bf7d8"
      },
      "source": [
        "alice._objects"
      ],
      "execution_count": 91,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{2751762222: (Wrapper)>[PointerTensor | alice:2751762222 -> bob:28829788838]}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 91
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PGM3-w2K4_8Z",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "1eaab00d-74e9-4c48-d99d-584e5ad230b6"
      },
      "source": [
        "x = x.get()\n",
        "x"
      ],
      "execution_count": 92,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(Wrapper)>[PointerTensor | me:2751762222 -> bob:28829788838]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 92
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2ptx5X7w4_8a",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "9be697ed-bee4-4513-9503-08b28ea54eca"
      },
      "source": [
        "bob._objects"
      ],
      "execution_count": 93,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{28829788838: tensor([1, 2, 3, 4, 5])}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 93
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BIqtU5fr4_8c",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "6f6e77b8-b176-4297-931d-ac23252164ee"
      },
      "source": [
        "alice._objects"
      ],
      "execution_count": 94,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 94
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LKLlNhkS4_8e",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "9fb0a9d2-c8ce-444e-e34a-82516af3608c"
      },
      "source": [
        "x = x.get()\n",
        "x"
      ],
      "execution_count": 95,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([1, 2, 3, 4, 5])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 95
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "w9B2Yhrt4_8g",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "faa48430-7f55-4ac7-b0d8-470eed7c0570"
      },
      "source": [
        "bob._objects"
      ],
      "execution_count": 96,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 96
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vTiEmN_a4_8h",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "bob.clear_objects()\n",
        "alice.clear_objects()\n",
        "\n",
        "x = th.tensor([1,2,3,4,5]).send(bob).send(alice)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sXvNqr_74_8i",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "6a6b8713-340c-451f-8e47-5b87f0e3786b"
      },
      "source": [
        "bob._objects"
      ],
      "execution_count": 98,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{59551609408: tensor([1, 2, 3, 4, 5])}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 98
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tbsYvLOX4_8j",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "3bcc5f66-4f67-44b1-9a83-661706b99685"
      },
      "source": [
        "alice._objects"
      ],
      "execution_count": 99,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{85416971553: (Wrapper)>[PointerTensor | alice:85416971553 -> bob:59551609408]}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 99
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OTIROPE_4_8n",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "del x"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gb9LU-z24_8o",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "d3d387c2-ec26-4c82-be8a-16c60c872f59"
      },
      "source": [
        "bob._objects"
      ],
      "execution_count": 101,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 101
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1iZKkmat4_8q",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "ebdecd3e-907e-4bb0-bdc0-527db7a55680"
      },
      "source": [
        "alice._objects"
      ],
      "execution_count": 102,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 102
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5OhbF8n-4_8u",
        "colab_type": "text"
      },
      "source": [
        "# Lesson: Pointer Chain Operations"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VYBUDKAl4_8v",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "9e5a958f-dee1-41c4-be01-94336711e1d9"
      },
      "source": [
        "bob.clear_objects()\n",
        "alice.clear_objects()"
      ],
      "execution_count": 119,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<VirtualWorker id:alice #objects:0>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 119
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "U0OGvpmT4_8x",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "x = th.tensor([1,2,3,4,5]).send(bob)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gXIPh_tF4_8z",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "55bfd8e5-bf42-4efd-efdc-edb958e40e03"
      },
      "source": [
        "bob._objects"
      ],
      "execution_count": 121,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{5811013067: tensor([1, 2, 3, 4, 5])}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 121
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5mijIzTG4_82",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "45d3146f-e45d-4f9b-f241-112e58f725af"
      },
      "source": [
        "alice._objects"
      ],
      "execution_count": 122,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 122
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cP--jgIQ4_84",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "ac66d825-b4f5-4d60-f960-0a2986be6db3"
      },
      "source": [
        "x.move(alice)"
      ],
      "execution_count": 123,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(Wrapper)>[PointerTensor | me:58008875625 -> alice:58008875625]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 123
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ocxu-Sll4_86",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "d8cf1daa-0612-451b-8b0f-8fdb964c6ea6"
      },
      "source": [
        "bob._objects"
      ],
      "execution_count": 124,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 124
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "aRWOW-sL4_88",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "b8ab0276-62e5-4f09-a916-950f09175381"
      },
      "source": [
        "alice._objects"
      ],
      "execution_count": 125,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{58008875625: tensor([1, 2, 3, 4, 5])}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 125
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "a9ZQ_KPL4_9B",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "x = th.tensor([1,2,3,4,5]).send(bob).send(alice)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zwxxH96F4_9C",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "49ebe06a-d92a-47ed-cba2-96a5939b8391"
      },
      "source": [
        "bob._objects"
      ],
      "execution_count": 127,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{81090552116: tensor([1, 2, 3, 4, 5])}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 127
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qRoV614S4_9E",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 52
        },
        "outputId": "6d62daf5-980c-485f-960d-44b0422cb5ad"
      },
      "source": [
        "alice._objects"
      ],
      "execution_count": 128,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{58008875625: tensor([1, 2, 3, 4, 5]),\n",
              " 70864437181: (Wrapper)>[PointerTensor | alice:70864437181 -> bob:81090552116]}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 128
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OHr0cvXB4_9F",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "4625ab31-9807-49be-849f-46ec26573945"
      },
      "source": [
        "x.remote_get()"
      ],
      "execution_count": 129,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(Wrapper)>[PointerTensor | me:99079322792 -> alice:70864437181]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 129
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iBJ62YsW4_9H",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "dac7ba04-19b4-4240-fc42-4fe9c92d96a7"
      },
      "source": [
        "bob._objects"
      ],
      "execution_count": 130,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 130
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bfB_JP904_9K",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "53d17e73-f12f-40e8-f1f8-f9728826f817"
      },
      "source": [
        "alice._objects"
      ],
      "execution_count": 131,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{58008875625: tensor([1, 2, 3, 4, 5]), 70864437181: tensor([1, 2, 3, 4, 5])}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 131
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sRM5GaxH4_9L",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "723b3b77-f99b-47c2-d416-128aceb8723d"
      },
      "source": [
        "x.move(bob)"
      ],
      "execution_count": 132,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(Wrapper)>[PointerTensor | me:99079322792 -> bob:99079322792]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 132
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "17nF93_n4_9M",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "ed3253c9-2518-4856-9dd6-e3a64ed6f14a"
      },
      "source": [
        "x"
      ],
      "execution_count": 133,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(Wrapper)>[PointerTensor | me:99079322792 -> bob:99079322792]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 133
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PikHTXzX4_9P",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "2764b219-2ac8-4b1b-c88a-b2706cd6867a"
      },
      "source": [
        "bob._objects"
      ],
      "execution_count": 134,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{99079322792: tensor([1, 2, 3, 4, 5])}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 134
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "M_VQuy354_9R",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "73d3526d-8b2a-44de-b8eb-2107ae9493a6"
      },
      "source": [
        "alice._objects"
      ],
      "execution_count": 135,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{58008875625: tensor([1, 2, 3, 4, 5])}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 135
        }
      ]
    }
  ]
}