05_deploy_lookup_and_scann_caip.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Part 5: Deploying the Embedding Lookup and ScaNN Index to AI Platform\n",
    "\n",
    "This tutorial shows how to use Matrix Factorization algorithm in BigQuery ML to generate embeddings for items based on their cooccurrence statistics. The generated item embeddings can be then used to find similar items.\n",
    "\n",
    "Part 5 covers the following steps:\n",
    "1. Deploying the Embedding Lookup SavedModel to AI Platform Prediction. \n",
    "2. Deploying the ScaNN index to AI Platform Prediction, using a Custom Container, for real-time similar item matching, which works as follows:\n",
    "    1. Accepts a query item Id.\n",
    "    2. Looks up the embedding of the query item Id from Embedding Lookup Model in AI Platform Prediction.\n",
    "    3. Uses the ScaNN index to find similar item Ids for the given query item embedding.\n",
    "    4. Returns a list of the similar item Ids to the query item Id.\n",
    "3. (Optional) Exporting and deploying the BigQuery ML matrix Factorization model to AI Platform for exact matching.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Import libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import tensorflow as tf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Configure GCP environment settings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "PROJECT_ID = 'chernikov-playground'\n",
    "BUCKET = 'cooltacular-gcp1'\n",
    "REGION = 'us-west1'\n",
    "\n",
    "ARTIFACTS_REPOSITORY_NAME = 'ml-serving'\n",
    "\n",
    "EMBEDDNIG_LOOKUP_MODEL_OUTPUT_DIR = f'gs://{BUCKET}/bqml/embedding_lookup_model'\n",
    "EMBEDDNIG_LOOKUP_MODEL_NAME = 'item_embedding_lookup'\n",
    "EMBEDDNIG_LOOKUP_MODEL_VERSION = 'v1'\n",
    "\n",
    "INDEX_DIR = f'gs://{BUCKET}/bqml/scann_index'\n",
    "SCANN_MODEL_NAME = 'index_server'\n",
    "SCANN_MODEL_VERSION = 'v1'\n",
    "\n",
    "KIND = 'song'\n",
    "\n",
    "!gcloud config set project $PROJECT_ID"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Authenticate your GCP account\n",
    "This is required if you run the notebook in Colab"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "try:\n",
    "  from google.colab import auth\n",
    "  auth.authenticate_user()\n",
    "  print(\"Colab user is authenticated.\")\n",
    "except: pass"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Serve an Embedding Lookup Model on AI Platform Prediction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!gcloud ai-platform models create {EMBEDDNIG_LOOKUP_MODEL_NAME} --region={REGION}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!gcloud ai-platform versions create {EMBEDDNIG_LOOKUP_MODEL_VERSION} \\\n",
    "  --region={REGION} \\\n",
    "  --model={EMBEDDNIG_LOOKUP_MODEL_VERSION} \\\n",
    "  --origin={EMBEDDNIG_LOOKUP_MODEL_OUTPUT_DIR} \\\n",
    "  --runtime-version=2.2 \\\n",
    "  --framework=TensorFlow \\\n",
    "  --python-version=3.7 \\\n",
    "  --machine-type=n1-standard-2\n",
    "\n",
    "print(\"The model version is deployed to AI Platform Prediciton.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Test the deployed embedding lookup AI Platform Prediction model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import googleapiclient.discovery\n",
    "from google.api_core.client_options import ClientOptions\n",
    "\n",
    "api_endpoint = f'https://{REGION}-ml.googleapis.com'\n",
    "client_options = ClientOptions(api_endpoint=api_endpoint)\n",
    "service = googleapiclient.discovery.build(\n",
    "    serviceName='ml', version='v1', client_options=client_options)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def caip_embedding_lookup(input_items):\n",
    "  request_body = {'instances': input_items}\n",
    "  service_name = f'projects/{PROJECT_ID}/models/{EMBEDDNIG_LOOKUP_MODEL_NAME}/versions/{EMBEDDNIG_LOOKUP_MODEL_VERSION}'\n",
    "  print(f'Calling : {service_name}')\n",
    "  response = service.projects().predict(\n",
    "    name=service_name, body=request_body).execute()\n",
    "\n",
    "  if 'error' in response:\n",
    "    raise RuntimeError(response['error'])\n",
    "\n",
    "  return response['predictions']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "input_items = ['2114406', '2114402 2120788', 'abc123']\n",
    "\n",
    "embeddings = caip_embedding_lookup(input_items)\n",
    "print(f'Embeddings retrieved: {len(embeddings)}')\n",
    "for idx, embedding in enumerate(embeddings):\n",
    "  print(f'{input_items[idx]}: {embedding[:5]}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Build a Custom Prediction Container for the ScaNN Index\n",
    "The custom container runs a [gunicorn](https://gunicorn.org/) web server hosting a [Flask](https://flask.palletsprojects.com/en/1.1.x/quickstart/) application. The app loads the ScaNN index to use it for similar items matching.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create an Artifact Registry for the Docker container images."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!gcloud beta artifacts repositories create {ARTIFACTS_REPOSITORY_NAME} \\\n",
    "  --location={REGION} \\\n",
    "  --repository-format=docker"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!gcloud beta auth configure-docker {REGION}-docker.pkg.dev --quiet"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Use Cloud Build to build the Docker container image\n",
    "\n",
    "You need to enable the `Compute Engine` service account in [Cloud Build setting](https://console.cloud.google.com/cloud-build/settings/service-account)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "IMAGE_URL = f'{REGION}-docker.pkg.dev/{PROJECT_ID}/{ARTIFACTS_REPOSITORY_NAME}/{SCANN_MODEL_NAME}:{SCANN_MODEL_VERSION}'\n",
    "PORT=5001\n",
    "\n",
    "SUBSTITUTIONS = ''\n",
    "SUBSTITUTIONS += f'_IMAGE_URL={IMAGE_URL},'\n",
    "SUBSTITUTIONS += f'_PORT={PORT}'\n",
    "\n",
    "!gcloud builds submit --config=index_server/cloudbuild.yaml \\\n",
    "  --substitutions={SUBSTITUTIONS} \\\n",
    "  --timeout=1h"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "repository_id = f'{REGION}-docker.pkg.dev/{PROJECT_ID}/{ARTIFACTS_REPOSITORY_NAME}'\n",
    "\n",
    "!gcloud beta artifacts docker images list {repository_id}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create a Service Account for the Container to Access Cloud Resources"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "SERVICE_ACCOUNT_NAME = 'caip-serving'\n",
    "SERVICE_ACCOUNT_EMAIL = f'{SERVICE_ACCOUNT_NAME}@{PROJECT_ID}.iam.gserviceaccount.com'\n",
    "!gcloud iam service-accounts create {SERVICE_ACCOUNT_NAME} \\\n",
    "  --description=\"Service account for AI Platform Prediction to access cloud resources.\" "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We need to grant the Cloud ML Engine (AI Platform) service account the **iam.serviceAccountAdmin** privileges, and grant the new service account the privileges required by the ScaNN matching service: **storage.objectViewer** and **ml.developer**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!gcloud projects describe {PROJECT_ID} --format=\"value(projectNumber)\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "PROJECT_NUMBER=900786220115 # Change to your project number\n",
    "\n",
    "!gcloud projects add-iam-policy-binding {PROJECT_ID} \\\n",
    "  --role=roles/iam.serviceAccountAdmin \\\n",
    "  --member=serviceAccount:service-{PROJECT_NUMBER}@cloud-ml.google.com.iam.gserviceaccount.com\n",
    "\n",
    "!gcloud projects add-iam-policy-binding {PROJECT_ID} \\\n",
    "  --role=roles/storage.objectViewer \\\n",
    "  --member=serviceAccount:{SERVICE_ACCOUNT_EMAIL}\n",
    "    \n",
    "!gcloud projects add-iam-policy-binding {PROJECT_ID} \\\n",
    "  --role=roles/ml.developer \\\n",
    "  --member=serviceAccount:{SERVICE_ACCOUNT_EMAIL}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Serve the ScaNN Index Custom Container on AI Platform Prediction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!gcloud ai-platform models create {SCANN_MODEL_NAME} --region={REGION}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "HEALTH_ROUTE=f'/v1/models/{SCANN_MODEL_NAME}/versions/{SCANN_MODEL_VERSION}'\n",
    "PREDICT_ROUTE=f'/v1/models/{SCANN_MODEL_NAME}/versions/{SCANN_MODEL_VERSION}:predict'\n",
    "\n",
    "ENV_VARIABLES = f'PROJECT_ID={PROJECT_ID},'\n",
    "ENV_VARIABLES += f'REGION={REGION},'\n",
    "ENV_VARIABLES += f'INDEX_DIR={INDEX_DIR},'\n",
    "ENV_VARIABLES += f'EMBEDDNIG_LOOKUP_MODEL_NAME={EMBEDDNIG_LOOKUP_MODEL_NAME},'\n",
    "ENV_VARIABLES += f'EMBEDDNIG_LOOKUP_MODEL_VERSION={EMBEDDNIG_LOOKUP_MODEL_VERSION}'\n",
    "\n",
    "!gcloud beta ai-platform versions create {SCANN_MODEL_VERSION} \\\n",
    "  --region={REGION} \\\n",
    "  --model={SCANN_MODEL_NAME} \\\n",
    "  --image={IMAGE_URL} \\\n",
    "  --ports={PORT} \\\n",
    "  --predict-route={PREDICT_ROUTE} \\\n",
    "  --health-route={HEALTH_ROUTE} \\\n",
    "  --machine-type=n1-standard-4 \\\n",
    "  --env-vars={ENV_VARIABLES} \\\n",
    "  --service-account={SERVICE_ACCOUNT_EMAIL}\n",
    "\n",
    "print(\"The model version is deployed to AI Platform Prediction.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test the Deployed ScaNN Index Service"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from google.cloud import datastore\n",
    "import requests\n",
    "client = datastore.Client(PROJECT_ID)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def caip_scann_match(query_items, show=10):\n",
    "  request_body = {\n",
    "      'instances': [{\n",
    "          'query':' '.join(query_items), \n",
    "          'show':show\n",
    "      }]\n",
    "   }\n",
    "  \n",
    "  service_name = f'projects/{PROJECT_ID}/models/{SCANN_MODEL_NAME}/versions/{SCANN_MODEL_VERSION}'\n",
    "  print(f'Calling: {service_name}')  \n",
    "  response = service.projects().predict(\n",
    "    name=service_name, body=request_body).execute()\n",
    "\n",
    "  if 'error' in response:\n",
    "    raise RuntimeError(response['error'])\n",
    "\n",
    "  match_tokens = response['predictions']\n",
    "  keys = [client.key(KIND, int(key)) for key in match_tokens]\n",
    "  items = client.get_multi(keys)\n",
    "  return items\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "songs = {\n",
    "    '2114406': 'Metallica: Nothing Else Matters',\n",
    "    '2114402': 'Metallica: The Unforgiven',\n",
    "    '2120788': 'Limp Bizkit: My Way',\n",
    "    '2120786': 'Limp Bizkit: My Generation',\n",
    "    '1086322': 'Jacques Brel: Ne Me Quitte Pas',\n",
    "    '3129954': 'Édith Piaf: Non, Je Ne Regrette Rien',\n",
    "    '53448': 'France Gall: Ella, Elle l\\'a',\n",
    "    '887688': 'Enrique Iglesias: Tired Of Being Sorry',\n",
    "    '562487': 'Shakira: Hips Don\\'t Lie',\n",
    "    '833391': 'Ricky Martin: Livin\\' la Vida Loca',\n",
    "    '1098069': 'Snoop Dogg: Drop It Like It\\'s Hot',\n",
    "    '910683': '2Pac: California Love',\n",
    "    '1579481': 'Dr. Dre: The Next Episode',\n",
    "    '2675403': 'Eminem: Lose Yourself',\n",
    "    '2954929': 'Black Sabbath: Iron Man',\n",
    "    '625169': 'Black Sabbath: Paranoid',\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for item_Id, desc in songs.items():\n",
    "  print(desc)\n",
    "  print(\"==================\")\n",
    "  similar_items = caip_scann_match([item_Id], 5)\n",
    "  for similar_item in similar_items:\n",
    "    print(f'- {similar_item[\"artist\"]}: {similar_item[\"track_title\"]}')\n",
    "  print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## (Optional) Export and Deploy the BigQuery Model to AI Platform"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Import the model from BigQuery ML to Cloud Storage as a SavedModel\n",
    "This will export the matrix factorization as an exact matching model. The model takes Item1_Id as an input and outputs the top 50 recommended item2_Ids."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "BQ_DATASET_NAME = 'recommendations'\n",
    "BQML_MODEL_NAME = 'item_matching_model'\n",
    "BQML_MODEL_VERSION = 'v1' \n",
    "BQML_MODEL_OUTPUT_DIR = f'gs://{BUCKET}/bqml/item_matching_model'\n",
    "\n",
    "!bq --quiet extract -m {BQ_DATASET_NAME}.{BQML_MODEL_NAME} {BQML_MODEL_OUTPUT_DIR}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!saved_model_cli show --dir {BQML_MODEL_OUTPUT_DIR} --tag_set serve --signature_def serving_default"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Deploy the item matching SavedModel to AI Platform Prediction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!gcloud ai-platform models create {BQML_MODEL_NAME} --region={REGION}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!gcloud ai-platform versions create {MODEL_VERSION} \\\n",
    "  --region={REGION} \\\n",
    "  --model={BQML_MODEL_NAME} \\\n",
    "  --origin={BQML_MODEL_OUTPUT_DIR} \\\n",
    "  --runtime-version=2.2 \\\n",
    "  --framework=TensorFlow \\\n",
    "  --python-version=3.7 \\\n",
    "  --machine-type=n1-standard-2\n",
    "\n",
    "print(\"The model version is deployed to AI Platform Prediciton.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def caip_bqml_matching(input_items, show):\n",
    "  request_body = {'instances': input_items}\n",
    "  service_name = f'projects/{PROJECT_ID}/models/{BQML_MODEL_NAME}/versions/{BQML_MODEL_VERSION}'\n",
    "  print(f'Calling : {service_name}')\n",
    "  response = service.projects().predict(\n",
    "    name=service_name, body=request_body).execute()\n",
    "\n",
    "  if 'error' in response:\n",
    "    raise RuntimeError(response['error'])\n",
    "    \n",
    "  \n",
    "  match_tokens = response['predictions'][0][\"predicted_item2_Id\"][:show]\n",
    "  keys = [client.key(KIND, int(key)) for key in match_tokens]\n",
    "  items = client.get_multi(keys)\n",
    "  return items\n",
    "\n",
    "  return response['predictions']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for item_Id, desc in songs.items():\n",
    "  print(desc)\n",
    "  print(\"==================\")\n",
    "  similar_items = caip_bqml_matching([int(item_Id)], 5)\n",
    "  for similar_item in similar_items:\n",
    "    print(f'- {similar_item[\"artist\"]}: {similar_item[\"track_title\"]}')\n",
    "  print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## License\n",
    "\n",
    "Copyright 2020 Google LLC\n",
    "\n",
    "Licensed under the Apache License, Version 2.0 (the \"License\");\n",
    "you may not use this file except in compliance with the License. You may obtain a copy of the License at: http://www.apache.org/licenses/LICENSE-2.0\n",
    "\n",
    "Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. \n",
    "\n",
    "See the License for the specific language governing permissions and limitations under the License.\n",
    "\n",
    "**This is not an official Google product but sample code provided for an educational purpose**"
   ]
  }
 ],
 "metadata": {
  "environment": {
   "name": "tf2-2-3-gpu.2-3.m59",
   "type": "gcloud",
   "uri": "gcr.io/deeplearning-platform-release/tf2-2-3-gpu.2-3:m59"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}