notebook_examples/train-register-deploy-huggingface-pipeline.ipynb

{
 "cells": [
  {
   "cell_type": "raw",
   "id": "d9633263",
   "metadata": {},
   "source": [
    "@notebook{train-register-deploy-huggingface-pipeline.ipynb,\n",
    "    title: Train, register, and deploy HuggingFace Pipeline,\n",
    "    summary: Train, register, and deploy a huggingface pipeline.,\n",
    "    developed_on: pytorch110_p38_cpu_v1,\n",
    "    keywords: huggingface, deploy model, register model, train model,\n",
    "    license: Universal Permissive License v 1.0\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c90177eb",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Upgrade Oracle ADS to pick up latest features and maintain compatibility with Oracle Cloud Infrastructure.\n",
    "\n",
    "!pip install -U oracle-ads"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "87ff4b41",
   "metadata": {},
   "source": [
    "<font color=gray>Oracle Data Science service sample notebook.\n",
    "\n",
    "Copyright (c) 2023 Oracle, Inc.  All rights reserved.\n",
    "Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl.\n",
    "</font>\n",
    "\n",
    "***\n",
    "# <font color=red>Train, Register, and Deploy HuggingFace Model</font>\n",
    "<p style=\"margin-left:10%; margin-right:10%;\">by the <font color=teal> Oracle Cloud Infrastructure Data Science Service Team </font></p>\n",
    "\n",
    "***\n",
    "\n",
    "## Overview:\n",
    "\n",
    "The `HuggingFacePipelineModel` class in Accelerated Data Science (ADS) is designed to allow you to rapidly get a HuggingFace pipeline into production. The `.prepare()` method creates the model artifacts that are needed to deploy a functioning model without you having to configure it or write code. However, it does allow you to customize the `score.py` file as needed. Simulate a call to a deployed model with the `.verify()` method. This method calls the `load_model()` and `predict()` functions in the `score.py` file. Using `.verify()` allows you to debug your `score.py` file without having to deploy a model. The `.save()` method pushes your `HuggingFacePipelineModel` and the model artifacts to the model catalog. The `.deploy()` method deploys the model to a REST endpoint for you. Finally, the `.predict()` method allows you to call the endpoint to perform model inference.\n",
    "\n",
    "Compatible conda pack: [PyTorch](https://docs.oracle.com/en-us/iaas/data-science/using/conda-pytorch-fam.htm) for CPU on Python 3.8 (version 1.0)\n",
    "\n",
    "### Prerequisites\n",
    "\n",
    "This notebook requires authorization to work with the OCI Data Science Service. Details can be found [here](https://accelerated-data-science.readthedocs.io/en/latest/user_guide/cli/authentication.html#). For the purposes of this notebook what is important to to know is that resource principals will be used absent api_key authentication.\n",
    "\n",
    "---\n",
    "\n",
    "Datasets are provided as a convenience. Datasets are considered third-party content and are not considered materials under your agreement with Oracle.\n",
    "      \n",
    "You can access the `orcl_attrition` dataset license [here](https://oss.oracle.com/licenses/upl).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0129630a",
   "metadata": {},
   "outputs": [],
   "source": [
    "import ads\n",
    "import logging\n",
    "import tempfile\n",
    "import warnings\n",
    "import PIL.Image\n",
    "import requests\n",
    "import cloudpickle\n",
    "\n",
    "from transformers import pipeline\n",
    "from shutil import rmtree\n",
    "from ads.model import HuggingFacePipelineModel\n",
    "\n",
    "logging.basicConfig(format=\"%(levelname)s:%(message)s\", level=logging.ERROR)\n",
    "warnings.filterwarnings(\"ignore\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e03bafd8",
   "metadata": {},
   "source": [
    "<a id='intro'></a>\n",
    "# Introduction"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "28a3ac83",
   "metadata": {},
   "source": [
    "## Authenticate\n",
    "\n",
    "Authentication to the OCI Data Science service is required. Here we default to resource principals."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7606068d",
   "metadata": {},
   "outputs": [],
   "source": [
    "ads.set_auth(auth=\"resource_principal\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2452c577",
   "metadata": {},
   "source": [
    "<a id=\"intro_dataset\"></a>\n",
    "## Download Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "803e8103",
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "image_url = \"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/pipeline-cat-chonk.jpeg\"\n",
    "\n",
    "image = PIL.Image.open(requests.get(image_url, stream=True).raw)\n",
    "image_bytes = cloudpickle.dumps(image)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ecc4059c",
   "metadata": {},
   "source": [
    "## Download a Pretrained Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7c999282",
   "metadata": {},
   "outputs": [],
   "source": [
    "segmenter = pipeline(task=\"image-segmentation\", model=\"facebook/detr-resnet-50-panoptic\", revision=\"fc15262\")\n",
    "preds = segmenter(\n",
    "    \"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/pipeline-cat-chonk.jpeg\"\n",
    ")\n",
    "\n",
    "preds"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e50f9c20",
   "metadata": {},
   "source": [
    "## Prepare Conda Pack\n",
    "You can start with the PyTorch conda pack with slug `pytorch110_p38_cpu_v1`. \n",
    "- Run `pip install timm` since image segmentation model requires `timm`. \n",
    "- Then use `odsc conda init -b your_bucket_name -n bucket_namespace` to config where to store the published conda pack if you have not done this yet. \n",
    "- Lastly, run `odsc conda publish -s pytorch110_p38_cpu_v1`.\n",
    "- Once it's done, you can find the path of the pusblished conda pack under the Environment Explorer `Published` tab. Refresh the page if you cannot find it."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8312cd97",
   "metadata": {},
   "source": [
    "## Prepare the model\n",
    "Now paste the conda pack path here. Initiate the `HuggingFacePipelineModel` instance and call `prepare` to generate the artifact."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c3c77e4e",
   "metadata": {},
   "outputs": [],
   "source": [
    "conda_pack_path = \"<replace with the path of your published conda pack>\" # it looks like this \"oci://bucket_name@namespace/path\"\n",
    "\n",
    "huggingface_pipeline_model = HuggingFacePipelineModel(\n",
    "    segmenter, artifact_dir=tempfile.mkdtemp()\n",
    ")\n",
    "huggingface_pipeline_model.prepare(inference_conda_env=conda_pack_path, force_overwrite=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "266818d3",
   "metadata": {},
   "outputs": [],
   "source": [
    "huggingface_pipeline_model.summary_status()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "da10fc61",
   "metadata": {},
   "source": [
    "## Verify\n",
    "\n",
    "The verify method invokes the ``predict`` function defined inside ``score.py`` in the artifact_dir\n",
    "\n",
    "#### PIL.Image\n",
    "`HuggingFacePipelineModel` class supports images directly. It by default uses `cloudpickle` to serialize and deserialize the input data. You can pass in the image to `.verify` function to test locally if everything works."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "48548edd",
   "metadata": {},
   "outputs": [],
   "source": [
    "preds = huggingface_pipeline_model.verify(image)[\"prediction\"]\n",
    "[{\"score\": round(pred[\"score\"], 4), \"label\": pred[\"label\"]} for pred in preds]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "60bcd875",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(type(preds[0]['mask']))\n",
    "preds[0]['mask'][0][:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a45f0faa",
   "metadata": {},
   "source": [
    "#### Bytes\n",
    "If you are not using `HuggingFacePipelineModel` to invoke your model, then you need to pass in either Json Serializable data or bytes to the endpoint since Model Deployment only supports these two types currently. Again, you can use `.verify` to test locally. When `auto_serialize_data=True`, the data will be serialized by the default model input serializer which is `cloudpickle`. Set `auto_serialize_data=False` since the image is already serialized to bytes by cloudpickle. And in `score.py`, the payload will be deserialized back to image by `cloudpickle`. If `auto_serialize_data=True`, the image will be serialized twice and cause a problem."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "699f65d3",
   "metadata": {},
   "outputs": [],
   "source": [
    "preds = huggingface_pipeline_model.verify(image_bytes, auto_serialize_data=False)[\"prediction\"]\n",
    "[{\"score\": round(pred[\"score\"], 4), \"label\": pred[\"label\"]} for pred in preds]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1d889758",
   "metadata": {},
   "source": [
    "## Save"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2ccb0e6c",
   "metadata": {},
   "outputs": [],
   "source": [
    "huggingface_pipeline_model.save(display_name=\"HuggingFace Pipeline Model\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6b3bcc85",
   "metadata": {},
   "outputs": [],
   "source": [
    "huggingface_pipeline_model.summary_status()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c75513f6",
   "metadata": {},
   "source": [
    "## Deploy\n",
    "\n",
    "When the model is in the model catalog, you can use the model's `.deploy()` method to deploy it. This method allows you to specify the attributes of the deployment such as the display name, description, instance type and count, the maximum bandwidth, and logging groups. The next cell deploys the model with the default settings except for the custom bandwidth mbps, display name and logging. The `.deploy()` method returns a `ModelDeployment` object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6668c7d2",
   "metadata": {},
   "outputs": [],
   "source": [
    "deploy = huggingface_pipeline_model.deploy(\n",
    "    deployment_bandwidth_mbps = 100,\n",
    "    wait_for_completion = False,\n",
    "    display_name = \"HuggingFace Pipeline Model For Image Segmentation\",\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8c05bcd4",
   "metadata": {},
   "outputs": [],
   "source": [
    "huggingface_pipeline_model.summary_status()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "08f26233",
   "metadata": {},
   "source": [
    "## Predict\n",
    "\n",
    "After the deployment is active, you can call `predict()` on the model object to send request to the deployed endpoint. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "20061da9",
   "metadata": {},
   "outputs": [],
   "source": [
    "preds = huggingface_pipeline_model.predict(image)[\"prediction\"]\n",
    "[{\"score\": round(pred[\"score\"], 4), \"label\": pred[\"label\"]} for pred in preds]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f5f2c220",
   "metadata": {},
   "source": [
    "## Invoke Your Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "787711f2",
   "metadata": {},
   "outputs": [],
   "source": [
    "headers = {\"Content-Type\": \"application/octet-stream\"} \n",
    "endpoint = huggingface_pipeline_model.model_deployment.url + \"/predict\"\n",
    "\n",
    "preds = requests.post(endpoint, data=image_bytes, auth=ads.common.auth.default_signer()['signer'], headers=headers).json()\n",
    "[{\"score\": round(pred[\"score\"], 4), \"label\": pred[\"label\"]} for pred in preds['prediction']]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "70256e2f",
   "metadata": {},
   "source": [
    "<a id='clean_up'></a>\n",
    "# Clean Up\n",
    "\n",
    "This notebook created a model deployment and a model. This section cleans up those resources. \n",
    "\n",
    "The model deployment must be deleted before the model can be deleted. You use the `.delete_deployment()` method on the `HuggingFacePipelineModel` object to do this."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ed4b363c",
   "metadata": {},
   "outputs": [],
   "source": [
    "delete = huggingface_pipeline_model.delete_deployment(wait_for_completion=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9436bb27",
   "metadata": {},
   "source": [
    "After the model deployment has been deleted, the `.summary_status()` method shows that the model has been deleted and that the `predict()` method is not available."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d23fa29e",
   "metadata": {},
   "outputs": [],
   "source": [
    "huggingface_pipeline_model.summary_status()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "4cc37a71",
   "metadata": {},
   "source": [
    "Use the `.delete()` method to delete the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b98aeaba",
   "metadata": {},
   "outputs": [],
   "source": [
    "huggingface_pipeline_model.delete()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dbe448b2",
   "metadata": {},
   "source": [
    "The next cell removes the model artifacts that were stored on your local drive."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ddff12fb",
   "metadata": {},
   "outputs": [],
   "source": [
    "rmtree(huggingface_pipeline_model.artifact_dir)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6871fcdf",
   "metadata": {},
   "source": [
    "<a id='ref'></a>\n",
    "# References\n",
    "- [ADS Library Documentation](https://accelerated-data-science.readthedocs.io/en/latest/index.html)\n",
    "- [Data Science YouTube Videos](https://www.youtube.com/playlist?list=PLKCk3OyNwIzv6CWMhvqSB_8MLJIZdO80L)\n",
    "- [OCI Data Science Documentation](https://docs.cloud.oracle.com/en-us/iaas/data-science/using/data-science.htm)\n",
    "- [Oracle Data & AI Blog](https://blogs.oracle.com/datascience/)\n",
    "- [Understanding Conda Environments](https://docs.cloud.oracle.com/en-us/iaas/data-science/using/use-notebook-sessions.htm#conda_understand_environments)\n",
    "- [Use Resource Manager to Configure Your Tenancy for Data Science](https://docs.cloud.oracle.com/en-us/iaas/data-science/using/orm-configure-tenancy.htm)\n",
    "- [`runtime.yaml`](https://docs.content.oci.oracleiaas.com/en-us/iaas/data-science/using/model_runtime_yaml.htm#model_runtime_yaml)\n",
    "- [`score.py`](https://docs.content.oci.oracleiaas.com/en-us/iaas/data-science/using/model_score_py.htm#model_score_py)\n",
    "- [Model artifact](https://docs.content.oci.oracleiaas.com/en-us/iaas/data-science/using/models_saving_catalog.htm#create-models)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.1"
  },
  "vscode": {
   "interpreter": {
    "hash": "1a1af0ee75eeea9e2e1ee996c87e7a2b11a0bebd85af04bb136d915cefc0abce"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}