workshop/5-Conversational/5.1-LexChatbot.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Retail Demo Store - Conversational AI Workshop\n",
    "\n",
    "Welcome to the Retail Demo Store Conversational AI Workshop. In this module we're going to implement a conversational chatbot using [Amazon Lex](https://aws.amazon.com/lex/) and integrate it into the Retail Demo Store's web UI. We'll provide some basic functionality to our chatbot such as being able to provide a return policy to users as well as wiring up the chatbot to the Amazon Personalize ML models we created in the [Personalization](../1-Personalization/Lab-01-(Core)-Introduction-and-data-preparation.ipynb) workshop to provide personalized product recommendations to our users. When you combine this workshop with the Personalization and [Messaging](../4-Messaging/4.1-Pinpoint.ipynb) workshops, you have a compelling example of how to deliver omnichannel personalization.\n",
    "\n",
    "Although we will highlight some relevant code examples, this workshop will mostly involve working in the AWS console.\n",
    "\n",
    "Recommended Time: 30 Minutes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Foundation - What's Already Setup\n",
    "\n",
    "When the Retail Demo Store was deployed in this AWS account, some of the foundational pieces needed to implement the conversational AI interface we are going to cover in this workshop were put in place. We will quickly review those pieces now before proceeding."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### User Interface Components\n",
    "\n",
    "The chatbot interface is wired into the Retail Demo Store web user interface on the Help view. You can access the Help view by clicking on the \"?\" icon in the upper right corner of the navigation. As you see from the screenshot below, the Help view is unable to communicate with our chatbot because it has not been setup yet. That is what we will be completing in this workshop.\n",
    "\n",
    "![Initial Chatbot State](images/retaildemostore-help-start.png)\n",
    "\n",
    "The Help view is implemented in [src/web-ui/src/public/Help.vue](https://github.com/aws-samples/retail-demo-store/blob/master/src/web-ui/src/public/Help.vue) file in the Retail Demo Store repository.\n",
    "\n",
    "Since we are using the AWS Amplify framework for Vue.js, adding the interaction chatbot to the Help component is a simple matter of adding the `amplify-chatbot` component.\n",
    "\n",
    "```html\n",
    "<amplify-chatbot v-bind:chatbotConfig=\"chatbotConfig\" id=\"chatBot\"></amplify-chatbot>\n",
    "```\n",
    "\n",
    "The chatbot configuration is provided by the following `chatbotConfig` object.\n",
    "\n",
    "```javascript\n",
    "chatbotConfig: function () {\n",
    "  let config = {\n",
    "    bot: import.meta.env.VITE_BOT_NAME,\n",
    "    clearComplete: false,\n",
    "    botTitle: \"Retail Demo Store Support\",\n",
    "    conversationModeOn: false,\n",
    "    voiceEnabled: false,\n",
    "    textEnabled: true\n",
    "  }\n",
    "  return config\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### IAM Roles\n",
    "\n",
    "In order for the chatbot component to be able to communicate with Amazon Lex in our AWS account, the Cognito Authorized and Unauthorized IAM roles require the following policy statements. **The following policy has already been setup for you in IAM.**\n",
    "\n",
    "```javascript\n",
    "{\n",
    "    \"Action\": [\n",
    "        \"lex:PostText\"\n",
    "    ],\n",
    "    \"Resource\": [\n",
    "        \"arn:aws:lex:[REGION]:[ACCOUNT_ID]:bot:RetailDemoStore:*\"\n",
    "    ],\n",
    "    \"Effect\": \"Allow\"\n",
    "}\n",
    "```\n",
    "\n",
    "This allows the chatbot component to call `PostText` on the Lex bot named `RetailDemoStore`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create RetailDemoStore Bot\n",
    "\n",
    "Let's create the `RetailDemoStore` Bot in Amazon Lex.\n",
    "\n",
    "1. Sign in to the AWS console for the account where the Retail Demo Store is deployed.\n",
    "2. Browse to the [Amazon Lex](https://console.aws.amazon.com/lex/home) console page.\n",
    "3. Click the \"Actions\" dropdown and then \"Import\".\n",
    "\n",
    "![Import Bot](images/retaildemostore-import-bot.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From the Import dialog, browse to the directory on your local machine where you cloned the Retail Demo Store repository and locate the file `src/workshop/5-Conversational/RetailDemoStore_Lex.zip`. Alternatively, you can download the `RetailDemoStore_Lex.zip` file from the SageMaker notebook instance to your local machine and then browse to the zip file in your \"Downloads\" directory.\n",
    "\n",
    "![Import Bot](images/retaildemostore-import-bot2.png)\n",
    "\n",
    "4. Click \"Import\" to import the bot configuration file. Once the bot has been imported, click on the \"RetailDemoStore\" bot name."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You will see three intents for our bot in the left naviation. Intents represent an action that the user wants to perform. For this workshop we have three intents: \"Greeting\", \"RecommendedProduct\", and \"ReturnPolicy\"\n",
    "\n",
    "![Bot Intents](images/retaildemostore-bot-intents.png)\n",
    "\n",
    "For the \"Greeting\" intent, we have several sample utterances that the user can say to initiate the intent. Scroll down to the \"Response\" section and you will see the response that Lex will return as a result of any of the utterances. Fairly simple but static.\n",
    "\n",
    "5. Take a look at the utterances and responses for the other two intents, \"RecommendProduct\" and \"ReturnPolicy\"."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "6. Let's Build and Publish our bot and see how the Help view in the Retail Demo Store's web user interface changes. When you publish the bot, enter `development` for the alias.\n",
    "\n",
    "![Build and Publish Bot](images/retaildemostore-bot-build-publish.png)\n",
    "\n",
    "You can also test the bot within the Amazon Lex console by clicking on \"Test Chatbot\" vertically oriented panel along the right side of the Lex console. Enter some intent utterances to test out the greeting, product recommendation, and return policy intents. You will notice that the \"RecommendProduct\" intent is not fully functional yet. Will get to that soon."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "7. Return to the Retail Demo Store web user interface and reload/refresh the Help view. You should see the chatbot widget now. Enter some utterances for our three intents.\n",
    "\n",
    "![Bot Active](images/retaildemostore-bot-active.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Personalized Product Recommendations\n",
    "\n",
    "If you tested the \"RecommendProduct\" intent, you noticed that it currently does not return product recommendations. Let's change that now by wiring up an AWS Lambda function to our \"RecommendProduct\" intent that will return personalized product recommendations for the current user.\n",
    "\n",
    "1. In the AWS console, browse to the Amazon Lex console page and select the \"RetailDemoStore\" bot.\n",
    "2. Click on the \"RecommendProduct\" intent in the left navigation.\n",
    "3. Under \"Lambda initialization and validation\", check the \"Initialization and validation code hook\" box.\n",
    "\n",
    "![Bot Lambda](images/retaildemostore-rec-lambda.png)\n",
    "\n",
    "4. Select the \"RetailDemoStore-Chat-Recommendations\" Lambda function and \"Latest\" for the version/alias. This function was deployed as part of the Retail Demo Store deployment. You can view the complete source for this function in the [GitHub Repository](https://github.com/aws-samples/retail-demo-store/blob/master/src/aws-lambda/bot-intent-recommendations/bot-intent-recommendations.py).\n",
    "5. Scroll down and save the intent.\n",
    "6. Build and Publish the bot."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Return to the Retail Demo Store web user interface and ask the bot to return product recommendations using one of the utterances. If you are not signed in, you should receive a message asking you to sign in. Once you're signed in as a user, ask the bot again for recommendations.\n",
    "\n",
    "![Bot Recommendatins](images/retaildemostore-bot-personalize.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Inspecting Web UI and Lambda Source\n",
    "\n",
    "Let's take a closer look at the relevant source code in the [\"RetailDemoStore-Chat-Recommendations\"](https://github.com/aws-samples/retail-demo-store/blob/master/src/aws-lambda/bot-intent-recommendations/bot-intent-recommendations.py) Lambda function.\n",
    "\n",
    "When the function is called, we are passed an event that specifies the intent name (\"RecommendProduct\"), a userId, and session information. Since this intent returns recommendations for the current user, it does not use Slots (used to gather additional input from the user).\n",
    "\n",
    "The `userId` is how we bind the call to the Lambda function via Lex to the user in the Retail Demo Store's Users microservice. Since the AWS Amplify framework sends the Cognito EntityId from the Congnito Entity Pool as the `userId`, we need to map the EntityId to a user ID that we can use to fetch recommendations. It's important to emphasize that the `userId` provided by Lex is *NOT* the same user ID used in the Retail Demo Store. We need to map the user IDs somehow. The Retail Demo Store web app makes this possible by updating the user in the Users microservice with the entityId after the user signs in. This allows us to later map the EntityId to a user.\n",
    "\n",
    "Here is the relevant code from [index.js](https://github.com/aws-samples/retail-demo-store/blob/master/src/web-ui/src/router/index.js) in the Web UI.\n",
    "\n",
    "```javascript\n",
    "const authListener = async (data) => {\n",
    "  switch (data.payload.event) {\n",
    "    case 'signOut':\n",
    "      // ...\n",
    "    case 'signIn': {\n",
    "    // ...\n",
    "    // Get entityId for the signed in user.\n",
    "    const credentials = await Credentials.get();\n",
    "    store.commit('setIdentityId',credentials.identityId)\n",
    "    \n",
    "    // Update user with entityId\n",
    "    UsersRepository.updateUser(storeUser)\n",
    "    //...\n",
    "    }\n",
    "  }\n",
    "}\n",
    "Hub.listen('auth', authListener);\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `RetailDemoStore-Chat-Recommendations` Lambda function can then take the EntityId provided by Lex (confusingly named \"userId\") to look up the user. The [Users](https://github.com/aws-samples/retail-demo-store/tree/master/src/users) service provides an endpoint for mapping an entityId to a user. Here is the relevant code from the Lambda function that makes this call.\n",
    "\n",
    "```python\n",
    "def lookup_user(identity_id):\n",
    "    url = f'{users_service_base_url}/users/identityid/{identity_id}'\n",
    "    response = requests.get(url)\n",
    "    \n",
    "    user = None\n",
    "\n",
    "    if response.ok:\n",
    "        user_check = response.json()\n",
    "        if user_check.get('id') and len(user_check.get('id')) > 0:\n",
    "            user = user_check\n",
    "\n",
    "    return user\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can tie it all together to show how recommendations are returned as [Response Cards](https://docs.aws.amazon.com/lex/latest/dg/howitworks-manage-prompts.html#msg-prompts-resp-card) to the Web UI.\n",
    "\n",
    "The function's entry point delegates control to the `dispatch` function.\n",
    "\n",
    "```python\n",
    "def lambda_handler(event, context):\n",
    "    if not users_service_base_url:\n",
    "        raise ValueError(\"Missing required environment value for 'users_service_base_url'\")\n",
    "\n",
    "    if not recommendations_service_base_url:\n",
    "        raise ValueError(\"Missing required environment value for 'recommendations_service_base_url'\")\n",
    "\n",
    "    return dispatch(event)\n",
    "```\n",
    "\n",
    "The `dispatch` function validates the intent passed from Lex and calls `recommend_products`\n",
    "\n",
    "```python\n",
    "def dispatch(intent_request):\n",
    "    \"\"\"\n",
    "    Called when the user specifies an intent for this bot.\n",
    "    \"\"\"\n",
    "\n",
    "    intent_name = intent_request['currentIntent']['name']\n",
    "\n",
    "    # Dispatch to bot's intent handlers\n",
    "    if intent_name == 'RecommendProduct':\n",
    "        return recommend_products(intent_request)\n",
    "\n",
    "    raise Exception('Intent with name ' + intent_name + ' not supported')\n",
    "```\n",
    "\n",
    "The `recommend_products` function looks to see if we already have a user attached to the bot session. If not it calls the `lookup_user` function we saw above.\n",
    "\n",
    "If we successfully mapped the entityId to a Retail Demo Store user, recommendations are retrieved for the user and returned as Response Cards.\n",
    "\n",
    "```python\n",
    "def recommend_products(intent_request):\n",
    "    user_id = intent_request['userId']\n",
    "    output_session_attributes = intent_request['sessionAttributes'] if intent_request['sessionAttributes'] is not None else {}\n",
    "\n",
    "    store_user = None\n",
    "    if output_session_attributes.get('storeUser'):\n",
    "        store_user = json.loads(output_session_attributes.get('storeUser'))\n",
    "        logger.debug('Found user {} ({}) as session attribute'.format(store_user['username'], store_user['id']))\n",
    "\n",
    "    if not store_user:\n",
    "        store_user = lookup_user(user_id)\n",
    "        if store_user:\n",
    "            output_session_attributes['storeUser'] = json.dumps(store_user)\n",
    "\n",
    "    if store_user:\n",
    "        recommendations = get_recommendations(store_user['id'], 4)\n",
    "\n",
    "        if recommendations and len(recommendations) > 0:\n",
    "            attachments = []\n",
    "\n",
    "            for recommendation in recommendations:\n",
    "                product = recommendation['product']\n",
    "                attachments.append(build_response_card_attachment(product['name'], product['description'], product['image'], product['url']))\n",
    "\n",
    "            response = {\n",
    "                'sessionAttributes': output_session_attributes,\n",
    "                'dialogAction': {\n",
    "                    'type': 'Close',\n",
    "                    'fulfillmentState': 'Fulfilled',\n",
    "                    'message': {\n",
    "                        'contentType': 'PlainText',\n",
    "                        'content': 'Hi {}. Based on your shopping trends, I think you may be interested in the following products.'.format(store_user['first_name'])\n",
    "                    },\n",
    "                    'responseCard': build_response_card(attachments)\n",
    "                }\n",
    "            }\n",
    "        else:\n",
    "            response = close(output_session_attributes, 'Failed', 'Sorry, I was unable to find any products to recommend.')\n",
    "    else:\n",
    "        response = close(output_session_attributes, 'Failed', 'Before I can make personalized recommendations, I need to know more about you. Please sign in or create an account and try again.')\n",
    "\n",
    "    return response\n",
    "```\n",
    "\n",
    "The `get_recommendations` function calls the [Recommendations](https://github.com/aws-samples/retail-demo-store/tree/master/src/recommendations) to retrieve recommendations.\n",
    "\n",
    "```python\n",
    "def get_recommendations(user_id, max_items = 10):\n",
    "    url = f'{recommendations_service_base_url}/recommendations?userID={user_id}&fullyQualifyImageUrls=1&numResults={max_items}'\n",
    "    response = requests.get(url)\n",
    "    \n",
    "    recommendations = None\n",
    "\n",
    "    if response.ok:\n",
    "        recommendations = response.json()\n",
    "        logger.debug(recommendations)\n",
    "\n",
    "    return recommendations\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Unfortunately the `<amplify-chatbot/>` component does not display response cards from Lex responses. So for the Retail Demo Store response cards are rendered in the Help view alongside the chatbot component. This allows the user to interact with the recommended products by viewing them for more details, adding them to their shopping cart, and so on. When using this Lex bot with external messaging tools such as Facebook Messenger and Slack, the response cards will be rendered directly within those applications."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Workshop Complete\n",
    "\n",
    "Congratulations! You have completed the conversational AI workshop. In this module we demonstrated how to add another communication channel with users of the Retail Demo Store. This is a powerful example of how to implement omnichannel user experiences where the same back-end personalization machine learning models are used to provide recommendations across web/mobile apps, messaging (from the Pinpoint workshop), and now a conversational interface.\n",
    "\n",
    "Additional functionality can be easily added to the chatbot interface to allow users to request information such as order and inventory status, store locations & hours, and so on. This bot framework can also be used to power conversational interfaces in Facebook Messenger, Slack, and Twilio."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "conda_python3",
   "language": "python",
   "name": "conda_python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}