ENG-424: initial port of moodbot to flows / command pattern

docs/docs/start-here.mdx

@@ -0,0 +1,211 @@
+---
+id: start-here
+sidebar_label: Start Here
+title: Start Here
+hide_table_of_contents: true
+---
+
+
+## Getting Started with Rasa
+
+This page provides an introduction to Rasa. It's intended for newcomers and for people
+familiar with earlier versions of Rasa who want to understand the new DM2 approach (name TBD). 
+DM2 gives you the best of both worlds: the time-to-value and generality of LLMs, and the 
+controllability of intent-based approaches. 
+
+### Command Line Interface Basics
+
+Once you have installed Rasa, you can run the `init` command to create a starter project:
+
+```bash
+rasa init --dm2
+```
+
+Run the `train` command at any time to build an assistant from the current state of your project:
+
+```bash
+rasa train
+```
+
+:::note Model Training
+
+While the command is called `rasa train`, building a new version of your assistant doesn't always
+require training a model. Rasa uses a cache and only trains when necessary.
+More information on the [`rasa train` command reference](./command-line-interface.mdx#rasa-train).
+
+:::
+
+
+Run the `shell` command to talk to your assistant on the command line:
+
+```bash
+rasa shell
+```
+
+To stop the conversation, type `/stop` or use `control+C`.
+
+
+:::note Debugging
+
+Adding the flag `rasa shell --debug` can be very helpful if you want to understand what is happening in detail.
+More information on the [`rasa shell` command reference](./command-line-interface.mdx#rasa-shell)
+
+:::
+
+
+### Exploring what your assistant can do
+
+Assistants often have to collect a few pieces of information from the user in order to complete a task.
+This starter project contains an example flow which recommends restaurants. To do so, it asks the user
+for their preferred cuisine, city, and price range.
+
+To try it out, start a conversation using `rasa shell`, and say something like "I'm looking for a restaurant". 
+
+Out of the box, this assistant can already handle a variety of conversations. The first is what we call the "happy path",
+where the user always provides the information the assistant requests.
+But if users change their mind, answer indirectly, or interject with questions, this assistant can handle those cases as well.
+Try out some of these conversations yourself to get a feel for things.
+If you want your assistant to sound a bit more natural, you can activate [contextual rephrasing](./llms/llm-nlg.mdx) of responses.
+
+<Tabs values={[{"label": "Happy path", "value": "happy-path"}, {"label": "Complete request", "value": "all-upfront"}, {"label": "Change of mind", "value": "change-mind"}, {"label": "Interjection", "value": "interjection"}]}  groupId="example-conversations" defaultValue="happy-path">
+  <TabItem value="happy-path">
+    <Chat caption="happy path">
+    <ChatUserText>Can you recommend somewhere to eat?</ChatUserText>
+    <ChatBotText>What kind of food are you looking for?</ChatBotText>
+    <ChatUserText>Indian food</ChatUserText>
+    <ChatBotText>in what price range?</ChatBotText>
+    <ChatUserText>cheap</ChatUserText>
+    <ChatBotText>and in which city?</ChatBotText>
+    <ChatUserText>Berlin</ChatUserText>
+    <ChatBotText>Here's a recommendation: ...</ChatBotText>    
+    </Chat>
+  </TabItem>
+  <TabItem value="all-upfront">
+    <Chat caption="All info given">
+    <ChatUserText>I'm looking for a cheap Chinese restaurant in Amsterdam?</ChatUserText>
+    <ChatBotText>Here's a recommendation: ...</ChatBotText>
+    </Chat>
+  </TabItem>  
+  <TabItem value="change-mind">
+    <Chat caption="user changes their mind">
+    <ChatUserText>Can you recommend somewhere to eat?</ChatUserText>
+    <ChatBotText>What kind of food are you looking for?</ChatBotText>
+    <ChatUserText>Indian food</ChatUserText>
+    <ChatBotText>in what price range?</ChatBotText>
+    <ChatUserText>actually no I want Italian. A cheap spot</ChatUserText>
+    <ChatBotText>and in which city?</ChatBotText>
+    <ChatUserText>Berlin</ChatUserText>
+    <ChatBotText>Here's a recommendation: ...</ChatBotText>    
+    </Chat>
+  </TabItem>
+  <TabItem value="interjection">
+    <Chat caption="user changes their mind">
+    <ChatUserText>Can you recommend somewhere to eat?</ChatUserText>
+    <ChatBotText>What kind of food are you looking for?</ChatBotText>
+    <ChatUserText>wait are you a bot?</ChatUserText>
+    <ChatBotText>I am a bot, powered by Rasa.</ChatBotText>
+    <ChatBotText>What kind of food are you looking for?</ChatBotText>    
+    </Chat>
+  </TabItem>  
+</Tabs>
+
+
+### Understanding flows
+
+Given the range of conversations this assistant can handle, you might expect the implementation to be complex.
+In fact, there are only two small files that provide Rasa what it needs. 
+
+The `data/flows/restaurants.yml` file defines the logic for this flow. In this example the logic is linear
+and walks the user through each of the steps in order. Each of the `question` steps fills the corresponding slot.
+
+```yaml
+flows:
+  recommend_restaurant:
+    description: This flow recommends a restaurant
+    steps:
+      - id: "0"
+        question: cuisine
+        skip_if_filled: true
+        next: "1"
+      - id: "1"
+        question: price_range
+        next: "2"
+      - id: "2"
+        question: part_of_town
+        next: "3"
+      - id: "3"
+        action: search_restaurants
+```
+
+To build more advanced flows, you can add conditional logic, link to other flows, and more. Read more on how to handle [Business Logic with Flows](./flows.mdx)
+
+In addition to the flows, the `domain.yml` file contains definitions of the slots and responses used in this flow. 
+
+### Understanding DM2
+
+Rasa uses a new approach to building AI assistants called DM2 (name TBD). 
+If you've built AI assistants before, you might look at your project and think that many things are missing.
+
+* There is no NLU data with intents and entities
+* There is no logic mapping an intent like "restaurant_search" to the start of the restaurant flow
+* There are no slot mappings
+* There is no explicit logic handling corrections, interruptions, or other unhappy paths.
+
+DM2 doesn't need any of these things to be able to handle the example conversations above. So, how does that work?
+
+Many developers are familiar with dialogue systems made up of separate NLU, dialogue, and NLG components
+(this is also how Rasa worked previously).
+
+In DM2, we have a different set of modules. 
+
+* The *conversation handling* component (name TBD) interprets the conversation so far and predicts 
+a series of commands to progress the state of the conversation.
+* The *business logic* component executes those commands and the logic of your flows. 
+
+The NLU systems you might be familiar with take a single user message as input, and aim to represent
+the meaning of that message by predicting intents and entities. 
+Instead, *conversation handling* considers the conversation as a whole (not just one message),
+and predicts the *intended effect* of the user's message. 
+
+As an example, let's look at using a yes/no question to fill a slot called `late_delivery`:
+
+<Chat caption="yes/no question">
+<ChatBotText>Has it been more than 10 business days since you placed your order?</ChatBotText>
+</Chat>
+
+
+When a user answers "yes" or "no", a traditional NLU model predicts an intent like `affirm` or `deny`.
+A second step (usually handled by the dialogue manager) then maps the intents to
+the `True/False` values of the `late_delivery` slot. 
+
+Instead, the *conversation handling* component directly outputs a command to set the `late_delivery` slot to `True`.
+
+The *conversation handling* approach requires much less work to set up, since you don't need to worry about
+intents and entities and slot mappings. It is also more powerful because it allows us to 
+[break free from intents](https://rasa.com/blog/its-about-time-we-get-rid-of-intents/).
+
+For example, intent-based approaches struggle when context is required to understand what the user means:
+
+<Chat caption="pragmatic understanding">
+<ChatBotText>Has it been more than 10 business days since you placed your order?</ChatBotText>
+<ChatUserText>sadly</ChatUserText>
+</Chat>
+
+This kind of conversation illustrates the limitations of working with intents. It's perfectly clear 
+what the user means in this context, but in general the word "sadly" does not mean `affirm`.
+The *conversation handling* component will correctly output `SetSlot("late_delivery", True)`.
+
+The example project above includes a definition of the core business logic for recommending a restaurant
+and not much else. Yet, it can handle a number of advanced conversations right away. 
+The advantage of DM2 is that it makes chatbots much smarter and much easier to build, while still giving
+you full control over your business logic, and the ability to override and customize all behavior.
+
+Learn more about how to use DM2 to build advanced assistants:
+
+* Advanced flow logic
+* Search-based question-answering
+* Context switching
+* Disambiguation
+* Contextual understanding and negation
+* Chitchat and digressions
+

docs/sidebars.js

@@ -265,6 +265,7 @@ module.exports = {
         },
     ],
     "llms": [
+        'start-here',
         'llms/large-language-models',
         'llms/llm-setup',
         {

rasa/cdu/command_generator/command_prompt_template.jinja2

@@ -25,7 +25,7 @@ Here are the slots of the currently active flow with their names and values:
 {% endif %}
 {% else %}
 You are currently not in any flow and so there are no active slots.
-In order to fill a slot, you first have to start the flow and then fill the slot.
+This means you can only set a slot if you first start a flow that requires that slot.
 {% endif %}
 If you start a flow, first start the flow and then optionally fill that flow's slots with information the user provided in their message.
 

rasa/cdu/command_generator/llm_command_generator.py

@@ -21,8 +21,6 @@
 from rasa.engine.storage.resource import Resource
 from rasa.engine.storage.storage import ModelStorage
 from rasa.shared.core.constants import (
-    MAPPING_TYPE,
-    SlotMappingType,
     MAPPING_CONDITIONS,
     ACTIVE_LOOP,
 )
@@ -245,15 +243,14 @@ def is_extractable(q: QuestionFlowStep, tracker: DialogueStateTracker) -> bool:
             return False
 
         for mapping in slot.mappings:
-            if mapping.get(MAPPING_TYPE) == str(SlotMappingType.FROM_ENTITY):
-                conditions = mapping.get(MAPPING_CONDITIONS, [])
-                if len(conditions) == 0:
-                    return True
-                else:
-                    for condition in conditions:
-                        active_loop = condition.get(ACTIVE_LOOP)
-                        if active_loop and active_loop == tracker.active_loop_name:
-                            return True
+            conditions = mapping.get(MAPPING_CONDITIONS, [])
+            if len(conditions) == 0:
+                return True
+            else:
+                for condition in conditions:
+                    active_loop = condition.get(ACTIVE_LOOP)
+                    if active_loop and active_loop == tracker.active_loop_name:
+                        return True
         return False
 
     def render_template(

rasa/cli/initial_project_dm2/actions/actions.py

@@ -0,0 +1,27 @@
+# This files contains your custom actions which can be used to run
+# custom Python code.
+#
+# See this guide on how to implement these action:
+# https://rasa.com/docs/rasa/custom-actions
+
+
+# This is a simple example for a custom action which utters "Hello World!"
+
+# from typing import Any, Text, Dict, List
+#
+# from rasa_sdk import Action, Tracker
+# from rasa_sdk.executor import CollectingDispatcher
+#
+#
+# class ActionHelloWorld(Action):
+#
+#     def name(self) -> Text:
+#         return "action_hello_world"
+#
+#     def run(self, dispatcher: CollectingDispatcher,
+#             tracker: Tracker,
+#             domain: Dict[Text, Any]) -> List[Dict[Text, Any]]:
+#
+#         dispatcher.utter_message(text="Hello World!")
+#
+#         return []

rasa/cli/initial_project_dm2/config.yml

@@ -0,0 +1,13 @@
+recipe: default.v1
+language: en
+pipeline:
+  - name: LLMCommandGenerator
+    # llm:
+    #   model_name: gpt-4
+
+policies:
+  - name: rasa.core.policies.flow_policy.FlowPolicy
+#  - name: rasa_plus.ml.DocsearchPolicy
+#  - name: RulePolicy
+
+assistant_id: 20230405-114328-tranquil-mustard

rasa/cli/initial_project_dm2/credentials.yml

@@ -0,0 +1,33 @@
+# This file contains the credentials for the voice & chat platforms
+# which your bot is using.
+# https://rasa.com/docs/rasa/messaging-and-voice-channels
+
+rest:
+#  # you don't need to provide anything here - this channel doesn't
+#  # require any credentials
+
+
+#facebook:
+#  verify: "<verify>"
+#  secret: "<your secret>"
+#  page-access-token: "<your page access token>"
+
+#slack:
+#  slack_token: "<your slack token>"
+#  slack_channel: "<the slack channel>"
+#  slack_signing_secret: "<your slack signing secret>"
+
+#socketio:
+#  user_message_evt: <event name for user message>
+#  bot_message_evt: <event name for bot messages>
+#  session_persistence: <true/false>
+
+#mattermost:
+#  url: "https://<mattermost instance>/api/v4"
+#  token: "<bot token>"
+#  webhook_url: "<callback URL>"
+
+# This entry is needed if you are using Rasa Enterprise. The entry represents credentials
+# for the Rasa Enterprise "channel", i.e. Talk to your bot and Share with guest testers.
+rasa:
+  url: "http://localhost:5002/api"

rasa/cli/initial_project_dm2/data/flows.yml

@@ -0,0 +1,46 @@
+flows:
+  say_goodbye:
+    description: say goodbye to the user
+    steps:
+      - id: "0"
+        action: utter_goodbye
+  bot_challenge:
+    description: explain to the user that they are talking to a bot, if they ask
+    steps:
+      - id: "0"
+        action: utter_iamabot
+  greet:
+    description: greet the user and ask how they are doing. cheer them up if needed.
+    steps:
+      - id: "0"
+        question: good_mood
+        description: "can be true or false"
+        next:
+          - if: good_mood
+            then: "doing_great"
+          - else: "cheer_up"
+      - id: "doing_great"
+        action: utter_happy
+      - id: "cheer_up"
+        action: utter_cheer_up
+        next: "did_that_help" 
+      - id: "did_that_help"
+        action: utter_did_that_help
+  recommend_restaurant:
+    description: This flow recommends a restaurant
+    steps:
+      - id: "0"
+        question: cuisine
+        skip_if_filled: true
+        next: "1"
+      - id: "1"
+        question: price_range
+        skip_if_filled: true        
+        next: "2"
+      - id: "2"
+        question: city
+        skip_if_filled: true        
+        next: "3"
+      - id: "3"
+        action: utter_recommend_restaurant
+

rasa/cli/initial_project_dm2/data/nlu.yml

@@ -0,0 +1,11 @@
+version: "3.1"
+
+nlu:
+  - intent: affirm
+    examples: |
+      - yes
+      - yup
+  - intent: deny
+    examples: |
+      - no
+      - nope