Memory order out of Astra DB Chat Memory is incorrect

[mieslep]

Bug Description

I have injected Chat Memory into a Prompt Context, with the Chat Memory connected to Astra DB Chat Memory as external memory. My chat order is (with timestamps):

1722414538.8927608   hello, what is my name?
1722414543.1207514   I'm not sure what your name is! How would you like me to address you?
1722414553.4024212   my name is phil

The prompt (captured via LangSmith) is:

Continue the following conversation:
AI: I'm not sure what your name is! How would you like me to address you?
User: my name is phil
User: hello, what is my name

Which is clearly not in the right order. It is in database order (select * from ...):

1722414543.1207514   I'm not sure what your name is! How would you like me to address you?
1722414553.4024212   my name is phil
1722414538.8927608   hello, what is my name?

The ordering is by token (as the table is partitioned by _id with no clustering, and each message is in its own partition).

Reproduction

1. Install the linked flow via API call, with LangSmith or some other means of capturing prompts
2. Run the linked flow with at least two user messages sent with the same session_id
3. Review order in database table default_keyspace.memories. If select * from shows the correct order, truncate the table and re-run the flow. Eventually the order in the database will not match conversation order
4. Review the prompt text: you will find it in database order, not time order.

Expected behavior

The prompt should be ordered by time.

Who can help?

No response

Operating System

Langflow Docker Container

Langflow Version

1.0.14

Python Version

3.12

Screenshot

No response

Flow File

https://raw.githubusercontent.com/DataStax-Examples/langflow-demo-prod/7c231fcfbfd2fb52fbec1b78da9c7fc0e31b6e5e/flows/AstraMemoryChatbot.json

[dosubot]

To ensure that the prompt is ordered by time instead of database order when using Chat Memory with Astra DB in Langflow, you need to modify the retrieval logic to sort the messages by their timestamp. This can be done by adjusting the query or sorting the messages after retrieval.

In the provided code, the AstraDBChatMessageHistory class is used to manage chat messages. You would need to ensure that the messages are sorted by their timestamp when they are retrieved from the database. Here is an example of how you might modify the retrieval logic to sort by time:

from langchain_astradb.chat_message_histories import AstraDBChatMessageHistory

class AstraDBMessageWriterComponent(BaseMemoryComponent):
    # ... existing code ...

    def get_sorted_messages(self, session_id: str, collection_name: str, token: str, api_endpoint: str, namespace: Optional[str] = None):
        memory = AstraDBChatMessageHistory(
            session_id=session_id,
            collection_name=collection_name,
            token=token,
            api_endpoint=api_endpoint,
            namespace=namespace,
        )

        messages = memory.get_messages()
        sorted_messages = sorted(messages, key=lambda msg: msg.timestamp)
        return sorted_messages

    # ... existing code ...

In this example, the get_sorted_messages method retrieves the messages and sorts them by their timestamp attribute. You would need to ensure that each message has a timestamp attribute that accurately reflects the time it was created.

Additionally, in Langflow, you can configure the order of messages directly in the MemoryComponent class by setting the order attribute. Here is an example configuration from the MemoryComponent class:

class MemoryComponent(Component):
    display_name = "Chat Memory"
    description = "Retrieves stored chat messages."
    icon = "message-square-more"

    inputs = [
        # ... other inputs ...
        DropdownInput(
            name="order",
            display_name="Order",
            options=["Ascending", "Descending"],
            value="Ascending",
            info="Order of the messages.",
            advanced=True,
        ),
        # ... other inputs ...
    ]

    def retrieve_messages(self) -> Data:
        sender = self.sender
        sender_name = self.sender_name
        session_id = self.session_id
        n_messages = self.n_messages
        order = "DESC" if self.order == "Descending" else "ASC"

        if sender == "Machine and User":
            sender = None

        messages = get_messages(
            sender=sender,
            sender_name=sender_name,
            session_id=session_id,
            limit=n_messages,
            order=order,
        )
        self.status = messages
        return messages

By setting the order attribute to "Ascending" or "Descending", you can control the order in which messages are retrieved and displayed. The retrieve_messages method in the MemoryComponent class uses this order attribute to fetch messages in the desired order.

In the Langflow configuration, you can see that the Memory node has an order attribute that can be set to "Ascending" or "Descending". This attribute is used to determine the order in which messages are retrieved and displayed in the prompt:

{
  "data": {
    "nodes": [
      {
        "data": {
          "id": "Memory-rvcL5",
          "node": {
            "custom_fields": {
              "order": {
                "advanced": true,
                "display_name": "Order",
                "dynamic": false,
                "info": "Order of the messages.",
                "name": "order",
                "options": [
                  "Ascending",
                  "Descending"
                ],
                "placeholder": "",
                "required": false,
                "show": true,
                "title_case": false,
                "type": "str",
                "value": "Ascending"
              }
            }
          },
          "type": "Memory"
        }
      }
    ]
  }
}

By configuring the order attribute in the Langflow UI, you can ensure that the messages are ordered by time when they are retrieved and used in the prompt.

Additionally, the get_messages function in src/backend/base/langflow/memory.py provides a way to retrieve messages ordered by a specified field, defaulting to "timestamp". You can use this function to ensure messages are ordered by their timestamp:

def get_messages(
    sender: Optional[str] = None,
    sender_name: Optional[str] = None,
    session_id: Optional[str] = None,
    order_by: Optional[str] = "timestamp",
    order: Optional[str] = "DESC",
    limit: Optional[int] = None,
):
    monitor_service = get_monitor_service()
    messages_df = monitor_service.get_messages(
        sender=sender,
        sender_name=sender_name,
        session_id=session_id,
        order_by=order_by,
        limit=limit,
        order=order,
    )

    messages: list[Message] = []
    messages_df = messages_df[::-1] if order == "DESC" else messages_df
    for row in messages_df.itertuples():
        msg = Message(
            text=row.text,
            sender=row.sender,
            session_id=row.session_id,
            sender_name=row.sender_name,
            timestamp=row.timestamp,
        )

        messages.append(msg)

    return messages

This function retrieves messages from the monitor service and orders them by the specified field, which defaults to "timestamp". By setting the order parameter to "ASC" or "DESC", you can control the order of the messages ^{[1][[2]](https://github.com/langflow-ai/langflow/blob/dev/src/backend/base/langflow/initial_setup/starter_projects/Memory Chatbot.json)[[3]](https://github.com/langflow-ai/langflow/blob/main/src/backend/base/langflow/initial_setup/starter_projects/Memory Chatbot.json)[[4]](https://github.com/langflow-ai/langflow/blob/dev/src/backend/base/langflow/initial_setup/starter_projects/Langflow Memory Conversation.json)[5]}.

To continue talking to Dosu, mention @dosu.

[mieslep]

Fixed with #3170