Langchain message history with message length limiting

  Kiến thức lập trình

My goal is to limit the last N messages in a message history so I don’t overload the LLM. My plan is to use use RunnableWithMessageHistory in conjunction with a filter function. Unfortunately I seem to have two problems: 1) getting the limiting function to work; and 2) passing the actual user message into the model.

I’m using these two docs as reference:
https://python.langchain.com/v0.2/docs/how_to/message_history/
https://python.langchain.com/v0.2/docs/tutorials/chatbot/#message-history

My questions:
What am I doing wrong? And specifically:

  1. Why is the RunnablePassthrough not applying to the message history?
  2. Am I using HumanMessage("{user_message_key}") correctly with input_messages_key? I understand there’s HumanMessagePromptTemplate but I’m not sure why I must use it.

If I apply _filter_messages within get_session_history it solves #1 but I’d still like to understand why my existing approach does not.

Here’s MRS:

from typing import List, Union
from langchain_openai import ChatOpenAI
from langchain_community.chat_message_histories import ChatMessageHistory
from langchain_core.chat_history import BaseChatMessageHistory
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_core.messages import HumanMessage, AIMessage, SystemMessage
from langchain_core.runnables.history import RunnableWithMessageHistory
from langchain_core.runnables import RunnablePassthrough


store = {}
model_name = "gpt-3.5-turbo"
system_message = "You're a helpful, friendly AI"
model = ChatOpenAI(model=model_name)

# Function to get chat message history
def get_session_history(session_id: str) -> BaseChatMessageHistory:
    if session_id not in store:
        store[session_id] = ChatMessageHistory()
    history: BaseChatMessageHistory = store[session_id]
    # history.messages = self._filter_messages(history.messages)
    return history

def filter_messages(messages: List[Union[HumanMessage, AIMessage]]) -> List[Union[HumanMessage, AIMessage]]:
    return messages[-5:]

def process_text(text: str, session_history: BaseChatMessageHistory, session_id: str) -> AIMessage:
    # Create the prompt template
    prompt = ChatPromptTemplate.from_messages(
        [
            SystemMessage(system_message),
            MessagesPlaceholder(variable_name="chat_hist"),
            HumanMessage("{user_message_key}")
        ]
    )

    runnable = RunnablePassthrough.assign(chat_hist=lambda x: filter_messages(x["chat_hist"])) | prompt | model

    with_message_history = RunnableWithMessageHistory(
        runnable=runnable,
        get_session_history=get_session_history,
        history_messages_key="chat_hist",
        input_messages_key="usr_msg",
    )

    resp: AIMessage = with_message_history.invoke(
        {"usr_msg": [HumanMessage(content=text)]},
        config={"configurable": {"session_id": session_id}},
    )
    print(resp.content)
    return resp


session_id = "test_session"
previous_messages = [
    HumanMessage(content="I love the color Pink. It's my favorite color."),
    AIMessage(content="That's great to know!"),
    HumanMessage(content="Here's a placeholder message for testing"),
    HumanMessage(content="Here's a placeholder message for testing"),
    HumanMessage(content="Here's a placeholder message for testing"),
    HumanMessage(content="Here's a placeholder message for testing"),
    HumanMessage(content="Here's a placeholder message for testing"),
    HumanMessage(content="Here's a placeholder message for testing"),
    HumanMessage(content="Here's a placeholder message for testing"),
    HumanMessage(content="Hello, my name is John."),
    AIMessage(content="Hi John, great to meet you."),
    HumanMessage(content="I'm sick of this rain."),
]
session_history = get_session_history(session_id)
for message in previous_messages:
    session_history.add_message(message)

user_input = "Repeat exactly: GOODBYE"
response:AIMessage = process_text(user_input, session_history, session_id)
# answer: I'm sorry to hear that you're feeling down because of the rain. Is there anything I can do to help cheer you up?

LEAVE A COMMENT