Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Warning when using the structured output method 'json_schema'. #27252

Open
5 tasks done
thoffmann-artidis opened this issue Oct 9, 2024 · 8 comments
Open
5 tasks done
Labels
🤖:bug Related to a bug, vulnerability, unexpected error with an existing feature investigate

Comments

@thoffmann-artidis
Copy link

Checked other resources

  • I added a very descriptive title to this issue.
  • I searched the LangGraph/LangChain documentation with the integrated search.
  • I used the GitHub search to find a similar question and didn't find it.
  • I am sure that this is a bug in LangGraph/LangChain rather than my code.
  • I am sure this is better as an issue rather than a GitHub discussion, since this is a LangGraph bug and not a design question.

Example Code

import operator
from typing import TypedDict, Annotated, Sequence, Literal

from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.messages import BaseMessage
from langgraph.graph import END, StateGraph
from pydantic import BaseModel


llm = ChatOpenAI(model='gpt-4o-mini')

class Decision(BaseModel):
    next: Literal['animals', 'car_dealer']

class AgentState(TypedDict):
    """Defining the internal state structure of the agent graph."""
    messages: Annotated[Sequence[BaseMessage], operator.add]
    next: str


prompt1 = ChatPromptTemplate.from_messages([('placeholder', '{messages}'),
                                           ('system', 'Given the question, which agent should act next?')])

supervisor = prompt1 | llm.with_structured_output(Decision, method="json_schema")

prompt2 = ChatPromptTemplate.from_messages(
        [('system', 'you are an animal expert agent'),
         ('placeholder', '{messages}')])

animal_agent = prompt2 | llm | (lambda x: {'messages': [x.content]})

prompt3 = ChatPromptTemplate.from_messages(
        [('system', 'you are a car expert agent'),
         ('placeholder', '{messages}')])

car_agent = prompt3 | llm | (lambda x: {'messages': [x.content]})

workflow = StateGraph(AgentState)
workflow.add_node("supervisor", supervisor)
workflow.add_node("animals", animal_agent)
workflow.add_node("car_dealer", car_agent)

conditional_map = {k: k for k in ['car_dealer', 'animals']}
conditional_map["FINISH"] = END
workflow.add_conditional_edges("supervisor",
                                lambda x: x["next"],
                                conditional_map)

workflow.set_entry_point("supervisor")

graph = workflow.compile(debug=False)

Error Message and Stack Trace (if applicable)

Failed to use model_dump to serialize <class 'pydantic._internal._model_construction.ModelMetaclass'> to JSON: TypeError("BaseModel.model_dump() missing 1 required positional argument: 'self'")
Failed to use dict to serialize <class 'pydantic._internal._model_construction.ModelMetaclass'> to JSON: TypeError("BaseModel.dict() missing 1 required positional argument: 'self'")
Failed to use model_dump to serialize <class 'pydantic._internal._model_construction.ModelMetaclass'> to JSON: TypeError("BaseModel.model_dump() missing 1 required positional argument: 'self'")
Failed to use dict to serialize <class 'pydantic._internal._model_construction.ModelMetaclass'> to JSON: TypeError("BaseModel.dict() missing 1 required positional argument: 'self'")

Description

Hi, I'm trying to use OpenAI's structured output feature as a supervisor node in Langgraph. However, when I invoke the graph for the first time, I get the warnings above. They disappear when I invoke the graph a second time. Here's a minimal example to reproduce the issue.

System Info

langchain==0.3.2
langchain-community==0.3.1
langchain-core==0.3.9
langchain-openai==0.2.2
langchain-text-splitters==0.3.0
langgraph==0.2.34

Python 3.12.6
MacOS

@hinthornw
Copy link
Collaborator

Hello! Could you please share your pydantic version as well? Thank you!

@thoffmann-artidis
Copy link
Author

thoffmann-artidis commented Oct 9, 2024

Sure, it's 2.9.2

Edit: Same behaviour with the following setup
langchain==0.3.3
langchain-community==0.3.1
langchain-core==0.3.10
langchain-openai==0.2.2
langchain-text-splitters==0.3.0
langgraph==0.2.35
langgraph-checkpoint==2.0.1
pydantic==2.9.2
pydantic-settings==2.5.2
pydantic_core==2.23.4

@yoch
Copy link
Contributor

yoch commented Oct 10, 2024

Hi,

Same bug here, here is the code responsible of that :

class UserInfos(BaseModel):
    "User informations to retrieve"
    first_name: Optional[str] = Field(description="user first name")
    last_name: Optional[str] = Field(description="user last name")
    email: Optional[str] = Field(description="user email")
    phone: Optional[str] = Field(description="user phone")

class GetUserInfosReturnType(BaseModel):
    user_infos: UserInfos = Field(description="User informations")
    status: Literal['STARTED', 'FINISHED', 'CANCELLED'] = Field(description='Discussion status')
    response: str = Field(description="Message from the assistant to continue")

llm = ChatOpenAI(model='gpt-4o-mini')
template = ChatPromptTemplate.from_messages([
    ("placeholder", "{history}"),
    ('system', GET_USER_INFOS_SYSTEM_MESSAGE),
])
chain = template | llm.with_structured_output(GetUserInfosReturnType, method='json_schema')
out = chain.invoke({'history': messages})

Here is the lib versions (pretty same as the OP) :

langchain                                0.3.3
langchain-community                      0.3.2
langchain-core                           0.3.10
langchain-openai                         0.2.2
openai                                   1.51.2
pydantic                                 2.9.2
pydantic_core                            2.23.4

The warnings

Failed to use model_dump to serialize <class 'pydantic._internal._model_construction.ModelMetaclass'> to JSON: TypeError("BaseModel.model_dump() missing 1 required positional argument: 'self'")
Failed to use dict to serialize <class 'pydantic._internal._model_construction.ModelMetaclass'> to JSON: TypeError("BaseModel.dict() missing 1 required positional argument: 'self'")
Failed to use model_dump to serialize <class 'pydantic._internal._model_construction.ModelMetaclass'> to JSON: TypeError("BaseModel.model_dump() missing 1 required positional argument: 'self'")
Failed to use dict to serialize <class 'pydantic._internal._model_construction.ModelMetaclass'> to JSON: TypeError("BaseModel.dict() missing 1 required positional argument: 'self'")

@yoch
Copy link
Contributor

yoch commented Oct 10, 2024

NOTES

This works without any warning : (EDIT: same bug occurs, see below)

chain = template | llm.bind(response_format=GetUserInfosReturnType)
msg = chain.invoke({'history': messages})
out = msg.additional_kwargs.get('parsed')

Or even simpler since I only want JSON output :

chain = template | llm.bind(response_format=GetUserInfosReturnType) | JsonOutputParser()
out = chain.invoke({'history': messages})

@gbaian10
Copy link
Contributor

I can reproduce this issue. I reduced the code to the following.
I think it's an issue within the langchain_openai package.

from dotenv import load_dotenv
from langchain_openai import ChatOpenAI
from pydantic import BaseModel, Field

load_dotenv()
model = ChatOpenAI(model="gpt-4o-mini")


class UserInfo(BaseModel):
    "User informations"

    name: str = Field(description="user first name")


chain = model.with_structured_output(UserInfo, method="json_schema")
chain.invoke("Hi, I'm Bob.")

@thoffmann-artidis
Copy link
Author

bind(response_format=GetUserInfosReturnType) | JsonOutputParser()

Not sure this is true. The following code yields the same warning:

from dotenv import load_dotenv
from langchain_core.output_parsers import JsonOutputParser
from langchain_openai import ChatOpenAI
from pydantic import BaseModel, Field

load_dotenv()
model = ChatOpenAI(model="gpt-4o-mini")


class UserInfo(BaseModel):
    "User informations"

    name: str = Field(description="user first name")


chain = model.bind(response_format=UserInfo) | JsonOutputParser()
chain.invoke("Hi, I'm Bob.")

@MiloParigi
Copy link

Hi, same issue in my code, and using the minimal example from @gbaian10

langgraph==0.2.34
langchain-core==0.3.9
langchain-openai==0.2.2
langgraph-cli==0.1.52
pydantic=2.9.2

@yoch
Copy link
Contributor

yoch commented Oct 10, 2024

@thoffmann-artidis

you're right, I was wrong because the warning doesn't appear on the second use.

In fact, this minimal code is able to reproduce without chaining :

from dotenv import load_dotenv
from langchain_openai import ChatOpenAI
from pydantic import BaseModel, Field

load_dotenv()

class UserInfo(BaseModel):
    "User informations"
    name: str = Field(description="user first name")

model = ChatOpenAI(model="gpt-4o-mini", model_kwargs={'response_format': UserInfo})
model.invoke("Hi, I'm Bob.")

@vbarda vbarda transferred this issue from langchain-ai/langgraph Oct 10, 2024
@langcarl langcarl bot added the investigate label Oct 10, 2024
@dosubot dosubot bot added the 🤖:bug Related to a bug, vulnerability, unexpected error with an existing feature label Oct 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🤖:bug Related to a bug, vulnerability, unexpected error with an existing feature investigate
Projects
None yet
Development

No branches or pull requests

5 participants