type object 'dummy' has no attribute 'model_json_schema'

[thongtr-dev]

Description of the bug:

When using Pydantic BaseModel or TypedDict from typing_extensions as the response_schema in model.generate_content_async, the following error is raised:

type object 'dummy' has no attribute 'model_json_schema'

This error occurs consistently, regardless of whether I use Pydantic BaseModel or TypedDict.

Steps to Reproduce

1. Define the following models using Pydantic BaseModel or TypedDict:

from pydantic import BaseModel
from typing import Optional, TypedDict

class Question(BaseModel):  # Alternatively, replace with TypedDict for testing
    question_text: Optional[str]
    options: list[str] = [""] * 4
    correct_option: int
    exam_section: str

class Passage(BaseModel):  # Alternatively, replace with TypedDict for testing
    passage_text: str
    questions: list[Question]

2. Use the model as response_schema in a call to model.generate_content_async:

import os
import google.generativeai as genai
from google.generativeai.types import HarmCategory, HarmBlockThreshold

genai.configure(api_key=os.getenv("GEMINI_API_KEY"))
model = genai.GenerativeModel("gemini-1.5-flash-latest")

response_schema = Passage  # Using Pydantic BaseModel here; same issue with TypedDict

prompt = "Generate some data in the required schema."
result = await model.generate_content_async(
    contents=prompt,
    safety_settings={
        HarmCategory.HARM_CATEGORY_HATE_SPEECH: HarmBlockThreshold.BLOCK_LOW_AND_ABOVE,
        HarmCategory.HARM_CATEGORY_HARASSMENT: HarmBlockThreshold.BLOCK_LOW_AND_ABOVE,
        HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT: HarmBlockThreshold.BLOCK_LOW_AND_ABOVE,
        HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT: HarmBlockThreshold.BLOCK_LOW_AND_ABOVE,
    },
    generation_config=genai.GenerationConfig(
        temperature=1.0,
        top_k=3,
        top_p=0.9,
        response_mime_type="application/json",
        response_schema=response_schema,
    ),
)

3. Observe the error:

type object 'dummy' has no attribute 'model_json_schema'

Actual vs expected behavior:

Expected Behavior
The response_schema should validate the generated content against the specified schema (Pydantic BaseModel or TypedDict) without throwing an error like the example in https://ai.google.dev/gemini-api/docs/structured-output?lang=python and in https://discuss.ai.google.dev/t/response-schema-from-pydantic/50028

Actual Behavior
An AttributeError is raised: type object 'dummy' has no attribute 'model_json_schema

Any other information you'd like to share?

Environment

• Python Version: >=3.10
• google-generativeai: >=0.8.3

Workaround
Remove the response_schema from the GenerationConfig or use response_schema with JSON schema instead of Pydantic or TypedDict.

[thongtr-dev]

Update
The exact reason is still unclear, but relocating the script to a completely separate repository with a new Python environment and dependencies resolved the issue.

[manojssmk]

Hi @thongtr-dev,

It seems the error might be due to dependency issues. Here's the gist of the code. Colab may not handle async functions properly, so I recommend trying to run the code in Kaggle instead.