Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: v2.5.0 run text_embedding.py example tip error:pymilvus.exceptions.DataNotMatchException #2480

Open
1 task done
danerlt opened this issue Dec 19, 2024 · 0 comments
Labels
kind/bug Something isn't working

Comments

@danerlt
Copy link

danerlt commented Dec 19, 2024

Is there an existing issue for this?

  • I have searched the existing issues

Describe the bug

当我运行example目录下的text_embedding.py,报错:

pymilvus.exceptions.DataNotMatchException: <DataNotMatchException: (code=1, message=Insert missed an field `dense` to collection without set nullable==true or set default_value)>

下面是我调整之后的代码,加入了创建数据库的逻辑:

# hello_text_embedding.py demonstrates how to insert raw data only into Milvus and perform
# dense vector based ANN search using TextEmbedding.
# 1. connect to Milvus
# 2. create collection
# 3. insert data
# 4. create index
# 5. search
# 6. drop collection
import time

from pymilvus import (
    MilvusClient,
    utility,
    FieldSchema, CollectionSchema, Function, DataType, FunctionType,
    Collection,
)

from pymilvus import connections, db

conn = connections.connect(host="127.0.0.1", port=19530)

databases = db.list_database()
print(databases)
db_name = "my_db"
if db_name not in databases:
    print("需要创建数据库")
    database = db.create_database(db_name)
    print("创建数据库成功")

collection_name = "text_embedding"

milvus_client = MilvusClient("http://127.0.0.1:19530", db_name=db_name)

has_collection = milvus_client.has_collection(collection_name, timeout=5)
schema = milvus_client.create_schema()
schema.add_field("id", DataType.INT64, is_primary=True, auto_id=False)
schema.add_field("document", DataType.VARCHAR, max_length=9000)
schema.add_field("dense", DataType.FLOAT_VECTOR, dim=1536)

text_embedding_function = Function(
    name="openai",
    function_type=FunctionType.TEXTEMBEDDING,
    input_field_names=["document"],
    output_field_names="dense",
    params={
        "provider": "openai",
        "model_name": "text-embedding-3-small",
    }
)

schema.add_function(text_embedding_function)

index_params = milvus_client.prepare_index_params()
index_params.add_index(
    field_name="dense",
    index_name="dense_index",
    index_type="AUTOINDEX",
    metric_type="IP",
)

ret = milvus_client.create_collection(collection_name, schema=schema, index_params=index_params, consistency_level="Strong")

rows = [
        {"id": 1, "document": "Artificial intelligence was founded as an academic discipline in 1956."},
        {"id": 2, "document": "Alan Turing was the first person to conduct substantial research in AI."},
        {"id": 3, "document": "Born in Maida Vale, London, Turing was raised in southern England."},
]

insert_result = milvus_client.insert(collection_name, rows, progress_bar=True)


# -----------------------------------------------------------------------------
search_params = {
    "params": {"nprobe": 10},
}
queries = ["When was artificial intelligence founded",
           "Where was Alan Turing born?"]

start_time = time.time()
result = milvus_client.search(collection_name, data=queries, anns_field="dense", search_params=search_params, limit=3, output_fields=["document"], consistency_level="Strong")
end_time = time.time()

for hits, text in zip(result, queries):
    print(f"result of text: {text}")
    for hit in hits:
        print(f"\thit: {hit}, document field: {hit.get('document')}")

错误详细信息:

RPC error: [insert_rows], <DataNotMatchException: (code=1, message=Insert missed an field `dense` to collection without set nullable==true or set default_value)>, <Time:{'RPC start': '2024-12-19 17:54:08.846455', 'RPC error': '2024-12-19 17:54:08.895140'}>
Traceback (most recent call last):
  File "E:\work\incremental-learning\api\tests\t_milvus.py", line 69, in <module>
    insert_result = milvus_client.insert(collection_name, rows, progress_bar=True)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\work\incremental-learning\api\.venv\Lib\site-packages\pymilvus\milvus_client\milvus_client.py", line 225, in insert
    raise ex from ex
  File "E:\work\incremental-learning\api\.venv\Lib\site-packages\pymilvus\milvus_client\milvus_client.py", line 221, in insert
    res = conn.insert_rows(
          ^^^^^^^^^^^^^^^^^
  File "E:\work\incremental-learning\api\.venv\Lib\site-packages\pymilvus\decorators.py", line 141, in handler
    raise e from e
  File "E:\work\incremental-learning\api\.venv\Lib\site-packages\pymilvus\decorators.py", line 137, in handler
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "E:\work\incremental-learning\api\.venv\Lib\site-packages\pymilvus\decorators.py", line 176, in handler
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\work\incremental-learning\api\.venv\Lib\site-packages\pymilvus\decorators.py", line 116, in handler
    raise e from e
  File "E:\work\incremental-learning\api\.venv\Lib\site-packages\pymilvus\decorators.py", line 86, in handler
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "E:\work\incremental-learning\api\.venv\Lib\site-packages\pymilvus\client\grpc_handler.py", line 494, in insert_rows
    request = self._prepare_row_insert_request(
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\work\incremental-learning\api\.venv\Lib\site-packages\pymilvus\client\grpc_handler.py", line 520, in _prepare_row_insert_request
    return Prepare.row_insert_param(
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\work\incremental-learning\api\.venv\Lib\site-packages\pymilvus\client\prepare.py", line 601, in row_insert_param
    return cls._parse_row_request(request, fields_info, enable_dynamic, entities)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\work\incremental-learning\api\.venv\Lib\site-packages\pymilvus\client\prepare.py", line 462, in _parse_row_request
    raise DataNotMatchException(
pymilvus.exceptions.DataNotMatchException: <DataNotMatchException: (code=1, message=Insert missed an field `dense` to collection without set nullable==true or set default_value)>

Expected Behavior

运行结果不报错

Steps/Code To Reproduce behavior

如上

Environment details

- Hardware/Softward conditions (OS, CPU, GPU, Memory): 
- Method of installation (Docker, or from source): Docker
- Milvus version (v0.3.1, or v0.4.0): v2.4.1 
- PyMilvus Version: v2.5.0
- Milvus configuration (Settings you made in `server_config.yaml`):

Anything else?

No response

@danerlt danerlt added the kind/bug Something isn't working label Dec 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant