Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cosmos: query_items_change_feed reture same resualt with is_start_from_beginning parameters #11689

Closed
nonokangwei opened this issue May 28, 2020 · 8 comments
Assignees
Labels
bug This issue requires a change to an existing behavior in the product in order to be resolved. Client This issue points to a problem in the data-plane of the library. Cosmos customer-reported Issues that are reported by GitHub users external to the Azure organization. Service Attention Workflow: This issue is responsible by Azure service team.
Milestone

Comments

@nonokangwei
Copy link

nonokangwei commented May 28, 2020

  • Package Name: azure-cosmos
  • Package Version: 4.0.0
  • Operating System: Linux
  • Python Version: 3.5.3

Describe the bug
query_items_change_feed fucntion return the same result with the parameter is_start_from_beginning=True and False

To Reproduce
Steps to reproduce the behavior:

python -m venv venv
source venv/bin/activate
pip install azure-cosmos==4.0.0

Query

client = cosmos_client.CosmosClient(HOST, {'masterKey': MASTER_KEY})
container = client.get_database_client(DATABASE_ID).get_container_client(CONTAINER_ID)
container.query_items_change_feed(is_start_from_beginning=True)
# with is_start_from_beginning=false
container.query_items_change_feed()

Expected behavior
Read the change feed from the last checkpoint, tried with the parameter continunation, did not find the sample to fetch the continunation token.

@ghost ghost added needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. customer-reported Issues that are reported by GitHub users external to the Azure organization. question The issue doesn't require a change to the product in order to be resolved. Most issues start as that labels May 28, 2020
@kaerm kaerm added the Cosmos label May 28, 2020
@ghost ghost removed the needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. label May 28, 2020
@kaerm kaerm added the Client This issue points to a problem in the data-plane of the library. label May 28, 2020
@annatisch
Copy link
Member

Thanks @nonokangwei - I will try to repro today :)

@annatisch
Copy link
Member

annatisch commented Jun 1, 2020

Hi @nonokangwei - I put together some sample code based on our tests demonstrating the continuation support that is hopefully helpful:

document_definition = {'pk': 'pk', 'id':'doc1'}
collection.create_item(body=document_definition)

query_iterable = collection.query_items_change_feed(
    partition_key_range_id=pkRangeId,
    is_start_from_beginning=True,
)
iter_list = list(query_iterable)
self.assertEqual(len(iter_list), 1)

continuation = collection.client_connection.last_response_headers['etag']

# Create two new documents. Verify that change feed contains the 2 new documents
document_definition = {'pk': 'pk', 'id': 'doc2'}
collection.create_item(body=document_definition)
document_definition = {'pk': 'pk', 'id': 'doc3'}
collection.create_item(body=document_definition)

# verify iterator
 query_iterable = created_collection.query_items_change_feed(
    partition_key_range_id=pkRangeId,
    continuation=continuation,
    max_item_count=pageSize
)

Here is the actual test with more detail:
https://github.com/Azure/azure-sdk-for-python/blob/master/sdk/cosmos/azure-cosmos/test/test_query.py#L50

As for the behaviour of is_start_from_beginning - @southpolesteve could you please weigh in with regards to the expected service behaviour of this options as presenting in the OP snippet?

@southpolesteve
Copy link
Contributor

@nonokangwei Is this a partitioned collection? You'll need to pass the partition key range id as the example above

@annatisch Thanks for investigating. The API is a bit strange here. Partition Key Range ID is an integer representing the physical partition that a logical partition key maps to. But it's not something we normally expose to the user. I don't see any methods in the SDK that let a user convert a partition key to a partition key range id. This might just be a bug. Instead the SDK will need to accept the partition key and set that as the header. This is what JS does and forces the user to supply the partition key: https://github.com/Azure/azure-sdk-for-js/blob/master/sdk/cosmosdb/cosmos/samples/ChangeFeed.ts#L73-L78

@nonokangwei
Copy link
Author

@annatisch @southpolesteve i do some test based on the @annatisch sample code, if i do not give the partition key range id paramterer it seems not works. as @southpolesteve said i review the SDK, it seems no function for customer to get the partition key range id. So what is the suggested method to get the partition key range id, i only saw the description in rest-api interface.

@southpolesteve
Copy link
Contributor

Relabeling this as a bug. Partition Key Range Id shouldn't be exposed here. We should instead be accepting a partition key and change feed is scoped to a single partition key value. Cross partition change feed is something we would eventually like to support, but it requires some additional work.

@southpolesteve southpolesteve added bug This issue requires a change to an existing behavior in the product in order to be resolved. and removed question The issue doesn't require a change to the product in order to be resolved. Most issues start as that labels Jul 2, 2020
@lmazuel lmazuel added this to the Backlog milestone Jul 14, 2020
@Petermarcu Petermarcu added the Service Attention Workflow: This issue is responsible by Azure service team. label Jul 30, 2020
@ghost
Copy link

ghost commented Jul 30, 2020

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @anfeldma-ms, @zfoster, @southpolesteve.

@southpolesteve
Copy link
Contributor

@srinathnarayanan Can you look into this issue?

@srinathnarayanan
Copy link
Contributor

Fixed with #13857
We now expose partitionKey param in the query_items_change_feed API (can be passed as a named parameter)

openapi-sdkautomation bot pushed a commit to AzureSDKAutomation/azure-sdk-for-python that referenced this issue Nov 25, 2020
[NetAppFiles] Anf 8187 update swagger rest api to 2020 08 01 (Azure#11689)

* Add 2020-08-01 folder

* Update api version

* update readme

* update readme

* issue in readme

* issue in readme

* issue in readme

* touch

Co-authored-by: Audunn Baldvinsson <audunn.baldvinsson@netapp.com>
@github-actions github-actions bot locked and limited conversation to collaborators Apr 12, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug This issue requires a change to an existing behavior in the product in order to be resolved. Client This issue points to a problem in the data-plane of the library. Cosmos customer-reported Issues that are reported by GitHub users external to the Azure organization. Service Attention Workflow: This issue is responsible by Azure service team.
Projects
None yet
Development

No branches or pull requests

7 participants