DataKiosk Query NextToken - Create new query to retrieve all paginated data? #4384
-
|
DataKiosk requires paginated data retrieval to be called by creating a new query using the createQuery operation, until there are no tokens returned. I don't see this as a viable solution for production queries, given the extremely limited throttling of one call per minute on DataKiosk create query requests. Can you confirm that this is the case? and if so, is there a way around this limitation? |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 1 reply
-
|
Hello @kevnhowe so the rate limits are set to match the Reports API and paginated requests should be somewhat rare. Did you face any throttling issue when using tokens? Most datasets as of now have a data refresh rate of more than one day which should not cause any limitations on your end to fetch data. Best, |
Beta Was this translation helpful? Give feedback.
-
|
Hey Chris, we are only experiencing paging when asking for large datasets. I am trying to query daily data for 1200 skus going back two years. Since this is a one time project, I don't see a real issue with it. That is, unless paging limits become smaller. |
Beta Was this translation helpful? Give feedback.
Hello @kevnhowe
so the rate limits are set to match the Reports API and paginated requests should be somewhat rare. Did you face any throttling issue when using tokens?
Most datasets as of now have a data refresh rate of more than one day which should not cause any limitations on your end to fetch data.
Best,
Chris Khoury
Solution Architect