Datastore: add retry param to page iterator. #8547
Conversation
pchauhan-qlogic
requested review from
crwilcox,
frankyn and
tseaver
as code owners
googlebot
added
the
cla: yes
AVaksman
added
the
kokoro:force-run
yoshi-kokoro
removed
the
kokoro:force-run
tseaver
changed the title
| call = retry(call) | ||
| return call() | ||
|
|
||
|
|
There was a problem hiding this comment.
Please add explicit unit tests for _call_api, including covering the case when retry is None.
| def _call_api(fnc_call, retry, *args, **kwargs): | ||
|
|
||
| call = functools.partial(fnc_call, *args, **kwargs) | ||
| if retry: |
There was a problem hiding this comment.
If retry is None, then avoide the partial and just return fnc_call(*args, **kwargs).
| To modify the default retry behavior, call a ``with_XXX`` method | ||
| on ``DEFAULT_RETRY``. For example, to change the deadline to 30 seconds, | ||
| pass ``retry=bigquery.DEFAULT_RETRY.with_deadline(30)``. | ||
| """ |
There was a problem hiding this comment.
Just copying this file in wholesale from bigquery isn't appropriate: we should be refactoring to share this implementation.
datastore/tests/unit/test_retry.py
Outdated
| from google.api_core.exceptions import BadGateway | ||
|
|
||
| exc = BadGateway("testing") | ||
| self.assertTrue(self._call_fut(exc)) |
There was a problem hiding this comment.
Likewise here for the copy-paste from BigQuery.
tseaver
added
the
kokoro:force-run
yoshi-kokoro
removed
the
kokoro:force-run
|
@pchauhan-qlogic Please run This PR still needs to avoid blindly copying the BigQuery @tswast WDYT? |
|
That's right. BigQuery is definitely a special snowflake with respect to retry codes. I recommend choosing the codes to retry independently for Datastore. |
tseaver
added
api: datastore
yoshi-kokoro
removed
the
kokoro:force-run
|
please look into @tswast |
|
I think to bring this back to an earlier comment by @tseaver, we likely want shared code, but the list of retry codes is likely to be special per client. We have a higher level work item that needs to be done which is to decide on the best way forward for providing timeout and retry functionality in our clients in a way that is consistent across products. |
|
Overall design of retries for manual clients is tracked in #9298. |
Fixes #6119