Prefer AsyncMock to as_future

[pquentin]

In Python, calling an async function f() can be done in two steps:

1. call the function using regular function call f()
2. await the result, possibly later

The driver tests took advantage of this detail to make MagicMock work with async functions by using as_future. With that trick, mocked functions return an awaitable result, which are then awaited in the tested code.

However, coroutines are best thought as an implementation detail, and it's better to always call an async function f using await f(), never separating the two steps mentioned above. Thankfully, Python 3.8 introduced AsyncMock that allows removing as_future by just specifying a return value, which avoids thinking about coroutines, which is what we want.

There's just one wrinkle: while mock.patch() replaces the target with an AsyncMock, it does not work recursively. So while we would like to rewrite

es.cluster.health.return_value = as_future({"status": "green", "relocating_shards": 0})

to

es.cluster.health.return_value = {"status": "green", "relocating_shards": 0}

we need to use

es.cluster.health = mock.AsyncMock(return_value={"status": "green", "relocating_shards": 0})

which is still an improvement as it avoids the as_future code smell.

[danielmitterdorfer]

Good catch! I noticed that there are still places where we use as_future and I assume it's because the mock.patch annotation uses regular mocks? In any case I think we should add pydocs with guidance when to use as_future and state explicitly that mock.AsyncMock is preferred. Also, can you please add a description in the PR body documenting the motivation for this PR?

[pquentin]

@danielmitterdorfer I ended up completely removing as_future, and explained in the description why the second part was harder. Do we still need guidance now?

[danielmitterdorfer]

I ended up completely removing as_future [...] Do we still need guidance now?

Thanks! As there is now only one approach to test async code there's no need for any guidance anymore.

[danielmitterdorfer]

Nice cleanup! LGTM