Add more backends

[dcharbon]

• NumPy/SciPy
• PyTorch
• Xnd
• Dask (add initial implementation for dask backend uarray#142)
• CuPy (Add CuPy (testing locally for now) uarray#146)
• Sparse (Add sparse backend. uarray#147)
• TensorFlow
• More to be added...

[hameerabbasi]

@MridulS is working on the Dask backend! (Thanks!)

[hameerabbasi]

I added the CuPy and PyData/Sparse backends!

[MridulS]

@hameerabbasi Have you ran any benchmarks for CuPy yet?

[hameerabbasi]

Time for calculating the exponential of an array of length 10 ** 8:

float64: 119 ms vs 25.1 ms
float32: 54 ms vs 5.76 ms

[grlee77]

Another potential candidate for inclusion is MXNet which also provides some subset of the NumPy API. I was recently contacted by their team as a potential user, but have not yet tried it out. There is a nice overview given in this post
https://medium.com/apache-mxnet/a-new-numpy-interface-for-apache-mxnet-incubating-dbb4a4096f9f?

Specifically there appears to be CPU and GPU implementations of NumPy functions (in C++/CUDA) here:
https://github.com/apache/incubator-mxnet/tree/master/src/operator/numpy

[dhirschfeld]

I'd personally prefer depth rather than breadth at this point in development - i.e. implement a small number of backends well rather than trying to implement (and support) every possible framework out there.

[jbogaardt]

This conceptually seems like an awesome project and I look forward to seeing its development. I'm wondering if I am trying to jump onto it too soon though or I am using it wrong.

Trying to elaborate on your docs with the following example, I cannot get it to work with a cupy backend. Are the backends available in v0.6.0?

import uarray as ua
import unumpy as np
import unumpy.cupy_backend as cupy_backend
import unumpy.numpy_backend as numpy_backend

def main():
    x = np.zeros(5)
    return np.exp(x)

print(ua.__version__)
print(np.__version__)
# works
with ua.set_backend(numpy_backend):
    print(main())
# does not work
with ua.set_backend(cupy_backend):
    print(main())

0.6.0
0.6.0
[1. 1. 1. 1. 1.]

AttributeError Traceback (most recent call last)
in ()
14 print(main())
15 # does not work
---> 16 with ua.set_backend(cupy_backend):
17 print(main())

~\AppData\Local\Continuum\anaconda3\lib\site-packages\uarray_backend.py in set_backend(backend, coerce, only)
270 pass
271
--> 272 ctx = _SetBackendContext(backend, coerce, only)
273 backend.ua_cache["set", coerce, only] = ctx
274 return ctx

AttributeError: module 'unumpy.cupy_backend' has no attribute 'ua_domain'

[hameerabbasi]

@jbogaardt Unfortunately, I cannot test CuPy on CI, so it tends to lag behind. #34 fixes this. Thanks, @grlee77.

[jbogaardt]

Excellent. I look forward to the next release.

[jbogaardt]

follow up question @hameerabbasi , dask and cupy are starting to become interoperable:
https://matthewrocklin.com/blog//work/2019/01/03/dask-array-gpus-first-steps

Do you envision the backend setup ever supporting combo backends like dask + cupy or will dask_backend always be a chunked numpy array?

[hameerabbasi]

@jbogaardt

Do you envision the backend setup ever supporting combo backends like dask + cupy or will dask_backend always be a chunked numpy array?

This is already possible. You just do the following:

with ua.set_backend(cupy_backend), ua.set_backend(dask_backend):
    ...

The order is reversed from what one would normally expect, but think of it in the following manner: The inner backend gets set last, and then dominates, and then, inside its own code, skips "itself", and dispatches to the next prominent backend. I guess one could provide a utility in uarray to do this. 😄

[jbogaardt]

That is awesome. This is truly an amazing library you're building. Thank you @hameerabbasi !

[hameerabbasi]

@jbogaardt unumpy 0.6.1 is up on PyPI and conda-forge.