WIP: Setup GitHub Action Continuous Integration tests #45
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Running tests on Ubuntu-22.04 and Python 3.10/3.12.
for more information, see https://pre-commit.ci
Remove the requirements_test.txt file, and put `dask` and `pytest` dependencies directly in setup.py instead.
weiji14
changed the title
Try pip installing CuPy without CUDA.
Easier to install CuPy (without CUDA libs) from conda-forge instead of PyPI.
Fixes `ModuleNotFoundError: No module named 'pooch'` when trying to call `xr.tutorial.open_dataset`.
Fixes `ImportError: opening tutorial dataset air_temperature requires either scipy or netCDF4 to be installed` when trying to read the air_temperature.nc file.
|
Yeah, was probably too optimistic to think that we can test CuPy on standard GitHub Actions runners: =================================== FAILURES ===================================
____________________________ test_data_set_accessor ____________________________
tutorial_ds_air = <xarray.Dataset> Size: 31MB
Dimensions: (lat: 25, time: 2920, lon: 53)
Coordinates:
* lat (lat) float32 100B 7...). These a...
platform: Model
references: http://www.esrl.noaa.gov/psd/data/gridded/data.ncep.reanaly...
def test_data_set_accessor(tutorial_ds_air):
ds = tutorial_ds_air
assert hasattr(ds, "cupy")
assert not ds.cupy.is_cupy
> ds = ds.as_cupy()
cupy_xarray/tests/test_accessors.py:37:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../../micromamba/envs/cupy-xarray/lib/python3.10/site-packages/cupy_xarray/accessors.py:182: in as_cupy
return ds.cupy.as_cupy(*args, **kwargs)
../../../micromamba/envs/cupy-xarray/lib/python3.10/site-packages/cupy_xarray/accessors.py:136: in as_cupy
data_vars = {var: da.as_cupy() for var, da in self.ds.data_vars.items()}
../../../micromamba/envs/cupy-xarray/lib/python3.10/site-packages/cupy_xarray/accessors.py:136: in <dictcomp>
data_vars = {var: da.as_cupy() for var, da in self.ds.data_vars.items()}
../../../micromamba/envs/cupy-xarray/lib/python3.10/site-packages/cupy_xarray/accessors.py:168: in as_cupy
return da.cupy.as_cupy(*args, **kwargs)
../../../micromamba/envs/cupy-xarray/lib/python3.10/site-packages/cupy_xarray/accessors.py:70: in as_cupy
data=cp.asarray(self.da.data),
../../../micromamba/envs/cupy-xarray/lib/python3.10/site-packages/cupy/_creation/from_data.py:88: in asarray
return _core.array(a, dtype, False, order, blocking=blocking)
cupy/_core/core.pyx:2383: in cupy._core.core.array
???
cupy/_core/core.pyx:2410: in cupy._core.core.array
???
cupy/_core/core.pyx:2553: in cupy._core.core._array_default
???
cupy/_core/core.pyx:135: in cupy._core.core.ndarray.__new__
???
cupy/_core/core.pyx:223: in cupy._core.core._ndarray_base._init
???
cupy/cuda/memory.pyx:738: in cupy.cuda.memory.alloc
???
cupy/cuda/memory.pyx:1424: in cupy.cuda.memory.MemoryPool.malloc
???
cupy/cuda/memory.pyx:1444: in cupy.cuda.memory.MemoryPool.malloc
???
cupy/cuda/device.pyx:40: in cupy.cuda.device.get_device_id
???
cupy_backends/cuda/api/runtime.pyx:202: in cupy_backends.cuda.api.runtime.getDevice
??
?
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> ???
E cupy_backends.cuda.api.runtime.CUDARuntimeError: cudaErrorInsufficientDriver: CUDA driver version is insufficient for CUDA runtime version
cupy_backends/cuda/api/runtime.pyx:146: CUDARuntimeError
___________________________ test_data_array_accessor ___________________________
tutorial_da_air = <xarray.DataArray 'air' (time: 2920, lat: 25, lon: 53)> Size: 31MBAnyone want to sponsor some GPU runners? Or know of others ways to run GPU-based CI tests, e.g. https://docs.cirun.io/reference/yaml#gpu-gpu? |
weiji14
changed the title
Blog post on how scikit-learn set up GPU CI - https://blog.scientific-python.org/scikit-learn/gpu-ci/ (see also discussion at scikit-learn/scikit-learn#24491). Need to ask if we could get this set up for cupy-xarray via NumFOCUS. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Minimal configuration to run unit tests on GitHub Actions CI. Using pytest to run the test suite.
Running matrix tests on Ubuntu-22.04 and Python 3.10/3.12.
TODO:
- [ ] Fix broken unit tests as much as possible(done in Fix broken doctest and tests on accessors #46)Fixes #8