Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add basic geometry conversion #260

Merged
merged 10 commits into from
Oct 8, 2021
Merged

Add basic geometry conversion #260

merged 10 commits into from
Oct 8, 2021

Conversation

aulemahal
Copy link
Contributor

@aulemahal aulemahal commented Sep 3, 2021

This adds a geometry submodule with 2 main functions : cf_to_geometry and geometry_to_cf. Instead of implementing something geopandas-dependent, I realized I could just assume everything was xarray. That's just an added gdf.to_xarray() in my use case.

geometry_to_cf takes a 1D DataArray of shapely objects and translates it to a dataset that follows the CF conventions (specifically this part). In fact, it simply wraps points_to_cf.

"Features" (as in individual geometries) have nodes and coordinates. Nodes names are easy : 'x' and 'y'. On the other hand, coordinates would normally have names depending on the grid mapping. Also, many attributes would depend on the grid mapping (units and standard_name for example, in nodes as well as coordinates). That's something we lost when not using geopandas, so I added a grid_mapping keyword, but only the basic longitude_latitude is implemented.

cf_to_geometry implements the inverse conversion. shapely is explicitly needed for this, but I didn't want to add a dependency, so import is done within the function. A DataArray of shapely objects is returned.

For a complete cfxr.geometry, I think we would need:

  • Some mapping (json?) of all CF grid mappings, giving the attributes and standard variable names to be used by geometry_to_cf.
  • lines_to_cf, polygons_to_cf and vice-versa.

Bonus

I don't know where to put reshape_unique_geometries... The idea is that our main use case of all this is to translate point timeseries to xarray objects. The input GeoDataframe is 1D, so if, for example, we get multiple timesteps for the same geometry, that geometry is repeated. The reshape_unique_geometries takes care of putting all unique geometries in a new axis. I guess it could be useful for people, but not sure if cf_xarray is the best place. And neither is xarray directly...

How to use

import cf_xarray as cfxr
import geopandas as gpd

# Example using a web service.
gdf = gpd.read_file("https://api.weather.gc.ca/collections/climate-daily/items?datetime=2010-03-01%2000:00:00/2010-06-02%2000:00:00&bbox=-115,50,-112,51&sortby=PROVINCE_CODE,STN_ID,LOCAL_DATE&f=json&limit=150000&startindex=0")

gdfr = gdf.rename(columns={'LOCAL_DATE': 'time'}).set_index('time')
ds_raw = gdfr.to_xarray()

ds_uniq = cfxr.geometry.reshape_unique_geometries(ds_raw)

outcf = cfxr.geometry_to_cf(ds_uniq.geometry, grid_mapping='longitude_latitude')
outcf = xr.merge([ds_uniq.drop_vars('geometry'), outcf])

And the way back:

out_geom = outcf.drop_vars(['x', 'y', 'geometry_container'])
out_geom['geometry']  = cfxr.cf_to_geometry(outcf)

I'll add tests next week.

@codecov
Copy link

codecov bot commented Sep 3, 2021

Codecov Report

Merging #260 (992755d) into main (1fde526) will increase coverage by 0.18%.
The diff coverage is 98.83%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #260      +/-   ##
==========================================
+ Coverage   96.54%   96.72%   +0.18%     
==========================================
  Files          16       18       +2     
  Lines        1939     2110     +171     
==========================================
+ Hits         1872     2041     +169     
- Misses         67       69       +2     
Flag Coverage Δ
unittests 96.72% <98.83%> (+0.18%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
cf_xarray/geometry.py 97.84% <97.84%> (ø)
cf_xarray/__init__.py 100.00% <100.00%> (ø)
cf_xarray/tests/__init__.py 61.90% <100.00%> (+0.92%) ⬆️
cf_xarray/tests/test_geometry.py 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1fde526...992755d. Read the comment docs.

Copy link
Contributor

@dcherian dcherian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just took a quick look; we'll need

  • update api.rst
  • tests
  • make sure ci passes no-optional-deps

I think we can push better docs until we redo the documentation (I have a WIP branch)

cf_xarray/geometry.py Outdated Show resolved Hide resolved
cf_xarray/geometry.py Outdated Show resolved Hide resolved
cf_xarray/geometry.py Outdated Show resolved Hide resolved
cf_xarray/geometry.py Outdated Show resolved Hide resolved
cf_xarray/geometry.py Outdated Show resolved Hide resolved
cf_xarray/geometry.py Outdated Show resolved Hide resolved
cf_xarray/geometry.py Outdated Show resolved Hide resolved
cf_xarray/__init__.py Outdated Show resolved Hide resolved
Copy link
Contributor

@dcherian dcherian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey I can't really review this so you should merge when you think it's ready. It would be good to fix some of those codecov comments.

doc/whats-new.rst Outdated Show resolved Hide resolved
Co-authored-by: Deepak Cherian <dcherian@users.noreply.github.com>
@aulemahal
Copy link
Contributor Author

Good! I'll continue some more testing in the next week. I see that we have no release coming soon, so I guess there is no rush!

@aulemahal aulemahal merged commit a5aa1d6 into main Oct 8, 2021
@aulemahal aulemahal deleted the basic-geometry-conversion branch October 8, 2021 21:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Convert GeoPandas to CF datasets
2 participants