Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

unable to read nc file exported by xee #265

Closed
kongdd opened this issue Sep 3, 2024 · 6 comments
Closed

unable to read nc file exported by xee #265

kongdd opened this issue Sep 3, 2024 · 6 comments

Comments

@kongdd
Copy link

kongdd commented Sep 3, 2024

using NCDatasets
f = "xee-test.nc"
NCDataset(f)

"xee-test.nc" is exported by xee, and available at: xee-test.zip

Details can be found at: google/Xee#171

Gives the error:

julia> NCDataset(f)
ERROR: MethodError: Cannot `convert` an object of type Vector{Float64} to an object of type String

Closest candidates are:
  convert(::Type{String}, ::Base.JuliaSyntax.Kind)
   @ Base C:\workdir\base\JuliaSyntax\src\kinds.jl:975
  convert(::Type{String}, ::String)
   @ Base essentials.jl:321
  convert(::Type{String}, ::FilePathsBase.AbstractPath)
   @ FilePathsBase C:\Users\hydro\.julia\packages\FilePathsBase\Laclt\src\path.jl:117
  ...

Stacktrace:
 [1] setindex!(h::Dict{String, String}, v0::String, key0::Vector{Float64})
   @ Base .\dict.jl:367
 [2] initboundsmap!(ds::NCDataset{Nothing, Missing})
   @ CommonDataModel C:\Users\hydro\.julia\packages\CommonDataModel\G3moc\src\dataset.jl:284
 [3] NCDataset(ncid::Int32, iswritable::Bool, isdefmode::Base.RefValue{Bool}; parentdataset::Nothing, maskingvalue::Missing)
   @ NCDatasets C:\Users\hydro\.julia\packages\NCDatasets\PkTUf\src\dataset.jl:89
 [4] NCDataset
   @ C:\Users\hydro\.julia\packages\NCDatasets\PkTUf\src\dataset.jl:65 [inlined]
 [5] NCDataset(filename::String, mode::String; format::Symbol, share::Bool, diskless::Bool, persist::Bool, memory::Nothing, maskingvalue::Missing, attrib::Vector{…})
   @ NCDatasets C:\Users\hydro\.julia\packages\NCDatasets\PkTUf\src\dataset.jl:235
 [6] NCDataset
   @ C:\Users\hydro\.julia\packages\NCDatasets\PkTUf\src\dataset.jl:170 [inlined]
 [7] NCDataset(filename::String)
   @ NCDatasets C:\Users\hydro\.julia\packages\NCDatasets\PkTUf\src\dataset.jl:170
 [8] top-level scope
   @ c:\Users\hydro\github\CUG-hydro\xee_abc\test-NcData.jl:11

While, it is normal with NetCDF.jl and R terra.

using NetCDF
f = "xee-test.nc"
lon = ncread(f, "lon")
lat = ncread(f, "lat")
A = ncread(f, "surface_net_solar_radiation")

NCDatasets: v0.14.5

@Alexander-Barth
Copy link
Owner

Alexander-Barth commented Sep 4, 2024

Thank you for your reproducible bug report.
In fact, according to the CF convention the bounds attribute is expected to be a string (a containing a variable name with related metadata):
https://cfconventions.org/Data/cf-conventions/cf-conventions-1.11/cf-conventions.html#cell-boundaries

This is what I see with the tool ncdump:

netcdf xee-test {
dimensions:
        time = 10 ;
        lon = 78 ;
        lat = 48 ;
variables:
        float surface_net_solar_radiation(time, lon, lat) ;
                surface_net_solar_radiation:_FillValue = NaNf ;
                surface_net_solar_radiation:id = "surface_net_solar_radiation" ;
                surface_net_solar_radiation:data_type = "{\'type\': \'PixelType\', \'precision\': \'double\'}" ;
                surface_net_solar_radiation:dimensions = 3601LL, 1801LL ;
                surface_net_solar_radiation:crs = "EPSG:4326" ;
                surface_net_solar_radiation:crs_transform = 0.1, 0., -180.05, 0., -0.1, 90.05 ;
                surface_net_solar_radiation:scale_factor = 0.1 ;
                surface_net_solar_radiation:bounds = 108.5, 28.75, 116.25, 33.5603826424351 ;
[...]

xee seem to reuse the attribute bounds for something else (probably the bounding box).
I will update NCDatasets/CommonDataModel to ignore the bounds attribute if it has not an expected type.

@Alexander-Barth
Copy link
Owner

Alexander-Barth commented Sep 4, 2024

Also a coordinate variable must not have the _FillValue or missing_value attributes:

https://github.com/cf-convention/cf-conventions/blob/main/conformance.adoc#5-coordinate-systems-and-domain

        double lon(lon) ;
                lon:_FillValue = NaN ;
        double lat(lat) ;
                lat:_FillValue = NaN ;

@kongdd
Copy link
Author

kongdd commented Sep 4, 2024

Thanks for your quick response. I think ignore the wrong format bounds attribute is a acceptable option.

@kongdd
Copy link
Author

kongdd commented Sep 4, 2024

Also a coordinate variable must not have the _FillValue or missing_value attributes:

https://github.com/cf-convention/cf-conventions/blob/main/conformance.adoc#5-coordinate-systems-and-domain

        double lon(lon) ;
                lon:_FillValue = NaN ;
        double lat(lat) ;
                lat:_FillValue = NaN ;

Thank you. Maybe I need to post those information to xee developer.

Alexander-Barth added a commit to JuliaGeo/CommonDataModel.jl that referenced this issue Sep 4, 2024
@Alexander-Barth
Copy link
Owner

Maybe I need to post those information to xee developer.

Yes, that would be great and reduce some troubles with other software.

@Alexander-Barth
Copy link
Owner

Alexander-Barth commented Oct 31, 2024

Fixed via JuliaGeo/CommonDataModel.jl@46f22f4

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants