WIP: 109 layer descriptor shape parameters #110
Conversation
In order to allow for layers that have different dimensions from the main grid (specifically the VRRefinements and VRNode layers), added support for dimensions in the LayerDescriptor class, and propagated support to the derived classes.
VR data in field BAG files appears to contain 2D arrays rather than the 1D version that they should; modified code to follow this model, rather than the standard so that source data can be read. Replaced deprecated sprintf() with snprintf().
WIP debugging to confirm location of memory leak causing heap-bombs elsewhere (commit purely to allow publication and other support activities).
brian-r-calder
added
bug
| hsize_t dims[2]; | ||
| int ndims = h5dataSet->getSpace().getSimpleExtentDims(dims, nullptr); | ||
| if (ndims != 2) { | ||
| throw InvalidVRRefinementDimensions{}; |
There was a problem hiding this comment.
@brian-r-calder Here we are checking for there to be two dimensions, if there aren't, an InvalidVRRefinementDimensions exception is thrown. However, the message from InvalidVRRefinementDimensions currently reads: "The variable resolution refinement layer is not 1 dimensional.", which is confusing. Can you clarify?
There was a problem hiding this comment.
@brian-r-calder Relatedly, test_bag_valuetable.cpp is failing in test case TEST_CASE("test georeferenced metadata layer readVR/writeVR", "[georefMetadatalayer][readvr][writevr]") because the test VR dataset is being used is one-dimensional, which is triggering the above exception. Does the test need to be updated, or the code?
There was a problem hiding this comment.
This is a case of adapting to the reality of BAG files in the field. Although the specification says that VR refinements is a one-dimensional array, and should be written into the HDF5 file as a single dimensional dataset, BAG files retrieved from NCEI (from NOAA surveys) do not have this form. Although not entirely proven, I suspect that this is because the GDAL drivers are not following the specification as written.
We have a couple of options here: we can attempt to get the GDAL drivers re-written and then re-write ever BAG file in the US national archive (and presumably everywhere else that BAG files are created using GDAL -- meaning everything that uses CARIS software, most likely); or, we can adapt the reference library to match the field experience. Given the level of complexity in the former, I've chosen to pursue the latter.
To fix this, I've changed the exception to say that the dimensions are "inconsistent with specification", but this also means that we'll have to change the test to reflect this.
| uint32_t numRows = 0, numColumns = 0; | ||
| std::tie(numRows, numColumns) = pDataset->getDescriptor().getDims(); | ||
| std::tie(numRows, numColumns) = m_pLayerDescriptor->getDims(); | ||
|
|
||
| if (columnEnd >= numColumns || rowEnd >= numRows) |
There was a problem hiding this comment.
@brian-r-calder This branch is being taken when running test_bag_vrmetadata.cpp, specifically when the test attempts to read back the VR metadata. Is the test wrong, or is there a mistake in Layer::read()?
There was a problem hiding this comment.
I don't see a branch here, but the code is correct --- we have to read the dimensions specifically from the layer rather than from the dataset, since we need to allow each layer to have its own dimensions.
When reporting exceptions for VR refinements not having the right number of dimensions, we have both 1D and 2D arrays that have to be handled (the 2D simply because we're patching around a bug in BAG creation in other software that doesn't use this code that's not following the specification correctly). The reporting of InvalidVRRefinementDimensions therefore has to be neutral on number of dimensions and report that the number is inconsistent.
Issue #109: layers do not currently have separate shape parameters (except Geometadata layer), but should in order for VR refinements to work appropriately. This pull request adds this functionality, and also bug-fixes a number of issues in the VR implementation, and other deprecation issues (e.g., snprintf() rather than sprintf()).