Upstream failing due to Xarray dims ordering change

[tomwhite]

Xarray no longer sorts dimension by name (pydata/xarray#4753). This affects the display representation, which is causing some upstream tests to fail. Example failure:

Differences (unified diff with -expected +actual):
    @@ -1,5 +1,5 @@
     <xarray.Dataset>
    -Dimensions:                  (alphas: 5, blocks: 2, contigs: 2, outcomes: 5, samples: 50)
    -Dimensions without coordinates: alphas, blocks, contigs, outcomes, samples
    +Dimensions:                  (blocks: 2, alphas: 5, samples: 50, outcomes: 5, contigs: 2)
    +Dimensions without coordinates: blocks, alphas, samples, outcomes, contigs
     Data variables:
         regenie_base_prediction  (blocks, alphas, samples, outcomes) float64 0.33...

I'm not sure how to fix this so that the tests work against both Xarray 0.18.2 and the main branch.

The test_gwas_linear_regression__validate_statistics test is also failing due to the same change, but the error is a numerical mismatch, which suggests the dimensions are somehow getting mixed up.

[hammer]

From pydata/xarray#4753:

Currently this PR retains the sorting in reprs.

Could we use repr() in the tests as a medium term fix?

[tomwhite]

Could we use repr() in the tests as a medium term fix?

I think that comment is out of date, since the repr was changed in the end.

[hammer]

🤔 this reminds me of discussions around https://github.com/pystatgen/sgkit/issues/43. I suppose we might want some kind of schema-aware comparison approach within the library?

[hammer]

@eric-czech suggests using http://xarray.pydata.org/en/stable/generated/xarray.Dataset.transpose.html to enforce dimension order.

[tomwhite]

I don't think transpose is what we want since it transposes the underlying arrays, not the order the dimensions are listed in the dataset/