Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use micromamba 1.5.10 where conda is needed #9737

Merged
merged 8 commits into from
Nov 8, 2024

Conversation

kmuehlbauer
Copy link
Contributor

@kmuehlbauer kmuehlbauer commented Nov 7, 2024

Updated behaviour

Since micromamba 2.0 installing conda seems to be broken in CI. In #9732 conda was removed from the install where not absolutely needed and the micromamba version was pinned to 2.0.2-0 due to broken 2.0.3-0.

The broken version was yanked and this PR reverts to the default micromamba version (now 2.0.2-0). In CI runs where conda is absolutely needed the micromamba-version is set to 1.5.10-0 (last version of v1-release line).

Former ramblings:

conda.api and libmambapy are not API compatible

In https://github.com/pydata/xarray/blob/main/ci/min_deps_check.py we use conda.api to gather package versions from ["conda-forge", "defaults"] channel (shouldn't we remove defaults, because of the legal stuff?).

The replacement for micromamba v2 is libmambapy which unfortunately isn't API compatible. It should have all the capabilities to do the same, but I wasn`t successful ( 🤷‍♂️ ) in gathering the correct incantations from the docs:

If figured out the correct way, this can eventually be replaced.

Tested micromamba/libmamba 2.0.2-0 also for:

  • ✅ upstream
  • ❌ benchmark, asv doesn't work, need to step back here, too
  • ✅ hypothesis
  • ❌ pyright, currently disabled

@kmuehlbauer kmuehlbauer added run-benchmark Run the ASV benchmark workflow run-upstream Run upstream CI run-slow-hypothesis Run slow hypothesis tests run-pyright Run pyright type checker labels Nov 7, 2024
@kmuehlbauer kmuehlbauer removed run-pyright Run pyright type checker run-slow-hypothesis Run slow hypothesis tests run-upstream Run upstream CI labels Nov 7, 2024
@kmuehlbauer
Copy link
Contributor Author

@keewis I've missed this yesterday, so checked it today.

asv is also not yet ready for micromamba 2 , so I've reverted min-deps-check and benchmark CI runs to use last 1.5.10-0 version of mamba/micromamba. All other CI runs look good.

WOuld be nice, if you could have another look here. Thanks!

Copy link
Collaborator

@keewis keewis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good, just a couple of comments:

  • I don't have knowledge about the benchmarks, so I can't comment on that
  • according to the issue you opened we might be able to remove most of the micromamba version pins again?
  • regarding libmambapy, I think it is time to rewrite the min-versions script anyways, so I think pinning for now is fine

shouldn't we remove defaults, because of the legal stuff?

we're firmly open source, so that shouldn't affect us (I think?). However, we don't really use the defaults channel anywhere else, so my guess is that this was to get more accurate python release dates? All that would change, though, is that we would get to advance the bump date by a few days – which we usually miss by a couple of weeks anyways, so doesn't really matter either way. And finally, see my note above about rewriting the script.

.github/workflows/ci-additional.yaml Show resolved Hide resolved
@keewis
Copy link
Collaborator

keewis commented Nov 7, 2024

I think we can use py-rattler's version of query instead, which will get rid of the conda dependency, is much faster, and we can avoid the undocumented libmambapy API.

I'll open a PR for the rewrite of the script once it's close to being ready.

@kmuehlbauer
Copy link
Contributor Author

@keewis I've followed your suggestion to use default settings where applicable, as the problematic version was removed.

Let's see how this works.

@kmuehlbauer kmuehlbauer added run-upstream Run upstream CI run-slow-hypothesis Run slow hypothesis tests labels Nov 8, 2024
@kmuehlbauer
Copy link
Contributor Author

Nice, array-api-strict 2.1.2 (released 5 hours ago) breaks the tests.

@kmuehlbauer kmuehlbauer changed the title Use micromamba 1.5.10 together with conda for min-version-policy Use micromamba 1.5.10 where conda is needed Nov 8, 2024
@kmuehlbauer kmuehlbauer enabled auto-merge (squash) November 8, 2024 10:52
@kmuehlbauer kmuehlbauer merged commit 61596a7 into pydata:main Nov 8, 2024
26 of 28 checks passed
@kmuehlbauer kmuehlbauer deleted the min-version-check branch November 8, 2024 11:12
dcherian added a commit to scharlottej13/xarray that referenced this pull request Nov 9, 2024
* main: (125 commits)
  http:// → https:// (pydata#9748)
  Discard useless `!s` conversion in f-string (pydata#9752)
  Apply ruff/flake8-simplify rule SIM401 (pydata#9749)
  Use micromamba 1.5.10 where conda is needed (pydata#9737)
  pin array-api-strict<=2.1 (pydata#9751)
  Reorganise ruff rules (pydata#9738)
  use new conda-forge package pydap-server (pydata#9741)
  Enforce ruff/flake8-pie rules (PIE) (pydata#9740)
  Enforce ruff/flake8-comprehensions rules (C4) (pydata#9724)
  Enforce ruff/Perflint rules (PERF)  (pydata#9730)
  Apply ruff rule RUF007 (pydata#9739)
  chmod -x (pydata#9725)
  Aplpy ruff rules (RUF) (pydata#9731)
  Fix typos found by codespell (pydata#9721)
  support for additional scipy nd interpolants  (pydata#9599)
  Apply ruff/flake8-simplify rules (SIM) (pydata#9727)
  Apply ruff/flake8-implicit-str-concat rules (ISC) (pydata#9722)
  Apply ruff/flake8-pie rules (PIE) (pydata#9726)
  Enforce ruff/pygrep-hooks rules (PGH) (pydata#9729)
  Move to micromamba 2 (pydata#9732)
  ...
dcherian added a commit to dcherian/xarray that referenced this pull request Nov 19, 2024
* main:
  fix html repr indexes section (pydata#9768)
  Bump pypa/gh-action-pypi-publish from 1.11.0 to 1.12.2 in the actions group (pydata#9763)
  unpin array-api-strict, as issues are resolved upstream (pydata#9762)
  rewrite the `min_deps_check` script (pydata#9754)
  CI runs ruff instead of pep8speaks (pydata#9759)
  Specify copyright holders in main license file (pydata#9756)
  Compress PNG files (pydata#9747)
  Dispatch to Dask if nanquantile is available (pydata#9719)
  Updates to Dask page in Xarray docs (pydata#9495)
  http:// → https:// (pydata#9748)
  Discard useless `!s` conversion in f-string (pydata#9752)
  Apply ruff/flake8-simplify rule SIM401 (pydata#9749)
  Use micromamba 1.5.10 where conda is needed (pydata#9737)
  pin array-api-strict<=2.1 (pydata#9751)
  Reorganise ruff rules (pydata#9738)
  use new conda-forge package pydap-server (pydata#9741)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
run-benchmark Run the ASV benchmark workflow run-slow-hypothesis Run slow hypothesis tests run-upstream Run upstream CI
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants