Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add rust unstable with the dev label #31

Closed
rth opened this issue Dec 22, 2018 · 10 comments · Fixed by #52
Closed

Add rust unstable with the dev label #31

rth opened this issue Dec 22, 2018 · 10 comments · Fixed by #52

Comments

@rth
Copy link
Member

rth commented Dec 22, 2018

While the ability to install rust with conda is quite nice, something I have found limiting so far is the ability to install rust unstable needed for some packages/crates.

As far as I understood, features currently in rust unstable have no guarantee of making into the stable, and so when those are needed by a crate, waiting for the next release of rust is not a viable solution.

In particular, rust unstable is currently required by https://github.com/PyO3/pyo3 (bindings for CPython C API) and as a result https://github.com/rust-numpy/rust-numpy (binding for numpy C API). Given the amount of Python package on conda-forge, I think it is quite important to be able to build a Python package with a "C" extension written in Rust.

If you think it's a good idea, I can (try) to make a PR to add rust unstable under e.g. dev label (or some other non-default label).

@rth rth changed the title Add rust unstable to the dev channel Add rust unstable with the dev label Dec 22, 2018
@luizirber
Copy link

luizirber commented Dec 22, 2018

What if a new feedstock for rust-nightly is created, and follow CalVer? The main issue of using nightly in other feedstocks is that it is not uncommon to break code with new updates, but using the date for a "known good version" will keep the dependant feedstock working.

@rth
Copy link
Member Author

rth commented Dec 22, 2018

What if a new feedstock for rust-nightly is created, and follow CalVer?

What would be the advantage of that compared e.g. to having a separate brach in this repo? I know that some repos (e.g. https://github.com/conda-forge/qt-feedstock) have two parallel branches, but I'm not sure if anything specific needs to be done to publish the built packages from the non master branch to the server as well.

The main issue of using nightly in other feedstocks is that it is not uncommon to break code with new updates, but using the date for a "known good version" will keep the dependant feedstock working.

Right, but if it has a separate label, it will only be installed when the user explicitly provides

conda install --channel "conda-forge/label/dev" package

and a different version scheme could be done there I imagine without impacting the rust package on the main conda-forge channel (unless I'm missing something)?

@jakirkham
Copy link
Member

Publishing under a different label using a different branch is how we prefer to handle RCs and such.

@pkgw
Copy link
Contributor

pkgw commented Dec 22, 2018

How often would you want to update the unstable feedstock? Each set of Rust artifacts is about 750 MB, so daily updates might start getting to be a bit much even in these glorious days of The Cloud.

@rth
Copy link
Member Author

rth commented Dec 22, 2018

Definitely not daily. Even a couple of times a year might do I think. For instance pyo3 can be run with rust unstable 1.30 even if the latest stable version is 1.31. So it's not about getting the latest nightly, but rather having a compiler which has the unstable features even if it's from some time ago.

@pkgw
Copy link
Contributor

pkgw commented Dec 22, 2018

OK, cool. In that case I think this is definitely a good idea.

If you want a reference for how to start packaging a new label, here's the patch I used for cairo not so long ago.

If the autotick bot starts submitting PR's for the dev branch every day, we'll need some way to get it to chill out, but we can cross that bridge if/when we get there.

@rth
Copy link
Member Author

rth commented Jan 18, 2019

Thanks.

The other solution is to wait for PyO3 to get rid of specialization feature which seems to be on the roadmap for the next release. That would allow building Python extensions with rust stable, and will make this issue no longer necessary as far as I am concerned..

@mariusvniekerk
Copy link
Member

@pkgw Should we have a specialized rust-nightly package for things that need rust nightly instead?

@pkgw
Copy link
Contributor

pkgw commented Jun 25, 2019

@mariusvniekerk Hmm, I think that might be the best approach. Could we implement a rust-nightly package as a branch of this same feedstock, but with a different package name in the meta.yaml? Having two different feedstock repos wouldn't be the end of the world, but their contents would be awfully similar so I think it would be better to keep them close together if possible.

hartb added a commit to hartb/powerai that referenced this issue Mar 9, 2020
Add a feedstock recipe for building a 'rust-nightly' package from the
upstream binary distribution tarballs. This recipe is based on:

https://github.com/conda-forge/rust-feedstock

And note some discussion there about building nightlies:

conda-forge/rust-feedstock#31

One complication with the nightlies is that the source URLs for rust
nightly builds are only versioned with the word 'nightly'. They don't
contain an indication of the actual rust version that's included,
and presumably they'll change from time to time.

That leads to 2 issues:

1) The SHA hashes in the recipe may often be invalidated.

"Fix" this by removing the hashes from meta.yaml.

2) The 'rust-nightly' package version is hard-coded in meta.yaml and
so may fall out of sync with the internal rust version in the
download. In this case the build would succeed, but the package
version would be misleading.

Fix this by including a build test to confirm that the rust version
in the built package actually matches the package version. That
will at least force a build break, rather than allowing creation
of a mis-versioned package.
hartb added a commit to IBM/powerai that referenced this issue Mar 10, 2020
Add a feedstock recipe for building a 'rust-nightly' package from the
upstream binary distribution tarballs. This recipe is based on:

https://github.com/conda-forge/rust-feedstock

And note some discussion there about building nightlies:

conda-forge/rust-feedstock#31

One complication with the nightlies is that the source URLs for rust
nightly builds are only versioned with the word 'nightly'. They don't
contain an indication of the actual rust version that's included,
and presumably they'll change from time to time.

That leads to 2 issues:

1) The SHA hashes in the recipe may often be invalidated.

"Fix" this by removing the hashes from meta.yaml.

2) The 'rust-nightly' package version is hard-coded in meta.yaml and
so may fall out of sync with the internal rust version in the
download. In this case the build would succeed, but the package
version would be misleading.

Fix this by including a build test to confirm that the rust version
in the built package actually matches the package version. That
will at least force a build break, rather than allowing creation
of a mis-versioned package.
@pkgw
Copy link
Contributor

pkgw commented Apr 29, 2020

Fixed in #52, merged to the new dev branch as per new conda-forge recommendations for this sort of thing.

@pkgw pkgw closed this as completed Apr 29, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants