-
Notifications
You must be signed in to change notification settings - Fork 3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Financing arm64 runners #2002
Comments
One day after I complained, Apple silicon powered M1 macOS Github runners were released as a public beta. I think I will create a PR trying to use them. |
If you want to continue using GCP with custom runners, I can try ask internally at 2i2c if we can help cover some of the cost. |
Alternatively we can use QEMU in the regular runners! It's a bit slower, but costs no money and doesn't require us to maintain more infra. We use it through the jupyterhub project (https://github.com/jupyterhub/jupyterhub/blob/70717dc9ab163b568c51f70369374d35b91ba56f/.github/workflows/release.yml#L111, https://github.com/jupyterhub/zero-to-jupyterhub-k8s/blob/78bec62d591b7a97560c37edd79c7a15186fcfe9/.github/workflows/publish.yml#L71, etc). Would you be open to trying that out here? |
That would be nice, thank you! |
We had QEMU before and this is gonna be going backwards, not forward. At least one image doesn't build using it, maintaining qemu is more difficult than maintaining self-hosted runners, we also had qemu related hard-to-fix bug, the build time is much slower - our CI takes 1h30min, if I remember correctly it's gonna take at least ~2.5 times longer with QEMU. |
Ah TIL #1703. I'll reach out internally and try figure something out! |
FWIW, I've used CircleCI a few places for free-tier native ARM builders, pyzmq for wheels, and some docker images for work. I believe Travis CI also has free-tier aarch46 builders. |
Thanks, I'll take a look. I kind of like having everything in GitHub, and it works well. |
Yeah, it's all trade-offs. Spreading builds across CIs definitely has a cost, if not a financial one. There isn't a single great answer. Paying some $$$ to keep things simple is definitely sensible, if we can find a way to do it. |
I fully agree, i think 2i2c can fund this, otherwise my "sundell open source consulting" will! This is the most advanced use of githubs CI system i've seen, so maintaining it in two systems isn't an option i think. |
@mathbunnyru I'm working internally in making this happen. Do you have a sense of what total cost would be? I am currently asking for Google Cloud support, capped at $100 a month for the next year (and check-in at that point). Do you think that'll cover it? |
@mathbunnyru good news! I've $2000 approved for use over the next 12 months at least (and then check-in)! We'd like to be credited with a simple line in the docs and README as sponsoring the CI for ARM images. If this sounds ok to you, I can work with you on setting this up. If you already have a google cloud project going, perhaps the easiest is for us to work together on putting that on a different payment method that comes from us? Then we can set budget alerts to monitor. Thank you for all the amazing tireless work you do <3 |
That's great!
Sure, I think it's fair. Send a PR 🙂
Well, to be honest, I'm using my personal and main Google account currently. I think the best approach would be if you or I created a separate Google (and the Google Cloud) account, shared permissions, and you tied your payment method to this account. |
@mathbunnyru oh yeah that makes total sense! Here's the actions I will take:
Does that sound good? I'll do this once the card gets approved for my use. |
@yuvipanda sounds wonderful! 👍 |
@mathbunnyru this is set up! Can you share your email address with me (privately if you prefer - |
Done |
@yuvipanda I recreated the setup in the new project. |
@yuvipanda I also created a PR mentioning 2i2c project. Please, take a look, and then I'll be able to completely close this issue. |
yay, thanks for getting this all set up @mathbunnyru - and thanks for all the work you do here :) |
We somehow got rid of curl errors after we re-created aarch64 runners 🎉 We tried to fix it here, but it didn't actually work and I was still getting some failures during downloads in aarch64 builds (and had to rerun failed builds from Web UI). |
My Google Cloud trial is over, Tau T2A VMs are available for a free trial until March 31, 2024.
And, there is already some charge for networking - our building process consumes lots of networking (uploading/downloading images as artifacts). Also some daily fees for SSD ($0.34) and static IP addresses ($0.19).
I'm glad I introduced
zstd
3 months ago - it definitely saved me some money 😆My costs for August and September were $24.28 and $26.42 respectively.
I'm okay to pay for these runners out of my pocket (for now), but it would be nice to hear from Jupyter community if there is something we can do 😄
The text was updated successfully, but these errors were encountered: