YJIT: Allow upgrading YJIT in released Ruby versions

[k0kubun]

Problem

Core and 99% of SFR use Ruby 3.2 for YJIT deployment. As a result, you need to wait until Ruby 3.3 is released to test the latest YJIT on Core or with more workloads on SFR. A shorter feedback loop would be desirable to improve the quality of YJIT releases.

Motivation

I wish we had this feature in Ruby 3.2 when I looked at SFR production metrics.

On average, Ruby 3.2 generates smaller code on SFR than Ruby 3.3 because Ruby 3.3 has better ratio_in_yjit, which is a good thing.

However, if you look at the maximum, Ruby 3.2 generates a lot more code than Ruby 3.3. I believe this is because Ruby 3.3 is exposed to only 1% of the traffic and serves fewer varieties of requests. The more traffic, the better testing.

Because of that, code GC runs regularly on Ruby 3.2 while it never runs on Ruby 3.3.

Since Ruby 3.3 is not tested under SFR's worst case scenario yet, I don't know what will happen to that workload when it's actually released.

For Core, while we tuned Ruby 3.2 YJIT after the release and adjusted the default --yjit-exec-mem-size much later, it would have been a better release if we were able to experiment with that before released.

Background

One might think you should just upgrade the main Ruby version in those repositories to Ruby 3.3. It's technically possible, but it would leave the problem of significantly slowing down bundle install in local development, which would be bad for the productivity of developers.

Some C extensions, especially the ones that are slow to build, have binary releases for released Rubies. You need to build them from scratch for unreleased Rubies. When a new release of a third-party gem comes out and a developer tries to upgrade the version locally, there's no way the gem publisher have prepared a binary release for a random Ruby sha we're using.

Solutions

I'm not married to any specific approach yet, but right now I think it's worth experimenting with building a yjit.gem to get more ideas about the actual blockers for Approaches 1 and 2. Then, if we learn it's practical enough from the experiment, we could take Approach 2 as a small start.

1. Implement YJIT as a default gem

A default gem is a gem that is usable without extra steps and hard-copied to the ruby/ruby repository. Even if we make YJIT a default gem, it should not change how YJIT is built when installing Ruby. You need rustc when building Ruby and it should be available if you pass --yjit. Default gems are upgraded when newer Rubies are released. You don't need to treat them as a gem unless you need to override the gem version that is released by core developers.

If we go with this path, required_ruby_version of yjit.gem should be very constrained. YJIT for Ruby 3.3 should be allowed only on Ruby 3.3.x. @paracycle even suggested to lock it at a teeny level (e.g. 3.3.0), and I think it's a good option too. This means that we'll need to maintain different YJIT branches for every Ruby minor versions, but note that we already do that in ruby_3_1 and ruby_3_2 branches of ruby/ruby. This doesn't necessarily introduce extra maintenance targets unless we wish to do so.

pros

• Easy to upgrade YJIT independently from the interpreter
• Easy to manage backports
  • Commit history in ruby/yjit will include only YJIT changes. Easier to read them, and less likely to conflict. We could automate the process for no-conflict changes too.
• Binary releases of yjit.gem could make rustc installation optional (@paracycle's idea)
  • Instead of building Ruby with rustc available, users may choose to retrofit YJIT by writing gem "yjit" in Gemfile.

cons

• We need to spend some time developing this feature
  • Export functions used by YJIT, rework how cruby_bindings.inc.rs is generated, update invalidation hooks to call a replaceable gem, implement a Rust extension environment, and propagate Ruby build flags to the gem.
• More moving parts that could break YJIT
  • As Ufuk suggested, it's probably safer to lock versions at a teeny level for this reason.

2. Implement YJIT as an extension library

Even if we don't gemify YJIT, we could tweak the interpreter to lazily load the YJIT implementation through require "yjit.so". This is how YARP currently works in ruby/ruby. If the interpreter doesn't require "yjit.so" on --yjit-pause and lazily calls it on RubyVM::YJIT.resume, we could mutate $LOAD_PATH to replace the yjit.so implementation using an arbitrary gem. Then we could develop and use yjit.gem like Approach 1 while never gemifying YJIT in the interpreter implementation.

The main difference between Approaches 1 and 2 is that yjit.so cannot be replaced by just writing gem "yjit" in Gemfile. You need to do some extra setup on $LOAD_PATH to replace yjit.so, which is fine for our team but not convenient for most other people. Anyway, if you want to avoid gemifying YJIT in Ruby releases while still allowing us to replace it with a yjit.gem, this is how.

pros

• It shares the same pros as Approach 1.

cons

• It shares the same cons as Approach 1.
  • The YJIT extension library will not need functions to be exported, but yjit.gem will need them to be exported anyway.
• It's tricky to replace the YJIT implementation with a gem unlike Approach 1.

3. Use pshopify to backport more YJIT features

We kind of address the problem already by "pshopify" releases in Shopify/ruby-definitions. In fact, our Ruby 3.2 build has an optimization that was backported from Ruby 3.3 and then stayed while it was supposed to be a temporary experiment.

This could work as a last resort. We might actually want to try this for Ruby 3.2. But for future Rubies, taking other approaches would make the backport work simpler and easier, which would lead us to test YJIT changes in those environments more often.

pros

• This is already possible today.

cons

• Backport management seems more tedious than other approaches.

[casperisfine]

I'd like to add one pro to 1 (and potentially 2): it would allow to ship a secondary yjit-stats gems, allowing to get extra info without having to compile ruby with extra flags. Most users get Ruby via various binary releases, so compiling Ruby with extra flags is a bit out of reach. It's also make it harder to enable it on a small % of your infra.

[k0kubun]

The most important thing we want to switch with a Ruby build flag is rb_yjit_collect_vm_usage_insn, which is called inside vm_exec_core. It's part of the interpreter, so it might be still out of reach.

This is an off topic here, but I gave a thought to the stats build problem again. I came up with an idea of duplicating vm_exec_core and it seems like we can build Ruby that supports ratio_in_yjit without an interpreter slowdown this way.

[maximecb]

I don't want to sound overly critical, maybe gemifying YJIT would be a good solution, but since this would be a big change, I would like to voice some concerns, and also think about potential alternatives to make sure we look at all possible options.

Some C extensions, especially the ones that are slow to build, have binary releases for released Rubies. You need to build them from scratch for unreleased Rubies.

I know this may be impractical, but I do have to wonder if there wouldn't be some way to have a centralized cache of precompiled gems for prerelease rubies. In theory, there could be a machine associated with rubygems.org which periodically rebuilds the most common gems with Ruby head. It might not be necessary to rebuild said gems for every commit. I know, more infrastructure, sounds hard, but it could also make using unreleased rubies in production much easier?

Instead of building Ruby with rustc available, users may choose to retrofit YJIT by writing gem "yjit" in Gemfile.

Does that mean not enabling YJIT by default? We have to be careful because every extra step that we require end users to do to enable YJIT means less people are going to use YJIT. If we want the broadest possible YJIT adoption, we have to make enabling it as simple as possible, and make YJIT be available by default on every system where it's supported.

Backporting

Regarding having YJIT as a default gem. It's not clear to me what the backporting process would look like to build a YJIT gem for older Ruby releases. The difficulty seems to be that a priori, every released Ruby version would have its own API that it exposes to the YJIT gem, with minor API differences for each version. That would mean we'd need to build multiple YJIT gem versions that work with different Ruby releases. I guess that's a bit simpler than what we currently do in terms of backporting, because it wouldn't be the whole YJIT codebase that changes, but it still means we have to build multiple versions of YJIT. Possibly we would want to use conditional compilation or such? It's non-trivial effort.

There are also YJIT features that require changes to CRuby. Things like object shapes, and Alan's work on a faster calling convention. Things like that can't easily be backported, which could make things very iffy in the YJIT codebase. It would be super awkward to have to use conditional compilation to deal with multiple different possible object layouts in the YJIT codebase, for example.

I think in theory, having an API for a pluggable JIT sounds nice, but in practice it could turn out to be fairly complex and not that practical. That's the big risk. I would want to know how the CRuby build process changes, whether we could still count on having YJIT be enabled by default (and eventually turned on by default) what the API would look like from the YJIT end, and also to have a clearer picture of what backporting would look like.

Alternatives

I think it would also be good to keep thinking about other solutions for making the use of unreleased/prereleased Rubies in production easier. Be that some kind of networked build cache... Or just being willing to run 3.3+ on the CI and in production even though that's not what developers are running locally?

I do want to point out that we've been running prereleased Ruby commits on SFR for over a year and it's been very reliable, it works. The developers are presumably running 3.2 on their machines and it hasn't been a problem. In theory we could deploy the same newer Ruby commit on all of SFR and it seems like it should work fine, based on past experience. Core is bigger, but could the same thing not be true there?

[k0kubun]

In theory, there could be a machine associated with rubygems.org which periodically rebuilds the most common gems with Ruby head.

Given that nobody else runs Ruby head in production (even GitHub doesn't), they would not build it just for us. We need to build and operate the system ourselves.

One practical compromise I thought of now is having a scheduled workflow that polls a set of known slow-build extension gems and pushes binaries for Ruby shas we're currently using to our gem server.

Another thing we should consider is binary gem distributions for preview Ruby releases, which seems more practical when we need to persuade the community. It would take time for the community to adopt it on rubygems.org, but we could start doing that in our gem server.

I'm open to taking those paths if Approaches 1 and 2 turn out to be impractical, but the gem capability would allow a quicker feedback loop and make us more productive than not having it.

Does that mean not enabling YJIT by default?

No, this doesn't change. As the name suggests, default gems are built/installed by default. As long as rustc is present when building Ruby, YJIT build would be enabled by default.

It's not clear to me what the backporting process would look like to build a YJIT gem for older Ruby releases.

have a clearer picture of what backporting would look like.

Approaches 1 and 2 do not really change how we backport patches from master to stable branches. Currently, we backport patches from master to ruby_3_1 / ruby_3_2 on ruby/ruby. Imagine doing the same thing but on ruby/yjit, which will be automatically synchronized to the corresponding branches on ruby/ruby.

With Approach 2, the release process will not change because we don't ship any gem in the first place. With Approach 1, the release managers may bump a gem version if something in yjit.gem has been changed and not released yet.

What's different is that we can optionally have a non-stable branch that we don't include in Ruby releases but we use for applications like Core. Besides stable branches like ruby_3_3, you can maintain ruby_3_3-master branch, for example, and backport non-bugfix changes to it. We could discard the branch at any time since it's not part of releases. The ruby_3_3-master branch should be abandoned once Ruby 3.4 is released. Also, if we have something to experiment with on Core, we could only commit changes to ruby_3_3-master and nothing needs to be backported from master for it.

Things like that can't easily be backported, which could make things very iffy in the YJIT codebase. It would be super awkward to have to use conditional compilation to deal with multiple different possible object layouts in the YJIT codebase, for example.

I did not propose to use a single branch for multiple Ruby versions. YJIT master should only support Ruby master, and each released YJIT version can require a specific Ruby teeny version. You should only backport what makes sense to backport. Memory optimization or code changes that are closed in codegen.rs should be possible. If we're not interested in testing a next version in more production workloads, we can just leave the branch. Right now it's just impossible even if we want to.

it could turn out to be fairly complex and not that practical. That's the big risk. I would want to know how the CRuby build process changes
what the API would look like from the YJIT end

which is why I'm suggesting "experimenting with building a yjit.gem to get more ideas" in this issue.

whether we could still count on having YJIT be enabled by default (and eventually turned on by default)

Whether we take Approach 1 or 2, we should be able to do that.

I think it would also be good to keep thinking about other solutions for making the use of unreleased/prereleased Rubies in production easier. Be that some kind of networked build cache... Or just being willing to run 3.3+ on the CI and in production even though that's not what developers are running locally?

I agree. In particular, as I mentioned early in this comment, we should start thinking about how we could have binary gem distributions for Ruby releases. This would be useful whether YJIT is gemified or not. Once we start using a preview Ruby, we can use it like a new Ruby release. Whenever a preview release is made, we can abandon a non-stable branch and re-fork a preview release, which would make backports easier. When we need to test something that cannot be backported, we can ask the release managers to cut a new preview release.

This would be much better than today's situation, so we should definitely do that. At the same time, you'd still need to wait for preview releases when you have something to experiment with in production. Allowing yjit.gem seems like the only practical solution for Shopify to fully control when to unblock a YJIT change in production.

I do want to point out that we've been running prereleased Ruby commits on SFR for over a year and it's been very reliable, it works. The developers are presumably running 3.2 on their machines and it hasn't been a problem. In theory we could deploy the same newer Ruby commit on all of SFR and it seems like it should work fine, based on past experience. Core is bigger, but could the same thing not be true there?

We're on the same page about this. If it's not safe, we shouldn't test newer YJIT changes in production in those environments anyway. There's no hard blocker, but I don't want to make developers unhappy (with slow bundle install) for running newer YJIT.

[k0kubun]

TL;DR for the previous comment: yjit.gem support still seems like the only solution that solves everything we want. We should experiment with the idea to verify its practicality. At the same time, preparing binary gem distributions for preview Ruby releases might be more practical, so it's worth exploring too.

[flavorjones]

a centralized cache of precompiled gems for prerelease rubies

I initially proposed a build farm for security reasons in early 2021, and I brought it up again just this week.

Luis Lavena set up a proof of concept in 2020 for this, but abandoned the work because it's hard to make a general solution for all platforms. I think if we limit our initial goals to "just Shopify arm64-darwin and x86_64-linux systems" it's probably doable, and would address both security and ruby-head use cases.

I don't want to get too far into the weeds, but a likely implementation would involved rake-compiler-dock, which I'm co-maintainer of. My experience setting up a build container for a the last few releases of Ruby has been challenging; and I'm not at all sure that it would be easy to keep that environment continuously up-to-date with ruby head as changes are being made. But it might be worth exploring.

see rake-compiler/rake-compiler-dock#103

[maximecb]

This would be much better than today's situation, so we should definitely do that. At the same time, you'd still need to wait for preview releases when you have something to experiment with in production. Allowing yjit.gem seems like the only practical solution for Shopify to fully control when to unblock a YJIT change in production.

The thing is, we're already deploying unreleased Ruby commits to SFR. It's a process that's relatively simple (from our end) and works well. Core is a bigger app, but in theory it should be possible to have a similar canary setup there too. That IMO, is the simplest solution from a YJIT development standpoint, in terms of developer ergonomics, because there is no backporting, we don't have to maintain multiple branches or make sure that our newer changes work with older Ruby binaries.

I did not propose to use a single branch for multiple Ruby versions. YJIT master should only support Ruby master, and each released YJIT version can require a specific Ruby teeny version. You should only backport what makes sense to backport. Memory optimization or code changes that are closed in codegen.rs should be possible. If we're not interested in testing a next version in more production workloads, we can just leave the branch.

What really makes me hesitate with this is that it adds complexity, more moving parts, to the YJIT development process. It means we have to keep track to some degree of multiple CRuby JIT gem API versions, we have to have multiple YJIT branches for different CRuby releases. There's a non-trivial backporting process. Some changes are backportable and some are not. Once changes are backported, we need a CI process to test our backported YJIT gem with older CRuby binary versions before deployment. Would we also need to publish the YJIT gems that we compile to rubygems? If so, we'd also need to document which YJIT gem versions go with which CRuby versions. I guess the YJIT gems could check that the CRuby version is compatible and give you an error message if not.

We should experiment with the idea to verify its practicality.

For me the big pain point is how difficult/annoying the backporting, and having to deal with the coupling of multiple CRuby releases and multiple YJIT gem versions is going to be.

I also want to point out. I think we both have our own biases. My own bias is that I think the status quo that we have is not that bad and could be improved in other ways that would possibly be more seamless from a YJIT-development standpoint. You and Aaron are working on RJIT/TenderJIT. You love the idea of a JIT gem because it puts your JIT projects on a more even footing, and it makes it easier to build a Ruby JIT in Ruby. You view the idea of a pluggable JIT gem as a desirable feature in and of itself, whereas I tend think of it as added complexity that we should properly justify before we make such a big change. You seem very motivated to work on this and I think it will most likely happen, but my words of caution are: I think what you are suggesting is definitely feasible, but it may not solve our problems as well as you think it will in practice. Don't just think of the upsides, think of the potential downsides as well. Particularly the human factor and developer experience.

which is why I'm suggesting "experimenting with building a yjit.gem to get more ideas" in this issue.

Would you be ok with writing a document in which you try to sketch what the JIT gem API would look like? What are all the things the gem exposes to CRuby? How does CRuby expose functions, constants and struct layouts to the JIT gem? How does one go about running CRuby with an alternative JIT gem version? How do we handle checking for JIT API compatibility? It doesn't need to be perfect, but it would give everyone a more clear picture.

Maxime's proposed solution

Implement a canary deployment system for Shopify Core that works like our SFR canary/experimental system, which we can point at recent Ruby commits. This solution should work as long as the CI machine(s) have build caches for the gems? Presumably developers don't necessarily have to run the same CRuby sha, because that is not the case with SFR and we've been managing very well.

If you couple that with more frequent preview releases that have prebuilt gems, it means we could cut down the time until our YJIT changes make it into production down to 4-6 months, without worrying about backporting.

Not saying this solution is perfect, but presumably, it is feasible. It could be a viable alternative to having a JIT gem. The benefits are that the process for testing YJIT changes on Core becomes more like SFR. We don't have to worry about JIT gem compatibility issues. We can also deploy/test CRuby changes regularly. We also test Core on Ruby head regularly, which makes sure that Core remains working on Ruby head, even though this is not what the Core developers are using.

[k0kubun]

The thing is, we're already deploying unreleased Ruby commits to SFR. It's a process that's relatively simple (from our end) and works well.

Did you read the "Details" of the Motivation section? This issue is trying to solve a problem that is not addressed yet. We could possibly increase the canary % or have another cluster to improve it a bit, but it comes with a different trade-off.

What really makes me hesitate with this is that it adds complexity, more moving parts, to the YJIT development process.

This is true and I take it into account carefully, but:

There's a non-trivial backporting process.
For me the big pain point is how difficult/annoying the backporting

I'd like to point out that the backport process for what we publicly release will not necessarily change and only branches for experimental deployments/versions will have a non-trivial backporting process, which is optional and can be skipped when you don't need it. The way it's solved today is that we simply can't solve it even if we want to.

having to deal with the coupling of multiple CRuby releases and multiple YJIT gem versions is going to be.

I clarified in past comments that this doesn't happen with teeny-level version locking.

You and Aaron are working on RJIT/TenderJIT. You love the idea of a JIT gem because it puts your JIT projects on a more even footing, and it makes it easier to build a Ruby JIT in Ruby.

I actually didn't realize that it'd be easier to build TenderJIT after this since RJIT doesn't really benefit from it. RJIT is already as easy to build as YJIT, so nothing is improved by this project. While Aaron could be biased by that, I am not.

I tend think of it as added complexity that we should properly justify before we make such a big change.

I do too.

You seem very motivated to work on this

because of the "Motivation" I wrote. I want to improve production performance, not for making it easier to build RJIT or TenderJIT. You seem to care more about how Ruby JITs are built, but my first priority is how well YJIT performs in production. I think the difference in our bias comes from the fact that Jean and I spend more time in tuning and monitoring Ruby in production than other folks. I take making ratio_in_yjit available by default more seriously than you, for example. I am a former SRE and have as much interest in (especially YJIT's) production performance as in optimizing compilers.

If you couple that with more frequent preview releases that have prebuilt gems, it means we could cut down the time until our YJIT changes make it into production down to 4-6 months, without worrying about backporting.

Yeah. We usually have 3 preview releases per version, so some interval may be even shorter. Plus, we'll also have rc releases with relatively short intervals when nearing the release, and that'd be convenient for making sure the next release is in a good shape. This would be the only option for testing Ruby 3.3 release anyway (since we didn't gemify it in 3.2), so we should explore that this year.

[maximecb]

I actually didn't realize that it'd be easier to build TenderJIT after this since RJIT doesn't really benefit from it. RJIT is already as easy to build as YJIT, so nothing is improved by this project. While Aaron could be biased by that, I am not.

Fair.

I take making ratio_in_yjit available by default more seriously than you

I don't think that's necessarily fair. I just view engineering work as trying to find a good compromise/tradeoff between multiple different concerns, that are sometimes pulling in slightly different directions. The YJIT team is a small team, so for me, the amount of complexity that we have to manage vs the time/energy we have is a high priority concern. My main worry is that we make YJIT so complex that we struggle to maintain and develop it. That's not something that happens instantly. It happens over dozens of technical decisions that we make over months and years.

Maxime: You seem very motivated to work on this
Kokubun: because of the "Motivation" I wrote. I want to improve production performance, not for making it easier to build RJIT or TenderJIT.

Just to clarify because it can be hard to read tone in text. Me saying that you seem very motivated to work on this is not any kind of attack or criticism. It just means: you seem to really like this idea and you seem excited to just go ahead and do it. I haven't said no. I'm not trying to stop you from doing it. I just want to make sure that we look at all the options and we're all satisfied with the solution(s) we decide to go for, and how we go about it. What I actually think is that we'll likely move forward with the JIT gem solution.

I clarified in past comments that this doesn't happen with teeny-level version locking.

Why not? I don't really understand all the details of how all of this would work. I'm assuming you would end up with one or multiple YJIT gems. Like, say, there could be a yjit-ruby33-v3 gem, or something, and that gem only works with Ruby 3.3. Then you would need a separate gem with its own versioning scheme for Ruby 3.4. You still need to make sure that you pair the correct YJIT gems with the correct Ruby releases. It does introduce (some) potential for user confusion if you allow end users to swap the YJIT gem by changing one line in a gem file, no?

Let's try to imagine how this would look from the perspective of end users. What would it look like to deploy a newer YJIT gem for a released version of Ruby 3.3?

One thing that I think could potentially be very cool, if it were possible, is if we could seamlessly deploy a custom Ruby 3.3 binary that comes with an upgraded YJIT gem. Maybe even with YJIT enabled by default, but also works with all the precompiled Ruby 3.3 gems. Could we get to a point where someone can use ruby-build or ruby-install to install/deploy a version of CRuby with an upgraded YJIT gem (and maybe YJIT enabled by default)?

If that's not viable, how would it work then? Could you specify which YJIT gem to use through an environment variable? Would you have to edit a gem file?

[k0kubun]

The YJIT team is a small team, so for me, the amount of complexity that we have to manage vs the time/energy we have is a high priority concern. My main worry is that we make YJIT so complex that we struggle to maintain and develop it. That's not something that happens instantly. It happens over dozens of technical decisions that we make over months and years.

Very true 👍

I just want to make sure that we look at all the options and we're all satisfied with the solution(s) we decide to go for, and how we go about it.

Yeah, I appreciate all your input. You're making my effort of filing this issue really meaningful.

Like, say, there could be a yjit-ruby33-v3 gem, or something, and that gem only works with Ruby 3.3.

I admit that I'm discussing something not straightforward, so let me rephrase it. Basically, I imagine to have two different kinds of YJIT releases for past Ruby versions:

• A: Bug-fix only. This is shipped as part of stable Ruby releases.
  • For Approach 1, "teeny-level version locking" means that we specify something like required_ruby_version = "3.3.1", which cannot be installed on 3.3.0 or 3.3.2+.
  • For Approach 2, code included in the Ruby release is not gemified, so it's not reusable by other versions.
• B (totally optional): Experimental YJIT builds, which exist for the latest Ruby on SFR and Core. Not part of Ruby releases.
  • This is for Shopify or any other YJIT developers to test their changes. It's very similar to our pshopify distribution. End-users are not supposed to use this. We could say "own your risk" to use this distribution. Imagine installing TenderJIT over RJIT.
  • Because of that, we only need to support the latest Ruby version. If Core and SFR use Ruby 3.3.1, required_ruby_version can be 3.3.1.

Does this make sense?

[maximecb]

I think I was imagining something different, because at some point you (or someone else) mentioned loading an alternative JIT gem by changing a line in a gem file. So, I was imagining that there would be publicly available YJIT gem releases that match up with specific Ruby release versions.

B (totally optional): Experimental YJIT builds, which exist for the latest Ruby on SFR and Core. Not part of Ruby releases.
This is for Shopify or any other YJIT developers to test their changes.

I got the impression that this is really the main goal. For us to deploy the latest YJIT advances to Core/SFR faster. There is still some backporting effort required, but let's assume that the effort is less because the JIT API exposed by CRuby would presumably be fairly stable over time.

I do still want to ask though (just humor me), why is it that much better to have a YJIT gem and backport than to just backport the latest cruby/yjit subtree onto the CRuby 3.3.1 pshopify branch? Do you feel that having a mostly stable JIT API will make the backporting work that much easier?

I'm also wondering if there is some way that we can somehow make the broader community (e.g. GitHub) benefit as well, by somehow having our own CRuby 3.3.1 + yjit_gem_latest release.

In terms of actionable steps, let's assume that we move forward with a YJIT gem. I still think it would be good to sketch the JIT API (what is exposed by YJIT, what is exposed by CRuby) in a document, and also to answer questions about what the backporting and deployment process would look like. If you go through this exercise, I think it will give us all a clearer picture, and assuming we do go forward, you will write better code once you've taken the time to sketch and map out the idea first.

[k0kubun]

I think I was imagining something different, because at some point you (or someone else) mentioned loading an alternative JIT gem by changing a line in a gem file.

It is an optional benefit Approach 1 can achieve with relative ease. It can still have required_ruby_version = "3.3.1" with the cost of cutting a (often no-diff) YJIT release for each teeny Ruby release. And yes, (B) is the main goal here, not that.

why is it that much better to have a YJIT gem and backport than to just backport the latest cruby/yjit subtree onto the CRuby 3.3.1 pshopify branch?

Hmm, I assumed we don't have subtree git history like https://github.com/ruby/ruby/commits/master/yjit, but we do. It might not be as inconvenient as I initially imagined. Being able to use the standard procedure on ruby/yjit just like other ruby/xxx repositories (e.g. ruby/yarp) would be still more intuitive/convenient for some folks, but maybe only slightly.

Another thing is, it currently feels uncomfortable to backport experimental stuff to pshopify tags because pshopify is shared by all services. If our experimental YJIT is managed as an experimental gem distribution or just a random sha on GitHub, we can start from deploying an experimental branch to only a canary cluster on SFR, and then gradually rollout that to other clusters or services. In pshopify, everybody would think it's a stable release and use it right away.

We could maybe have an experimental pyjit distribution and tweak dev and spin to make it work transparently. I wasn't sure if the team would be open to that idea, but yeah, we should think about that as an option too.

[k0kubun]

I discussed the idea of deploying a preview release (maybe preview2 when it's released since preview1 is already too old) to Core and SFR globally before BFCM with Mike and Jean today. Given that CI and spin cache precompiled extensions, we think it's worth a shot, even without crafting a gem server with precompiled binaries for head/preview versions. The time taken for building gRPC and nokogiri will still be a problem for dev users, but we assume few people use dev (Mike, Jean, and I use dev for Core, but it's because we're Ruby Infra?), and it happens only when the version is bumped anyway. As an experiment, we'll also compare the performance between (non-pshopify) 3.2 and 3.3 later on Core (Jean will do after the reforking project) and SFR (I can do myself), which may contribute to justifying this change on BFCM.

So for the problem described in the issue description, I'm going to use the combination of Solution 3 (#525), which I've been doing in the last three weeks, and the preview release idea for the foreseeable future.

---

@tenderworks Since I lost the use case for it, I'll leave it for you to file an issue about the design of YJIT gemification if you're still interested. I have ideas, but I bet you'd think of something similar.