Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFC 0099] Change default shell from bash to oil #99

Closed
Closed
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
121 changes: 121 additions & 0 deletions 0099-Replacing bash as the default shell.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
---
feature: oil-shell
start-date: 2021-08-11
author: Raphael Megzari (happysalada)
co-authors: (find a buddy later to help out with the RFC)
shepherd-team: (names, to be nominated and accepted by RFC steering committee)
shepherd-leader: (name to be appointed by RFC steering committee)
related-issues:
---

# Summary

[summary]: #summary

Use oilshell as the default shell for nixos. Oilshell aims to be compatible with bash while trying to bring real programming language features (like hashmap or dicts) to the shell.
happysalada marked this conversation as resolved.
Show resolved Hide resolved

## Motivation

[motivation]: #motivation

Two primary motivations

- Removing the footguns in bash that even experienced programmers find painful
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How a shell 100% compatible with bash can remove bash footguns?

- Making shell scripts more powerfull to remove the need to bring additional languages for scripting.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What occasions are you referring to here where we bring in other languages for scripting? As far as I am aware stdenv and setup hooks are 100% bash.

Some core nixpkgs builders are implemented in Perl, the question here is, what we'd gain from rewriting them in oil. As it stands the user base of Oil is quite small.

In any case, I'd appreciate a clarification here, maybe even some examples!

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The main thing I'm referring to is the findInputs() in stdenv.
that function is super involved and is doing a very weird recursion with dynamic references all over the place. I feel that the best was done with the tools that bash has to offer, I keep on thinking that it would have been written differently with a programming language that gives you a little more power. Perhaps that function will appear crystal clear to you in that case, feel free to ignore this first point :-)

One more point I have against bash is the error handling. If we had a better way of handling errors, it wouldn't be so bad to try to debug why a modification in stdenv breaks something. I'm not 100% that oil will be able to solve this, but normal programming language give you a little easier interface to work with error handling.

The last point I have is about trying to restrict the numbers of things you need to learn by heart on a language. There are so many pitfals, that we run into experienced programmers making mistakes in nixos modules or packages. I'm hoping that there are less things to learn with oil. (less rules, make it harder to mess up).


## Detailed design

[design]: #detailed-design

The [oil shell](https://www.oilshell.org/) has two parts osh and oil. Osh is compatible with bash and posix and it's goal is to run existing shell scripts. The oil language is a brand new incompatible language. The idea is to fix more than four decades of accumulated warts in the Unix shell. Many Unix users are angry that shell is so difficult, and Oil aims to fix that. Those definitions were taken verbatim from [reference](https://www.oilshell.org/blog/2021/01/why-a-new-shell.html)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do osh and oil interoperate?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You have two binaries, and depending on the one you use, you have different features. osh can run bash script almost as is. In order for oil to run bash scripts, there needs to run scripts, those need to be reworked to tolerate the strict err_exit. Running a script with oil enables you to use additional features of the oil language. Where scripts run with osh, only have access to bash shell features. Let me know if this doesn't answer your question.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So you can access functions defined in an Oil script I assume?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the same way you can access bash function defined in a script.
oil introduce procs which are functions that don't mess with their outer scope and take parameters. I don't know if the same interoperability would apply to those. The way to call a proc is to pass it arguments. I think those would be fine too but this is something that has to be confirmed.

Copy link

@jakeisnt jakeisnt Aug 14, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To clarify, "oilshell" is a standardization of the POSIX shell protocol, and "Oil" (the programming language) is oilshell augmented with Oil language features, correct?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

osh is a bit more as it is meant to run bash things which have some non POSIX behaviours in it. But yes, oil is osh augmented with new language features.


Regarding oil

A high level overview is that it has a syntax similar to python, it brings dictionaries (hashmaps). For more details check the following posts

- [This post](https://www.oilshell.org/blog/2021/01/why-a-new-shell.html) aims at describing the goals in trying to create a new shell and the alternatives.
- [This post](https://www.oilshell.org/blog/2020/01/simplest-explanation.html) aims to provide a simple explanation of what oil is.
- [This post](https://www.oilshell.org/release/latest/doc/idioms.html) shows different oil idioms and how they fix some of bash problems.

There would be two steps to make the transition from bash to oil.

- From bash to osh. There is one incompatibility that came up so far. osh does not handle the `-i` flag on local and declare. The transition involves rewriting some shell scripts to get rid of the `-i` flag. There are about 5-10 uses in stdenv, and I've started PRs to remove them. After those changes, it would be possible to switch from bash to osh and test this for a while to verify there are no regression.
- From osh to oil. It will require to modify stdenv to be able to run with the `strict_errexit` flag. The changes will be a little more involved, but as long as nobody uses oil specific features, we could retain compatibility with bash and posix. The exact quantity of work involved is unknown.
Copy link

@nrdxp nrdxp Sep 14, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps you can clarify the meaning of this statement, as the way it is currently written, it seems to defeat the purpose of this whole idea. Why rewrite the standard env if we don't actually wish to leverage any of oils specific features? If that's not what you meant to convey, then perhaps this segment should just be removed? (I'm referring to line 43 specifically)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I mean by this is that the changes to make the switch possible are a little more involved.
The changes to osh only include removing a -i flag so far (potentially a couple of lines change in stdenv).
The changes to enable a switch to oil are more involved (hard to know the extent yet).
The referring to maintaining compatibility is related to having a period where things are in-between and we can evaluate if we want to switch to oil or not. Potentially retain compatibility for some time and when we feel more comfortable then start using oil exclusive feature.
I do agree with you, it's not very well formulated, how would the following work ?

- From osh to oil requires more heavy changes to stdenv. In particular it requires changing the logic so it can run with `-e` (bash equivalent, strict_errexit). After those changes are made, compatibility with bash can be retained for an arbitrary period of time until we are more confident we want to make the full switch (when we start using oil exclusive features).


One more things to consider here is that changes to stdenv are relatively slow and costly. Even for some seemingly simple changes they can break things in unexpected ways. One example was trying to split buildFlagsArray on whitespace, some packages use newlines for the split. The tests seemed to pass and breakage was discovered much later. For this reason, changes in stdenv can only be made sure to work by triggering a full hydra rebuild which takes several days. Even after that because of flaky tests, it's hard to be sure nothing was broken by the actual changes.

[reference](https://github.com/oilshell/oil/wiki/Migration-Guide)

Some more background context. Originally this is coming from this [PR](https://github.com/NixOS/nixpkgs/pull/105233) by zimbatm. After seeing this, I thought I would implement the changes needed to make the switch and generate a discussion in a PR. Somebody brought to my attention that this change is significant enought that it needs to go through an RFC.

## Examples and Interactions

[examples-and-interactions]: #examples-and-interactions

Some extracts from the post [why use oil](https://www.oilshell.org/why.html)

- better error handling. One of the confusing things about bash is how to handle errors in scripts. `err_exit` was a try at improving that but it has many pitfalls. Oil aims to have more straightforward error handling. Any error will exit, if you do not want that behavior there is the `try my_function` form.

`mkdir /tmp/dest && cp foo /tmp/dest` becomes simply

```Shell
mkdir /tmp/dest
cp foo /tmp/dest
```

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The passage above demonstrates that the semantics of the former and the latter statements provided are equivalent with oilshell, correct? This could use some clarification - 'x becomes y', I don't think, is sufficiently clear.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right this isn't super clear.
Here is what I intended to convey.
The top example is "idiomatic" bash. The bottom example is the equivalent "idiom" in oil.
Let me know if that is too vague


- Oil has
happysalada marked this conversation as resolved.
Show resolved Hide resolved
- Introduce functions that need to explicitely define their parameters

```Shell
proc f(first, @rest) { # @ means "the rest of the arguments"
write -- $first
write -- @rest # @ means "splice this array"
}
```

instead of the traditional

```Shell
f() {
echo $1
shift 1
echo "$@"
}
```

those functions are called procs and their variables don't mess with their outter scope.
happysalada marked this conversation as resolved.
Show resolved Hide resolved

- Oil has proper associative arrays (dictionaries or hashmaps) that don't have the problems that bash's have.

- No more quotes to prevent splitting

## Drawbacks

[drawbacks]: #drawbacks
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I find a benefit of rougly-POSIX-shell, and maybe even Bash outright, is that upstream build system has a high chance of including some shell pieces, and sometimes we need to understand it, and switching between that and a similar but different language stdenv code sounds like a path to confusion — maybe slightly more confusion than using a conspiciously different language like Python or Lua. (But that would have higher migration costs)

Copy link

@jakeisnt jakeisnt Aug 14, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As stated earlier in the proposal, there is no need to modify most bash scripts used for build systems. Think of Oil as a superset of POSIX - it supports everything Bash does with the same semantics, then introduces new, useful language features without regressing on legacy code.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How is Oil's architecture support? I know there is some interest in running Nix on chips different from aarch-64 and x86-64, so we should reasonably ensure that there is interest in actively testing for and supporting select alternative architectures going forward (i.e. RISC-V) so as not to disparage those valuable efforts.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand the confusing part.

  • One thing that should improve the situation is that at the same time all the version compatibility tricks for bash will be slowly removed. Nobody should have to know again what flagsArray=(${distFlagsArray+"${distFlagsArray[@]}"}) mean (just a compatibility between 3 to 4). Those are being removed and so hopefully the bash code feels less foreign as it gets massaged.
  • Oil itself feels very different, so hopefully when we actually use it, the confusion should lessen

Regarding the architecture support I've never seen anything on it. As the code moves to c++ the story should get better in that regard.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One more thing that is probably not a big issue but I want just to mention it..

We know that bash → osh has few «loud» issues like -i. But are there no «quiet» issues where osh attempted to make most sense and ended up slightly incompatible by accident? Probably not so many that we'd hit one. However, we could want either to decide on the scope of our testing of the corner cases, or on the expectations of evidence that other people have checked whether differences are documented.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your concern makes sense. I think we can only make changes by testing the rebuild of the whole of stdenv to make sure nothing breaks silently. It won't provide any guarantees that nothing will break in the future, however, it should be a pretty good sign that the status quo is good.
I'll make sure to mention anything that is found and that cannot be resolved.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just stdenv per se is not much; there needs to be something cross-platform, and maybe ideally at least one three-platform test? Not sure what are the other corner cases, maybe someone will remember some more.


- Currently oilshell has a bus factor of 1. While Andrew has been very motivated and responsive thus far, this could change in the future.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC @zimbatm suggested to try out oil as language for the stdenv rather than bash and I actually like the idea. However I think that this is the biggest risk we have.

Shifts of interest or (unfortunately also) burnout do happen in open source. The worst case is that the "heart" of the nixpkgs project depends on an unmaintained, effectively dead language at some point in the future.

That said, I think that we should discuss how we can minimize this risk, e.g. by finding people who are willing to help out / can (co-)maintain this project or by sponsoring this project via the NixOS foundation (these are just two ideas from the top of my head, perhaps someone else has better ideas :))

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One thing to mitigate this could be that we could have a "long" period of keeping compatibility with both language and only make the switch when more people have been involved with oil.

After the flag feature was proposed, I think enabling package maintainers early to choose another shell for evaluation of their package could be an interesting experiment. As oil's user base increases, contributors should increase as well. We could make it a condition to switch stdenv only when a "sufficent" numbers of contributors have appeared.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An argument could be made for being worth the switch even if it gets stuck in some state — it's not like we eagerly adopt new features of Bash immediately. But then indeed there should be evaluation from this point of view (maybe including some estimation of the burden of maintaining buildability)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that this is the biggest concern. Sure, if Oil disappears we won't be "screwed" overnight but over time bugs will be found and security issues will need to be fixed. Oil's dependencies will need to be updated, including its build tools. Sure, in theory we can keep whatever old deps Oil needs forever but that is a significant maintenance burden. I don't want Oil to be forcing us to keep python 3 around when the world is on python 5.

There are alternatives to consider. ZSH avoids some of the biggest issues by not requiring variables to be quoted, but it isn't as nice as Oil. I think it is a clear improvement from bash but is it worth the migration? Oil seems clearly worth the migration given the current state of the project but again, I don't want to bet on a 1-dev project and need to migrate back to bash (or zsh) one day. I also don't think that long-term multi-shell support is a great option. I think if we are going to change the shell we need to expect to complete the migration without any significant "waiting" period. I'm not saying that it needs to be done overnight, or that we can't change our mind and rollback, but I am against the long-term multi shell state due to the confusion and maintenance burden it causes.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

security issues

Well, we are not talking CGI or user shell, intentional attacks on stdenv are a pretty niche consideration.

to keep python 3

Current track record of PyPy suggests that if there is nothing CPython-only, the risk is limited.

But of course even just a big mass of C++ having lost maintenance can become an annoying liability.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Completely correct. We just have to weigh the potential cost of carrying this project vs the savings of having a better shell that helps make nixpkgs more reliable and productive. In my mind it is a very tough call, but I think I am leaning in favour.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's unclear to me whether a better backwards compatible shell replacement exists - reading the oil shell blog reveals a ton of time and research sunk into the POSIX spec and the behavior of common shells (not always POSIX compliant!) to optimize for backwards compatibility above all else.

I don't see a better alternative shell for Nix.

That said, picking up Oil with a single developer and before a 1.0 release is a significant risk, so I agree that we should somehow ensure that development is funded and actively contributed to. Is the current developer aware of Nix and our interest in the project?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the alias andychu that commented on this RFC is the person behind oil.

There are also some nix users who have contributed to oil https://github.com/abathur is one of them.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this objection is reasonable. However, at the same time, GNU Bash has very few maintainers (I cannot say if there are more than one due to the unclear/opaque dev process and git commits). The main advantage with Bash in this respect is that so many people rely on it that it is likely some maintenance will happen indefinitely regardless of who does it.

I don't think this is necessarily a major problem.

- Oil shell is pre 1.0 so the syntax might change.
- Parts of oil are rewritten in c++, while this could be good for performance, new user might discover bugs that are really hard to debug.
- Oil has had little usage so far, there might be bugs that will just be discovered by having a large user base.
- Using osh, could be straightforward, but using oil might take more work as basically the existing bash code need to rewritten with `strict_errexit`.
- The oil documentation is lacking at the moment.
- Oil has really lofty goals, which I think it's good, but some potential solutions are not documented or not implemented. One example is eggex which are meant to be a replacement for reggexes.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How is the platform support for Oil? Does it work on all platforms that are supported to some extent in Nixpkgs?

Copy link
Author

@happysalada happysalada Aug 12, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Linux and darwin are both supported.

  • There seems to be no problem so far with the different architectures
  • Darwin has some heisenbugs related to the python implementation it seems.

## Alternatives
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This appears to be proposing a "flag-day" kind of switch to some degree. While it is expected that there is a period of oil+bash compatibility my understanding is that there is no true guarantee of this and at the end of the day once we think everything is compatible the shell will be flipped.

I wonder if we could somewhat ease this by adding a shell flag to stdenv which can be set to bash or oil. It would start defaulting to bash and we can slowly start moving some packages to oil. Once we have a substantial number of packages moved that we are fairly confident that the stdenv machinery and common hooks work we can flip the default. However for packages that fail on oil we can temporarily add the flag to switch back to bash.

This way we can get better, continuous, testing before the switch. Then we have a route to deal with the last remaining bugs without continuously flipping the shell between oil and bash for all packages.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really like the idea of the flag. I would even want to have it for specific packages for people to use their favorite shell. The only thing would be that if people can start using their own shell for a package and they use specific features of that shell, then we will never be able to remove that flag again (I don't necessarily think it's a bad idea). Perhaps you were thinking of a flag that isn't changeable by individual developpers ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would hesitate about "for specific packages for people to use their favorite shell". This puts long-term maintenance burden on stdenv and hooks to work with as many shells as possible, it also means that we can't take advantage of the convenience and safety features of oil. I don't think that is worth it. I see it purely as a migration tool between bash and osh. It could be set per-package so that the migration looks something like this.

  1. Set shell = "oil" for a small number of packages after updating them and stdenv to work.
  2. Expand this list of packages until we believe that just about everything in stdenv and popular hooks are working on oil.
  3. Flip the default shell and add shell = "bash" to any packages that this breaks.
  4. Burn down the list of shell = "bash" packages to zero.
  5. Remove the shell flag from stdenv. ("inline" oil as the default shell)
  6. We can now take advantage of oil features!

The ensures that each step is small and safe and enforces forward progress through the migration (accidental regressions on either shell can't happen as some builds will fail).

Again, I recommend this only as a migration tool. I think the long-term maintenance of multiple shells is likely more effort than it is worth.


[alternatives]: #alternatives

- I'm not aware of any other alternative shell that could enable a smooth transition with bash. [Here](https://github.com/oilshell/oil/wiki/Alternative-Shells) is a list of alternative shells.
Copy link
Member

@sternenseemann sternenseemann Aug 14, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there are two alternatives to this which should at least be seriously considered:

More provocative maybe than realistic, but POSIX shell could also be an alternative. It would also allow for a transition period where we could keep using bash and make our scripts POSIX compatible (granted, the changes necessary would be far greater than for osh). The two reasons for this would be:

  • Nix's env setup (without structuredAttrs) can't make use of any non-standard bash features like arrays. This leads to surprising behavior where setting *FlagsArray attributes which setup.sh understands will not work as expected.
  • Portability: There are far more POSIX shell implementation or POSIX shell compatible shells than bash compatible ones.

While I don't think me or anyone here actually wants to switch to POSIX shell (it's just not worth the effort as there's little to gain), I do think that answering this question could help us and the RFC to clarify what we want from stdenv.shell.

The second alternative would be to rely less on the shell in the future. The bulk of shell scripts subject to this RFC or discussion is not the build scripts which are quite simple execute-commands-in-sequence affairs, but the setup hooks and the many wrappers (binutils, cc, pkg-config, ...) contained in nixpkgs. setup hooks have always been considered an implementation detail of nixpkgs and consequently, we could consider replacing these shell scripts with an alternative approach, e. g. a mechanism implemented in pure Nix or using a DSL for manipulating the environment (which is the main thing the wrappers / setup hooks are doing after all).

Should we want to take such a direction in the future, it may not be considered worth it to invest time into Oil support in the short / medium term.


Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think you need a Bash-compatible programming language (not necessarily shell language) to obtain compatibility with the existing code, e.g., what is a function in stdenv could likely be replaced by an external program.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think having some kind of bash compatibility would be critical, as it would allow us to incrementally port the existing code. Otherwise we would be forced to rewrite everything at once which is a lot of work and would probably lead to a situation where it is harder to catch and debug regressions.

Also I don't think current shell functions could be replaced with external programs as we rely on bash-features like arrays which generally don't translate well to anything non-bash.

## Unresolved questions
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like to see some sort of performance estimate. I suspect it will be trivial but it would be nice to do a sanity check with some number of packages just to validate.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think some kind of performance eval is in order. As noted in https://matthewbauer.us/blog/avoid-subshells.html, this can really add up in the stdenv.


[unresolved]: #unresolved-questions

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One aspect is that it complicates the system bootstrap as it is now part of the low-level dependencies needed for the stdenv. It would be good to get an idea of how much that represents, and how hard it would be to port to various systems.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean in terms of size?
In the following PR, I've made the initial additions to stdenv in terms of allowedRequisites
NixOS/nixpkgs#131676
I haven't checked the size though. (this PR will fail until the local -i changes are merged to staging, so I don't know how much information it will give you.)

In terms of various systems, do you mean linux and darwin ? Or different architectures ? (the PR includes what I know for linux and darwin).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Size in terms of number of additional dependencies.

nixpkgs only officially supports 4(?) (kernel, cpu) combinations but there are efforts out there to port it to other types of systems. Each new tool added to the stdenv closure can introduce more friction for these porting efforts. Even though those systems are not fully supported, we usually try to keep them in mind for high-impact dependencies like the stdenv.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only 4? Reasonable way to get native stdenv should be a concern where we pay attention even to Tier 4, I think — and at that level we have quite a few more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From #99 (comment) I gather that currently python is required for Oil, this means that as it stands we'd need to redo all bootstrap tarballs to also include Python. OTOH this could also serve as a good indicator of the earliest viable point to switch to Oil: As soon as oil-native is ready, it would be way easier to add it to the bootstrap. Up to that point we could provide an oil-based stdenv, but wouldn't be able to bootstrap nixpkgs using oil.

- What are the pittfalls and bugs in oilshell?
- How much work exactly is required to make the switch?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another aspect is speed. What time impact on a mass rebuild would the Oil adoption cause?

Ideally that's something that can be answered before this RFC gets accepted.

## Future work

[future]: #future-work

- Remove the `-i` flag uses in stdenv. An example can be found in [PR](https://github.com/NixOS/nixpkgs/pull/130597)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The whole space where bash gets massaged to be compatible with Oil is probably fine to do outside of RFC scope. I don't think it would be too controversial.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with you, I think it can even be considered a benefit (having less potential bug in stdenv).
Only the final PR of either adding a flag to enable conditional switching or switching altogether would have to go through the RFC.
I wanted to give people an idea of what work needs to be done. The idea is that it's not so much work and it's valid in both bash and osh. But perhaps it wasn't very clear from the sentence. Let me know if you think I can remove this.