Install WSL and MSYS2 on Windows Images #342
Conversation
AndreyMaslennikov
requested review from
maxim-lobanov,
alepauly,
ivan-nosar,
al-cheb and
miketimofeev
| Import-Module -Name ImageHelpers | ||
|
|
||
| $msys2Path = "C:\tools\msys64" | ||
| choco install msys2 -y --params "/InstallDir:$msys2Path" |
There was a problem hiding this comment.
I'm rather sure that the mingw toolchains can be installed through msys2 so ideally we do that here and remove the separate installation of mingw. Also, I wonder if it's possible to tell GitForWindows to just use the installed msys2/mingw rather than bring in it's own. Any thoughts?
There was a problem hiding this comment.
Also, how does this affect existing builds that depend on the currently installed mingw?
There was a problem hiding this comment.
another comment. if avoiding chocolatey is not substantially difficult, I think we should avoid it since waiting for choco packages adds another layer of delay and potential problems that we can't control. This comment by @eine suggests it should be easy enough to just untar and execute setup.
There was a problem hiding this comment.
Re using the tar, I created an Install-MSYS2.ps1 script mentioned in Issue #30 comment. Also installs typical MSYS2 shell tools and basic 32 & 64 bit compilers.
Having used MSYS2 for a while, the main tar is not revised often, as the current release is dated 2019-May-24. Hence, the more important code is the update code, which hopefully can be run any time images are updated...
There was a problem hiding this comment.
Hence, the more important code is the update code, which hopefully can be run any time images are updated...
I beg to differ. Updating by default prevents users from testing their tools on a stable environment.
There was a problem hiding this comment.
| { | ||
| "type": "powershell", | ||
| "scripts":[ | ||
| "{{ template_dir }}/scripts/Installers/Install-WSL.ps1" |
There was a problem hiding this comment.
I don't think we should make these changes to the Windows2016 - I don't see any requests asking for them and it just adds one more element to destabilize an older setup. @thejoebourneidentity thoughts?
There was a problem hiding this comment.
As I can see there is a request for the both Windows2016 and 2019 images in the issue #50.
We also planned to use bash from the WSL as default . Wouldn't it be confused if we'll use different bash.exe for these images?
| Import-Module -Name ImageHelpers | ||
|
|
||
| $msys2Path = "C:\tools\msys64" | ||
| choco install msys2 -y --params "/InstallDir:$msys2Path" |
There was a problem hiding this comment.
another comment. if avoiding chocolatey is not substantially difficult, I think we should avoid it since waiting for choco packages adds another layer of delay and potential problems that we can't control. This comment by @eine suggests it should be easy enough to just untar and execute setup.
|
As commented by @alepauly, I believe that it'd be desirable to get the tarball from MSYS2's site (as done in setup-msys2), instead of using chocolatey. Not only it avoids an unnecessary layer of delay/complexity, but it is also faster.
Actually, if MSYS2 is installed, GitForWindows should not be required at all. MSYS2 provides three flavours: MSYS, MINGW64 or MINGW32. Hence, starting an MINGW64 shell from MSYS2 provides the same features as GitForWindows, plus Precisely, issue #30 (which should probably be closed by this PR) is about replacing the existing Furthermore, I think that the following points should also be considered:
|
|
This looks great! I don't mind the choice of Debian as long as I can install Ubuntu instead, but perhaps we could also install both at the same time? Would that be too wasteful? Ubuntu is much more popular than Debian in WSL so I think many people would be switching to it anyway, so it makes sense to have it out of the box. Thoughts? |
|
Above, I mentioned a ps1 script written to install MSYS2. Please see the following, which is a link to MSYS2's script that they use for CI on Azure Pipelines: https://github.com/msys2/MINGW-packages/blob/master/azure-pipelines.yml Also note that they have a repo with a base install of MSYS2, which is used in their script: |
No. From my perspective, a stable environment is the equivalent of a fresh MSYS2 install, since pacman does not normally allow older packages to be installed. If you want to download and install older packages, have at it. Most users will not want that. Given your statement, what then determines when MSYS2 (or anything for that matter) gets updated? |
That's exactly what I said. The tar should be uncompressed and any dummy command should be executed (say Conversely, you said that "the more important code is the update code, which hopefully can be run any time images are updated", which I understand as updating MSYS (executing
|
|
Ever used MSYS? One of the main changes in MSYS2 was the use of pacman.
Since pacman is able to update all of one's MSYS2 install, releases are not done as frequently as other software. Since you mentioned them, why does all their CI for packages run with a fully updated version? Why do all their docs mention performing the equivalent of Actions should try to update/refresh MSYS2 anytime a new image is released. People will prefer it, and the extra resources (time, downloads) needed to update it wouldn't be needed. |
These are two facts. What do you mean by stating them?
I believe that's an standard procedure to detect regression bugs in the development branch of a tool, innit? I.e. this question/discussion would better fit in their repo/forum, rather than here.
I'm afraid that's not true. It is not the first time you word your very personal point of view as if it was some objetive fact. I'd like to please ask you to do an effort to make it clear what it is, and what you'd like to be. On the one hand, I didn't find On the other hand, they suggest it to make it immediately after install, as long as you want to update the packages. It makes sense to suggest using updating commands in order to have packages updated. I acknowledge that the frontpage (https://www.msys2.org/) might be misleading. But I believe that the docs are clear enough:
I think you forgot to add "some" before "people". Having it updated by default involves a maintenance burden, due to the references above, and it will break multiple use cases. Those use cases are maintained by people, which will be forced to keep installing a separate "clean" MSYS2. Once again, I feel that you are ok with having your particular preferences placed, but seem to show little regard to other users' needs. Honestly, I suggest you to get involved in the development of setup-msys2. Many of your ideas are useful for many users indeed, and I'd be really happy to have those supported by a single action, instead of fragmenting the ecosystem. Nevertheless, I believe that most of them need to be optional; thus, should not be applied to this PR. Please, note that, regarding setup time (which you seem to be more concerned about), setup-msys2 can be combined with a caching approach, so that you have your custom MSYS2 install ready-to-use. |
|
It happened a while ago, so I can't remember all the details, but sometimes package updates cannot be applied piecemeal. There was an issue with the AppVeyor MSYS2 install, and I needed one package updated. Jumped thru all sorts of hoops trying to figure out how to do it, and couldn't force it with script. I was writing code and running it locally to show dependency chains, etc. The main reason for trying to do so was that the MSYS2 install at the time was outdated, and, given all the packages that were installed, running I asked about it in MSYS2, and the response was equivalent to "what you're trying to do should work, but there will be some rare package updates that can only be performed by Anyway, maybe we'll just agree that we disagree.
For now, I'm just waiting for an independent, normal install in the image(s). I have an action (https://github.com/MSP-Greg/msys2-action), which uses the embedded MSYS2, and is directed towards gcc use. It's also somewhat Ruby focused, as older Ruby versions (which I encourage people not to use) are compiled with OpenSSL 1.0.2. There's logic to detect the Ruby version and switch OpenSSL packages accordingly. Anyway, I'll have a look at |
I mostly agree, and I'd expect any average user to update MSYS2 periodically. However, this is not always true. Some users (which are not familiar with the environment) will just install it and use it straightaway. Moreover, in some "computer rooms" installations are preset, reset after each restart and do not provide admin permissions to the user. These are the use cases that (as maintainer of PKGBUILD recipes for MSYS2), I'd like to be able to test. That allows me to know whether it should work, or if users need to talk to their admins.
Just for the sake of completeness, should you remember or find a link, please do add it here or in #30. I believe it might be useful for someone in the future.
I think that situations such as this one you describe should be handled by requesting/requiring MSYS2 maintainers to release a new package. As you said, MSYS2 tarballs are updated once or twice a year. Overall, I'm ok with that. Nevertheless, if the latest is so outdated that 10min are required, I believe that's a fair point to nudge them. There should not be 10+ breaking changes to the core in a year anyway. Moreover, as you pointed, I believe they don't want thousands of potential CI jobs hitting their servers so hard.
I understand that Ruby-specific details might not fit in setup-msys2, so it would still make sense for you to keep your own action. Nonetheless, features such as having gcc and other build-essentials set up can be useful for many users. Hence, setup-msys2 might support some "preset" groups which are optionally cached separatedly. Of course, this is just me thinking loud. Regardless of our discussion, GitHub employees/maintainers might take any approach. |
Sorry, the main reason that But, the fact that they have all of those packages installed means something. As stated elsewhere, I feel that Actions should install standard compiler tools and leave individual packages up to the people using them.
What packages do you maintain? Just wondering what you normally work with...
It's not really Ruby specific, more about allowing installation of OpenSSL 1.0.2 when needed. I just provided an easy 'translation' for it into Ruby terms... |
GHDL, a FOSS analyser, compiler, simulator and synthesiser for VHDL: https://github.com/ghdl/ghdl/tree/master/dist/msys2-mingw It supports three different backends, so up to 6 different PKGBUILD files might exist. Currently, only two of them are maintained. Moreover, there are also variants to test 'stable' releases: https://github.com/eine/ghdl-packaging/tree/master/msys2-mingw. Hence, the full matrix might have up to 18 items. |
|
See draft PR #355 for a ps1 script installing MSYS2 via download from repo.msys2.org. |
|
@AndreyMaslennikov, any updates on this? |
|
@eine , we have reworked our approach a bit and still working on implementation / validation. |
|
Awesome work on MSYS2 and so glad to see that merged in. The reworked pull requests seem to no longer include WSL support though. Any chance that part can come back? Help needed on that? Seems fairly trivial to enable and would make me (and likely others) insanely happy. If there are concerns / risks around enabling it please let us know. Having WSL enabled seems to also be the only legitimate path for Linux containers on Docker for Windows now that LCOW seems deprecated. |
In this PR we enabled WSL Windows feature, requested in #50, and installed Debian Image to use it in the WSL.
We also installed MSYS2 tool