Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Assorted fixes/enhancements for the git-artifacts workflow #3053

Merged
merged 12 commits into from
Mar 5, 2021

Conversation

dscho
Copy link
Member

@dscho dscho commented Feb 24, 2021

The git-artifacts workflow can be used to generate Git for Windows installers, MinGits, etc.

The improvements in this PR mostly revolve around two topics: actually make the ARM64 stuff work (it didn't work when I tried it), and accelerate the entire thing, mostly by using the shiny new setup-git-for-windows-sdk GitHub Action.

@dscho dscho force-pushed the assorted-git-artifacts-fixes branch 2 times, most recently from 37e0f7e to bfb078c Compare February 24, 2021 22:06
@dscho
Copy link
Member Author

dscho commented Feb 24, 2021

Oh, sorry @dennisameling, I forgot to cc: you here... This should be interesting.

FWIW I saw something rather concerning: the ARM64 artifacts are super large! Likewise, nothing about mingit-arm64 is "minimal". The reason, most likely, is that the dashed built-ins (such as git-add.exe, which is usually a hard-linked version of git.exe) are copies. That'll bloat any Git for Windows.

I did start to investigate how to port 179227d to CMakeLists.txt, see #3056.

@dscho
Copy link
Member Author

dscho commented Feb 25, 2021

I saw something rather concerning: the ARM64 artifacts are super large!

@dennisameling to see what I mean, please compare the artifact sizes of https://github.com/dscho/git/actions/runs/596316620 (whose mingit-arm64 weighs 205MB!) to https://github.com/dscho/git/actions/runs/597692856 (mingit-arm64 weighs only 38.6MB, which however is still a lot larger than mingit-i686 that weighs 22.2MB).

@rimrul
Copy link
Member

rimrul commented Feb 25, 2021

mingit-arm64 weighs only 38.6MB, which however is still a lot larger than mingit-i686 that weighs 22.2MB

Isn't a size increase in roughly that ballpark region to be expected due to the current nature of our arm releases?

Or do we already exclude the x86 versions of the executables we compile natively?

@dscho
Copy link
Member Author

dscho commented Feb 25, 2021

mingit-arm64 weighs only 38.6MB, which however is still a lot larger than mingit-i686 that weighs 22.2MB

Isn't a size increase in roughly that ballpark region to be expected due to the current nature of our arm releases?

It's slightly larger than necessary because we don't exclude ARM64 files from MinGit as we do with i686 files. At least that's what I think is the issue.

Or do we already exclude the x86 versions of the executables we compile natively?

No, we specifically ship all of he i686 binaries in the ARM64 flavor.

@dennisameling
Copy link

FWIW I saw something rather concerning: the ARM64 artifacts are super large!

Yes, I mentioned that in #3017 (comment), but didn't know how to fix it back then. Definitely needed someone who's more familiar with the Git for Windows codebase. So, thank you for looking into this! Highly appreciated 😊

This seems to work correctly, but I'm currently stuck on #3071 (comment) so can't really test this further. I did see some msvcrt.dll-related crashes in cmd\git.exe in my Windows event logs, but not sure if that's related to this PR of the issue mentioned above.

dscho added 12 commits March 5, 2021 13:14
This is needed to allow for the artifact checksum to be computed.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Let's use a more verbose, slightly more readable conditional.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Let's skip this step if it isn't needed.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
... just removing two superfluous empty lines...

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Let's adjust the indentation to conform with the coding style of the
remainder of the file.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Since the directory can contain spaces, we need to quote it, but we
cannot do that directly (but have to store the option in a variable),
therefore we need to `eval` the command-line.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
GitHub workflows run in agents that have quite a bit of stuff in their
`PATH`, e.g. Chocolatey. To make sure that those bits and pieces are
_not_ used to build the artifacts, let's whittle down the `PATH` to
contain the bare minimum.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The workflow allows users to restrict what parts are being built. For
example, `installer-i686` will build only the 32-bit installer, not the
64-bit one nor any MinGit flavor.

However, this logic was not extended when introducing support for ARM64:
Instead, we _also_ built the ARM64 installer when the user asked for
`installer-i686`.

Let's allow restricting to `installer-i686` _without_ building the ARM64
version, and allow restricting to `installer-arm64` _just_ for the ARM64
version.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When the user asked for `installer-x86_64`, there is no point in
building `pkg-i686` or `build-arm64`; Let's be a bit smarter about this.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This workflow needs to be triggered manually, and it offers to specify a
couple input parameters. But none of them are required. Make that
explicit.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
…vision

We cannot just check out the current revision: The user might have
overridden `REPOSITORY` and `REF` via the workflow dispatch.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This simplifies the workflow dramatically.

Note that we have to reinstate that `/usr/bin/git` hack (a shell script
that simply redirects to `/mingw64/bin/git.exe`) in the `pkg` job
manually, since we no longer cache the `build-installers` artifact
_after_ installing that hack in `bundle-artifacts`.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
@dscho dscho force-pushed the assorted-git-artifacts-fixes branch from bfb078c to 2b11260 Compare March 5, 2021 12:23
dscho added a commit to dscho/git that referenced this pull request Mar 5, 2021
This corresponds to git-for-windows#3053.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
@dscho dscho merged commit d7cb837 into git-for-windows:main Mar 5, 2021
@dscho dscho deleted the assorted-git-artifacts-fixes branch March 5, 2021 21:24
@dscho dscho added this to the Next release milestone Mar 5, 2021
dscho added a commit to dscho/git that referenced this pull request Mar 5, 2021
This corresponds to git-for-windows#3053.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
dscho added a commit that referenced this pull request Mar 6, 2021
Assorted fixes/enhancements for the `git-artifacts` workflow
dscho added a commit that referenced this pull request Mar 6, 2021
Assorted fixes/enhancements for the `git-artifacts` workflow
dscho added a commit that referenced this pull request Mar 6, 2021
Assorted fixes/enhancements for the `git-artifacts` workflow
git-for-windows-ci pushed a commit that referenced this pull request Aug 2, 2021
Assorted fixes/enhancements for the `git-artifacts` workflow
dscho added a commit to dscho/git that referenced this pull request Aug 3, 2021
…acts-fixes

Assorted fixes/enhancements for the `git-artifacts` workflow
git-for-windows-ci pushed a commit that referenced this pull request Aug 3, 2021
Assorted fixes/enhancements for the `git-artifacts` workflow
dscho added a commit that referenced this pull request Aug 3, 2021
Assorted fixes/enhancements for the `git-artifacts` workflow
dscho added a commit that referenced this pull request Aug 3, 2021
Assorted fixes/enhancements for the `git-artifacts` workflow
dscho added a commit that referenced this pull request Aug 3, 2021
Assorted fixes/enhancements for the `git-artifacts` workflow
git-for-windows-ci pushed a commit that referenced this pull request Aug 4, 2021
Assorted fixes/enhancements for the `git-artifacts` workflow
git-for-windows-ci pushed a commit that referenced this pull request Aug 4, 2021
Assorted fixes/enhancements for the `git-artifacts` workflow
dscho added a commit that referenced this pull request Aug 4, 2021
Assorted fixes/enhancements for the `git-artifacts` workflow
dscho added a commit that referenced this pull request Aug 4, 2021
Assorted fixes/enhancements for the `git-artifacts` workflow
git-for-windows-ci pushed a commit that referenced this pull request Aug 4, 2021
Assorted fixes/enhancements for the `git-artifacts` workflow
git-for-windows-ci pushed a commit that referenced this pull request Aug 4, 2021
Assorted fixes/enhancements for the `git-artifacts` workflow
dscho added a commit that referenced this pull request Aug 6, 2021
Assorted fixes/enhancements for the `git-artifacts` workflow
git-for-windows-ci pushed a commit that referenced this pull request Aug 6, 2021
Assorted fixes/enhancements for the `git-artifacts` workflow
git-for-windows-ci pushed a commit that referenced this pull request Aug 6, 2021
Assorted fixes/enhancements for the `git-artifacts` workflow
dscho added a commit that referenced this pull request Aug 11, 2021
Assorted fixes/enhancements for the `git-artifacts` workflow
dscho added a commit that referenced this pull request Aug 13, 2021
Assorted fixes/enhancements for the `git-artifacts` workflow
dscho added a commit that referenced this pull request Aug 18, 2021
Assorted fixes/enhancements for the `git-artifacts` workflow
dscho added a commit that referenced this pull request Aug 23, 2021
Assorted fixes/enhancements for the `git-artifacts` workflow
git-for-windows-ci pushed a commit that referenced this pull request Aug 24, 2021
Assorted fixes/enhancements for the `git-artifacts` workflow
dscho added a commit that referenced this pull request Aug 31, 2021
Assorted fixes/enhancements for the `git-artifacts` workflow
git-for-windows-ci pushed a commit that referenced this pull request Sep 8, 2021
Assorted fixes/enhancements for the `git-artifacts` workflow
dscho added a commit that referenced this pull request Sep 13, 2021
Assorted fixes/enhancements for the `git-artifacts` workflow
dscho added a commit that referenced this pull request Oct 4, 2021
Assorted fixes/enhancements for the `git-artifacts` workflow
git-for-windows-ci pushed a commit that referenced this pull request Oct 12, 2021
Assorted fixes/enhancements for the `git-artifacts` workflow
git-for-windows-ci pushed a commit that referenced this pull request Oct 12, 2021
Assorted fixes/enhancements for the `git-artifacts` workflow
dscho added a commit that referenced this pull request Oct 13, 2021
Assorted fixes/enhancements for the `git-artifacts` workflow
git-for-windows-ci pushed a commit that referenced this pull request Oct 13, 2021
Assorted fixes/enhancements for the `git-artifacts` workflow
git-for-windows-ci pushed a commit that referenced this pull request Oct 13, 2021
Assorted fixes/enhancements for the `git-artifacts` workflow
git-for-windows-ci pushed a commit that referenced this pull request Oct 13, 2021
Assorted fixes/enhancements for the `git-artifacts` workflow
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants