-
Notifications
You must be signed in to change notification settings - Fork 133
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Offer to run CI/PR builds in Azure Pipelines #31
Commits on Jan 27, 2019
-
travis: fix skipping tagged releases
When building a PR, TRAVIS_BRANCH refers to the *target branch*. Therefore, if a PR targets `master`, and `master` happened to be tagged, we skipped the build by mistake. Fix this by using TRAVIS_PULL_REQUEST_BRANCH (i.e. the *source branch*) when available, falling back to TRAVIS_BRANCH (i.e. for CI builds, also known as "push builds"). Let's give it a new variable name, too: CI_BRANCH (as it is different from TRAVIS_BRANCH). This also prepares for the upcoming patches which will make our ci/* code a bit more independent from Travis and open it to other CI systems (in particular to Azure Pipelines). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for f553fd4 - Browse repository at this point
Copy the full SHA f553fd4View commit details -
ci: rename the library of common functions
The name is hard-coded to reflect that we use Travis CI for continuous testing. In the next commits, we will extend this to be able use Azure DevOps, too. So let's adjust the name to make it more generic. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for f0852de - Browse repository at this point
Copy the full SHA f0852deView commit details -
ci/lib.sh: encapsulate Travis-specific things
The upcoming patches will allow building git.git via Azure Pipelines (i.e. Azure DevOps' Continuous Integration), where variable names and URLs look a bit different than in Travis CI. Also, the configurations of the available agents are different. For example, Travis' and Azure Pipelines' macOS agents are set up differently, so that on Travis, we have to install the git-lfs and gettext Homebrew packages, and on Azure Pipelines we do not need to. Likewise, Azure Pipelines' Ubuntu agents already have asciidoctor installed. Finally, on Azure Pipelines the natural way is not to base64-encode tar files of the trash directories of failed tests, but to publish build artifacts instead. Therefore, that code to log those base64-encoded tar files is guarded to be Travis-specific. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 7c16d31 - Browse repository at this point
Copy the full SHA 7c16d31View commit details -
ci: inherit --jobs via MAKEFLAGS in run-build-and-tests
Let's not decide in the generic ci/ part how many jobs to run in parallel; different CI configurations would favor a different number of parallel jobs, and it is easy enough to hand that information down via the `MAKEFLAGS` variable. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for bf72fb1 - Browse repository at this point
Copy the full SHA bf72fb1View commit details -
ci: use a junction on Windows instead of a symlink
Symbolic links are still not quite as easy to use on Windows as on Linux (for example, on versions older than Windows 10, only administrators can create symlinks, and on Windows 10 you still need to be in developer mode for regular users to have permission), but NTFS junctions can give us a way out. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 681f8e6 - Browse repository at this point
Copy the full SHA 681f8e6View commit details -
test-date: add a subcommand to measure times in shell scripts
In the next commit, we want to teach Git's test suite to optionally output test results in JUnit-style .xml files. These files contain information about the time spent. So we need a way to measure time. While we could use `date +%s` for that, this will give us only seconds, i.e. very coarse-grained timings. GNU `date` supports `date +%s.%N` (i.e. nanosecond-precision output), but there is no equivalent in BSD `date` (read: on macOS, we would not be able to obtain precise timings). So let's introduce `test-tool date getnanos`, with an optional start time, that outputs preciser values. Note that this might not actually give us nanosecond precision on some platforms, but it will give us as precise information as possible, without the portability issues of shell commands. Granted, it is a bit pointless to try measuring times accurately in shell scripts, certainly to nanosecond precision. But it is better than second-granularity. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for ccf8bf5 - Browse repository at this point
Copy the full SHA ccf8bf5View commit details
Commits on Jan 28, 2019
-
tests: optionally write results as JUnit-style .xml
This will come in handy when publishing the results of Git's test suite during an automated Azure DevOps run. Note: we need to make extra sure that invalid UTF-8 encoding is turned into valid UTF-8 (using the Replacement Character, \uFFFD) because t9902's trace contains such invalid byte sequences, and the task in the Azure Pipeline that uploads the test results would refuse to do anything if it was asked to parse an .xml file with invalid UTF-8 in it. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for e9a869d - Browse repository at this point
Copy the full SHA e9a869dView commit details -
ci/lib.sh: add support for Azure Pipelines
This patch introduces a conditional arm that defines some environment variables and a function that displays the URL given the job id (to identify previous runs for known-good trees). Because Azure Pipeline's macOS agents already have git-lfs and gettext installed, we can leave `BREW_INSTALL_PACKAGES` empty (unlike in Travis' case). Note: this patch does not introduce an Azure Pipelines definition yet; That is left for the next patch. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 4c78085 - Browse repository at this point
Copy the full SHA 4c78085View commit details -
Add a build definition for Azure DevOps
This commit adds an azure-pipelines.yml file which is Azure DevOps' equivalent to Travis CI's .travis.yml. The main idea is to replicate the Travis configuration as faithfully as possible, to make it easy to compare the Azure Pipeline builds to the Travis ones (spoiler: some parts, especially the macOS jobs, are way faster in Azure Pileines). Meaning: the number and the order of the jobs added in this commit faithfully replicates what we have in .travis.yml. Note: Our .travis.yml configuration has a Windows part that is *not* replicated in the Azure Pipelines definition. The reason is easy to see: As Travis cannot support our Windws needs (even with the preliminary Windows support that was recently added to Travis after waiting for *years* for that feature, our test suite would simply hit Travis' timeout every single time). To make things a bit easier to understand, we refrain from using the `matrix` feature here because (while it is powerful) it can be a bit confusing to users who are not familiar with CI setups. Therefore, we use a separate phase even for similar configurations (such as GCC vs Clang on Linux, GCC vs Clang on macOS). Also, we make use of the shiny new feature we just introduced where the test suite can output JUnit-style .xml files. This information is made available in a nice UI that allows the viewer to filter by phase and/or test number, and to see trends such as: number of (failing) tests, time spent running the test suite, etc. (While this seemingly contradicts the intention to replicate the Travis configuration as faithfully as possible, it is just too nice to show off that capability here already.) Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for d34812b - Browse repository at this point
Copy the full SHA d34812bView commit details -
ci: add a Windows job to the Azure Pipelines definition
Previously, we did not have robust support for Windows in our CI definition, simply because Travis cannot accommodate our needs (even after Travis added experimental Windows support very recently, it takes longer than Travis' 50 minute timeout to build Git and run the test suite on Windows). Instead, we used a hack that started a dedicated Azure Pipeline from Travis and waited for the output, often timing out (which is quite fragile, as we found out). With this commit, we finally have first-class support for Windows in our CI definition (in the Azure Pipelines one, that is). Due to our reliance on Unix shell scripting in the test suite, combined with the challenges on executing such scripts on Windows, the Windows job currently takes a whopping ~1h20m to complete. Which is *far* longer than the next-longest job takes (linux-gcc, ~35m). Now, Azure Pipelines's free tier for open source projects (such as Git) offers up to 10 concurrent jobs for free, meaning that the overall run time will be dominated by the slowest job(s). Therefore, it makes sense to start the Windows job first, to minimize the time the entire build takes from start to end (which is now pretty safely the run time of the Windows job). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for d565131 - Browse repository at this point
Copy the full SHA d565131View commit details -
ci: use git-sdk-64-minimal build artifact
Instead of a shallow fetch followed by a sparse checkout, we are better off by using a separate, dedicated Pipeline that bundles the SDK as a build artifact, and then consuming that build artifact here. In fact, since this artifact will be used a lot, we spent substantial time on figuring out a minimal subset of the Git for Windows SDK, just enough to build and test Git. The result is a size reduction from around 1GB (compressed) to around 55MB (compressed). This also comes with the change where we now call `usr\bin\bash.exe` directly, as `git-cmd.exe` is not included in the minimal SDK. That reduces the time to initialize Git for Windows' SDK from anywhere between 2m30s-7m to a little over 1m. Note: in theory, we could also use the DownloadBuildArtifacts@0 task here. However, restricted permissions that are in effect when building from forks would let this fail for PR builds, defeating the whole purpose of the Azure Pipelines support for git.git. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 1ab1d14 - Browse repository at this point
Copy the full SHA 1ab1d14View commit details -
mingw: be more generous when wrapping up the setitimer() emulation
Every once in a while, the Azure Pipeline fails with some semi-random error: timer thread did not terminate timely This error message means that the thread that is used to emulate the setitimer() function did not terminate within 1,000 milliseconds. The most likely explanation (and therefore the one we should assume to be true, according to Occam's Razor) is that the timeout of one second is simply not enough because we try to run so many tasks in parallel. So let's give it ten seconds instead of only one. That should be enough. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for c1ab8df - Browse repository at this point
Copy the full SHA c1ab8dfView commit details -
README: add a build badge (status of the Azure Pipelines build)
Just like so many other OSS projects, we now also have a build badge. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for b6316e1 - Browse repository at this point
Copy the full SHA b6316e1View commit details -
tests: avoid calling Perl just to determine file sizes
It is a bit ridiculous to spin up a full-blown Perl instance (especially on Windows, where that means spinning up a full POSIX emulation layer, AKA the MSYS2 runtime) just to tell how large a given file is. So let's just use the test-tool to do that job instead. This command will also be used over the next commits, to allow for cutting out individual test cases' verbose log from the file generated via --verbose-log. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 7a5caa2 - Browse repository at this point
Copy the full SHA 7a5caa2View commit details -
tests: include detailed trace logs with --write-junit-xml upon failure
The JUnit XML format lends itself to be presented in a powerful UI, where you can drill down to the information you are interested in very quickly. For test failures, this usually means that you want to see the detailed trace of the failing tests. With Travis CI, we passed the `--verbose-log` option to get those traces. However, that seems excessive, as we do not need/use the logs in almost all of those cases: only when a test fails do we have a way to include the trace. So let's do something different when using Azure DevOps: let's run all the tests with `--quiet` first, and only if a failure is encountered, try to trace the commands as they are executed. Of course, we cannot turn on `--verbose-log` after the fact. So let's just re-run the test with all the same options, adding `--verbose-log`. And then munging the output file into the JUnit XML on the fly. Note: there is an off chance that re-running the test in verbose mode "fixes" the failures (and this does happen from time to time!). That is a possibility we should be able to live with. Ideally, we would label this as "Passed upon rerun", and Azure Pipelines even know about that outcome, but it is not available when using the JUnit XML format for now: https://github.com/Microsoft/azure-pipelines-agent/blob/master/src/Agent.Worker/TestResults/JunitResultReader.cs Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 2593b9b - Browse repository at this point
Copy the full SHA 2593b9bView commit details -
mingw: try to work around issues with the test cleanup
It seems that every once in a while in the Git for Windows SDK, there are some transient file locking issues preventing the test clean up to delete the trash directory. Let's be gentle and try again five seconds later, and only error out if it still fails the second time. This change helps Windows, and does not hurt any other platform (normally, it is highly unlikely that said deletion fails, and if it does, normally it will fail again even 5 seconds later). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 991b41a - Browse repository at this point
Copy the full SHA 991b41aView commit details -
tests: add t/helper/ to the PATH with --with-dashes
We really need to be able to find the test helpers... Really. This change was forgotten when we moved the test helpers into t/helper/ Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 77896b2 - Browse repository at this point
Copy the full SHA 77896b2View commit details
Commits on Jan 29, 2019
-
t0061: workaround issues with --with-dashes and RUNTIME_PREFIX
When building Git with RUNTIME_PREFIX and starting a test helper from t/helper/, it fails to detect a system prefix. The reason is that the RUNTIME_PREFIX feature wants to use the location of the Git executable to determine where the support files can be found, e.g. system-wide Git config or the translations. This does not make any sense for the test helpers, though, as they are distinctly not in a directory structure resembling the final installation location of Git. That is the reason why the test helpers rely on environment variables to indicate the location of the needed support files, e.g. GIT_TEXTDOMAINDIR. If this information is missing, the output will contain warnings like this one: RUNTIME_PREFIX requested, but prefix computation failed. [...] In t0061, we did not expect that to happen, and it actually does not happen in the regular case, because bin-wrappers/test-tool specifically sets GIT_TEXTDOMAINDIR (and as a consequence, nothing in test-tool needs to know anything about any runtime prefix). However, with --with-dashes, bin-wrappers/test-tool is no longer called, but t/helper/test-tool is called directly instead. So let's just ignore the RUNTIME_PREFIX warning. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 4ec6cc8 - Browse repository at this point
Copy the full SHA 4ec6cc8View commit details -
tests: optionally skip bin-wrappers/
This speeds up the tests by a bit on Windows, where running Unix shell scripts (and spawning processes) is not exactly a cheap operation. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 248473d - Browse repository at this point
Copy the full SHA 248473dView commit details -
As Unix shell scripting comes at a hefty price on Windows, we have to see where we can save some time to run the test suite. Let's skip the chain linting and the bin-wrappers/ redirection on Windows; this seems to shave of anywhere between 10-30% from the overall runtime. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 3532811 - Browse repository at this point
Copy the full SHA 3532811View commit details -
ci: parallelize testing on Windows
The fact that Git's test suite is implemented in Unix shell script that is as portable as we can muster, combined with the fact that Unix shell scripting is foreign to Windows (and therefore has to be emulated), results in pretty abysmal speed of the test suite on that platform, for pretty much no other reason than that language choice. For comparison: while the Linux build & test is typically done within about 8 minutes, the Windows build & test typically lasts about 80 minutes in Azure Pipelines. To help with that, let's use the Azure Pipeline feature where you can parallelize jobs, make jobs depend on each other, and pass artifacts between them. The tests are distributed using the following heuristic: listing all test scripts ordered by size in descending order (as a cheap way to estimate the overall run time), every Nth script is run (where N is the total number of parallel jobs), starting at the index corresponding to the parallel job. This slicing is performed by a new function that is added to the `test-tool`. To optimize the overall runtime of the entire Pipeline, we need to move the Windows jobs to the beginning (otherwise there would be a very decent chance for the Pipeline to be run only the Windows build, while all the parallel Windows test jobs wait for this single one). We use Azure Pipelines Artifacts for both the minimal Git for Windows SDK as well as the built executables, as deduplication and caching close to the agents makes that really fast. For comparison: while downloading and unpacking the minimal Git for Windows SDK via PowerShell takes only one minute (down from anywhere between 2.5 to 7 when using a shallow clone), uploading it as Pipeline Artifact takes less than 30s and downloading and unpacking less than 20s (sometimes even as little as only twelve seconds). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 1572444 - Browse repository at this point
Copy the full SHA 1572444View commit details