Git on Windows is buggy on CI

[meteorcloudy]

Context:

@irengrig has a PR to refactor the git_repository and new_git_repository rules so that the rules call git binary directly instead of executing git in a bash script.

But the tests fail on CI with weird error:
https://buildkite.com/bazel/bazel-bazel-github-presubmit/builds/2869#a74532d6-3819-49da-8b12-480613a55cbb

error running 'C:/Program Files/Git/cmd/git.exe fetch origin tags/mytag:tags/mytag' while working with @withbuild:
      1 [main] sh 11000 C:\Program Files\Git\usr\bin\sh.exe: *** fatal error - add_item ("\??\C:\Program Files\Git", "/", ...) failed, errno 1
Stack trace:
Frame        Function    Args
000FFFF9BC0  0018006021E (001802572AA, 001802340B9, 00600010000, 000FFFF8B40)
000FFFF9BC0  00180048859 (000FFFFABF0, 000FFFF9BC0, 00000000000, 00000000000)
000FFFF9BC0  00180048892 (000FFFF9BC0, 00000000001, 00600010000, 505C3A435C3F3F5C)
000FFFF9BC0  001800D1C1B (00000000000, 00040000024, 00000000000, 00000000000)
1D5073F62DBCE65  0018011F5F5 (001800B8E3C, 00000000000, 00000000000, 00000000000)
000FFFFCCD0  00180048E53 (00000000000, 00000000000, 00000000000, 00000000000)
00000000000  00180047963 (00000000000, 00000000000, 00000000000, 00000000000)
000FFFFFFF0  00180047A14 (00000000000, 00000000000, 00000000000, 00000000000)

However, the test only fails on CI when running by the Buildkite agent. I tried to reproduce locally and logging into a CI machine, but in both situations, the test passed.

Following the error message, I found git-for-windows/git#1135, which described an extremely similar issue. But the solution doesn't look applicable in our case.

@philwo Any idea on this? I really don't know what we could do..

[philwo]

Seems like it's our bug:

This symptom is common when running MSYS2 programs (such as ssh.exe) from an account of an unhandled account type. We fixed something similar a while ago, where the MSYS2 runtime was ill-prepared to handle a "NETWORK" account.

Our Buildkite Agent runs as SYSTEM. This also explains, why you can't repro it on your own machine.

[philwo]

I guess there's no other fix available, so maybe we can try to change our setup again to run CI jobs as a different user - we had that in the past, but it was very complex. But maybe I can bring back the logic from that time...

[meteorcloudy]

Hmm,, sounds like the only way. Thanks for helping on this!

[meteorcloudy]

@philwo Thanks for the fix! Please ping me when the Windows slaves are updated. ;)

[philwo]

@meteorcloudy Rolled out!

[philwo]

Seems like we need to explicitly grant SECreateSymbolicLinkPrivilege to the Buildkite Agent user... rolling out a fix now: 8404c92

[philwo]

Verified that the fix worked - postsubmit is green again :)