Fix race condition in ServiceProcess tests

[ericstj]

Fixes #59297

[danmoseley]

LGTM, thank you. Why did this take so long to appear I wonder.

[azure-pipelines]

No pipelines are associated with this pull request.

[ericstj]

/azp run outerloop

[azure-pipelines]

No pipelines are associated with this pull request.

[ericstj]

/azp run runtime-coreclr outerloop

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[danmoseley]

@BruceForstall should the format job be clean? also, it has a bunch of errors like

Running: clang-tidy -fix -checks=-*,readability-braces*,modernize-use-nullptr  -header-filter=jit/.* -p=D:\a\1\s\src\coreclr\..\..\artifacts\obj\coreclr\windows.x64.Release\compile_commands.json D:\a\1\s\src\coreclr\jit\alloc.cpp
Unhandled exception. System.ComponentModel.Win32Exception (193): The specified executable is not a valid application for this OS platform.

[danmoseley]

Arm32 failures are known: #58481

[danmoseley]

@ericstj do the outerloop tests even run as admin (so these tests actually run)?

I actually can't find the config in the Azdo UI that is outerloop. Looking at each of the Windows libraries test configs, I see a line like this:

2021-09-27T22:48:26.1877349Z ./build.sh -subset libs.pretest -rc release -configuration Debug -ci -arch x64 -framework net7.0  -testscope innerloop  /p:RuntimeOS=linux-musl   /p:RuntimeArtifactsPath=/home/vsts/work/1/s/artifacts/transport/coreclr  /p:RuntimeFlavor=coreclr /bl:/home/vsts/work/1/s/artifacts/log/Debug/overrideRuntimeFromLiveDrop.binlog

Note -testscope innerloop. So I'm not sure where the outerloop ones are, so I couldn't try to figure out if they run elevated.

[ericstj]

CHecking the logs. @safern recommended this trigger to me. Not sure if the Test UI is good for checking on passing tests because there are so many. I might be able to find the console logs from build log spew.

[ericstj]

I actually don't see where libraries outerloop tests are even run. I see coreclr runtime pri1, but no libraries tests.

[safern]

Oops this was my bad, there is a runtime-coreclr outerloop and a runtime-libraries-coreclr outerloop and when I searched for outerloop I saw both but when telling @ericstj the command to run it I accidentally missed the -libraries portion of the pipeline name 😞 . Sorry about that.

[ericstj]

No worries, next time I'll use /azp run runtime-libraries-coreclr outerloop-windows

[danmoseley]

Good info. @safern do you know (or know how to know) whether we run elevated?

[safern]

@safern do you know (or know how to know) whether we run elevated?

IIRC helix do run elevated, but I'd like @MattGal to confirm that to be on the safe side.

[ericstj]

I opened #59712 to find out (and get test coverage I was hoping to have in this PR). Interestingly since its the same hash it doesn't retrigger the main validation builds, bonus resource savings :)

[MattGal]

@safern do you know (or know how to know) whether we run elevated?

IIRC helix do run elevated, but I'd like @MattGal to confirm that to be on the safe side.

Yes, you have the ability to start processes lower but the default is prompt-less elevated. In the earliest days, we had some machines set up to run the helix client in Medium IL mode and it hit tons of problems with people's work items.

[ericstj]

🧬 == 🍯

[ericstj]

@MattGal were those issues in regular runs or just outerloop? I remember at one point we said outerloop would get elevation but inner loop wouldn't.

[MattGal]

@MattGal were those issues in regular runs or just outerloop? I remember at one point we said outerloop would get elevation but inner loop wouldn't.

The helix agents don't know or care about your concepts of inner/outer-loopiness.

[danmoseley]

Yes, you have the ability to start processes lower but the default is prompt-less elevated.

How is this done? It might be good to add one such run in the mix.

Also, is this the case for both Windows and Unix (elevated or su respectively)

[BruceForstall]

@BruceForstall should the format job be clean? also, it has a bunch of errors like

Running: clang-tidy -fix -checks=-*,readability-braces*,modernize-use-nullptr  -header-filter=jit/.* -p=D:\a\1\s\src\coreclr\..\..\artifacts\obj\coreclr\windows.x64.Release\compile_commands.json D:\a\1\s\src\coreclr\jit\alloc.cpp
Unhandled exception. System.ComponentModel.Win32Exception (193): The specified executable is not a valid application for this OS platform.

It should be, and is in subsequent runs.

The real failure was earlier:

Invoke-WebRequest : Unable to read data from the transport connection: An existing connection was forcibly closed by 
the remote host.
At D:\a\1\s\eng\formatting\download-tools.ps1:20 char:23
+ ...   $status = Invoke-WebRequest -Uri "$baseUri/$toolName.exe" -OutFile  ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Invoke-WebRequest], IOException
    + FullyQualifiedErrorId : System.IO.IOException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand

@jkoritzinsky Is this your new downloader from #59374 not handling failure well?

[jkoritzinsky]

Yeah I think there’s a problem with the new downloader script. I’ll take a look into it next time I have cycles