New arcade causes "install-cmake.sh: Permission denied" failures with version 1.0.0-beta.19251.6

[michellemcdaniel]

We are seeing the following when building the linux legs in the optimization repo with the latest arcade drop:

[command]/usr/bin/bash --noprofile --norc /__w/_temp/02f4ab40-7769-4597-b591-29a457d1831e.sh
/__w/1/s/eng/common/init-tools-native.sh: line 120: /__w/1/s/eng/common/native/install-cmake.sh: Permission denied
Execution Failed
##[error]Bash exited with code '1'.
##[section]Finishing: Build Linux

I believe this is because build.sh was changed with the following code:

if [[ "$restore" == true ]]; then
  InitializeNativeTools
fi

Does a file need to be chmod +x'd?

[riarenas]

cc @dotnet/dnceng

[chcosta]

Ack, looking

[MattGal]

Thanks @chcosta !

[chcosta]

Do you have a repo I can clone @adiaaida and some repro instructions? I tried this on Arcade and it installed the native tools just fine.

[michellemcdaniel]

@chcosta This is the PR where we are seeing this failure: https://dnceng.visualstudio.com/internal/_git/dotnet-optimization/pullrequest/1128?_a=overview
Direct link to job: https://dnceng.visualstudio.com/internal/_build/results?buildId=176397&_a=summary

Repo: https://dnceng.visualstudio.com/internal/_git/dotnet-optimization
Branch: darc-master-e9ae2268-9dd8-49cd-921b-87030a76c832

We run ./build.sh --configuration Release on Linux

[chcosta]

Do you require native tools to be installed for Linux or are you expecting those to be in your Docker image?

I suspect that you don't really care about the Linux native tool installation, but this was recently enabled by default. It looks to me like the Linux native tools haven't received the same care that the Windows native tools (likely because of the Docker stuff) and that it's busted in a few ways.

1. We need to be able to opt out of native tools installations for some OS's / distros. Currently, it just tries to run always.
2. I suspect that, even if you got past the "permission denied" issue, you'd fail because there's no install script for python
3. The execute bit permissions for install-cmake.sh look fine to me... I'm not sure how to address that.

The immediate course of action here is to provide an option to disable non-Windows from native toolset installation. That will need to be followed up with an actual investment in this area. I'll look into preparing a PR for Arcade that will unblock you. If you need changes from Arcade, then you'll have to hack up eng/common/tools.sh until there's a better solution

[michellemcdaniel]

That's right. We do not need the linux native tool installation.

[chcosta]

Filed Native toolset installation scripts between Windows / Linux are not at parity

Filed Permit defining native tools installation / OS

I'll work on preparing an interim fix

[michellemcdaniel]

Thank you so much!

[chcosta]

@adiaaida , #2675 should unblock you

[michellemcdaniel]

Do I need to change our yaml/build scripts in anyway to get the behavior?

[chcosta]

Yes, you'll need to the env block to your Linux build step. Something like this...

      - ${{ if ne(parameters.agentOS, 'Windows_NT') }}:
        - script: ./build.sh
            --configuration Release
          name: build_linux
          displayName: 'Build Linux'
          condition: succeeded()
          env:
            DisableNativeToolsetInstalls: true

[michellemcdaniel]

thanks!