Update .NET SDK to Preview 2 #8624

mmitche · 2022-03-16T16:16:31Z

To double check:

The right tests are in and and the right validation has happened. Guidance: https://github.com/dotnet/arcade/tree/main/Documentation/Validation

mmitche · 2022-03-16T17:47:20Z

@carlossanlop @hoyosjs Does that core dump when running p2 on ubuntu-latest look familiar at all?

Welcome to .NET 7.0!
---------------------
SDK Version: 7.0.100-preview.2.22153.17

----------------
Installed an ASP.NET Core HTTPS development certificate.
To trust the certificate run 'dotnet dev-certs https --trust' (Windows and macOS only).
Learn about HTTPS: https://aka.ms/dotnet-https
----------------
Write your first app: https://aka.ms/dotnet-hello-world
Find out what's new: https://aka.ms/dotnet-whats-new
Explore documentation: https://aka.ms/dotnet-docs
Report issues and find source on GitHub: https://github.com/dotnet/core
Use 'dotnet --help' to see available commands or visit: https://aka.ms/dotnet-cli
--------------------------------------------------------------------------------------
Cannot get required symbol SSL_set_alpn_protos from libssl
/__w/1/s/eng/common/tools.sh: line 474:   488 Aborted                 (core dumped) "$_InitializeBuildTool" "$@"
Build failed with exit code 134. Check errors above.
Killing running build processes...

hoyosjs · 2022-03-16T19:01:01Z

Not really. Also @bartonjs, given the missing libssl symbol... It's 20.04 with OpenSSL 1.1.1f, which I expect to have said symbols...

hoyosjs · 2022-03-16T19:51:13Z

I also can't repro with openssl 1.1.1f-1ubuntu2.12 amd64 on 20.04:

root@5c2c8e61a500:/# readelf --dyn-syms /lib/x86_64-linux-gnu/libssl.so.1.1 | grep SSL_set_alpn_protos
   907: 000000000003a100   147 FUNC    GLOBAL DEFAULT   15 SSL_set_alpn_protos@@OPENSSL_1_1_0

That missing method would definitely cause that... but seems like a custom build of libssl in the agent? Is there a way to get a hold of the image?

mmitche · 2022-03-16T20:01:48Z

I also can't repro with openssl 1.1.1f-1ubuntu2.12 amd64 on 20.04:
root@5c2c8e61a500:/# readelf --dyn-syms /lib/x86_64-linux-gnu/libssl.so.1.1 | grep SSL_set_alpn_protos
   907: 000000000003a100   147 FUNC    GLOBAL DEFAULT   15 SSL_set_alpn_protos@@OPENSSL_1_1_0
That missing method would definitely cause that... but seems like a custom build of libssl in the agent? Is there a way to get a hold of the image?

Interesting. @MattGal @dotnet/dnceng, I know we had a set of images created that mirror the hosted pool images. Is there one for ubuntu-latest or whatever that maps too?

MattGal · 2022-03-16T20:18:47Z

I also can't repro with openssl 1.1.1f-1ubuntu2.12 amd64 on 20.04:
root@5c2c8e61a500:/# readelf --dyn-syms /lib/x86_64-linux-gnu/libssl.so.1.1 | grep SSL_set_alpn_protos
   907: 000000000003a100   147 FUNC    GLOBAL DEFAULT   15 SSL_set_alpn_protos@@OPENSSL_1_1_0
That missing method would definitely cause that... but seems like a custom build of libssl in the agent? Is there a way to get a hold of the image?
Interesting. @MattGal @dotnet/dnceng, I know we had a set of images created that mirror the hosted pool images. Is there one for ubuntu-latest or whatever that maps too?

I didn't make a "latest" tag though that's possible if we think it's really valuable. You can see them at the bottom of https://helix.dot.net/. 20.04 is currently the same as latest per https://github.com/actions/virtual-environments.

You want 1es-ubuntu-2004-open for the public project, and 1es-ubuntu-2004 for internal. Ping me if you hit any trouble; we've already got a fair bit of builds moved over here.

am11 · 2022-03-21T08:43:12Z

We are using Ubuntu 14:

which is out of support in .NET 6: https://github.com/dotnet/core/blob/main/release-notes/6.0/supported-os.md#linux

I think we should be using 18.04 or 20.04, to match runtime pipelines?

MattGal · 2022-03-21T17:12:10Z

I think we should be using 18.04 or 20.04, to match runtime pipelines?

Ideally yes, but note that the docker image you build inside is independent of the build agent... that is, if you have some valid reason to be using Ubuntu 14.04 (I'm interested in knowing it) you can update the agent and keep using the ancient container as long as you like. Barring a good reason, you should be updating to at least a "Pro" supported 16.04 or 18.04 as soon as reasonably possible though.

am11 · 2022-03-21T17:29:25Z

If the build is failing inside Ubuntu 14, that will explain the reason of missing symbol. See this workaround for Ubuntu 14: https://gist.github.com/am11/11b8f526747c5ad12c9a74e7765e14a4#file-dockerfile-L12. We already deleted trusty setup in #7694, just need to update container references in pipelines to use something more recent (i.e. one of the supported version of Ubuntu).

mmitche · 2022-03-21T18:02:45Z

The SDK itself no longer supports 14.04, so running inside the 14.04 container wouldn't necessarily be expected to work. I've updated to the cotnainer the runtime uses for linux x64

hoyosjs · 2022-03-21T18:02:53Z

Actually - we should just update. Arcade has no native deps and it's using ubuntu-14.04-cross-0cd4667-20170319080304 (I somehow managed to overlook it was containerized). That's a cross-build image and it's not needed here.

mmitche · 2022-03-22T20:33:53Z

@MattGal This looks a bit puzzling. It's looking for aspnetcore but using the version of the core runtime. I think this is because this line https://github.com/dotnet/arcade/pull/8624/files#diff-8ab8c282f3d9822870eee0d25497d5ed8a335820fe03b6194b2dce937e1ca321L9 has an empty NetCoreSdkVersion property.

Does this look familiar or should I dig further?

MattGal · 2022-03-22T21:24:18Z

@MattGal This looks a bit puzzling. It's looking for aspnetcore but using the version of the core runtime. I think this is because this line https://github.com/dotnet/arcade/pull/8624/files#diff-8ab8c282f3d9822870eee0d25497d5ed8a335820fe03b6194b2dce937e1ca321L9 has an empty NetCoreSdkVersion property.

Does this look familiar or should I dig further?

Not instantly familiar but I'm taking a peek

MattGal · 2022-03-22T21:27:40Z

src/Microsoft.DotNet.Helix/Sdk/tools/dotnet-cli/DotNetCli.props

@@ -2,7 +2,7 @@
  <PropertyGroup>
    <IncludeDotNetCli Condition=" '$(IncludeDotNetCli)' != 'true' ">false</IncludeDotNetCli>
    <!-- TODO (https://github.com/dotnet/arcade/issues/7787): When bumping this, we should do #7787 -->
-    <AspNetCoreRuntimeVersion>6.0.0</AspNetCoreRuntimeVersion>
+    <AspNetCoreRuntimeVersion>7.0.0-preview.2.22153.2</AspNetCoreRuntimeVersion>


This isn't meant to be a preview / unstable version, rather it's what Xharness expects. I don't think you should rev the version here until 7.0.0 is GA

MattGal · 2022-03-22T21:28:30Z

src/Microsoft.DotNet.Helix/Sdk/tools/xharness-runner/XHarnessRunner.targets

@@ -4,7 +4,7 @@
    <IncludeDotNetCli>true</IncludeDotNetCli>
    <IncludeXHarnessCli>true</IncludeXHarnessCli>
    <XHarnessTargetFramework Condition=" '$(XHarnessTargetFramework)' == '' ">net6.0</XHarnessTargetFramework>
-    <DotNetCliVersion Condition=" '$(DotNetCliVersion)' == '' ">6.0.100</DotNetCliVersion>
+    <DotNetCliVersion Condition=" '$(DotNetCliVersion)' == '' ">7.0.100-preview.2.22153.17</DotNetCliVersion>


Same comment here... this is specific to Xharness, it doesn't need to be on the preview bleeding edge.

MattGal · 2022-03-22T21:46:38Z

@mmitche I think you should just revert the lines from the Xharness Helix SDK, and that what you're going for (assuming that build # is correct) will be brought in when #8336 is done.

mmitche · 2022-03-23T17:16:20Z

@MattGal No dice on reverting the xharness SDK (either both DotNetCli.props and XHarnessRunner.targets or only XHarnessRunner.targets.

It appears that the SDK knows the runtime tied to the SDK in the global.json, and that's ending up as DotNetCliVersion.

tkapin · 2022-03-23T18:31:18Z

/cc @premun to make sure you are aware of this once you are back

MattGal · 2022-03-23T18:44:46Z

@MattGal No dice on reverting the xharness SDK (either both DotNetCli.props and XHarnessRunner.targets or only XHarnessRunner.targets.

It appears that the SDK knows the runtime tied to the SDK in the global.json, and that's ending up as DotNetCliVersion.

Rats, I thought we honored the version in the .targets. It might just be that you need the other work coming in from @chcosta , but I defer to @premun for the XHarness Xpertise

premun · 2022-03-28T11:15:50Z

XHarness flows are in no way special really. They just utilize the IncludeDotNetCli property like any client can do. The only way it is special is probably that it wants aspnetcore-runtime and no other flows really want that flavour so it is actually catching an edge case where the versions of the runtime do not really match with what's coming from global.json.

There should also be some unit test somewhere testing that the aspnetcore-runtime is fine (#7227). You can also check how we were doing these bumps before:
9fc83fb

We can also use the full SDK with XHarness and things will work as well (seems to be hinted in #7787). We can try taking this:
https://github.com/dotnet/arcade/blob/main/src/Microsoft.DotNet.Helix/Sdk/tools/xharness-runner/XHarnessRunner.targets#L2-L10
and reducing it to this:

  <PropertyGroup Condition=" '$(IncludeXHarnessCli)' == 'true' ">
    <IncludeDotNetCli>true</IncludeDotNetCli>
    <XHarnessTargetFramework Condition=" '$(XHarnessTargetFramework)' == '' ">net6.0</XHarnessTargetFramework>
    <DotNetCliVersion Condition=" '$(DotNetCliVersion)' == '' ">6.0.100</DotNetCliVersion>
    <DotNetCliPackageType Condition=" '$(DotNetCliPackageType)' != 'sdk' ">sdk</DotNetCliPackageType>
  </PropertyGroup>

We can then bump DotNetCliVersion once XHarness consumes this and is built with .NET 7 preview 1. However, if we could get this version from global.json, we can get rid of this need for manual bumps. I know it's available in some MSBuild property somewhere.

src/Microsoft.DotNet.Helix/Sdk/tools/xharness-runner/XHarnessRunner.targets

…unner.targets Co-authored-by: Přemek Vysoký <premek.vysoky@microsoft.com>

mmitche · 2022-03-29T15:52:07Z

@premun No dice :/

premun · 2022-03-29T16:13:49Z

Unable to find dotnet cli sdk version 7.0.0-preview.2.22152.2 from any of the specified feeds.

Where does that number come from?

mmitche · 2022-03-29T16:16:18Z

Unable to find dotnet cli sdk version 7.0.0-preview.2.22152.2 from any of the specified feeds.

Where does that number come from?

That's the dotnet runtime that is within the SDK specified in global.json

premun · 2022-03-29T16:16:30Z

Is this it?

I think this boils down again to our FindDotNetVersion target (or how it's called) that is getting the info from some json somewhere and it is why we needed to type down this ASP.NET Core runtime version in the first place. I remember a discussion here: #7022

premun · 2022-03-29T16:17:46Z

That's the dotnet runtime that is within the SDK specified in global.json

Why does it want the runtime version when we specify SDK?

mmitche · 2022-03-29T16:18:07Z

@premun DotNetCliVersion is set here. I think maybe the right thing to do is to set it explicitly with 6.0.100, regardless of whethe it's already set.

premun · 2022-03-29T16:18:58Z

@premun DotNetCliVersion is set here. I think maybe the right thing to do is to set it explicitly with 6.0.100, regardless of whethe it's already set.

@mmitche I am afraid your last solution won't work once XHarness consumes newest Arcade, it will be built with .NET 7 and start requiring .NET 7 runtime/sdk to be run

mmitche · 2022-03-29T16:26:12Z

That's the dotnet runtime that is within the SDK specified in global.json

Why does it want the runtime version when we specify SDK?

In DotNetCli.props:

<DotNetCliPackageType Condition=" '$(DotNetCliPackageType)' == '' ">runtime</DotNetCliPackageType>
    <DotNetCliVersion Condition=" '$(DotNetCliVersion)' == '' AND '$(DotNetCliPackageType)' == 'runtime' ">$(BundledNETCoreAppPackageVersion)</DotNetCliVersion>
    <!-- TODO (https://github.com/dotnet/arcade/issues/7022): We are hardcoding this version to use the one tied to the SDK version from global.json -->
    <DotNetCliVersion Condition=" '$(DotNetCliVersion)' == '' AND '$(DotNetCliPackageType)' == 'aspnetcore-runtime' ">$(AspNetCoreRuntimeVersion)</DotNetCliVersion>
    <DotNetCliVersion Condition=" '$(DotNetCliVersion)' == '' AND '$(DotNetCliPackageType)' == 'sdk' ">$(NETCoreSdkVersion)</DotNetCliVersion>
    <DotNetCliChannel Condition=" '$(DotNetCliChannel)' == '' ">Current</DotNetCliChannel>

mmitche · 2022-03-29T16:34:01Z

@premun DotNetCliVersion is set here. I think maybe the right thing to do is to set it explicitly with 6.0.100, regardless of whethe it's already set.

@mmitche I am afraid your last solution won't work once XHarness consumes newest Arcade, it will be built with .NET 7 and start requiring .NET 7 runtime/sdk to be run

When is this slated to happen? Maybe we just wait till that flows into here and then we update?

premun · 2022-03-29T17:11:53Z

@mmitche I am afraid your last solution won't work once XHarness consumes newest Arcade, it will be built with .NET 7 and start requiring .NET 7 runtime/sdk to be run

When is this slated to happen? Maybe we just wait till that flows into here and then we update?

We can totally do it in sync, this is not a big deal at all. I am more worried about the non-XHarness usages of Helix SDK which will probably still suffer from this behaviour when trying to install the SDK because something is probably broken there?

Feel free to merge this, I can do the XHarness bump tomorrow and bump it here as well.

mmitche · 2022-03-29T17:37:33Z

k just waiting for review then.

riarenas · 2022-03-31T14:49:02Z

@mmitche fyi that this is stuck in arcade-validation: dotnet/arcade-validation#3022

I'm going to try and .net 7 all the things there real quick and if it's not straightforward I'll have FR take a look

riarenas · 2022-03-31T15:28:42Z

It's unblocked now

mmitche added 2 commits March 16, 2022 09:14

Update .NET SDK to Preview 2

17d3802

Two missing places

b7a5928

Update to use same container as runtime's linux x64 build

9c03915

MattGal reviewed Mar 22, 2022

View reviewed changes

mmitche added 2 commits March 22, 2022 14:57

Revert xharness to 6.0.0

d4fa7d8

Update the dotnet cli props

074280c

Apply @premun suggestions

8662955

premun reviewed Mar 29, 2022

View reviewed changes

src/Microsoft.DotNet.Helix/Sdk/tools/xharness-runner/XHarnessRunner.targets Outdated Show resolved Hide resolved

Update src/Microsoft.DotNet.Helix/Sdk/tools/xharness-runner/XHarnessR…

98d7b51

…unner.targets Co-authored-by: Přemek Vysoký <premek.vysoky@microsoft.com>

Force dotnet cli version

ed11d8f

mmitche enabled auto-merge (squash) March 29, 2022 17:37

premun approved these changes Mar 29, 2022

View reviewed changes

mmitche merged commit cfdac35 into dotnet:main Mar 29, 2022

mmitche deleted the update-to-net7 branch March 31, 2022 16:45

ryanbrandenburg mentioned this pull request May 19, 2022

[main] Update dependencies from dotnet/arcade dotnet/razor#6247

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update .NET SDK to Preview 2 #8624

Update .NET SDK to Preview 2 #8624

mmitche commented Mar 16, 2022

mmitche commented Mar 16, 2022

hoyosjs commented Mar 16, 2022

hoyosjs commented Mar 16, 2022

mmitche commented Mar 16, 2022

MattGal commented Mar 16, 2022

am11 commented Mar 21, 2022

MattGal commented Mar 21, 2022

am11 commented Mar 21, 2022

mmitche commented Mar 21, 2022

hoyosjs commented Mar 21, 2022

mmitche commented Mar 22, 2022

MattGal commented Mar 22, 2022

MattGal Mar 22, 2022

MattGal Mar 22, 2022

MattGal commented Mar 22, 2022

mmitche commented Mar 23, 2022

tkapin commented Mar 23, 2022

MattGal commented Mar 23, 2022

premun commented Mar 28, 2022 •

edited

Loading

mmitche commented Mar 29, 2022

premun commented Mar 29, 2022

mmitche commented Mar 29, 2022

premun commented Mar 29, 2022

premun commented Mar 29, 2022

mmitche commented Mar 29, 2022

premun commented Mar 29, 2022

mmitche commented Mar 29, 2022

mmitche commented Mar 29, 2022

premun commented Mar 29, 2022

mmitche commented Mar 29, 2022

riarenas commented Mar 31, 2022

riarenas commented Mar 31, 2022

Update .NET SDK to Preview 2 #8624

Update .NET SDK to Preview 2 #8624

Conversation

mmitche commented Mar 16, 2022

To double check:

mmitche commented Mar 16, 2022

hoyosjs commented Mar 16, 2022

hoyosjs commented Mar 16, 2022

mmitche commented Mar 16, 2022

MattGal commented Mar 16, 2022

am11 commented Mar 21, 2022

MattGal commented Mar 21, 2022

am11 commented Mar 21, 2022

mmitche commented Mar 21, 2022

hoyosjs commented Mar 21, 2022

mmitche commented Mar 22, 2022

MattGal commented Mar 22, 2022

MattGal Mar 22, 2022

Choose a reason for hiding this comment

MattGal Mar 22, 2022

Choose a reason for hiding this comment

MattGal commented Mar 22, 2022

mmitche commented Mar 23, 2022

tkapin commented Mar 23, 2022

MattGal commented Mar 23, 2022

premun commented Mar 28, 2022 • edited Loading

mmitche commented Mar 29, 2022

premun commented Mar 29, 2022

mmitche commented Mar 29, 2022

premun commented Mar 29, 2022

premun commented Mar 29, 2022

mmitche commented Mar 29, 2022

premun commented Mar 29, 2022

mmitche commented Mar 29, 2022

mmitche commented Mar 29, 2022

premun commented Mar 29, 2022

mmitche commented Mar 29, 2022

riarenas commented Mar 31, 2022

riarenas commented Mar 31, 2022

premun commented Mar 28, 2022 •

edited

Loading