How to build this repo from scratch on Banana Unix?

[ghost]

Suppose there exists a Banana Unix OS (e.g. some BSD or Solaris like).

On such an OS, how do we build this project fully offline, such that it does not rely on anything from the internet during the build?

git clone https://github.com/dotnet/cli
cli/build.sh <what goes here?>

Are there required components to build CLI project, which haven't been open sourced yet to accomplish this task? If there are, can we expect them to land on GitHub soon?

---

Before dotnet/cli (in dnx days), we were able to build most of the CoreFX assemblies on FreeBSD and even configured the test runner to work for the successfully built assemblies. Now CLI is a required dependency, we can't build it on FreeBSD nor NetBSD.

Refs: https://github.com/dotnet/corefx/issues/1626, https://github.com/dotnet/coreclr/issues/1633 and https://github.com/dotnet/coreclr/issues/3178

cc @ghuntley, @josteink, @krytarowski and @akoeplinger

[blackdwarf]

@jasonwilliams200OK why is internet a problem?

[ghost]

@blackdwarf, if there is a dependency on native binary and the source is available, we can essentially build on [m]any platforms.

Besides, if "build everything from source" is also an option (in addition to packaged binaries), that will make things more simpler for us and new platform outreach would be easier for dotnet maintainers as well IMVHO.

[davidfowl]

There's only a single native thing in the CLI (for the managed codez) and that's corhost. It's the native CLR host (https://github.com/dotnet/cli/tree/rel/1.0.0/src/corehost)

[ghost]

@davidfowl, thanks. I am able to build corehost by running cli/src/corehost/build.sh on FreeBSD. Now I have cli/src/corehost/cli/corehost. Is there any way I can get dotnet produced (given CoreCLR and Roslyn, both build on FreeBSD at this moment)?

[davidfowl]

2 questions:

• Why does Roslyn need freebsd build? It's just managed code.
• Do you produce FreeBSD packages for CoreCLR and CoreFx?

[krytarowski]

The corehost part already builds on NetBSD:

$ uname -a
NetBSD compaq 7.99.25 NetBSD 7.99.25 (GENERIC) #0: Fri Dec 25 20:05:36 CET 2015  root@chieftec:/tmp/netbsd-tmp/sys/arch/amd64/compile/GENERIC amd64
$ ./build.sh                                                                                                                                          
Building Corehost from /public/cli/src/corehost to /public/cli/src/corehost
-- The C compiler identification is GNU 4.8.5
-- The CXX compiler identification is GNU 4.8.5
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring done
-- Generating done
-- Build files have been written to: /public/cli/src/corehost
Scanning dependencies of target corehost
[  7%] Building CXX object cli/CMakeFiles/corehost.dir/__/corehost.cpp.o
[ 14%] Building CXX object cli/CMakeFiles/corehost.dir/__/common/trace.cpp.o
[ 21%] Building CXX object cli/CMakeFiles/corehost.dir/__/common/utils.cpp.o
[ 28%] Building CXX object cli/CMakeFiles/corehost.dir/__/common/pal.unix.cpp.o
[ 35%] Linking CXX executable corehost
[ 35%] Built target corehost
Scanning dependencies of target hostpolicy
[ 42%] Building CXX object cli/dll/CMakeFiles/hostpolicy.dir/__/__/common/trace.cpp.o
[ 50%] Building CXX object cli/dll/CMakeFiles/hostpolicy.dir/__/__/common/utils.cpp.o
[ 57%] Building CXX object cli/dll/CMakeFiles/hostpolicy.dir/__/args.cpp.o
[ 64%] Building CXX object cli/dll/CMakeFiles/hostpolicy.dir/__/hostpolicy.cpp.o
[ 71%] Building CXX object cli/dll/CMakeFiles/hostpolicy.dir/__/coreclr.cpp.o
[ 78%] Building CXX object cli/dll/CMakeFiles/hostpolicy.dir/__/deps_resolver.cpp.o
[ 85%] Building CXX object cli/dll/CMakeFiles/hostpolicy.dir/__/servicing_index.cpp.o
[ 92%] Building CXX object cli/dll/CMakeFiles/hostpolicy.dir/__/__/common/pal.unix.cpp.o
[100%] Linking CXX shared library libhostpolicy.so
[100%] Built target hostpolicy

[ghost]

Why does Roslyn need freebsd build? It's just managed code.

I was just highlighting that the project builds on FreeBSD with mono and PCL assemblies installed.

Do you produce FreeBSD packages for CoreCLR and CoreFx?

I have just compiled CoreCRL on FreeBSD and NetBSD, then compiled CoreFX managed assemblies on Windows for these paltforms, scp'd over to VMs and ran tests. On FreeBSD, running the tests is throwing assertions.

If we get CLI building with confirmed RIDs etc., it would be easier for us to make progress (exempting the part where we need Windows machine to cross-compile managed assemblies for these targets)

[krytarowski]

@jasonwilliams200OK Can you run tests / managed code on NetBSD?

[ghost]

@krytarowski, I haven't built CoreFX assemblies for NetBSD targets yet. It takes about ~20 minutes to cross-compile those assemblies for FreeBSD on Windows. The entire size of corefx/bin folder that I copied over to FreeBSD VM was 7GB (FreeBSD_Debug target)

[krytarowski]

Keep us posted on the progress of trying it out.

[ghost]

Keep us posted on the progress of trying it out.

This issue is not tracking the progress of CoreFX build in particular, but it is focusing on steps required to build this repo from scratch on a new platform.

[ghost]

@blackdwarf For example, net access is not allowed from the build hosts on Debian during a package build.

[blackdwarf]

/cc @bleroy

[omajid]

In case anyone is curious, https://github.com/dotnet/source-build/ is where this effort - building all of .NET Core on a particular *nix distro - is being handled.

[krytarowski]

Does it handle NetBSD?

[omajid]

Honestly, no idea. But if you want to contribute patches, that would be the right place.

[trungnt2910]

I am currently working on dotnet/runtime#55803 to add .NET support to Haiku and facing the same problem.

The process to cross-compile CoreCLR (dotnet/runtime) is quite straightforward, but it is not for this repo.

I did manage to get a working .NET SDK on Haiku, but it involves manually editing the RID (from linux-x64 to haiku-x64) in the generated artifact and changing a few version numbers in Microsoft.NETCoreSdk.BundledVersions.props.

What is the correct way to cross-compile this repo? It is not clearly documented in the developer guide.

[omajid]

@uweigand @janani66 : can you share your cross-build scripts publicly? Maybe @trungnt2910 can use them to cross-building the entire SDK for this new platform?

[Sapana-Khemkar]

I am currently working on dotnet/runtime#55803 to add .NET support to Haiku and facing the same problem.

The process to cross-compile CoreCLR (dotnet/runtime) is quite straightforward, but it is not for this repo.

I did manage to get a working .NET SDK on Haiku, but it involves manually editing the RID (from linux-x64 to haiku-x64) in the generated artifact and changing a few version numbers in Microsoft.NETCoreSdk.BundledVersions.props.

What is the correct way to cross-compile this repo? It is not clearly documented in the developer guide.

@trungnt2910 Here is blog describing how we cross build dotnet on x86 for power and here are scripts uploaded. Check if this helps you

[trungnt2910]

I am currently working on dotnet/runtime#55803 to add .NET support to Haiku and facing the same problem.
The process to cross-compile CoreCLR (dotnet/runtime) is quite straightforward, but it is not for this repo.
I did manage to get a working .NET SDK on Haiku, but it involves manually editing the RID (from linux-x64 to haiku-x64) in the generated artifact and changing a few version numbers in Microsoft.NETCoreSdk.BundledVersions.props.
What is the correct way to cross-compile this repo? It is not clearly documented in the developer guide.

@trungnt2910 Here is blog describing how we cross build dotnet on x86 for power and here are scripts uploaded. Check if this helps you

Thanks for the guide, but the problem I'm dealing with is entirely different (different OS, same architecture, instead of same OS (Linux), different architecture).

I tried:

 ./build.sh --pack /p:OSName=haiku

because of this:

sdk/src/Layout/redist/targets/GetRuntimeInformation.targets

Lines 6 to 9 in 86db944

	<PropertyGroup>
	<OSName Condition=" '$(OSName)' == '' AND $([MSBuild]::IsOSPlatform(`Windows`)) ">win</OSName>
	<OSName Condition=" '$(OSName)' == '' AND $([MSBuild]::IsOSPlatform('OSX')) ">osx</OSName>
	<OSName Condition=" '$(OSName)' == '' ">linux</OSName>

but still got linux-x64 instead of haiku-x64.

[Sapana-Khemkar]

I am currently working on dotnet/runtime#55803 to add .NET support to Haiku and facing the same problem.
The process to cross-compile CoreCLR (dotnet/runtime) is quite straightforward, but it is not for this repo.
I did manage to get a working .NET SDK on Haiku, but it involves manually editing the RID (from linux-x64 to haiku-x64) in the generated artifact and changing a few version numbers in Microsoft.NETCoreSdk.BundledVersions.props.
What is the correct way to cross-compile this repo? It is not clearly documented in the developer guide.

@trungnt2910 Here is blog describing how we cross build dotnet on x86 for power and here are scripts uploaded. Check if this helps you

Thanks for the guide, but the problem I'm dealing with is entirely different (different OS, same architecture, instead of same OS (Linux), different architecture).

I tried:

 ./build.sh --pack /p:OSName=haiku

because of this:

sdk/src/Layout/redist/targets/GetRuntimeInformation.targets

Lines 6 to 9 in 86db944

	<PropertyGroup>
	<OSName Condition=" '$(OSName)' == '' AND $([MSBuild]::IsOSPlatform(`Windows`)) ">win</OSName>
	<OSName Condition=" '$(OSName)' == '' AND $([MSBuild]::IsOSPlatform('OSX')) ">osx</OSName>
	<OSName Condition=" '$(OSName)' == '' ">linux</OSName>

but still got linux-x64 instead of haiku-x64.

ohh.. seems some other issue. I did not face this.

[trungnt2910]

The current .NET SDK build procedure uses some files, namely Microsoft.NETCoreSdk.BundledVersions.props, from the old SDK and then patches the files. The patch replaces the existing versions with a hard coded one and uses the existing runtime identifier of the host SDK.

To cross compile the SDK for Haiku, I had to replace the hard coded .NET runtime version with the one I recently built. I also had to replace the runtime identifier from linux-x64 to haiku-x64. The full code for this process is here.

It would be nice if Versions.props could be overridden, and also nice if OverrideAndCreateBundeldNETCoreAppPackageVersion could override the RID based on the OSName property.

[trungnt2910]

There is also a scarcely documented source-build mode, but this mode does not seem to create enough files for the SDK to function. It seems to be related to the source build feature, which currently only supports Linux.

[baronfel]

Most actual building of an SDK for any OS should happen in the dotnet/installer repo, or in the dotnet/dotnet repo for .NET 9+. That's where all of the actual bundle construction occurs - this repo is focused on the logic of the SDK specifically.