Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[LoongArch64] add build-config for LoongArch64. #8271

Merged
merged 2 commits into from
Dec 20, 2021
+5 −1
Merged

[LoongArch64] add build-config for LoongArch64. #8271

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
c297282
[LoongArch64] add build-config for LoongArch64.
Dec 17, 2021
a38f81a
[LoongArch64] revert the `eng/common/cross/toolchain.cmake`.
Dec 20, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions eng/common/cross/toolchain.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,9 @@ elseif(TARGET_ARCH_NAME STREQUAL "arm64")
if("$ENV{__DistroRid}" MATCHES "tizen.*")
set(TIZEN_TOOLCHAIN "aarch64-tizen-linux-gnu/9.2.0")
endif()
elseif(TARGET_ARCH_NAME STREQUAL "loongarch64")
Copy link
Member

@am11 am11 Dec 17, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@shushanhf, updating this file alone is not enough as we will also need setup bits in build-rootfs.sh.
The build-rootf.sh is then used in https://github.com/dotnet/dotnet-buildtools-prereqs-docker/ repo, to produce the cross-build docker container for various platforms/architectures (e.g. ubuntu-18.04-cross-arm-alpine-xxx, ubuntu-18.04-cross-illumos-xxx etc. full list of tags). This makes it easier for anyone (including runtime CI pipelines) to consume the container and build product.

Are all packages listed in build-rootfs.sh script available on loongarch64, and which distro will you be targeting? I am asking because for Debian riscv64 architecture, I am still waiting on a patch from lldb team for System-V support (meanwhile I unset lldb like s390x below). Other than that, riscv64 is still in Debian ports so I had to modify this script a little to add sid support (dotnet/runtime@main...am11:feature/riscv-rootfs, still WIP as I'm also waiting for llvm 13.0.1, which has a default fix for .NET ML dependencies, so I can remove the TODO).
For local testing:

# sudo is only required for debbotstrap command, other platforms like alpine and illumos don't require root
sudo eng/common/cross/build-rootfs.sh riscv64
sudo eng/common/cross/build-rootfs.sh freebsd
# etc.

then in runtime repo, pass cross arg: ROOTFS_DIR=$(pwd)/.tools/rootfs/loongarch64 ./build.sh clr -cross -arch loongarch64 ... (in the docker container, that ROOTFS_DIR will have a set path /crossrootfs/loongarch64)

Copy link
Contributor Author

@shushanhf shushanhf Dec 17, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@shushanhf, updating this file alone is not enough as we will also need setup bits in build-rootfs.sh.

Thanks!
I will commit LoongArch64 to this.
https://hub.fastgit.org/shushanhf/arcade/blob/main/eng/common/cross/build-rootfs.sh

Are all packages listed in build-rootfs.sh script available on loongarch64, and which distro will you be targeting? I am asking because for Debian riscv64 architecture, I am still waiting on a patch from lldb team for System-V support (meanwhile I unset lldb like s390x below). Other than that, riscv64 is still in Debian ports so I had to modify this script a little to add sid support (dotnet/runtime@main...am11:feature/riscv-rootfs, still WIP as I'm also waiting for llvm 13.0.1, which has a default fix for .NET ML dependencies, so I can remove the TODO). For local testing:

We can supply ubuntu20.04. Now we are preparing qemu supporting LoongArch64.

# sudo is only required for debbotstrap command, other platforms like alpine and illumos don't require root
sudo eng/common/cross/build-rootfs.sh riscv64
sudo eng/common/cross/build-rootfs.sh freebsd
# etc.

then in runtime repo, pass cross arg: ROOTFS_DIR=$(pwd)/.tools/rootfs/loongarch64 ./build.sh clr -cross -arch loongarch64 ... (in the docker container, that ROOTFS_DIR will have a set path /crossrootfs/loongarch64)

That means we have to supply a LoongArch64 rootfs tar-file and including a cross-compiler ?

Copy link
Member

@am11 am11 Dec 17, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That means we have to supply a LoongArch64 rootfs tar-file and including a cross-compiler ?

If we are on loongarch64 host, and building for loongarch64, then toolchain.cmake is not involved. I imagine it is your case, where you are locally using the baremetal loongarch64 machine. This cmake script is only involved when we cross compile the .NET runtime or SDK (e.g. in the CI where we only have x64 or arm64 host, so arm32, x86, s380x etc. binaries are cross-compiled then sent to remote machines with real hardware for testing). See https://github.com/dotnet/runtime/blob/b338078/eng/native/gen-buildsys.sh#L82 (the outer condition is if [[ "$CROSSCOMPILE" == "1" ]]).

build-rootfs.sh is a helper script which integrates seamlessly with rest of the engineering stack. If we can arrange the debootstrap setup for loongarch64 (e.g. I found some discussion here: https://gitlab.summer-ospp.ac.cn/summer2021/210080299/-/blob/main/week12_report.md), it would be nice, because after that we can request for a docker container (e.g. a PR like dotnet/dotnet-buildtools-prereqs-docker#324 with addition of qemu-loongarch64 package). With that docker container it will make things easier for everyone to build runtime for loongarch64 on any system where docker is installed.

Long story short, it is not required to have a cross compiler right now (it can be added later), and then we can update both toolchain.cmake and build-rootfs.sh together (as there might be other changes needed in toolchain.cmake, right now we can't test it or tell for sure).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That means we have to supply a LoongArch64 rootfs tar-file and including a cross-compiler ?

If we are on loongarch64 host, and building for loongarch64, then toolchain.cmake is not involved. I imagine it is your case, where you are locally using the baremetal loongarch64 machine. This cmake script is only involved when we cross compile the .NET runtime or SDK (e.g. in the CI where we only have x64 or arm64 host, so arm32, x86, s380x etc. binaries are cross-compiled then sent to remote machines with real hardware for testing). See https://github.com/dotnet/runtime/blob/b338078/eng/native/gen-buildsys.sh#L82 (the outer condition is if [[ "$CROSSCOMPILE" == "1" ]]).

Yes, now we just test and run all dotnet on our LoongArch64-linux machine.

build-rootfs.sh is a helper script which integrates seamlessly with rest of the engineering stack. If we can arrange the debootstrap setup for loongarch64 (e.g. I found some discussion here: https://gitlab.summer-ospp.ac.cn/summer2021/210080299/-/blob/main/week12_report.md), it would be nice, because after that we can request for a docker container (e.g. a PR like dotnet/dotnet-buildtools-prereqs-docker#324 with addition of qemu-loongarch64 package). With that docker container it will make things easier for everyone to build runtime for loongarch64 on any system where docker is installer.

Long story short, it is not required to have a cross compiler right now (it can be added later), and then we can update both toolchain.cmake and build-rootfs.sh together (as there might be other changes needed in toolchain.cmake, right now we can't test it or tell for sure).

Thanks.
Here is a link page for qemu and toolchain, https://github.com/loongson/build-tools.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we are on loongarch64 host, and building for loongarch64, then toolchain.cmake is not involved. I imagine it is your case, where you are locally using the baremetal loongarch64 machine. This cmake script is only involved when we cross compile the .NET runtime or SDK (e.g. in the CI where we only have x64 or arm64 host, so arm32, x86, s380x etc. binaries are cross-compiled then sent to remote machines with real hardware for testing).

By the way, do you need the real hardware of LoongArch64 for testing ?
If you need, we can supply the LoongArch64 machine for you.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By the way, do you need the real hardware of LoongArch64 for testing ?

I do know that question for not for me, but why do not ask. Can I get free treat? At least I can guarantee that I make NativeAOT work on this platform.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Long story short, it is not required to have a cross compiler right now (it can be added later), and then we can update both toolchain.cmake and build-rootfs.sh together

Agree. @shushanhf Could you please revert changes in toolchain.cmake so that we can get the rest of the changes in this PR merged?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@am11 If incremental infrastructure changes are ok, what do you feel about following future plan for cross-compile. That should be treated as stop-gap measurements in case debootstrap MR would take long time to be merged or other organizational issues arise.

Right now GCC and CLang cannot produce LoongArch64 execuables from released version and work just starting. so debootstrap is not enough (isn't it?). That mean that temporary cross compilation can be achieved via other means, like @shushanhf suggests above:

That means we have to supply a LoongArch64 rootfs tar-file and including a cross-compiler ?

Is this alternative viable for the future?

Copy link
Contributor Author

@shushanhf shushanhf Dec 20, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By the way, do you need the real hardware of LoongArch64 for testing ?

I do know that question for not for me, but why do not ask. Can I get free treat? At least I can guarantee that I make NativeAOT work on this platform.

Yes, complimentary a desktop host.

Long story short, it is not required to have a cross compiler right now (it can be added later), and then we can update both toolchain.cmake and build-rootfs.sh together

Agree. @shushanhf Could you please revert changes in toolchain.cmake so that we can get the rest of the changes in this PR merged?

Ok, I will do it right now.

@am11 If incremental infrastructure changes are ok, what do you feel about following future plan for cross-compile. That should be treated as stop-gap measurements in case debootstrap MR would take long time to be merged or other organizational issues arise.

Right now GCC and CLang cannot produce LoongArch64 execuables from released version and work just starting. so debootstrap is not enough (isn't it?). That mean that temporary cross compilation can be achieved via other means, like @shushanhf suggests above:

That means we have to supply a LoongArch64 rootfs tar-file and including a cross-compiler ?

Is this alternative viable for the future?

I had prepared a cross-comiler on x64-linux for LoongArch64.
If you needed, I can give you.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Although our company had finished all the comilers (gcc、clang、.NET、JVM、browser ...) and some tools liking docker, it still needs a little time to be accepted by the upstream.

There is another way for cross-compiling.

We can supply a docker image and a docker source suppling the related softwares before you get them from the upstream, liking getting the docker images about LoongArch64 from the docker-hub.

set(CMAKE_SYSTEM_PROCESSOR loongarch64)
set(TOOLCHAIN "loongarch64-linux-gnu")
elseif(TARGET_ARCH_NAME STREQUAL "s390x")
set(CMAKE_SYSTEM_PROCESSOR s390x)
set(TOOLCHAIN "s390x-linux-gnu")
Expand All @@ -49,7 +52,7 @@ elseif (ILLUMOS)
set(CMAKE_SYSTEM_PROCESSOR "x86_64")
set(TOOLCHAIN "x86_64-illumos")
else()
message(FATAL_ERROR "Arch is ${TARGET_ARCH_NAME}. Only armel, arm, arm64, s390x and x86 are supported!")
message(FATAL_ERROR "Arch is ${TARGET_ARCH_NAME}. Only armel, arm, arm64, loongarch64, s390x and x86 are supported!")
endif()

if(DEFINED ENV{TOOLCHAIN})
Expand Down Expand Up @@ -177,7 +180,7 @@ endif()

# Specify compile options

if((TARGET_ARCH_NAME MATCHES "^(arm|armel|arm64|s390x)$" AND NOT "$ENV{__DistroRid}" MATCHES "android.*") OR ILLUMOS)
if((TARGET_ARCH_NAME MATCHES "^(arm|armel|arm64|s390x|loongarch64)$" AND NOT "$ENV{__DistroRid}" MATCHES "android.*") OR ILLUMOS)
set(CMAKE_C_COMPILER_TARGET ${TOOLCHAIN})
set(CMAKE_CXX_COMPILER_TARGET ${TOOLCHAIN})
set(CMAKE_ASM_COMPILER_TARGET ${TOOLCHAIN})
Expand Down
3 changes: 3 additions & 0 deletions eng/common/dotnet-install.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,9 @@ case $cpuname in
aarch64)
buildarch=arm64
;;
loongarch64)
buildarch=loongarch64
;;
amd64|x86_64)
buildarch=x64
;;
Expand Down
3 changes: 2 additions & 1 deletion eng/common/native/CommonLibrary.psm1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -276,7 +276,8 @@ function Get-MachineArchitecture {
}
if (($ProcessorArchitecture -Eq "AMD64") -Or
($ProcessorArchitecture -Eq "IA64") -Or
($ProcessorArchitecture -Eq "ARM64")) {
($ProcessorArchitecture -Eq "ARM64") -Or
($ProcessorArchitecture -Eq "LOONGARCH64")) {
return "x64"
}
return "x86"
Expand Down