Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make s390x and multi-arch fedora PeerPod images via mkosi #1640

Open
2 of 8 tasks
huoqifeng opened this issue Dec 21, 2023 · 5 comments · Fixed by #1678 or #1694
Open
2 of 8 tasks

Make s390x and multi-arch fedora PeerPod images via mkosi #1640

huoqifeng opened this issue Dec 21, 2023 · 5 comments · Fixed by #1678 or #1694

Comments

@huoqifeng
Copy link

huoqifeng commented Dec 21, 2023

mkosi has been used to leverage nix to build x86 PeerPod images, we need make it available for s390x (and maybe multi-arch). Initial investigation indicates that the nix OS does not support s390x, and mkosi does not support s390x natively either.

To mitigate the nix problem on s390x, we can use the mkosi ToolsTree=default since v16 feature defined in mkosi.conf to download latest tools automatically.

[Host]
ToolsTree=default

There is also a discussion thread to track the s390x support in mkosi.
So far, it's OK to build a kernel bootable fedora image for s390x with PR Support booting Fedora default image on s390x, to support a bootsable device, might need cooperate with mkosi and zipl community to make it happen. A compromise goal might be:

  • In short term
    • We build a kernel bootable fedora s390x image via mkosi
    • We use a simple scripts/tool to make a bootable device/image by inject bootloader via zipl
  • In longer term (not in this issues scope)
    • We ask/cooperate with mkosi and zipl to make the bootable s390x image native in mkosi

Mean while, I noticed that the binaries built via fedora builder has only x86, so we need build s390x and maybe other arches binaries via fedora also in this issue.

So, the proposed approach in this issue is:

  • Build multi-arch PeerPod binaries via fedora builder
    • Either build s390x PeerPod binaries on an x86 fedora builder via gcc/rust cross platform build
    • Either build s390x PeerPod binaries on an simulated s390x fedora builder via qemu and docker buildx
    • Or, build s390x PeerPod binaries on a s390x fedora builder on s390x host
  • Use mkosi ToolsTree=default to download necessary packages to build fedora PeerPod image and use it to replace nix
  • Build kernel bootable fedora s390x PeerPod image
  • Add a simple tool/scripts to make a bootable Fedora s390x device/image
  • Considering the multi-arch framework to include other arches also.
@huoqifeng
Copy link
Author

huoqifeng pushed a commit to huoqifeng/cloud-api-adaptor that referenced this issue Jan 5, 2024
Fixes: confidential-containers#1640

Signed-off-by: Qi Feng Huo <huoqif@cn.ibm.com>
huoqifeng pushed a commit to huoqifeng/cloud-api-adaptor that referenced this issue Jan 9, 2024
Fixes: confidential-containers#1640

Signed-off-by: Qi Feng Huo <huoqif@cn.ibm.com>
huoqifeng pushed a commit to huoqifeng/cloud-api-adaptor that referenced this issue Jan 9, 2024
Fixes: confidential-containers#1640

Signed-off-by: Qi Feng Huo <huoqif@cn.ibm.com>
huoqifeng pushed a commit to huoqifeng/cloud-api-adaptor that referenced this issue Jan 17, 2024
Fixes: confidential-containers#1640

Signed-off-by: Qi Feng Huo <huoqif@cn.ibm.com>
huoqifeng pushed a commit to huoqifeng/cloud-api-adaptor that referenced this issue Jan 17, 2024
Fixes: confidential-containers#1640

Signed-off-by: Qi Feng Huo <huoqif@cn.ibm.com>
@huoqifeng
Copy link
Author

huoqifeng commented Jan 17, 2024

I'm facing an error when build s390x PeerPod binaries on fedora 39. Which seems related with cross platform compiler:

checking for s390x-gcc... s390x-linux-gnu-gcc
checking whether the C compiler works... no
configure: error: in `/tmp/build-libseccomp.gPHpT/libseccomp-2.5.4':
configure: error: C compiler cannot create executables
See `config.log' for more details
make: *** [Makefile.inc:138: /src/cloud-api-adaptor/podvm/staticlib/kata-libseccomp/lib/libseccomp.a] Error 77

Whole logs looks like:

libseccomp-config-bad.txt

@bpradipt @wainersm @jensfr not sure whether you met this on RHEL or Fedora before?
Code I'm using is: huoqifeng@948f48d

@huoqifeng
Copy link
Author

huoqifeng commented Jan 17, 2024

After run dnf install -y gcc-s390x-linux-gnu again in the build container (it's weird why need install 2nd time as its installed already via huoqifeng@948f48d), new error is:

configure:3187: s390x-linux-gnu-gcc -V >&5
s390x-linux-gnu-gcc: error: unrecognized command-line option '-V'
s390x-linux-gnu-gcc: fatal error: no input files
compilation terminated.
configure:3198: $? = 1
configure:3187: s390x-linux-gnu-gcc -qversion >&5
s390x-linux-gnu-gcc: error: unrecognized command-line option '-qversion'; did you mean '--version'?
s390x-linux-gnu-gcc: fatal error: no input files
compilation terminated.
configure:3198: $? = 1
configure:3218: checking whether the C compiler works
configure:3240: s390x-linux-gnu-gcc -O2   conftest.c  >&5
/usr/bin/s390x-linux-gnu-ld: cannot find crt1.o: No such file or directory
/usr/bin/s390x-linux-gnu-ld: cannot find crti.o: No such file or directory
/usr/bin/s390x-linux-gnu-ld: cannot find -lc: No such file or directory
/usr/bin/s390x-linux-gnu-ld: cannot find crtn.o: No such file or directory
collect2: error: ld returned 1 exit status

Full log:
libseccomp-config-bad2.txt

@huoqifeng
Copy link
Author

This is quite a similar bug https://bugzilla.redhat.com/show_bug.cgi?id=1622375 although not same exactly.

@huoqifeng
Copy link
Author

Sounds like ubuntu is the good choice for cross platform build but not for all other distros: dotnet/source-build#2839 (comment)

huoqifeng pushed a commit to huoqifeng/cloud-api-adaptor that referenced this issue Jan 18, 2024
Fixes: confidential-containers#1640

Signed-off-by: Qi Feng Huo <huoqif@cn.ibm.com>
huoqifeng pushed a commit to huoqifeng/cloud-api-adaptor that referenced this issue Jan 18, 2024
Fixes: confidential-containers#1640

Signed-off-by: Qi Feng Huo <huoqif@cn.ibm.com>
huoqifeng pushed a commit to huoqifeng/cloud-api-adaptor that referenced this issue Jan 18, 2024
Fixes: confidential-containers#1640

Signed-off-by: Qi Feng Huo <huoqif@cn.ibm.com>
huoqifeng pushed a commit to huoqifeng/cloud-api-adaptor that referenced this issue Jan 19, 2024
Fixes: confidential-containers#1640

Signed-off-by: Qi Feng Huo <huoqif@cn.ibm.com>
huoqifeng pushed a commit to huoqifeng/cloud-api-adaptor that referenced this issue Jan 19, 2024
Fixes: confidential-containers#1640

Signed-off-by: Qi Feng Huo <huoqif@cn.ibm.com>
huoqifeng pushed a commit to huoqifeng/cloud-api-adaptor that referenced this issue Jan 24, 2024
Fixes: confidential-containers#1640

Signed-off-by: Qi Feng Huo <huoqif@cn.ibm.com>
huoqifeng pushed a commit to huoqifeng/cloud-api-adaptor that referenced this issue Jan 26, 2024
Fixes: confidential-containers#1640

Signed-off-by: Qi Feng Huo <huoqif@cn.ibm.com>
huoqifeng pushed a commit to huoqifeng/cloud-api-adaptor that referenced this issue Jan 26, 2024
Fixes: confidential-containers#1640

Signed-off-by: Qi Feng Huo <huoqif@cn.ibm.com>
huoqifeng pushed a commit to huoqifeng/cloud-api-adaptor that referenced this issue Jan 26, 2024
Fixes: confidential-containers#1640

Signed-off-by: Qi Feng Huo <huoqif@cn.ibm.com>
huoqifeng pushed a commit to huoqifeng/cloud-api-adaptor that referenced this issue Jan 26, 2024
Fixes: confidential-containers#1640

Signed-off-by: Qi Feng Huo <huoqif@cn.ibm.com>
huoqifeng pushed a commit to huoqifeng/cloud-api-adaptor that referenced this issue Jan 26, 2024
Fixes: confidential-containers#1640

Signed-off-by: Qi Feng Huo <huoqif@cn.ibm.com>
huoqifeng pushed a commit to huoqifeng/cloud-api-adaptor that referenced this issue Jan 29, 2024
Fixes: confidential-containers#1640

Signed-off-by: Qi Feng Huo <huoqif@cn.ibm.com>
huoqifeng pushed a commit that referenced this issue Jan 29, 2024
Fixes: #1640

Signed-off-by: Qi Feng Huo <huoqif@cn.ibm.com>
huoqifeng pushed a commit that referenced this issue Jan 29, 2024
Fixes: #1640

Signed-off-by: Qi Feng Huo <huoqif@cn.ibm.com>
@huoqifeng huoqifeng reopened this Jan 29, 2024
huoqifeng pushed a commit to huoqifeng/cloud-api-adaptor that referenced this issue Feb 19, 2024
Fixes: confidential-containers#1640

Signed-off-by: Qi Feng Huo <huoqif@cn.ibm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
1 participant