Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Setting QT_HOST_PATH is required for native build of qt6-main downstream projecs on osx-arm64 #273

Open
1 task done
traversaro opened this issue Jun 24, 2024 · 15 comments
Labels
bug Something isn't working

Comments

@traversaro
Copy link
Contributor

Solution to issue cannot be found in the documentation.

  • I checked the documentation.

Issue

It seems that find_package(Qt6 REQUIRED) fails on osx-arm64 (even on native builds) with the error:

-- Detecting CXX compile features - done
CMake Error at /Users/runner/miniconda3/envs/test/lib/cmake/Qt6/QtPublicDependencyHelpers.cmake:258 (message):
-- Configuring incomplete, errors occurred!
  To use a cross-compiled Qt, please set the QT_HOST_PATH cache variable to
  the location of your host Qt installation.
Call Stack (most recent call first):
  /Users/runner/miniconda3/envs/test/lib/cmake/Qt6/Qt6Dependencies.cmake:10 (_qt_internal_setup_qt_host_path)
  /Users/runner/miniconda3/envs/test/lib/cmake/Qt6/Qt6Config.cmake:124 (include)
  CMakeLists.txt:5 (find_package)


CMake Error at CMakeLists.txt:5 (find_package):
  Found package configuration file:

    /Users/runner/miniconda3/envs/test/lib/cmake/Qt6/Qt6Config.cmake

  but it set Qt6_FOUND to FALSE so package "Qt6" is considered to be NOT
  FOUND.

This was just triggered by the following CMakeLists.txt

cmake_minimum_required(VERSION 3.12...3.29)

project(CMakePackageCheckTestCMakeProject)

find_package(Qt6 REQUIRED)

or you can easily trigger this by the cmake-package-check script with:

cmake-package-check Qt6

See https://github.com/traversaro/debug-find-package-qt6-conda-failure-apple-silicon/actions/runs/9649808633/job/26614067008 for a simple reproducer (and a check that this works fine in osx-64).

It seems that somehow Qt6 wants you to manually set QT_HOST_PATH if Qt6 was cross-compiled, but I am not sure the rationale of this. Probably the hidden assumption is that they assume that a cross-compiled Qt6 will be consumed in cross-compilation builds?

Indeed, I saw that error a lot in conda-forge CI cross-compiled builds, and it was nice, but for regular native builds this is quite unexpected (and it happens also when Qt6 is included only transitively, I originally got it in a project that was doing find_package(PCL)).

Installed packages

# packages in environment at /Users/runner/miniconda3/envs/test:
#
# Name                    Version                   Build  Channel
bzip2                     1.0.8                h93a5062_5    conda-forge
c-ares                    1.28.1               h93a5062_0    conda-forge
c-compiler                1.7.0                h6aa9301_1    conda-forge
ca-certificates           2024.6.2             hf0a4a13_0    conda-forge
cairo                     1.18.0               hc6c324b_2    conda-forge
cctools                   986                  h4faf515_0    conda-forge
cctools_osx-arm64         986                  h62378fb_0    conda-forge
clang                     16.0.6          default_h095aff0_8    conda-forge
clang-16                  16.0.6          default_hb63da90_8    conda-forge
clang_impl_osx-arm64      16.0.6              hc421ffc_16    conda-forge
clang_osx-arm64           16.0.6              h54d7cd3_16    conda-forge
clangxx                   16.0.6          default_h095aff0_8    conda-forge
clangxx_impl_osx-arm64    16.0.6              hcd7bac0_16    conda-forge
clangxx_osx-arm64         16.0.6              h54d7cd3_16    conda-forge
cmake                     3.29.6               had79d8f_0    conda-forge
cmake-package-check       0.0.4              pyh4af843d_0    conda-forge
compiler-rt               16.0.6               h3808999_2    conda-forge
compiler-rt_osx-arm64     16.0.6               h3808999_2    conda-forge
compilers                 1.7.0                hce30654_1    conda-forge
cxx-compiler              1.7.0                h2ffa867_1    conda-forge
double-conversion         3.3.0                h13dd4ca_0    conda-forge
expat                     2.6.2                hebf3989_0    conda-forge
font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
font-ttf-inconsolata      3.000                h77eed37_0    conda-forge
font-ttf-source-code-pro  2.038                h77eed37_0    conda-forge
font-ttf-ubuntu           0.83                 h77eed37_2    conda-forge
fontconfig                2.14.2               h82840c6_0    conda-forge
fonts-conda-ecosystem     1                             0    conda-forge
fonts-conda-forge         1                             0    conda-forge
fortran-compiler          1.7.0                hafb19e3_1    conda-forge
freetype                  2.12.1               hadb7bae_2    conda-forge
gfortran                  12.3.0               h1ca8e4b_1    conda-forge
gfortran_impl_osx-arm64   12.3.0               h53ed385_3    conda-forge
gfortran_osx-arm64        12.3.0               h57527a5_1    conda-forge
gmp                       6.3.0                h7bae524_2    conda-forge
graphite2                 1.3.13            hebf3989_1003    conda-forge
harfbuzz                  8.5.0                h1836168_0    conda-forge
icu                       73.2                 hc8870d7_0    conda-forge
isl                       0.26            imath32_h347afa1_101    conda-forge
jinja2                    3.1.4              pyhd8ed1ab_0    conda-forge
krb5                      1.21.2               h92f50d5_0    conda-forge
ld64                      711                  h634c8be_0    conda-forge
ld64_osx-arm64            711                  ha4bd21c_0    conda-forge
lerc                      4.0.0                h9a09cb3_0    conda-forge
libclang-cpp16            16.0.6          default_hb63da90_8    conda-forge
libclang13                18.1.8          default_hfc66aa2_0    conda-forge
libcurl                   8.8.0                h7b6f9a7_0    conda-forge
libcxx                    17.0.6               he7857fb_1    conda-forge
libdeflate                1.20                 h93a5062_0    conda-forge
libedit                   3.1.20191231         hc8eb9b7_2    conda-forge
libev                     4.33                 h93a5062_2    conda-forge
libexpat                  2.6.2                hebf3989_0    conda-forge
libffi                    3.4.2                h3422bc3_5    conda-forge
libgfortran               5.0.0           13_2_0_hd922786_3    conda-forge
libgfortran-devel_osx-arm64 12.3.0               hc62be1c_3    conda-forge
libgfortran5              13.2.0               hf226fd6_3    conda-forge
libglib                   2.80.2               h535f939_0    conda-forge
libiconv                  1.17                 h0d3ecfb_2    conda-forge
libintl                   0.22.5               h8fbad5d_2    conda-forge
libjpeg-turbo             3.0.0                hb547adb_1    conda-forge
libllvm16                 16.0.6               haab561b_3    conda-forge
libllvm18                 18.1.8               h5090b49_0    conda-forge
libnghttp2                1.58.0               ha4dd798_1    conda-forge
libpng                    1.6.43               h091b4b1_0    conda-forge
libpq                     16.3                 h7afe498_0    conda-forge
libsqlite                 3.46.0               hfb93653_0    conda-forge
libssh2                   1.11.0               h7a5bd25_0    conda-forge
libtiff                   4.6.0                h07db509_3    conda-forge
libuv                     1.48.0               h93a5062_0    conda-forge
libwebp-base              1.4.0                h93a5062_0    conda-forge
libxml2                   2.12.7               ha661575_1    conda-forge
libzlib                   1.3.1                hfb2fe0b_1    conda-forge
llvm-openmp               18.1.8               hde57baf_0    conda-forge
llvm-tools                16.0.6               haab561b_3    conda-forge
markupsafe                2.1.5           py312he37b823_0    conda-forge
mpc                       1.3.1                h91ba8db_0    conda-forge
mpfr                      4.2.1                h41d338b_1    conda-forge
mysql-common              8.3.0                hd1853d3_4    conda-forge
mysql-libs                8.3.0                hf036fc4_4    conda-forge
ncurses                   6.5                  hb89a1cb_0    conda-forge
ninja                     1.12.1               h420ef59_0    conda-forge
openssl                   3.3.1                hfb2fe0b_0    conda-forge
pcre2                     10.43                h26f9a81_0    conda-forge
pip                       24.0               pyhd8ed1ab_0    conda-forge
pixman                    0.43.4               hebf3989_0    conda-forge
pkg-config                0.29.2            hab62308_1008    conda-forge
python                    3.12.4          h30c5eda_0_cpython    conda-forge
python_abi                3.12                    4_cp312    conda-forge
qt6-main                  6.7.2                hef23ce4_1    conda-forge
readline                  8.2                  h92ec313_1    conda-forge
rhash                     1.4.4                hb547adb_0    conda-forge
setuptools                70.1.0             pyhd8ed1ab_0    conda-forge
sigtool                   0.1.3                h44b9a77_0    conda-forge
tapi                      1100.0.11            he4954df_0    conda-forge
tk                        8.6.13               h5083fa2_1    conda-forge
tzdata                    2024a                h0c530f3_0    conda-forge
wheel                     0.43.0             pyhd8ed1ab_1    conda-forge
xz                        5.2.6                h57fd34a_0    conda-forge
zlib                      1.3.1                hfb2fe0b_1    conda-forge
zstd                      1.5.6                hb46c0d2_0    conda-forge

Environment info

active environment : test
    active env location : /Users/runner/miniconda3/envs/test
            shell level : 2
       user config file : /Users/runner/.condarc
 populated config files : /Users/runner/miniconda3/.condarc
                          /Users/runner/.condarc
          conda version : 24.3.0
    conda-build version : not installed
         python version : 3.10.14.final.0
                 solver : libmamba (default)
       virtual packages : __archspec=1=m1
                          __conda=24.3.0=0
                          __osx=14.5=0
                          __unix=0=0
       base environment : /Users/runner/miniconda3  (writable)
      conda av data dir : /Users/runner/miniconda3/etc/conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/osx-arm64
                          https://conda.anaconda.org/conda-forge/noarch
          package cache : /Users/runner/conda_pkgs_dir
       envs directories : /Users/runner/miniconda3/envs
                          /Users/runner/.conda/envs
               platform : osx-arm64
             user-agent : conda/24.3.0 requests/2.31.0 CPython/3.10.14 Darwin/23.5.0 OSX/14.5 solver/libmamba conda-libmamba-solver/24.1.0 libmambapy/1.5.8
                UID:GID : 501:20
             netrc file : None
           offline mode : False
@traversaro traversaro added the bug Something isn't working label Jun 24, 2024
@traversaro
Copy link
Contributor Author

traversaro commented Jun 24, 2024

It seems that somehow Qt6 wants you to manually set QT_HOST_PATH if Qt6 was cross-compiled, but I am not sure the rationale of this. Probably the hidden assumption is that they assume that a cross-compiled Qt6 will be consumed in cross-compilation builds?

That is exactly the case, see https://github.com/qt/qtbase/blob/53e44e753b7cfc5c873ef5a1fc577cfb6308d9d0/cmake/QtPublicDependencyHelpers.cmake#L236-L241 :

    # If a QT_HOST_PATH is provided when configuring qtbase, we assume it's a cross build
    # and thus we require the QT_HOST_PATH to be provided also when using the cross-built Qt.

Indeed, it seems that the QT_REQUIRE_HOST_PATH_CHECK option was explicitly added to address our use case: qt/qtbase@b04a0f0 :

Another use case is building a desktop Qt using the host tools of
another desktop Qt. The new desktop Qt can use its newly built tools
just fine and wouldn't need the original desktop Qt tools.

@traversaro
Copy link
Contributor Author

I think that the fix boils down to pass -DQT_REQUIRE_HOST_PATH_CHECK=OFF to the build. @hmaarrfk let me know if you are ok in me opening the PR or you want to include this change in some other PR (to avoid overloading the CI).

@hmaarrfk
Copy link
Contributor

I think that the fix boils down to pass -DQT_REQUIRE_HOST_PATH_CHECK=OFF to the build. @hmaarrfk let me know if you are ok in me opening the PR or you want to include this change in some other PR (to avoid overloading the CI).

Is this fix to be done at the qt-main level?

I think jschuler is better at this stuff than me. i would wait for their response.

@traversaro
Copy link
Contributor Author

I think that the fix boils down to pass -DQT_REQUIRE_HOST_PATH_CHECK=OFF to the build. @hmaarrfk let me know if you are ok in me opening the PR or you want to include this change in some other PR (to avoid overloading the CI).

Is this fix to be done at the qt-main level?

From what I understand yes.

I think jschuler is better at this stuff than me. i would wait for their response.

Ok, thanks! I mentioned you by checking just the latest PRs merged

@hmaarrfk
Copy link
Contributor

if its a 1 liner, you might as well make the PR.

Qt6 builds on the CIs so its pretty easy....

@traversaro
Copy link
Contributor Author

Great, done: #274 .

@jschueller
Copy link
Contributor

actually on conda-forge the osx-arm64 target is cross-compiled from osx-64 platform, so imho nothing wrong here

@traversaro
Copy link
Contributor Author

traversaro commented Jun 25, 2024

actually on conda-forge the osx-arm64 target is cross-compiled from osx-64 platform, so imho nothing wrong here

Yes, but why there should be a difference in consuming a cross-compiled library as opposed to consume a native build library as part of a native build of a downstream package? In a nutshell, why should find_package(Qt6 REQUIRED) succeed out of the box in native builds on osx-64 while fail on native builds in osx-arm64 .

@jschueller
Copy link
Contributor

now what happens if we omit QT_HOST_PATH when building a package that depends on qt6 on conda-forge forge for osx-arm64 (which is still cross-compiling) ?

@traversaro
Copy link
Contributor Author

Ok, let me try to recap a bit here, I think there are two main use cases (UC)

  • UC1: You build via native build the library lib-that-uses-qt that uses (even transitively) a cross-compiled build of qt6-main : in this case, I do not want to need to explicitly set QT_HOST_PATH, while at the moment it is compulsory
  • UC2: You build via cross-compilation (typical case in conda-forge CI scripts) the library lib-that-uses-qt that uses a cross-compiled build of qt6-main: at the moment you get a clear error if you do not set QT_HOST_NAME, while with the PR Pass -DQT_REQUIRE_HOST_PATH_CHECK=OFF in cross-compiled builds #274 you would not get a clear error anymore, and you probably get a "bad architecture" error if you actually use the tools (this is just an hypothesis, I did not actually tried) as you typically do for most packages in conda-forge that install compilation tools.

If I understood your concern, you want to make sure that in UC2 we still get the nice QT_HOST_PATH in cross-compiled builds, right?

If I got this correctly, I think the main problem is the logic used in qt to understand if a given build is done using cross-compilation: they actually use "qt6 has been build via cross-compilation" as a proxy for "any library that is using qt6 is being build via cross-compilation", that is clearly wrong in general in the conda context. Would it be ok for you to change the logic to detect if a project is being cross-compiled in some other way? For example by checking the CMAKE_CROSSCOMPILING variable?

@jschueller
Copy link
Contributor

ok, lets merge #274

@traversaro
Copy link
Contributor Author

ok, lets merge #274

Great, thanks!

@traversaro
Copy link
Contributor Author

My bad, in #273 (comment) I actually I had a big confusion between two different variables :

traversaro added a commit to traversaro/qt-main-feedstock that referenced this issue Jul 2, 2024
instead of QT_REQUIRE_HOST_PATH_CHECK

QT_NO_REQUIRE_HOST_PATH_CHECK is the variable that we need to set in qt build system, see conda-forge#273 (comment) for details
@traversaro
Copy link
Contributor Author

traversaro commented Jul 8, 2024

I think I underestimated the situation (one thing that make is complex is that the same code CMake code runs for qt build itself and for downstream build of projects that build against qt). At the moment I do not have a lot of time to look into this, so I propose:

@traversaro
Copy link
Contributor Author

Re-open the issue, as anyhow the problem is present and it may be useful for people experiencing the problem to have an issue that explains the workaround

@conda-forge/qt-main if you agree feel free to re-open the issue (I do not have the required permissions)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants