[s2n] build failure on macOS

[Mike4Online]

Host Environment

• OS: macOS 10.14 Mojave
• Compiler: AppleClang 11.0.0.11000033

To Reproduce
Steps to reproduce the behavior:
./vcpkg install aws-sdk-cpp[s3]:x64-osx --no-binary-caching

Failure logs
Build fails while building the s2n library, claiming LibCrypo cannot be found

Additional context
A work-around is to manually build OpenSSL 1.1.1 first, as follows:
./vcpkg install openssl[core]:x64-osx --no-binary-caching
./vcpkg install aws-sdk-cpp[s3]:x64-osx --no-binary-caching

I also found that curl was rejecting the SSL certificate of the server providing the OpenSSL source. I had to create a text file ~/.curlrc and add the following line to the file:

--insecure

in order for curl to successfully download the OpenSSL source. I am not sure why this was necessary. I tried installing macOS security updates for macOS Mojave, and also tried updating curl with HomeBrew, but I still needed to set --insecure in ~/.curlrc for vcpkg to build OpenSSL.

[JackBoosY]

Can you please provide the failure logs first?

[JackBoosY]

What's the difference between this and #21445?

[Mike4Online]

What's the difference between this and #21445?

This issue is primarily about OpenSSL failing to automatically build as a dependency for aws-sdk-cpp.

Issue #21445 is about the aws-sdk-cpp package failing to fully honor the MACOSX_DEPLOYMENT_TARGET environment variable, specifically MACOSX_DEPLOYMENT_TARGET=10.9 (but could be any value less that 10.13).

It is possible both issues are related to building this package on macOS 10.14 Mojave, rather than a newer macOS version such as Big Sur or Monterey. But I assume vcpkg is still supported on macOS Mojave.

I will upload a failure log for this issue tomorrow.

[dg0yt]

This issue is primarily about OpenSSL failing to automatically build as a dependency for aws-sdk-cpp.

What do you mean by that? Port aws-sdk-cpp is to use port openssl from vcpkg.

aws-sdk-cpp in vcpkg is not up-to-date, unfortunately. The version used here has a number of cmake issues. For example, it calls include(FindOpenSSL) instead of find_package(OpenSSL). This prevents vcpkg cmake wrapper taking effect.

So the first step needed is updating the port to a more recent version.

[dg0yt]

A work-around is to manually build OpenSSL 1.1.1 first, as follows:
./vcpkg install openssl[core]:x64-osx --no-binary-caching
./vcpkg install aws-sdk-cpp[s3]:x64-osx --no-binary-caching

What is suggested to do manually is already enforced by dependencies:

$ ./vcpkg depend-info aws-sdk-cpp[core,s3]:x64-osx --max-recurse=0
aws-sdk-cpp[s3]: aws-crt-cpp, curl, openssl, vcpkg-cmake, vcpkg-cmake-config, zlib

[Mike4Online]

What I mean is that building
./vcpkg install aws-sdk-cpp[s3]:x64-osx
will fail, but when i build
./vcpkg install openssl[core]:x64-osx
and then build
./vcpkg install aws-sdk-cpp[s3]:x64-osx
then they both succeed.

It should not be necessary to build OpenSSL first, because OpenSSL is one of the dependencies of aws-sdk-cpp, as you mentioned, but for me it is necessary to build OpenSSL first. So I assume something must be wrong with the aws-sdk-cpp port. The s2n library (another aws-sdk-cpp dependency) builds before OpenSSL, but s2n needs LibCrypto from OpenSSL, so s2n fails. So perhaps the build order of the dependencies is wrong.

I am doing these builds on macOS Mojave, in a vcpkg folder that is freshly prepared (git clone followed by running the bootstrap.sh script).

[dg0yt]

What I mean is that building ./vcpkg install aws-sdk-cpp[s3]:x64-osx will fail, but when i build ./vcpkg install openssl[core]:x64-osx and then build ./vcpkg install aws-sdk-cpp[s3]:x64-osx then they both succeed.

It should not be necessary to build OpenSSL first, because OpenSSL is one of the dependencies of aws-sdk-cpp, as you mentioned, but for me it is necessary to build OpenSSL first. So I assume something must be wrong with the aws-sdk-cpp port. The s2n library (another aws-sdk-cpp dependency) builds before OpenSSL, but s2n needs LibCrypto from OpenSSL, so s2n fails. So perhaps the build order of the dependencies is wrong.

Well:

$ ./vcpkg depend-info s2n:x64-osx --max-recurse=1vcpkg-cmake: 
vcpkg-cmake-config: 
s2n: vcpkg-cmake, vcpkg-cmake-config

So s2n doesn't declare an openssl dependency.

Failure logs
Build fails while building the s2n library, claiming LibCrypo cannot be found

In that case, you should have chosen a different title for the issue, even if aws-sdk-cpp is the ultimate goal. (And I assume that this was written in the instruction in the console output when the s2n port failed.)
Would you mind updating the title now?

[Mike4Online]

s2n doesn't declare an openssl dependency.

Perhaps s2n should declare a dependency on openssl. I am uploading vcpkg console output and an error log to this comment. These logs were generated by attempting to build aws-sdk-cpp. The logs show that the build of s2n that is triggered by building aws-sdk-cpp fails due to LibCrypto being missing. So s2n does depend on LibCrypto (which is one of two OpenSSL libraries).

I will change the title to reference the s2n port rather than the aws-sdk-cpp port.

aws-sdk-cpp_console_output.txt
s2n_config-x64-osx-dbg-err.log

.

[JackBoosY]

Could NOT find LibCrypto (missing: LibCrypto_LIBRARY LibCrypto_INCLUDE_DIR)

Can you pleae remove openssl and retry again?

[Mike4Online]

I deleted my entire vcpkg root folder, then grabbed it fresh with git clone, ran the bootstrap.sh script, and tried installing with

./vcpkg install aws-sdk-cpp[s3]:x64-osx --no-binarycaching

I did this two minutes after receiving an email with your latest comment. I got the same error when it got to building s2n:

MACBUILD2:vcpkg michael$ cat /Users/michael/vcpkg/buildtrees/s2n/config-x64-osx-dbg-err.log
CMake Error at /usr/local/Cellar/cmake/3.21.3_1/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
  Could NOT find LibCrypto (missing: LibCrypto_LIBRARY LibCrypto_INCLUDE_DIR)
Call Stack (most recent call first):
  /usr/local/Cellar/cmake/3.21.3_1/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
  cmake/modules/FindLibCrypto.cmake:71 (find_package_handle_standard_args)
  /Users/michael/vcpkg/scripts/buildsystems/vcpkg.cmake:788 (_find_package)
  CMakeLists.txt:389 (find_package)

[Mike4Online]

Note in my previous comment, in the error text, that the path

/usr/local/Cellar/cmake

is where HomeBrew installed its copy of CMake.

[JackBoosY]

It seems that aws-sdk-cpp[s3] finds the openssl in system side.

[Mike4Online]

Perhaps the aws-sdk-cpp port behaves improperly only when CMake has been installed via HomeBrew.

Let me know if you'd like me to rename this issue back to referencing the [aws-sdk-cpp] port, rather than the s2n port.

As for the other issue, #21445, it is also an issue with the aws-sdk-cpp port, but it is unrelated to this issue.

[dg0yt]

I can reproduce the s2n issue on osx:

    See logs for more information:
      /Users/dg0yt/dev/vcpkg/buildtrees/s2n/config-x64-osx-dbg-out.log
      /Users/dg0yt/dev/vcpkg/buildtrees/s2n/config-x64-osx-dbg-err.log
...
Error: Building package s2n:x64-osx failed with: BUILD_FAILED
Please ensure you're using the latest portfiles with `git pull` and `./vcpkg update`, then
submit an issue at https://github.com/Microsoft/vcpkg/issues including:
  package: s2n[core]:x64-osx -> 1.1.1
  vcpkg version: 2021-11-02-af04ebf6274fd6f7a941bff4662b3955c64f6f42
  vcpkg-tool version: 03f3d5262 2021-11-16 (10 hours ago)

Additionally, attach any relevant sections from the log files above.

@Mike4Online This is the output of vcpkg tool. Having the title right helps to focus attention - and it is also helpful for other users which may have a different ultimate goal but also stumble over s2n.

[Mike4Online]

Glad you can reproduce the issue. Looking forward to a corrected s2n port. Meanwhile, can do the work-around I have documented, above.

I'll leave the Title of this issue as it currently is. If you need me to edit it, let me know and I'll do so.