Windows build failed if the build path location is too long

[stormshield-gt]

Problem:

On every Windows target, the project doesn't compile if the path to the build directory is too long, because of the Windows maximum path length limitation.
To reproduce you have to build inside several nested folders, so the path will be greater than 260 characters.

The current workaround is to set the build directory closer to the root of the file system, but this is not easily discoverable and reliable enough.

Relevant details

AWS-LC commit: (47333e1) (through aws-lc-sys 0.20.0)

System information:

• CPU architecture: x86-64
• OS: Windows 10

Build log

 -- Selecting Windows SDK version 10.0.22621.0 to target Windows 10.0.19045.
  -- The C compiler identification is MSVC 19.40.33813.0
  -- Detecting C compiler ABI info
  -- Detecting C compiler ABI info - failed
  -- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2022/BuildTools/VC/Tools/MSVC/14.40.33807/bin/Hostx64/x64/cl.exe
  -- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2022/BuildTools/VC/Tools/MSVC/14.40.33807/bin/Hostx64/x64/cl.exe - broken
  -- Configuring incomplete, errors occurred!

  --- stderr
  CMake Deprecation Warning at CMakeLists.txt:4 (cmake_minimum_required):
    Compatibility with CMake < 3.5 will be removed from a future version of
    CMake.

    Update the VERSION argument <min> value or use a ...<max> suffix to tell
    CMake that the project does not need compatibility with older versions.


  CMake Error at C:/Program Files/CMake/share/cmake-3.30/Modules/CMakeTestCCompiler.cmake:67 (message):
    The C compiler

      "C:/Program Files (x86)/Microsoft Visual Studio/2022/BuildTools/VC/Tools/MSVC/14.40.33807/bin/Hostx64/x64/cl.exe"

    is not able to compile a simple test program.

    It fails with the following output:

      Change Dir: 'C:/Users/stormshield/Documents/Git/my-project/build/target/target-x86_64-pc-windows-msvc/x86_64-pc-windows-msvc/debug/build/aws-lc-sys-7068532a0b4ccf5c/out/build/CMakeFiles/C
MakeScratch/TryCompile-an6xjf'

      Run Build Command(s): "C:/Program Files (x86)/Microsoft Visual Studio/2022/BuildTools/MSBuild/Current/Bin/amd64/MSBuild.exe" cmTC_8400d.vcxproj /p:Configuration=Debug /p:Platform=x64 /
p:VisualStudioVersion=17.0 /v:n
      MSBuild version 17.10.4+10fbfbf2e for .NET Framework
      Build started 8/1/2024 10:06:36 AM.

      Project "C:\Users\stormshield\Documents\Git\my-project\build\target\target-x86_64-pc-windows-msvc\x86_64-pc-windows-msvc\debug\build\aws-lc-sys-7068532a0b4ccf5c\out\build\CMakeFiles\CMake
Scratch\TryCompile-an6xjf\cmTC_8400d.vcxproj" on node 1 (default targets).
      PrepareForBuild:
        Creating directory "cmTC_8400d.dir\Debug\".
        Structured output is enabled. The formatting of compiler diagnostics will reflect the error hierarchy. See https://aka.ms/cpp/structured-output for more details.
        Creating directory "C:\Users\stormshield\Documents\Git\my-project\build\target\target-x86_64-pc-windows-msvc\x86_64-pc-windows-msvc\debug\build\aws-lc-sys-7068532a0b4ccf5c\out\build\CMa
keFiles\CMakeScratch\TryCompile-an6xjf\Debug\".
        Creating directory "cmTC_8400d.dir\Debug\cmTC_8400d.tlog\".
      InitializeBuildStatus:
        Creating "cmTC_8400d.dir\Debug\cmTC_8400d.tlog\unsuccessfulbuild" because "AlwaysCreate" was specified.
        Touching "cmTC_8400d.dir\Debug\cmTC_8400d.tlog\unsuccessfulbuild".
      ClCompile:
        C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.40.33807\bin\HostX64\x64\CL.exe /c /Zi /nologo /W1 /WX- /diagnostics:column /Od /Ob0 /D _MBCS /D "CMAK
E_INTDIR=\"Debug\"" /RTC1 /MDd /GS /fp:precise /Zc:wchar_t /Zc:forScope /Zc:inline /Fo"cmTC_8400d.dir\Debug\\" /Fd"cmTC_8400d.dir\Debug\vc143.pdb" /external:W1 /Gd /TC /errorReport:queue  -B
repro "C:\Users\stormshield\Documents\Git\my-project\build\target\target-x86_64-pc-windows-msvc\x86_64-pc-windows-msvc\debug\build\aws-lc-sys-7068532a0b4ccf5c\out\build\CMakeFiles\CMakeScratch\
TryCompile-an6xjf\testCCompiler.c"
        testCCompiler.c
      Link:
        C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.40.33807\bin\HostX64\x64\link.exe /ERRORREPORT:QUEUE /OUT:"C:\Users\stormshield\Documents\Git\my-project\
build\target\target-x86_64-pc-windows-msvc\x86_64-pc-windows-msvc\debug\build\aws-lc-sys-7068532a0b4ccf5c\out\build\CMakeFiles\CMakeScratch\TryCompile-an6xjf\Debug\cmTC_8400d.exe" /INCREMENT
AL /ILK:"cmTC_8400d.dir\Debug\cmTC_8400d.ilk" /NOLOGO kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib /MANIFEST /MANIFEST
UAC:"level='asInvoker' uiAccess='false'" /manifest:embed /DEBUG /PDB:"C:/Users/stormshield/Documents/Git/my-project/build/target/target-x86_64-pc-windows-msvc/x86_64-pc-windows-msvc/debug/build
/aws-lc-sys-7068532a0b4ccf5c/out/build/CMakeFiles/CMakeScratch/TryCompile-an6xjf/Debug/cmTC_8400d.pdb" /SUBSYSTEM:CONSOLE /TLBID:1 /DYNAMICBASE /NXCOMPAT /IMPLIB:"C:/Users/stormshield/Docume
nts/Git/my-project/build/target/target-x86_64-pc-windows-msvc/x86_64-pc-windows-msvc/debug/build/aws-lc-sys-7068532a0b4ccf5c/out/build/CMakeFiles/CMakeScratch/TryCompile-an6xjf/Debug/cmTC_8400d
.lib" /MACHINE:X64  /machine:x64 cmTC_8400d.dir\Debug\testCCompiler.obj
      FileTracker : error FTK1011: could not create the new file tracking log file: C:\Users\stormshield\Documents\Git\my-project\build\target\target-x86_64-pc-windows-msvc\x86_64-pc-windows-ms
vc\debug\build\aws-lc-sys-7068532a0b4ccf5c\out\build\CMakeFiles\CMakeScratch\TryCompile-an6xjf\cmTC_8400d.dir\Debug\cmTC_8400d.tlog\link-cvtres.read.1.tlog. The system cannot find the path s
pecified. [C:\Users\stormshield\Documents\Git\my-project\build\target\target-x86_64-pc-windows-msvc\x86_64-pc-windows-msvc\debug\build\aws-lc-sys-7068532a0b4ccf5c\out\build\CMakeFiles\CMakeScra
tch\TryCompile-an6xjf\cmTC_8400d.vcxproj]



      FileTracker : error FTK1011: could not create the new file tracking log file: C:\Users\stormshield\Documents\Git\my-project\build\target\target-x86_64-pc-windows-msvc\x86_64-pc-windows-ms
vc\debug\build\aws-lc-sys-7068532a0b4ccf5c\out\build\CMakeFiles\CMakeScratch\TryCompile-an6xjf\cmTC_8400d.dir\Debug\cmTC_8400d.tlog\link-cvtres.write.1.tlog. The system cannot find the path
specified. [C:\Users\stormshield\Documents\Git\my-project\build\target\target-x86_64-pc-windows-msvc\x86_64-pc-windows-msvc\debug\build\aws-lc-sys-7068532a0b4ccf5c\out\build\CMakeFiles\CMakeScr
atch\TryCompile-an6xjf\cmTC_8400d.vcxproj]



      Done Building Project "C:\Users\stormshield\Documents\Git\my-project\build\target\target-x86_64-pc-windows-msvc\x86_64-pc-windows-msvc\debug\build\aws-lc-sys-7068532a0b4ccf5c\out\build\CM
akeFiles\CMakeScratch\TryCompile-an6xjf\cmTC_8400d.vcxproj" (default targets) -- FAILED.

      Build FAILED.

      "C:\Users\stormshield\Documents\Git\my-project\build\target\target-x86_64-pc-windows-msvc\x86_64-pc-windows-msvc\debug\build\aws-lc-sys-7068532a0b4ccf5c\out\build\CMakeFiles\CMakeScratch\
TryCompile-an6xjf\cmTC_8400d.vcxproj" (default target) (1) ->
      (Link target) ->
        FileTracker : error FTK1011: could not create the new file tracking log file: C:\Users\stormshield\Documents\Git\my-project\build\target\target-x86_64-pc-windows-msvc\x86_64-pc-windows-
msvc\debug\build\aws-lc-sys-7068532a0b4ccf5c\out\build\CMakeFiles\CMakeScratch\TryCompile-an6xjf\cmTC_8400d.dir\Debug\cmTC_8400d.tlog\link-cvtres.read.1.tlog. The system cannot find the path
 specified. [C:\Users\stormshield\Documents\Git\my-project\build\target\target-x86_64-pc-windows-msvc\x86_64-pc-windows-msvc\debug\build\aws-lc-sys-7068532a0b4ccf5c\out\build\CMakeFiles\CMakeSc
ratch\TryCompile-an6xjf\cmTC_8400d.vcxproj]
        FileTracker : error FTK1011: could not create the new file tracking log file: C:\Users\stormshield\Documents\Git\my-project\build\target\target-x86_64-pc-windows-msvc\x86_64-pc-windows-
msvc\debug\build\aws-lc-sys-7068532a0b4ccf5c\out\build\CMakeFiles\CMakeScratch\TryCompile-an6xjf\cmTC_8400d.dir\Debug\cmTC_8400d.tlog\link-cvtres.write.1.tlog. The system cannot find the pat
h specified. [C:\Users\stormshield\Documents\Git\my-project\build\target\target-x86_64-pc-windows-msvc\x86_64-pc-windows-msvc\debug\build\aws-lc-sys-7068532a0b4ccf5c\out\build\CMakeFiles\CMakeS
cratch\TryCompile-an6xjf\cmTC_8400d.vcxproj]

          0 Warning(s)
          2 Error(s)

      Time Elapsed 00:00:00.79


    CMake will not be able to correctly generate this project.
  Call Stack (most recent call first):
    CMakeLists.txt:7 (enable_language)

From my understanding, they are several solutions to this problem:

• Use file system library that appends the prefix "\\?\" to every path to remove the limitation. For instance, this was added to the rust standard library in Automatically convert paths to verbatim for filesystem operations that support it rust-lang/rust#89174
• Enable a registry key on the developer's machine and add a windows manifest to aws-lc-rs, as described in Windows doc. CMake since version 3.4 know how to handle *.manifest files listed as source files

Please let me know what is your thought on this.

[stormshield-gt]

If I force the CMake Generator to be Ninja instead of CMake's Visual Studio generator, it compiles but I get a linker error

[stormshield-gt]

The linker error was due to the fact that the CRT was embedded both by my rust internal library and by aws-lc.
So compiling with ninja works for me. Sorry for the noise.
I guess there is still the problem of max path with the CMake's Visual Studio generator, but the docs suggest that it's not well tested so it's fine.
Maybe this should be reported to aws-lc-rs that use this generator by default on windows to not force the install of ninja ?

[andrewhop]

Thanks for the detailed write up.

I guess there is still the problem of max path with the CMake's Visual Studio generator, but the docs suggest that it's not well tested so it's fine.

Do you have a link for this documentation?

Maybe this should be reported to aws-lc-rs that use this generator by default on windows to not force the install of ninja ?

We updated our documentation in aws/aws-lc-rs#465 and the aws-lc-rs build only requires Ninja for the FIPS build.

[justsmth]

I did some googling on this. The "MSBuild" (i.e., CMake generator) part of the problem has been resolved, but other tooling around MSVC might still fail when path names exceed 260 chars. This link appears to be the relevant top-level tracking issue for Visual Studio.

[stormshield-gt]

Do you have a link for this documentation?

I was referring to this part of the documentation that mention that the CMake's Visual Studio generator is not tested regularly:

On Windows, CMake's Visual Studio generator may also work, but it not tested regularly and requires recent versions of CMake for assembly support.

I did some googling on this. The "MSBuild" (i.e., CMake generator) part of the problem has dotnet/msbuild#53 (comment), but other tooling around MSVC might still fail when path names exceed 260 chars. This link appears to be the relevant top-level tracking issue for Visual Studio.

Thanks a lot for digging this up! So it's an upstream problem that we cannot fix here. Note that might happen more frequently with aws-lc-rs, because how nested build artifacts are by default in the Rust target directory, creating long paths.

The workaround of using Ninja, by setting the environment variable $env:CMAKE_GENERATOR="Ninja" and make sure we have not embedded the CRT twice is fine for us. Feel free to close the issue. Thanks a lot for your help

Edit: For completion, overriding the CMake generator used by aws-lc-rs is possible because the cmake crate used in the build script support this natively. That's why it's possible to do so even if there is no logic for it inside the build script of the aws-lc-sys crate.