MacOS AppleClang 11 segfaults at commit ee70a23

[hcho3]

It appears that #5880 (commit ee70a23) has triggers a compiler bug in AppleClang 11.0.3:

FAILED: src/CMakeFiles/objxgboost.dir/tree/updater_quantile_hist.cc.o 
/Library/Developer/CommandLineTools/usr/bin/c++  -DDMLC_CORE_USE_CMAKE -DDMLC_LOG_CUSTOMIZE=1 -DDMLC_USE_CXX11=1 -DXGBOOST_BUILTIN_PREFETCH_PRESENT=1 -DXGBOOST_MM_PREFETCH_PRESENT=1 -D_MWAITXINTRIN_H_INCLUDED -D__USE_XOPEN2K8 -I../include -I../dmlc-core/include -I../rabit/include -Idmlc-core/include -I../rabit/../dmlc-core/include -isystem /usr/local/include -O3 -DNDEBUG -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -fPIC   -funroll-loops -Xclang -fopenmp -std=gnu++14 -MD -MT src/CMakeFiles/objxgboost.dir/tree/updater_quantile_hist.cc.o -MF src/CMakeFiles/objxgboost.dir/tree/updater_quantile_hist.cc.o.d -o src/CMakeFiles/objxgboost.dir/tree/updater_quantile_hist.cc.o -c ../src/tree/updater_quantile_hist.cc
clang: error: unable to execute command: Segmentation fault: 11
clang: error: clang frontend command failed due to signal (use -v to see invocation)
Apple clang version 11.0.3 (clang-1103.0.32.62)
Target: x86_64-apple-darwin19.6.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin
clang: note: diagnostic msg: PLEASE submit a bug report to http://developer.apple.com/bugreporter/ and include the crash backtrace, preprocessed source, and associated run script.
clang: note: diagnostic msg: 
********************

PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT:
Preprocessed source(s) and associated run script(s) are located at:
clang: note: diagnostic msg: /var/folders/3j/q0p7yccj48z9y_rmnc4dv3nm0000gp/T/updater_quantile_hist-a6672f.cpp
clang: note: diagnostic msg: /var/folders/3j/q0p7yccj48z9y_rmnc4dv3nm0000gp/T/updater_quantile_hist-a6672f.sh
clang: note: diagnostic msg: Crash backtrace is located in
clang: note: diagnostic msg: /Users/phcho/Library/Logs/DiagnosticReports/clang_<YYYY-MM-DD-HHMMSS>_<hostname>.crash
clang: note: diagnostic msg: (choose the .crash file that corresponds to your crash)
clang: note: diagnostic msg: 

I'd like to find a way to work around the compiler bug.

Crash log:

System Integrity Protection: enabled                                                                                                                        
                                                                                                                                                            
Crashed Thread:        0  Dispatch queue: com.apple.main-thread                                                                                             
                                                                                                                                                            
Exception Type:        EXC_BAD_ACCESS (SIGSEGV)                                                                                                             
Exception Codes:       EXC_I386_GPFLT                                                                                                                       
Exception Note:        EXC_CORPSE_NOTIFY                                                                                                                    
                                                                                                                                                            
Termination Signal:    Segmentation fault: 11                                                                                                               
Termination Reason:    Namespace SIGNAL, Code 0xb                                                                                                           
Terminating Process:   exc handler [2650]                                                                                                                   
                                                                                                                                                            
Application Specific Information:                                                                                                                           
dyld2 mode

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread                                                                                                    
0   clang                           0x000000010a77c1f2 clang::ASTContext::getPointerType(clang::QualType) const + 370                                       
1   clang                           0x000000010a77c220 clang::ASTContext::getPointerType(clang::QualType) const + 416                                       
2   clang                           0x000000010a2ceba6 clang::Sema::getCurrentThisType() + 534                                                              
3   clang                           0x000000010a3e6f54 clang::Sema::isOpenMPCapturedDecl(clang::ValueDecl*, bool, unsigned int) + 1156                      
4   clang                           0x000000010a27094d captureInCapturedRegion(clang::sema::CapturedRegionScopeInfo*, clang::VarDecl*, clang::SourceLocation, bool, clang::QualType&, clang::QualType&, bool, clang::Sema&, bool) + 77                                                                                  
5   clang                           0x000000010a26e6bf clang::Sema::tryCaptureVariable(clang::VarDecl*, clang::SourceLocation, clang::Sema::TryCaptureKind, clang::SourceLocation, bool, clang::QualType&, clang::QualType&, unsigned int const*) + 1551                                                                
6   clang                           0x000000010a214594 clang::Sema::BuildDeclarationNameExpr(clang::CXXScopeSpec const&, clang::DeclarationNameInfo const&, clang::NamedDecl*, clang::NamedDecl*, clang::TemplateArgumentListInfo const*, bool) + 740                                                                   
7   clang                           0x000000010a601f3b clang::TreeTransform<(anonymous namespace)::TemplateInstantiator>::TransformDeclRefExpr(clang::DeclRefExpr*) + 427                                                                                                                                               
8   clang                           0x000000010a5f5542 clang::TreeTransform<(anonymous namespace)::TemplateInstantiator>::TransformCallExpr(clang::CallExpr*) + 34                                                                                                                                                      
9   clang                           0x000000010a5ec413 clang::TreeTransform<(anonymous namespace)::TemplateInstantiator>::TransformStmt(clang::Stmt*, clang::TreeTransform<(anonymous namespace)::TemplateInstantiator>::StmtDiscardKind) + 83                                                                          
10  clang                           0x000000010a60548b clang::TreeTransform<(anonymous namespace)::TemplateInstantiator>::TransformCompoundStmt(clang::CompoundStmt*, bool) + 219                                                                                                                                       
11  clang                           0x000000010a60ec5f clang::TreeTransform<(anonymous namespace)::TemplateInstantiator>::TransformForStmt(clang::ForStmt*) + 239                                                                                                                                                       
12  clang                           0x000000010a60548b clang::TreeTransform<(anonymous namespace)::TemplateInstantiator>::TransformCompoundStmt(clang::CompoundStmt*, bool) + 219                                                                                                                                       
13  clang                           0x000000010a60321d clang::TreeTransform<(anonymous namespace)::TemplateInstantiator>::TransformLambdaExpr(clang::LambdaExpr*) + 3437                                                                                                                                                
14  clang                           0x000000010a5f8498 (anonymous namespace)::TemplateInstantiator::TransformLambdaExpr(clang::LambdaExpr*) + 152           
15  clang                           0x000000010a5ee851 clang::TreeTransform<(anonymous namespace)::TemplateInstantiator>::TransformExprs(clang::Expr* const*, unsigned int, bool, llvm::SmallVectorImpl<clang::Expr*>&, bool*) + 897                                                                                    
16  clang                           0x000000010a5f5599 clang::TreeTransform<(anonymous namespace)::TemplateInstantiator>::TransformCallExpr(clang::CallExpr*) + 121                                                                                                                                                     
17  clang                           0x000000010a5ec413 clang::TreeTransform<(anonymous namespace)::TemplateInstantiator>::TransformStmt(clang::Stmt*, clang::TreeTransform<(anonymous namespace)::TemplateInstantiator>::StmtDiscardKind) + 83                                                                          
18  clang                           0x000000010a60548b clang::TreeTransform<(anonymous namespace)::TemplateInstantiator>::TransformCompoundStmt(clang::CompoundStmt*, bool) + 219                                                                                                                                       
19  clang                           0x000000010a611e17 clang::TreeTransform<(anonymous namespace)::TemplateInstantiator>::TransformOMPExecutableDirective(clang::OMPExecutableDirective*) + 407                                                                                                                         
20  clang                           0x000000010a61017c clang::TreeTransform<(anonymous namespace)::TemplateInstantiator>::TransformOMPParallelDirective(clang::OMPParallelDirective*) + 76                                                                                                                              
21  clang                           0x000000010a60548b clang::TreeTransform<(anonymous namespace)::TemplateInstantiator>::TransformCompoundStmt(clang::CompoundStmt*, bool) + 219                                                                                                                                       
22  clang                           0x000000010a636706 clang::Sema::InstantiateFunctionDefinition(clang::SourceLocation, clang::FunctionDecl*, bool, bool, bool) + 1766                                                                                                                                                 
23  clang                           0x000000010a63a865 clang::Sema::PerformPendingInstantiations(bool) + 277                                                
24  clang                           0x0000000109ebaaac clang::Sema::ActOnEndOfTranslationUnitFragment(clang::Sema::TUFragmentKind) + 156                    
25  clang                           0x0000000109ebb3db clang::Sema::ActOnEndOfTranslationUnit() + 91                                                        
26  clang                           0x0000000109e48179 clang::Parser::ParseTopLevelDecl(clang::OpaquePtr<clang::DeclGroupRef>&, bool) + 937                 
27  clang                           0x0000000109d6c95d clang::ParseAST(clang::Sema&, bool, bool) + 253                                                      
28  clang                           0x000000010915c0d4 clang::FrontendAction::Execute() + 52                                                                
29  clang                           0x000000010910cc40 clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) + 1168                                
30  clang                           0x00000001091abbf4 clang::ExecuteCompilerInvocation(clang::CompilerInstance*) + 4212                                    
31  clang                           0x000000010742ee51 cc1_main(llvm::ArrayRef<char const*>, char const*, void*) + 1697                                     
32  clang                           0x000000010742a53b driver_main(int, char const**) + 7755                                                                
33  clang                           0x0000000107427fd6 main + 678                                                                                           
34  libdyld.dylib                   0x00007fff729fccc9 start + 1

CMake log:

cmake .. -GNinja -DGOOGLE_TEST=ON -DUSE_DMLC_GTEST=ON
-- The CXX compiler identification is AppleClang 11.0.3.11030032
-- The C compiler identification is AppleClang 11.0.3.11030032
-- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++
-- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++ - works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Check for working C compiler: /Library/Developer/CommandLineTools/usr/bin/cc
-- Check for working C compiler: /Library/Developer/CommandLineTools/usr/bin/cc - works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- CMake version 3.17.0
-- Performing Test XGBOOST_MM_PREFETCH_PRESENT
-- Performing Test XGBOOST_MM_PREFETCH_PRESENT - Success
-- Performing Test XGBOOST_BUILTIN_PREFETCH_PRESENT
-- Performing Test XGBOOST_BUILTIN_PREFETCH_PRESENT - Success
-- xgboost VERSION: 1.2.0
-- Setting build type to 'Release' as none was specified.
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
-- Found OpenMP_C: -Xclang -fopenmp (found version "3.1") 
-- Found OpenMP_CXX: -Xclang -fopenmp (found version "3.1") 
-- Found OpenMP: TRUE (found version "3.1")  
-- Found OpenMP_C: -Xclang -fopenmp (found version "3.1") 
-- Found OpenMP_CXX: -Xclang -fopenmp (found version "3.1") 
-- Looking for clock_gettime in rt
-- Looking for clock_gettime in rt - not found
-- Looking for fopen64
-- Looking for fopen64 - not found
-- Looking for C++ include cxxabi.h
-- Looking for C++ include cxxabi.h - found
-- Looking for nanosleep
-- Looking for nanosleep - found
-- Looking for backtrace
-- Looking for backtrace - found
-- backtrace facility detected in default set of libraries
-- Found Backtrace: /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include  
-- Check if the system is big endian
-- Searching 16 bit integer
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Check size of unsigned short
-- Check size of unsigned short - done
-- Searching 16 bit integer - Using unsigned short
-- Check if the system is big endian - little endian
-- /Users/phcho/Desktop/xgboost/dmlc-core/cmake/build_config.h.in -> include/dmlc/build_config.h
-- Performing Test SUPPORT_MSSE2
-- Performing Test SUPPORT_MSSE2 - Success
-- /Users/phcho/Desktop/xgboost/dmlc-core/test/unittest/build_config.h.in -> /Users/phcho/Desktop/xgboost/dmlc-core/test/unittest/build_config.h
-- Could NOT find GTest (missing: GTEST_LIBRARY GTEST_INCLUDE_DIR GTEST_MAIN_LIBRARY) 
-- GTest not found, downloading GTest.
/Users/phcho/Desktop/xgboost/dmlc-core/cmake/gtest_cmake.in -> /Users/phcho/Desktop/xgboost/build/googletest-download/CMakeLists.txt
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/phcho/Desktop/xgboost/build/googletest-download
[1/9] Creating directories for 'googletest'
[2/9] Performing download step (git clone) for 'googletest'
Cloning into 'googletest-src'...
Note: switching to 'release-1.10.0'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 703bd9ca Googletest export
[3/9] No patch step for 'googletest'
[4/9] Performing update step for 'googletest'
[5/9] No configure step for 'googletest'
[6/9] No build step for 'googletest'
[7/9] No install step for 'googletest'
[8/9] No test step for 'googletest'
[9/9] Completed 'googletest'
-- Found PythonInterp: /usr/bin/python (found version "2.7.16") 
DMLC_ROOT point to /Users/phcho/Desktop/xgboost/rabit/../dmlc-core
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/phcho/Desktop/xgboost/build

[hcho3]

Note that the segfault disappears when I check out the previous commit (bd6b7f4).

[hcho3]

The crash does not happen when I switched to GCC from Homebrew.

[hcho3]

The crash is also unique to AppleClang 11. Travis CI used AppleClang 10 and built XGBoost without error: https://travis-ci.org/github/dmlc/xgboost/jobs/716850246

[hcho3]

I found a workaround. Will post a pull request soon.