Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

使用Docker编译源码生成Ubuntu的*whl包的时候，报错 #8931

Closed
yeyupiaoling opened this issue Mar 9, 2018 · 21 comments
Closed

使用Docker编译源码生成Ubuntu的*whl包的时候，报错 #8931

yeyupiaoling opened this issue Mar 9, 2018 · 21 comments
Labels
User 用于标记用户问题

Comments

@yeyupiaoling
Copy link
Contributor

yeyupiaoling commented Mar 9, 2018 •
edited
Loading

报错信息如下

root@ng9e65017-master-instance-6fs0oczg:~# cd /mnt/Paddle/
root@ng9e65017-master-instance-6fs0oczg:/mnt/Paddle# docker run -it -v $PWD:/paddle -e "WITH_GPU=OFF" -e "WITH_AVX=OFF" -e "-DWITH_TESTING=OFF" paddlepaddle/paddle_manylinux_devel:cuda8.0_cudnn5 bash -x /paddle/paddle/scripts/docker/build.sh
Unable to find image 'paddlepaddle/paddle_manylinux_devel:cuda8.0_cudnn5' locally
cuda8.0_cudnn5: Pulling from paddlepaddle/paddle_manylinux_devel
b26de5a391ad: Pull complete 
a860867675b2: Pull complete 
2bb9cff4191c: Pull complete 
3aba8ab86873: Pull complete 
9049a3eb854a: Pull complete 
2eeeee2cf356: Pull complete 
94b0cc7a7f56: Pull complete 
7a6ffb25b5c2: Pull complete 
aae8a9571a10: Pull complete 
98e38f3d9bd6: Pull complete 
227600d15519: Pull complete 
bfdc7823d165: Pull complete 
981a5388ad80: Pull complete 
3465c54c486b: Pull complete 
43271940fb74: Pull complete 
a96aedd5b88f: Pull complete 
162390d278bf: Pull complete 
487a7f044b26: Pull complete 
Digest: sha256:22b4a3f34f46b645d624f30f9b12cbafc6ee517c784029d50eb26a68be502b16
Status: Downloaded newer image for paddlepaddle/paddle_manylinux_devel:cuda8.0_cudnn5
+ set -xe
+ cmake_gen ''
+ mkdir -p /paddle/build
+ cd /paddle/build
+ rm '*.deb'
+ true
+ rm -rf /paddle/paddle/dist
+ PYTHON_FLAGS=
+ '[' '' '!=' '' ']'
+ cat
    ========================================
    Configuring cmake in /paddle/build ...
        -DCMAKE_BUILD_TYPE=Release
        
        -DWITH_DSO=ON
        -DWITH_DOC=OFF
        -DWITH_GPU=OFF
        -DWITH_DISTRIBUTE=OFF
        -DWITH_MKL=ON
        -DWITH_AVX=OFF
        -DWITH_GOLANG=OFF
        -DCUDA_ARCH_NAME=All
        -DWITH_SWIG_PY=ON
        -DWITH_C_API=OFF
        -DWITH_PYTHON=ON
        -DWITH_SWIG_PY=ON
        -DCUDNN_ROOT=/usr/
        -DWITH_STYLE_CHECK=ON
        -DWITH_TESTING=ON
        -DWITH_FAST_BUNDLE_TEST=ON
        -DCMAKE_EXPORT_COMPILE_COMMANDS=ON
    ========================================
+ cmake .. -DCMAKE_BUILD_TYPE=Release -DWITH_DSO=ON -DWITH_DOC=OFF -DWITH_GPU=OFF -DWITH_DISTRIBUTE=OFF -DWITH_MKL=ON -DWITH_AVX=OFF -DWITH_GOLANG=OFF -DCUDA_ARCH_NAME=All -DWITH_SWIG_PY=ON -DWITH_C_API=OFF -DWITH_PYTHON=ON -DCUDNN_ROOT=/usr/ -DWITH_STYLE_CHECK=ON -DWITH_TESTING=ON -DWITH_FAST_BUNDLE_TEST=ON -DCMAKE_EXPORT_COMPILE_COMMANDS=ON
-- Found Paddle host system: centos, version: 6.9
-- Found Paddle host system's CPU: 6 cores
-- The CXX compiler identification is GNU 4.8.2
-- The C compiler identification is GNU 4.8.2
-- Check for working CXX compiler: /opt/rh/devtoolset-2/root/usr/bin/c++
-- Check for working CXX compiler: /opt/rh/devtoolset-2/root/usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Check for working C compiler: /opt/rh/devtoolset-2/root/usr/bin/cc
-- Check for working C compiler: /opt/rh/devtoolset-2/root/usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- CXX compiler: /opt/rh/devtoolset-2/root/usr/bin/c++, version: GNU 4.8.2
-- C compiler: /opt/rh/devtoolset-2/root/usr/bin/cc, version: GNU 4.8.2
-- Could NOT find Sphinx (missing:  SPHINX_EXECUTABLE) 
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- Found Git: /usr/bin/git (found version "1.7.1") 
-- Performing Test MMX_FOUND
-- Performing Test MMX_FOUND - Success
-- Performing Test SSE2_FOUND
-- Performing Test SSE2_FOUND - Success
-- Performing Test SSE3_FOUND
-- Performing Test SSE3_FOUND - Success
-- Performing Test AVX_FOUND
-- Performing Test AVX_FOUND - Success
-- Performing Test AVX2_FOUND
-- Performing Test AVX2_FOUND - Success
-- Protobuf protoc executable: /paddle/build/third_party/install/protobuf/bin/protoc
-- Protobuf library: /paddle/build/third_party/install/protobuf/lib/libprotobuf.a
-- Protobuf version: 3.1
-- Could NOT find PythonInterp: Found unsuitable version "2.6.6", but required is at least "2.7" (found /usr/bin/python2)
-- Could NOT find PythonLibs (missing:  PYTHON_LIBRARIES PYTHON_INCLUDE_DIRS) (Required is at least version "2.7")
-- Found cblas and lapack in MKLML (include: /paddle/build/third_party/install/mklml/mklml_lnx_2018.0.1.20171007/include, library: /paddle/build/third_party/install/mklml/mklml_lnx_2018.0.1.20171007/lib/libmklml_intel.so)
-- BLAS library: /paddle/build/third_party/install/mklml/mklml_lnx_2018.0.1.20171007/lib/libmklml_intel.so
-- Set /paddle/build/third_party/install/mkldnn/lib to runtime path
-- Build MKLDNN with MKLML /paddle/build/third_party/install/mklml/mklml_lnx_2018.0.1.20171007
-- MKLDNN library: /paddle/build/third_party/install/mkldnn/lib/libmkldnn.so
-- Found SWIG: /usr/local/bin/swig (found version "2.0.12") 
-- warp-ctc library: /paddle/build/third_party/install/warpctc/lib/libwarpctc.so
-- Enable Intel OpenMP with /paddle/build/third_party/install/mklml/mklml_lnx_2018.0.1.20171007/lib/libiomp5.so
-- Looking for UINT64_MAX
-- Looking for UINT64_MAX - found
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Check size of pthread_spinlock_t
-- Check size of pthread_spinlock_t - done
-- Check size of pthread_barrier_t
-- Check size of pthread_barrier_t - done
-- Performing Test C_COMPILER_SUPPORT_FLAG__fPIC
-- Performing Test C_COMPILER_SUPPORT_FLAG__fPIC - Success
-- Performing Test CXX_COMPILER_SUPPORT_FLAG__fPIC
-- Performing Test CXX_COMPILER_SUPPORT_FLAG__fPIC - Success
-- Performing Test C_COMPILER_SUPPORT_FLAG__fno_omit_frame_pointer
-- Performing Test C_COMPILER_SUPPORT_FLAG__fno_omit_frame_pointer - Success
-- Performing Test CXX_COMPILER_SUPPORT_FLAG__fno_omit_frame_pointer
-- Performing Test CXX_COMPILER_SUPPORT_FLAG__fno_omit_frame_pointer - Success
-- Performing Test C_COMPILER_SUPPORT_FLAG__Wall
-- Performing Test C_COMPILER_SUPPORT_FLAG__Wall - Success
-- Performing Test CXX_COMPILER_SUPPORT_FLAG__Wall
-- Performing Test CXX_COMPILER_SUPPORT_FLAG__Wall - Success
-- Performing Test C_COMPILER_SUPPORT_FLAG__Wextra
-- Performing Test C_COMPILER_SUPPORT_FLAG__Wextra - Success
-- Performing Test CXX_COMPILER_SUPPORT_FLAG__Wextra
-- Performing Test CXX_COMPILER_SUPPORT_FLAG__Wextra - Success
-- Performing Test C_COMPILER_SUPPORT_FLAG__Werror
-- Performing Test C_COMPILER_SUPPORT_FLAG__Werror - Success
-- Performing Test CXX_COMPILER_SUPPORT_FLAG__Werror
-- Performing Test CXX_COMPILER_SUPPORT_FLAG__Werror - Success
-- Performing Test C_COMPILER_SUPPORT_FLAG__Wnon_virtual_dtor
-- Performing Test C_COMPILER_SUPPORT_FLAG__Wnon_virtual_dtor - Failed
-- Performing Test CXX_COMPILER_SUPPORT_FLAG__Wnon_virtual_dtor
-- Performing Test CXX_COMPILER_SUPPORT_FLAG__Wnon_virtual_dtor - Success
-- Performing Test C_COMPILER_SUPPORT_FLAG__Wdelete_non_virtual_dtor
-- Performing Test C_COMPILER_SUPPORT_FLAG__Wdelete_non_virtual_dtor - Failed
-- Performing Test CXX_COMPILER_SUPPORT_FLAG__Wdelete_non_virtual_dtor
-- Performing Test CXX_COMPILER_SUPPORT_FLAG__Wdelete_non_virtual_dtor - Success
-- Performing Test C_COMPILER_SUPPORT_FLAG__Wno_unused_parameter
-- Performing Test C_COMPILER_SUPPORT_FLAG__Wno_unused_parameter - Success
-- Performing Test CXX_COMPILER_SUPPORT_FLAG__Wno_unused_parameter
-- Performing Test CXX_COMPILER_SUPPORT_FLAG__Wno_unused_parameter - Success
-- Performing Test C_COMPILER_SUPPORT_FLAG__Wno_unused_function
-- Performing Test C_COMPILER_SUPPORT_FLAG__Wno_unused_function - Success
-- Performing Test CXX_COMPILER_SUPPORT_FLAG__Wno_unused_function
-- Performing Test CXX_COMPILER_SUPPORT_FLAG__Wno_unused_function - Success
-- Performing Test C_COMPILER_SUPPORT_FLAG__Wno_error_literal_suffix
-- Performing Test C_COMPILER_SUPPORT_FLAG__Wno_error_literal_suffix - Success
-- Performing Test CXX_COMPILER_SUPPORT_FLAG__Wno_error_literal_suffix
-- Performing Test CXX_COMPILER_SUPPORT_FLAG__Wno_error_literal_suffix - Success
-- Performing Test C_COMPILER_SUPPORT_FLAG__Wno_error_sign_compare
-- Performing Test C_COMPILER_SUPPORT_FLAG__Wno_error_sign_compare - Success
-- Performing Test CXX_COMPILER_SUPPORT_FLAG__Wno_error_sign_compare
-- Performing Test CXX_COMPILER_SUPPORT_FLAG__Wno_error_sign_compare - Success
-- Performing Test C_COMPILER_SUPPORT_FLAG__Wno_error_unused_local_typedefs
-- Performing Test C_COMPILER_SUPPORT_FLAG__Wno_error_unused_local_typedefs - Success
-- Performing Test CXX_COMPILER_SUPPORT_FLAG__Wno_error_unused_local_typedefs
-- Performing Test CXX_COMPILER_SUPPORT_FLAG__Wno_error_unused_local_typedefs - Success
-- Performing Test C_COMPILER_SUPPORT_FLAG__Wno_error_parentheses_equality
-- Performing Test C_COMPILER_SUPPORT_FLAG__Wno_error_parentheses_equality - Failed
-- Performing Test CXX_COMPILER_SUPPORT_FLAG__Wno_error_parentheses_equality
-- Performing Test CXX_COMPILER_SUPPORT_FLAG__Wno_error_parentheses_equality - Failed
-- Performing Test C_COMPILER_SUPPORT_FLAG__Wno_error_ignored_attributes
-- Performing Test C_COMPILER_SUPPORT_FLAG__Wno_error_ignored_attributes - Failed
-- Performing Test CXX_COMPILER_SUPPORT_FLAG__Wno_error_ignored_attributes
-- Performing Test CXX_COMPILER_SUPPORT_FLAG__Wno_error_ignored_attributes - Failed
-- Performing Test C_COMPILER_SUPPORT_FLAG__Wno_error_terminate
-- Performing Test C_COMPILER_SUPPORT_FLAG__Wno_error_terminate - Failed
-- Performing Test CXX_COMPILER_SUPPORT_FLAG__Wno_error_terminate
-- Performing Test CXX_COMPILER_SUPPORT_FLAG__Wno_error_terminate - Failed
-- Performing Test C_COMPILER_SUPPORT_FLAG__Wno_error_unused_function
-- Performing Test C_COMPILER_SUPPORT_FLAG__Wno_error_unused_function - Success
-- Performing Test C_COMPILER_SUPPORT_FLAG__Wno_error_array_bounds
-- Performing Test C_COMPILER_SUPPORT_FLAG__Wno_error_array_bounds - Success
-- Paddle version is 0.11.1a1
-- Compile with MKLDNNMatrix
-- Compile with MKLDNNLayers and MKLDNNActivations
-- Compile with MKLPackedLayers
-- Performing Test CXX_COMPILER_SUPPORT_FLAG__ftls_model_global_dynamic
-- Performing Test CXX_COMPILER_SUPPORT_FLAG__ftls_model_global_dynamic - Success
-- Performing Test CXX_COMPILER_SUPPORT_FLAG__Wno_parentheses_equality
-- Performing Test CXX_COMPILER_SUPPORT_FLAG__Wno_parentheses_equality - Success
-- Performing Test CXX_COMPILER_SUPPORT_FLAG__Wno_self_assign
-- Performing Test CXX_COMPILER_SUPPORT_FLAG__Wno_self_assign - Success
-- Performing Test CXX_COMPILER_SUPPORT_FLAG__Wno_maybe_uninitialized
-- Performing Test CXX_COMPILER_SUPPORT_FLAG__Wno_maybe_uninitialized - Success
-- Performing Test CXX_COMPILER_SUPPORT_FLAG__Wno_missing_field_initializers
-- Performing Test CXX_COMPILER_SUPPORT_FLAG__Wno_missing_field_initializers - Success
CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
PYTHON_LIBRARY (ADVANCED)
    linked by target "GemmConvOpTest" in directory /paddle/paddle/function
    linked by target "DepthwiseConvOpTest" in directory /paddle/paddle/function
    linked by target "Im2ColTest" in directory /paddle/paddle/function
    linked by target "test_StringUtils" in directory /paddle/paddle/utils/tests
    linked by target "test_Thread" in directory /paddle/paddle/utils/tests
    linked by target "test_Error" in directory /paddle/paddle/utils/tests
    linked by target "test_CustomStackTracePrint" in directory /paddle/paddle/utils/tests
    linked by target "test_CustomStackTrace" in directory /paddle/paddle/utils/tests
    linked by target "test_ThreadBarrier" in directory /paddle/paddle/utils/tests
    linked by target "test_SpinLock" in directory /paddle/paddle/utils/tests
    linked by target "test_SIMDFlags" in directory /paddle/paddle/utils/tests
    linked by target "test_CpuGpuVector" in directory /paddle/paddle/math/tests
    linked by target "test_Matrix" in directory /paddle/paddle/math/tests
    linked by target "test_sparseMatrixCompare" in directory /paddle/paddle/math/tests
    linked by target "test_matrixCompare" in directory /paddle/paddle/math/tests
    linked by target "test_perturbation" in directory /paddle/paddle/math/tests
    linked by target "test_TrainingAlgorithm" in directory /paddle/paddle/math/tests
    linked by target "test_SIMDFunctions" in directory /paddle/paddle/math/tests
    linked by target "test_RowBuffer" in directory /paddle/paddle/math/tests
    linked by target "test_SparseMatrix" in directory /paddle/paddle/math/tests
    linked by target "test_ExecViaCpu" in directory /paddle/paddle/math/tests
    linked by target "test_BaseMatrix" in directory /paddle/paddle/math/tests
    linked by target "test_Allocator" in directory /paddle/paddle/math/tests
    linked by target "test_FPException" in directory /paddle/paddle/math/tests
    linked by target "test_Tensor" in directory /paddle/paddle/math/tests
    linked by target "test_lazyAssign" in directory /paddle/paddle/math/tests
    linked by target "test_GpuProfiler" in directory /paddle/paddle/math/tests
    linked by target "test_CompareSparse" in directory /paddle/paddle/gserver/tests
    linked by target "test_CRFLayerGrad" in directory /paddle/paddle/gserver/tests
    linked by target "test_LayerGrad" in directory /paddle/paddle/gserver/tests
    linked by target "test_ConvTrans" in directory /paddle/paddle/gserver/tests
    linked by target "test_PriorBox" in directory /paddle/paddle/gserver/tests
    linked by target "test_RecurrentGradientMachine" in directory /paddle/paddle/gserver/tests
    linked by target "test_MultinomialSampler" in directory /paddle/paddle/gserver/tests
    linked by target "test_LinearChainCRF" in directory /paddle/paddle/gserver/tests
    linked by target "test_ActivationGrad" in directory /paddle/paddle/gserver/tests
    linked by target "test_RecurrentLayer" in directory /paddle/paddle/gserver/tests
    linked by target "test_DetectionOutput" in directory /paddle/paddle/gserver/tests
    linked by target "test_ConvUnify" in directory /paddle/paddle/gserver/tests
    linked by target "test_NetworkCompare" in directory /paddle/paddle/gserver/tests
    linked by target "test_MKLDNN" in directory /paddle/paddle/gserver/tests
    linked by target "test_WarpCTCLayer" in directory /paddle/paddle/gserver/tests
    linked by target "test_Evaluator" in directory /paddle/paddle/gserver/tests
    linked by target "test_BatchNorm" in directory /paddle/paddle/gserver/tests
    linked by target "test_CrossEntropyOverBeamGrad" in directory /paddle/paddle/gserver/tests
    linked by target "test_KmaxSeqScore" in directory /paddle/paddle/gserver/tests
    linked by target "test_Expand" in directory /paddle/paddle/gserver/tests
    linked by target "test_MaxPoolingWithMaskOutput" in directory /paddle/paddle/gserver/tests
    linked by target "test_PyDataProvider2" in directory /paddle/paddle/gserver/tests
    linked by target "test_PyDataProvider" in directory /paddle/paddle/gserver/tests
    linked by target "test_SeqSliceLayerGrad" in directory /paddle/paddle/gserver/tests
    linked by target "test_CompareTwoNets" in directory /paddle/paddle/gserver/tests
    linked by target "test_argument" in directory /paddle/paddle/parameter/tests
    linked by target "test_common" in directory /paddle/paddle/parameter/tests
    linked by target "paddle_pserver_main" in directory /paddle/paddle/pserver
    linked by target "socket_test" in directory /paddle/paddle/pserver/test
    linked by target "test_ParameterServer2" in directory /paddle/paddle/pserver/test
    linked by target "test_ProtoServer" in directory /paddle/paddle/pserver/test
    linked by target "paddle_trainer" in directory /paddle/paddle/trainer
    linked by target "paddle_merge_model" in directory /paddle/paddle/trainer
    linked by target "test_PyDataProviderWrapper" in directory /paddle/paddle/trainer/tests
    linked by target "test_Trainer" in directory /paddle/paddle/trainer/tests
    linked by target "test_Compare" in directory /paddle/paddle/trainer/tests
    linked by target "test_TrainerOnePass" in directory /paddle/paddle/trainer/tests
    linked by target "test_recurrent_machine_generation" in directory /paddle/paddle/trainer/tests
    linked by target "_swig_paddle" in directory /paddle/paddle/api
    linked by target "protobuf_equal" in directory /paddle/python/paddle/trainer_config_helpers/tests

-- Configuring incomplete, errors occurred!
See also "/paddle/build/CMakeFiles/CMakeOutput.log".
See also "/paddle/build/CMakeFiles/CMakeError.log".

在错误日志CMakeError.log信息如下：

Determining if the pthread_create exist failed with the following output:
Change Dir: /paddle/build/CMakeFiles/CMakeTmp

Run Build Command:"/usr/bin/gmake" "cmTC_ea34e/fast"
/usr/bin/gmake -f CMakeFiles/cmTC_ea34e.dir/build.make CMakeFiles/cmTC_ea34e.dir/build
gmake[1]: Entering directory `/paddle/build/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_ea34e.dir/CheckSymbolExists.c.o
/opt/rh/devtoolset-2/root/usr/bin/cc    -fPIC    -o CMakeFiles/cmTC_ea34e.dir/CheckSymbolExists.c.o   -c /paddle/build/CMakeFiles/CMakeTmp/CheckSymbolExists.c
Linking C executable cmTC_ea34e
/usr/local/bin/cmake -E cmake_link_script CMakeFiles/cmTC_ea34e.dir/link.txt --verbose=1
/opt/rh/devtoolset-2/root/usr/bin/cc  -fPIC     CMakeFiles/cmTC_ea34e.dir/CheckSymbolExists.c.o  -o cmTC_ea34e -rdynamic
CMakeFiles/cmTC_ea34e.dir/CheckSymbolExists.c.o: In function `main':
CheckSymbolExists.c:(.text+0x1b): undefined reference to `pthread_create'
collect2: error: ld returned 1 exit status
gmake[1]: *** [cmTC_ea34e] Error 1
gmake[1]: Leaving directory `/paddle/build/CMakeFiles/CMakeTmp'
gmake: *** [cmTC_ea34e/fast] Error 2

File /paddle/build/CMakeFiles/CMakeTmp/CheckSymbolExists.c:
/* */
#include <pthread.h>

int main(int argc, char** argv)
{
  (void)argv;
#ifndef pthread_create
  return ((int*)(&pthread_create))[argc];
#else
  (void)argc;
  return 0;
#endif
}

Determining if the function pthread_create exists in the pthreads failed with the following output:
@peterzhang2029 peterzhang2029 added the User 用于标记用户问题 label Mar 9, 2018
@typhoonzero
Copy link
Contributor

typhoonzero commented Mar 9, 2018 •
edited by wangkuiyi
Loading

用manylinux镜像编译的话需要指定一些环境变量，我一般是这么启动：

docker run -it --privileged -v $PWD:/paddle --rm \
  -e "PYTHON_ABI=cp27-cp27mu" \
  -e "https_proxy=你的代理的地址，可选"
  -v /home/work/wuyi/nvidia:/usr/local/nvidia/lib64 \
  paddlepaddle/paddle_manylinux_devel:cuda${CUDA_VER}_cudnn5 \
  /bin/bash -c \
  'export PATH=/opt/python/cp27-cp27mu/bin:$PATH; export LD_LIBRARY_PATH=/opt/python/cp27-cp27mu/lib:$LD_LI  > BRARY_PATH; bash'

其中，--privileged 会把/dev全部mount到镜像中这样才可以访问GPU的device文件，-e "https_proxy=你的代理的地址，可选"配置代理服务器，用于加速build过程中的下载，/home/work/wuyi/nvidia这个目录下我把nvidia driver用到的so都拷贝过来了，这样可以支持GPU执行。

@yeyupiaoling
Copy link
Contributor Author

@typhoonzero
我的GPU服务器上的/usr/local/有cuda和cuda-8.0这两文件夹，是把它们的lib下的so文件考进去类似你的文件夹/home/work/wuyi/nvidia吗？很多哦

@yeyupiaoling
Copy link
Contributor Author

@typhoonzero
有没有其他的镜像没有那么复杂的？

@typhoonzero
Copy link
Contributor

用paddlepaddle/paddle-dev:latest可以直接build。

运行程序或单测仍然需要挂在nvidia驱动（不是cuda目录），不过如果机器是centos7以上可以安装nvidia-docker.

@yeyupiaoling
Copy link
Contributor Author

@typhoonzero
您这个镜像不存在呢？我要编码的GPU版本的paddlepaddle

@typhoonzero
Copy link
Contributor

哦，抱歉是paddlepaddle/paddle:latest-dev

@yeyupiaoling
Copy link
Contributor Author

@typhoonzero
使用您的镜像出现的是这个问题

Total errors found: 0
[ 30%] Built target paddle_gserver
Makefile:160: recipe for target 'all' failed
make: *** [all] Error 2

@typhoonzero
Copy link
Contributor

具体的报错应该在前面。这些报错信息不足以判断。编译可以参考：http://www.paddlepaddle.org/docs/develop/documentation/en/build_and_install/build_from_source_en.html

@yeyupiaoling
Copy link
Contributor Author

@typhoonzero
前面目测没有问题，如

cpplint: Checking source code style
Done processing gradientmachines/GradientMachine.h
Done processing gradientmachines/RecurrentGradientMachine.h
Done processing gradientmachines/NeuralNetwork.h
Done processing gradientmachines/ParallelNeuralNetwork.h
Done processing gradientmachines/MultiNetwork.h
Done processing gradientmachines/GradientMachineMode.h
Done processing gradientmachines/MultiGradientMachine.h
Done processing dataproviders/DataProviderGroup.h
Done processing dataproviders/ProtoReader.h
Done processing dataproviders/DataProvider.h
Done processing dataproviders/PyDataProvider.h
Done processing activations/ActivationFunction.h
Done processing activations/MKLDNNActivation.h
Done processing evaluators/Evaluator.h
Done processing layers/MKLDNNBase.h
Done processing layers/CostLayer.h
Done processing layers/MKLDNNLayer.h
Done processing layers/SequenceToBatch.h
Done processing layers/FullyConnectedLayer.h
Done processing layers/Conv3DLayer.h
Done processing layers/MaxLayer.h
Done processing layers/CrossEntropyOverBeam.h
Done processing layers/RecurrentLayer.h
Done processing layers/ConvBaseProjection.h
Done processing layers/CudnnConvBaseLayer.h
Done processing layers/MultinomialSampler.h
Done processing layers/DataLayer.h
Done processing layers/MKLDNNLRNLayer.h
Done processing layers/ROIPoolLayer.h
Done processing layers/ValidationLayer.h
Done processing layers/TableProjection.h
Done processing layers/BatchNormBaseLayer.h
Done processing layers/CudnnPoolLayer.h
Done processing layers/L2DistanceLayer.h
Done processing layers/PoolLayer.h
Done processing layers/MKLDNNBatchNormLayer.h
Done processing layers/MaxOutLayer.h
Done processing layers/CRFDecodingLayer.h
Done processing layers/AverageLayer.h
Done processing layers/LinearChainCTC.h
Done processing layers/LinearChainCRF.h
Done processing layers/BatchNormalizationLayer.h
Done processing layers/MKLDNNConvLayer.h
Done processing layers/Projection.h
Done processing layers/RotateLayer.h
Done processing layers/ConvTransOperator.h
Done processing layers/ConvTransProjection.h
Done processing layers/AgentLayer.h
Done processing layers/ExpandLayer.h
Done processing layers/DetectionUtil.h
Done processing layers/PoolProjection.h
Done processing layers/ContextProjection.h
Done processing layers/MKLDNNAddtoLayer.h
Done processing layers/MKLDNNConcatLayer.h
Done processing layers/LstmLayer.h
Done processing layers/ExpandConvLayer.h
Done processing layers/SwitchOrderLayer.h
Done processing layers/CropLayer.h
Done processing layers/MKLPackedRecurrentLayer.h
Done processing layers/HierarchicalSigmoidLayer.h
Done processing layers/ConvProjection.h
Done processing layers/CRFLayer.h
Done processing layers/PadLayer.h
Done processing layers/MultiBoxLossLayer.h
Done processing layers/FullMatrixProjection.h
Done processing layers/MaxPoolWithMaskLayer.h
Done processing layers/Operator.h
Done processing layers/BlockExpandLayer.h
Done processing layers/DeConv3DLayer.h
Done processing layers/AddtoLayer.h
Done processing layers/TensorLayer.h
Done processing layers/DataNormLayer.h
Done processing layers/GruCompute.h
Done processing layers/MKLDNNPoolLayer.h
Done processing layers/WarpCTCLayer.h
Done processing layers/NormProjectionLayer.h
Done processing layers/Pool3DLayer.h
Done processing layers/PoolProjectionLayer.h
Done processing layers/RowConvLayer.h
Done processing layers/MixedLayer.h
Done processing layers/LstmCompute.h
Done processing layers/ConvOperator.h
Done processing layers/CosSimLayer.h
Done processing layers/MKLPackedWeight.h
Done processing layers/FactorizationMachineLayer.h
Done processing layers/SequencePoolLayer.h
Done processing layers/ConvBaseOperator.h
Done processing layers/GatedRecurrentLayer.h
Done processing layers/CTCLayer.h
Done processing layers/SelectiveFullyConnectedLayer.h
Done processing layers/NormLayer.h
Done processing layers/CudnnBatchNormLayer.h
Done processing layers/BilinearInterpLayer.h
Done processing layers/SpatialPyramidPoolLayer.h
Done processing layers/ScaleSubRegionLayer.h
Done processing layers/Layer.h
Done processing layers/ParameterReluLayer.h
Done processing layers/ConvBaseLayer.h
Done processing layers/MKLDNNFcLayer.h
Done processing layers/DetectionOutputLayer.h
Done processing layers/TransLayer.h
Total errors found: 0
[ 30%] Built target paddle_gserver
Makefile:160: recipe for target 'all' failed
make: *** [all] Error 2

@typhoonzero
Copy link
Contributor

应该还在前面。多线程编译得不停往前找到实际报错的地方。另外可以参考编译文档或者FAQ。

@yeyupiaoling
Copy link
Contributor Author

@typhoonzero
我找到这个错误

/paddle/paddle/fluid/framework/tensor_util.cu(1): error: this declaration has no storage class or type specifier

/paddle/paddle/fluid/framework/tensor_util.cu(1): error: expected a ";"

2 errors detected in the compilation of "/tmp/tmpxft_00001d37_00000000-17_tensor_util.compute_61.cpp1.ii".
CMake Error at tensor_generated_tensor_util.cu.o.cmake:262 (message):
  Error generating file
  /paddle/build/paddle/fluid/framework/CMakeFiles/tensor.dir//./tensor_generated_tensor_util.cu.o


paddle/fluid/framework/CMakeFiles/tensor.dir/build.make:188: recipe for target 'paddle/fluid/framework/CMakeFiles/tensor.dir/tensor_generated_tensor_util.cu.o' failed
make[2]: *** [paddle/fluid/framework/CMakeFiles/tensor.dir/tensor_generated_tensor_util.cu.o] Error 1
CMakeFiles/Makefile2:11335: recipe for target 'paddle/fluid/framework/CMakeFiles/tensor.dir/all' failed
make[1]: *** [paddle/fluid/framework/CMakeFiles/tensor.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

@typhoonzero
Copy link
Contributor

看下是否有类似"No space left on device"之类的错误，如果有，应该是我之前也遇到过的一个docker的bug，用-v mount进容器的那个host的目录，有的时候回错误的认为没有空间了，删掉build目录，然后重启一个容器可以解决

@yeyupiaoling
Copy link
Contributor Author

@typhoonzero
我也看到这些问题，您说的No space left on device没有发现

[ 11%] Building CXX object paddle/fluid/platform/CMakeFiles/device_tracer.dir/device_tracer.cc.o
In file included from /paddle/paddle/fluid/framework/type_defs.h:23:0,
                 from /paddle/paddle/fluid/framework/attribute.h:24,
                 from /paddle/paddle/fluid/framework/op_desc.h:19,
                 from /paddle/paddle/fluid/framework/block_desc.h:23,
                 from /paddle/paddle/fluid/platform/device_tracer.cc:23:
/paddle/paddle/fluid/platform/variant.h:30:28: fatal error: boost/config.hpp: No such file or directory
compilation terminated.
paddle/fluid/platform/CMakeFiles/device_tracer.dir/build.make:62: recipe for target 'paddle/fluid/platform/CMakeFiles/device_tracer.dir/device_tracer.cc.o' failed
make[2]: *** [paddle/fluid/platform/CMakeFiles/device_tracer.dir/device_tracer.cc.o] Error 1
CMakeFiles/Makefile2:8309: recipe for target 'paddle/fluid/platform/CMakeFiles/device_tracer.dir/all' failed
make[1]: *** [paddle/fluid/platform/CMakeFiles/device_tracer.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

@wangkuiyi
Copy link
Collaborator

wangkuiyi commented Mar 12, 2018 •
edited
Loading

@yeyupiaoling 我看到你贴的错误信息里有

/paddle/paddle/fluid/framework/tensor_util.cu(1): error: this declaration has no storage class or type specifier

/paddle/paddle/fluid/framework/tensor_util.cu(1): error: expected a ";"

这个是个C++语法错误？是不是你修改了 tensor_util.cu 的源码了？

@yeyupiaoling
Copy link
Contributor Author

@wangkuiyi
没呢？我直接clone的，您们有测试过吗？什么时候提供最新版本的PaddlePaddle-GPU，是在Accuracy去掉之后的版本。

@wangkuiyi
Copy link
Collaborator

wangkuiyi commented Mar 15, 2018 •
edited
Loading

PaddlePaddle 的 PR merge 都得经过 CI，所以不会有这样的C++语法错误。我每天都基于develop branch 开发，我确实也没有经历过 Github 上的源码编译不过的问题。真是有点儿奇怪。

确认一下，你是在Docker 里build的吗？

@yeyupiaoling
Copy link
Contributor Author

@wangkuiyi
是的，那我直接在本地测试试试看吧

@luotao1
Copy link
Contributor

luotao1 commented Mar 15, 2018

tensor_util.cu文件是tensor_util.cc的软连接，所以请确认是文件格式是否正确：

[luotao02@yq01: framework] merge_bn -> $ ls -al tensor_util.cu
lrwxrwxrwx 1 luotao02 luotao02 14 Mar  7 10:18 tensor_util.cu -> tensor_util.cc

@yeyupiaoling
Copy link
Contributor Author

@luotao1
有的呢

root@ngfc04c35-master-instance-hzo8xaky:/mnt/test/Paddle/paddle/fluid/framework# ls -al tensor_util.cu
lrwxrwxrwx 1 root root 14 Mar 15 13:08 tensor_util.cu -> tensor_util.cc

@yeyupiaoling
Copy link
Contributor Author

我应该明白的，我编译的是GPU版本的，但是我却使用docker，这个没有GPU驱动，所以就会报错，如果是使用docker的，就应该挂载本地机器的GPU驱动：

export CUDA_SO="$(\ls /usr/lib64/libcuda* | xargs -I{} echo '-v {}:{}') $(\ls /usr/lib64/libnvidia* | xargs -I{} echo '-v {}:{}')"
export DEVICES=$(\ls /dev/nvidia* | xargs -I{} echo '--device {}:{}')
docker run ${CUDA_SO} ${DEVICES} -it paddlepaddle/paddle:latest-gpu

否则就要使用nvidia-docker来执行镜像了，

我以上是否理解对了？ @luotao1 @typhoonzero @wangkuiyi

@1224wxwx 1224wxwx mentioned this issue Feb 21, 2020
Closed
@121786404
Copy link

太坑了，折腾了两天，官方的编译文档应该加上这个说明！！

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
User 用于标记用户问题
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@wangkuiyi @121786404 @luotao1 @peterzhang2029 @typhoonzero @yeyupiaoling