cuda9.0_cudnn7 编译报错

[Danceiny]

简要描述：使用dockerhub上的cuda9.0_cudnn7 从源码编译，报错

+ set -xe
+ cmake_gen ''
+ mkdir -p /paddle/build
+ cd /paddle/build
+ rm '*.deb'
+ true
+ rm -rf /paddle/paddle/dist
+ PYTHON_FLAGS=
+ '[' '' '!=' '' ']'
+ cat
    ========================================
    Configuring cmake in /paddle/build ...
        -DCMAKE_BUILD_TYPE=Release

        -DWITH_DSO=ON
        -DWITH_DOC=OFF
        -DWITH_GPU=OFF
        -DWITH_AMD_GPU=OFF
        -DWITH_DISTRIBUTE=OFF
        -DWITH_MKL=ON
        -DWITH_AVX=OFF
        -DWITH_GOLANG=OFF
        -DCUDA_ARCH_NAME=All
        -DWITH_SWIG_PY=ON
        -DWITH_C_API=OFF
        -DWITH_PYTHON=ON
        -DWITH_SWIG_PY=ON
        -DCUDNN_ROOT=/usr/
        -DWITH_STYLE_CHECK=ON
        -DWITH_TESTING=OFF
        -DWITH_FAST_BUNDLE_TEST=ON
        -DCMAKE_MODULE_PATH=/opt/rocm/hip/cmake
        -DCMAKE_EXPORT_COMPILE_COMMANDS=ON
        -DWITH_FLUID_ONLY=OFF
    ========================================
+ cmake .. -DCMAKE_BUILD_TYPE=Release -DWITH_DSO=ON -DWITH_DOC=OFF -DWITH_GPU=OFF -DWITH_AMD_GPU=OFF -DWITH_DISTRIBUTE=OFF -DWITH_MKL=ON -DWITH_AVX=OFF -DWITH_GOLANG=OFF -DCUDA_ARCH_NAME=All -DWITH_SWIG_PY=ON -DWITH_C_API=OFF -DWITH_PYTHON=ON -DCUDNN_ROOT=/usr/ -DWITH_STYLE_CHECK=ON -DWITH_TESTING=OFF -DWITH_FAST_BUNDLE_TEST=ON -DCMAKE_MODULE_PATH=/opt/rocm/hip/cmake -DWITH_FLUID_ONLY=OFF -DCMAKE_EXPORT_COMPILE_COMMANDS=ON
-- Found Paddle host system: centos, version: 6.9
-- Found Paddle host system's CPU: 2 cores
-- CXX compiler: /opt/rh/devtoolset-2/root/usr/bin/c++, version: GNU 4.8.2
-- C compiler: /opt/rh/devtoolset-2/root/usr/bin/cc, version: GNU 4.8.2
-- Could NOT find Sphinx (missing:  SPHINX_EXECUTABLE)
-- Protobuf protoc executable: /paddle/build/third_party/install/protobuf/bin/protoc
-- Protobuf library: /paddle/build/third_party/install/protobuf/lib/libprotobuf.a
-- Protobuf version: 3.1
-- Could NOT find PythonInterp: Found unsuitable version "2.6.6", but required is at least "2.7" (found /usr/bin/python2)
-- Could NOT find PythonLibs (missing:  PYTHON_LIBRARIES PYTHON_INCLUDE_DIRS) (Required is at least version "2.7")
-- Found cblas and lapack in MKLML (include: /paddle/build/third_party/install/mklml/include, library: /paddle/build/third_party/install/mklml/lib/libmklml_intel.so)
-- BLAS library: /paddle/build/third_party/install/mklml/lib/libmklml_intel.so
-- Set /paddle/build/third_party/install/mkldnn/lib to runtime path
-- Build MKLDNN with MKLML /paddle/build/third_party/install/mklml
-- MKLDNN library: /paddle/build/third_party/install/mkldnn/lib/libmkldnn.so
-- warp-ctc library: /paddle/build/third_party/install/warpctc/lib/libwarpctc.so
-- Enable Intel OpenMP with /paddle/build/third_party/install/mklml/lib/libiomp5.so
-- Paddle version is 0.11.1a2
-- Compile with MKLDNNMatrix
-- Compile with MKLDNNLayers and MKLDNNActivations
-- Compile with MKLPackedLayers
CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
PYTHON_LIBRARY (ADVANCED)
    linked by target "paddle_pserver_main" in directory /paddle/paddle/pserver
    linked by target "paddle_trainer" in directory /paddle/paddle/trainer
    linked by target "paddle_merge_model" in directory /paddle/paddle/trainer
    linked by target "_swig_paddle" in directory /paddle/paddle/api

-- Configuring incomplete, errors occurred!
See also "/paddle/build/CMakeFiles/CMakeOutput.log".
See also "/paddle/build/CMakeFiles/CMakeError.log".

宿主机：Linux version 4.4.0-63-generic (buildd@lgw01-03) (gcc version 4.8.4 (Ubuntu 4.8.4-2ubuntu1~14.04.3) ) #84~14.04.2-Ubuntu SMP Mon Feb 13 11:04:58 UTC 2017
Docker版本： Docker version 17.03.1-ce, build 0801b25

[kuke]

从报错来看，应该是paddle相关的依赖没有装好。请参考 http://www.paddlepaddle.org/docs/develop/documentation/fluid/zh/build_and_install/build_from_source_cn.html

[Danceiny]

使用docker run -it -v $PWD:/paddle -e "WITH_GPU=OFF" -e "WITH_TESTING=OFF" paddlepaddle/paddle_manylinux_devel:cuda9.0_cudnn7 bash -x /paddle/paddle/scripts/docker/build.sh编译，这个不是不需要依赖其他任何软件了。即便是 Python 和 GCC 都不需要，因为我们会把所有编译工具都安装进一个 Docker 镜像里。吗？

[kuke]

@Danceiny 是的，有了docker就不需要在本地装依赖了

[putcn]

might have something to do with python version

Could NOT find PythonInterp: Found unsuitable version "2.6.6", but required is at least "2.7" (found /usr/bin/python2)
-- Could NOT find PythonLibs (missing:  PYTHON_LIBRARIES PYTHON_INCLUDE_DIRS) (Required is at least version "2.7")

going to look into this issue.
please use paddlepaddle/paddle:dev instead to build paddle for now.

[Yancey1989]

Hi @Danceiny
I think You need to export some environment variable to choose which Python environment you want to use in your building:

Paddle/paddle/scripts/docker/build.sh

Lines 17 to 29 in 598035f

	if [ "$1" == "cp27-cp27m" ]; then
	export LD_LIBRARY_PATH=/opt/_internal/cpython-2.7.11-ucs2/lib:${LD_LIBRARY_PATH#/opt/_internal/cpython-2.7.11-ucs4/lib:}
	export PATH=/opt/python/cp27-cp27m/bin/:${PATH}
	PYTHON_FLAGS="-DPYTHON_EXECUTABLE:FILEPATH=/opt/python/cp27-cp27m/bin/python
	-DPYTHON_INCLUDE_DIR:PATH=/opt/python/cp27-cp27m/include/python2.7
	-DPYTHON_LIBRARIES:FILEPATH=/opt/_internal/cpython-2.7.11-ucs2/lib/libpython2.7.so"
	elif [ "$1" == "cp27-cp27mu" ]; then
	export LD_LIBRARY_PATH=/opt/_internal/cpython-2.7.11-ucs4/lib:${LD_LIBRARY_PATH#/opt/_internal/cpython-2.7.11-ucs2/lib:}
	export PATH=/opt/python/cp27-cp27mu/bin/:${PATH}
	PYTHON_FLAGS="-DPYTHON_EXECUTABLE:FILEPATH=/opt/python/cp27-cp27mu/bin/python
	-DPYTHON_INCLUDE_DIR:PATH=/opt/python/cp27-cp27mu/include/python2.7
	-DPYTHON_LIBRARIES:FILEPATH=/opt/_internal/cpython-2.7.11-ucs4/lib/libpython2.7.so"
	fi

[Danceiny]

@Yancey1989 But I do NOT care these python environment staff.... what env variables combination should I export?

[Yancey1989]

Hi @Danceiny
paddlepaddle/paddle_manylinux_devel:* is used for building PaddlePaddle Python wheel package follow the manylinux stander which allows PaddlePaddle running on most Linux platform, and you can get some detail message from https://github.com/PaddlePaddle/Paddle/tree/develop/tools/manylinux1 .
So you can specified the Python ABI as the env var of build.sh, for exmaple:
PYTHON_ABI=cp27-cp27mu /bin/bash /paddle/script/docker/build.sh

But if you only want to run PaddlePaddle in the Docker container, you can use paddlepaddle/paddle:latest-dev to build PaddlePaddle and don't need to specified any env variable.

[Danceiny]

@Yancey1989 Thanks a lot. I've successfully built this.

[Yancey1989]

You're welcome and I will supplement details about the manylinux Docker image.