-
Notifications
You must be signed in to change notification settings - Fork 5.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
use -rpath to fix libmklml_intel.so not found #11806
Changes from 1 commit
d12a386
3e4b0d7
ccda8e4
25a2b7c
37ee566
f0deb4c
446377e
31d6387
7604258
83034c1
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,15 @@ | ||
# Copyright (c) 2018 PaddlePaddle Authors. All Rights Reserved | ||
# | ||
# Licensed under the Apache License, Version 2.0 (the "License"); | ||
# you may not use this file except in compliance with the License. | ||
# You may obtain a copy of the License at | ||
# | ||
# http://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, software | ||
# distributed under the License is distributed on an "AS IS" BASIS, | ||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
# See the License for the specific language governing permissions and | ||
# limitations under the License. | ||
|
||
# used for setup.py.in to store the thirdparty shared libraries |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,7 @@ | ||
from setuptools import setup, Distribution, Extension | ||
import subprocess | ||
import shutil | ||
import os | ||
class BinaryDistribution(Distribution): | ||
def has_ext_modules(foo): | ||
return True | ||
|
@@ -62,6 +64,7 @@ write_version_py(filename='@PADDLE_BINARY_DIR@/python/paddle/version.py') | |
|
||
|
||
packages=['paddle', | ||
'paddle.libs', | ||
'paddle.utils', | ||
'paddle.dataset', | ||
'paddle.reader', | ||
|
@@ -113,12 +116,30 @@ package_dir={ | |
} | ||
if '${WITH_FLUID_ONLY}'== 'OFF': | ||
package_dir['py_paddle']='${PADDLE_BINARY_DIR}/python/py_paddle' | ||
|
||
|
||
paddle_rt_lib_dir = 'lib' | ||
paddle_rt_libs = ['${WARPCTC_LIBRARIES}'] | ||
if '${MKL_SHARED_LIBS}'!= '': | ||
paddle_rt_libs += '${MKL_SHARED_LIBS}'.split(';') | ||
# put all thirdparty libraries in paddle.libs | ||
package_data['paddle.libs']=['libwarpctc.so'] | ||
libs_path='${PADDLE_BINARY_DIR}/python/paddle/libs' | ||
shutil.copy('${WARPCTC_LIBRARIES}', libs_path) | ||
if '${WITH_MKL}'== 'ON': | ||
shutil.copy('${MKLML_LIB}', libs_path) | ||
shutil.copy('${MKLML_IOMP_LIB}', libs_path) | ||
package_data['paddle.libs']+=['libmklml_intel.so','libiomp5.so'] | ||
if '${WITH_MKLDNN}'== 'ON': | ||
package_data['paddle.libs']+=['libmkldnn.so.0'] | ||
shutil.copy('${MKLDNN_SHARED_LIB}', libs_path) | ||
package_dir['paddle.libs']=libs_path | ||
|
||
# change rpath of core.so, add $ORIGIN/../libs/ to it. | ||
# The reason is that libwarpctc.so, libiomp5.so etc are in paddle.libs, and | ||
# core.so is in paddle.fluid, thus paddle/fluid/../libs will pointer to above libraries. | ||
# This operation will fix https://github.com/PaddlePaddle/Paddle/issues/3213 | ||
os.environ['core_rpath']=str(os.popen("patchelf --print-rpath \ | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 为什么要用environ?直接存一个string可以吗? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 存一个string的写法一直没有搞定,所以换成environ了。 |
||
${PADDLE_BINARY_DIR}/python/paddle/fluid/core.so").read().strip('\n')) | ||
os.environ['core_rpath']=os.environ['core_rpath']+":'$ORIGIN/../libs/'" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 其实我觉得是不是别的路径都不需要,只要ORIGIN那个就好了,因为他的路径都是编译时就有的。安装之后很大可能也用不到。
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 有道理。 |
||
command = "patchelf --set-rpath " + os.environ['core_rpath'] +\ | ||
" ${PADDLE_BINARY_DIR}/python/paddle/fluid/core.so" | ||
os.system(command) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Same as above, need some checks. |
||
|
||
setup(name='${PACKAGE_NAME}', | ||
version='${PADDLE_VERSION}', | ||
|
@@ -128,6 +149,5 @@ setup(name='${PACKAGE_NAME}', | |
ext_modules=[Extension('_foo', ['stub.cc'])], | ||
package_data=package_data, | ||
package_dir=package_dir, | ||
scripts=paddle_bins, | ||
data_files=[(paddle_rt_lib_dir, paddle_rt_libs)] | ||
scripts=paddle_bins | ||
) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
为什么要加这个文件呢?我看numpy的lib下面只有so.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
一开始如果没有__init__.py的话,whl打包的时候,会找不到这个目录。不过可以尝试在打包后删除这个文件。