Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

Julia path patch #15561

Merged
merged 3 commits into from
Jul 18, 2019
Merged

Julia path patch #15561

merged 3 commits into from
Jul 18, 2019

Conversation

aaronmarkham
Copy link
Contributor

@aaronmarkham aaronmarkham commented Jul 16, 2019

Description

Another followup PR to fix the Julia site build and unblock website publishing...
Since the website builds so many versions, it does so in their own folders inside /mxnet/docs/build_version_doc/apache-mxnet/... and this makes it difficult for Julia to find when it's expecting the binaries in /work/mxnet/lib. After digging through the Julia source and testing a bunch of things I found that $LD_LIBRARY_PATH doesn't seem to do anything, but $MXNET_HOME is what Julia is looking for. It then manually appends lib.
Unfortunately, according to the MXNet environment variables page, this same env var is used elsewhere to store models in .mxnet, not a typical $HOME/incubator-mxnet or what is often called $MXNET_ROOT in tutorials.

Unblocking website publishing is my primary goal, not dealing with conflicting environment variables, so I'm patching this for now, but I think the Julia package should updated to use $MXNET_LIBRARY_PATH or $LD_LIBRARY_PATH instead.

My test run used a test CI pipeline that points to my fork:
http://jenkins.mxnet-ci.amazon-ml.com/job/test-website-build/

The "successful" run was here. It fails because there's no place to publish, which is fine. Everything else worked: http://jenkins.mxnet-ci.amazon-ml.com/job/test-website-build/3/console
We might want to keep this test pipeline around... it took me awhile to remember how to create one for testing, and I definitely should have done this thorough test in the first place.

It was able to find Julia, and it used the locally compiled binary based off of the target branch: this one.

I'm hoping this is the last surprise that the CI environment is going to give me for getting Julia to publish.

Previous PRs for the Julia docs publishing:
#15554
#15523
#15454

@iblislin
Copy link
Member

Okay, I can make a patch for rename MXNET_HOME to MXNET_ROOT, then add a search candidate for MXNET_LIBRARY_PATH.

In case of LD_LIBRARY_PATH, I think it works already..
This variable will be picked up by C's dlopen.

% LD_LIBRARY_PATH=/home/iblis/git/mxnet/lib julia -e 'using Libdl; println(Libdl.dlopen("libmxnet.so"))' 
Ptr{Nothing} @0x000055b0eca49c50

@iblislin iblislin added the Julia label Jul 17, 2019
iblislin added a commit that referenced this pull request Jul 17, 2019
- Add MXNET_LIBRARY_PATH support

Ref: #15561
docs/mxdoc.py Outdated Show resolved Hide resolved
Co-Authored-By: Iblis Lin <iblis@hs.ntnu.edu.tw>
@aaronmarkham aaronmarkham merged commit c8a1de4 into apache:master Jul 18, 2019
iblislin added a commit that referenced this pull request Jul 19, 2019
- Add MXNET_LIBRARY_PATH support

Ref: #15561
iblislin added a commit that referenced this pull request Jul 23, 2019
- Add MXNET_LIBRARY_PATH support

Ref: #15561
anirudhacharya pushed a commit to anirudhacharya/mxnet that referenced this pull request Aug 20, 2019
* fix path to be static

* add MXNET_HOME setting for Julia docs

* Update docs/mxdoc.py

Co-Authored-By: Iblis Lin <iblis@hs.ntnu.edu.tw>
iblislin added a commit that referenced this pull request Sep 8, 2019
* julia: rename build env var `MXNET_HOME` to `MXNET_ROOT`

- Add MXNET_LIBRARY_PATH support

Ref: #15561

* backward compatibility
larroy pushed a commit to larroy/mxnet that referenced this pull request Sep 28, 2019
* julia: rename build env var `MXNET_HOME` to `MXNET_ROOT`

- Add MXNET_LIBRARY_PATH support

Ref: apache#15561

* backward compatibility
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants