Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

julia-0.5.2 #3503

Merged
merged 55 commits into from
Aug 18, 2017
Merged
Show file tree
Hide file tree
Changes from 54 commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
9d1ec7c
Julia-0.5.2
dfornika Aug 10, 2017
ee165c0
Update meta.yaml
dfornika Aug 10, 2017
6e4a9ef
Update meta.yaml
dfornika Aug 10, 2017
a765a18
Update meta.yaml
dfornika Aug 10, 2017
329ecf4
Update meta.yaml
dfornika Aug 10, 2017
0399205
Update meta.yaml
dfornika Aug 10, 2017
b59d56c
Update meta.yaml
dfornika Aug 10, 2017
fb78a2f
Addressed review by @bgruening
dfornika Aug 11, 2017
09ea8b4
Removed libgit2 dependency
dfornika Aug 11, 2017
79a24de
Added libgit2 dependency, un-pinned version
dfornika Aug 11, 2017
cd8c0f3
Set JULIA_HISTORY inside conda env
dfornika Aug 11, 2017
060bf17
Pinned llvmdev 3.7.1. Run all tests
dfornika Aug 12, 2017
056cb20
Add TAGGED_RELEASE_BANNER
bgruening Aug 12, 2017
50c9974
Remove testing from build.sh script
dfornika Aug 12, 2017
73cbd34
Merge remote-tracking branch 'upstream/master'
dfornika Aug 13, 2017
abb9ead
Add tagged release banner
dfornika Aug 13, 2017
4315d8f
Merge branch 'bgruening-patch-1'
dfornika Aug 13, 2017
c23dafc
Merge branch 'master' into patch-1
dfornika Aug 13, 2017
9f461d6
Merge pull request #1 from bgruening/patch-1
dfornika Aug 13, 2017
a1e0094
Update meta.yaml
dfornika Aug 14, 2017
9a16cd9
Update meta.yaml
dfornika Aug 14, 2017
8d1870b
Update meta.yaml
dfornika Aug 14, 2017
e50a172
Removed rpath patch, build llvm & fftw
dfornika Aug 15, 2017
f58ef7c
Remove conda mpfr
dfornika Aug 15, 2017
b42eac4
Added toolchain and patchelf to build reqs
dfornika Aug 15, 2017
ea8152b
Remove fftw
dfornika Aug 15, 2017
feab085
Add fftw
dfornika Aug 15, 2017
4fda532
Added juliarc.jl file
dfornika Aug 15, 2017
7f5062a
Removed toolchain
dfornika Aug 15, 2017
2e8b807
Added toolchain
dfornika Aug 16, 2017
f967c10
Added libtool
dfornika Aug 16, 2017
7ad4d18
Removed libtool
dfornika Aug 16, 2017
3f4ada5
Added gcc
dfornika Aug 16, 2017
1b9c389
Added libgcc
dfornika Aug 16, 2017
461dda3
Added mpfr and fftw as runtime deps
dfornika Aug 16, 2017
8a38506
[skip appveyor] Removed juliarc.jl
dfornika Aug 16, 2017
1f5a6e7
[skip appveyor] Pipe to bar during testing to avoid timeout
dfornika Aug 16, 2017
bc91f58
[skip appveyor] Add timeout modifier to test phase
dfornika Aug 16, 2017
62a07eb
[skip appveyor] Fixed syntax for timeout modifier
dfornika Aug 16, 2017
0b5087b
[skip appveyor] Fixed syntax for timeout modifier
dfornika Aug 17, 2017
6feace4
[skip appveyor] Removed timeout modifier
dfornika Aug 17, 2017
c92d774
[skip appveyor] added perl as runtime dep
dfornika Aug 17, 2017
fc2f8d3
[skip appveyor] Added python2 as runtime dep
dfornika Aug 17, 2017
22c76f5
[skip appveyor] Moved perl & python2 to build-time
dfornika Aug 17, 2017
9be917f
[skip appveyor] Moved perl requirement to test section
dfornika Aug 17, 2017
b87fd4e
[skip appveyor] Removed python 3.6 from build reqirements
dfornika Aug 17, 2017
9e15f34
[skip appveyor] Removed automake, autoconf and libxml2
dfornika Aug 17, 2017
747d101
[skip appveyor] Added perl as build requirement
dfornika Aug 17, 2017
8c6810f
[skip appveyor] Missing space before package name perl
dfornika Aug 17, 2017
d15efd2
[skip appveyor travis] Added arpack
dfornika Aug 17, 2017
e41adf6
[skip appveyor] [skip travis] Added curl as runtime requirement
dfornika Aug 17, 2017
a2b0fbe
[skip appveyor] Removed libssh2
dfornika Aug 17, 2017
79e1f4e
[skip appveyor] Added suitesparse
dfornika Aug 17, 2017
0a95d04
[skip appveyor] Added timeout modifier
dfornika Aug 18, 2017
6cb50af
[skip appveyor] Removed timeout modifier
dfornika Aug 18, 2017
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions recipes/julia/build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
#!/bin/sh

export C_INCLUDE_PATH=${PREFIX}/include
export LD_LIBRARY_PATH=${PREFIX}/lib
export LIBRARY_PATH=${PREFIX}/lib
export CMAKE_PREFIX_PATH=${PREFIX}

make -j 4 prefix=${PREFIX} MARCH=core2 sysconfigdir=${PREFIX}/etc NO_GIT=1 \
USE_SYSTEM_LIBGIT2=1 USE_LLVM_SHLIB=0 USE_SYSTEM_CURL=1 USE_SYSTEM_MPFR=1 \
USE_SYSTEM_PATCHELF=1 USE_SYSTEM_LIBSSH2=1 USE_SYSTEM_LLVM=0 USE_SYSTEM_BLAS=1 \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this mean we should be requiring patchelf? Is it needed just for the build or at run time too?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My (very rudimentary) guess is that it's only needed at build time.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would be my guess too. We may want to add patchelf as a build requirement then.

USE_SYSTEM_FFTW=1 USE_SYSTEM_GMP=1 USE_SYSTEM_LAPACK=1 USE_SYSTEM_ARPACK=1 USE_SYSTEM_SUITESPARSE=1 \
LIBBLAS=-lopenblas LIBBLASNAME=libopenblas.so LIBLAPACK=-lopenblas LIBLAPACKNAME=libopenblas.so \
TAGGED_RELEASE_BANNER="conda-forge-julia release" \
install

mv "$PREFIX/bin/julia" "$PREFIX/bin/julia_"
cp "$RECIPE_DIR/julia-wrapper.sh" "$PREFIX/bin/julia"
chmod +x "$PREFIX/bin/julia"

24 changes: 24 additions & 0 deletions recipes/julia/julia-wrapper.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
#!/bin/bash

JULIA_HOME="$(dirname "${BASH_SOURCE[0]}")"

# Set JULIA_PKGDIR to $PREFIX/share/julia/site to avoid contaminating
# user's ~/.julia/. This script will modify Pkg.dir() and
# Base.LOAD_CACHE_PATH[1] appropriately. Note that wrapper script is
# used rather than /etc/julia/juliarc.jl because LOAD_CACHE_PATH is
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you should also be able to just replace its contents

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you clarify? I'm not sure what you're suggesting.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

modify LOAD_CACHE_PATH in a juliarc instead of doing this non portable wrapper script business

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Though juliarc is going to be in the user's home directory, right? It would be better if we had a way to set this without touching the user's home directory. Is there one?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there's a juliarc in the installation as well

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I missed this before and maybe you did as well @tkelman, but the comment here says that "LOAD_CACHE_PATH is configured in base/sysimg.jl which is loaded before juliarc.jl." Hence its manipulation in the wrapper script. Knowing very little about Julia's loading procedure I can't comment further, but it sounds like changes to LOAD_CACHE_PATH in juliarc will simply be ignored. Thoughts on this?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the sysimg.jl value is just a default, it's a mutable array so can be modified anywhere

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right. That much is clear. What is less clear is what has already been done based on LOAD_CACHE_PATH at this point. Are there any docs on Julia's loading procedure?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW opened issue ( conda-forge/julia-feedstock#10 ) to get rid of the wrapper script and @dfornika's PR ( conda-forge/julia-feedstock#11 ) is an attempt to solve this. Maybe you can advise on how best to proceed.

# configured in base/sysimg.jl which is loaded before juliarc.jl.
if [ -z ${JULIA_PKGDIR+x} ] # see: http://stackoverflow.com/a/13864829
then
JULIA_PKGDIR="$(dirname "$JULIA_HOME")/share/julia/site"
export JULIA_PKGDIR
fi

# Set JULIA_HISTORY to $PREFIX/.julia_history to avoid saving
# to user's $HOME
if [ -z ${JULIA_HISTORY+x} ]
then
JULIA_HISTORY="$(dirname "$JULIA_HOME")/.julia_history"
export JULIA_HISTORY
fi

exec "$JULIA_HOME/julia_" "$@"
70 changes: 70 additions & 0 deletions recipes/julia/meta.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
{% set name = "julia" %}
{% set version = "0.5.2" %}
{% set sha256 = "efa3303d63e39bf3ea06e23ef68807cca0095c7e0507bdf26eae1d5dee7868c7" %}
{% set variant = "openblas" %}

package:
name: {{ name|lower }}
version: {{ version }}

source:
fn: v{{ version }}.tar.gz
url: https://github.com/JuliaLang/julia/archive/v{{ version }}.tar.gz
sha256: {{ sha256 }}

build:
skip: True # [osx or win]
number: 0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we are using openblas, we should change this like so.

  number: 200
  features:
    - blas_{{ variant }}

features:
- blas_{{ variant }}

requirements:
build:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also please add toolchain as a build dependency.

- gcc # [linux]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need gcc on Linux? There should be a compiler in the docker image.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The build fails at a very early stage if I don't add gcc.

- toolchain
- patchelf
- cmake
- zlib 1.2.8
- python 2.7.*
- perl 5.20.*
- openblas 0.2.19|0.2.19.*
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also should add

    - blas 1.1 {{ variant }}

- blas 1.1 {{ variant }}
- fftw
- gmp 6.1.*
- mpfr 3.1.*
- curl
- libgit2
- arpack
- suitesparse

run:
- zlib 1.2.8
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tkelman, is zlib used? Or is it used by a dependency?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

indirect I believe

- libgcc # [linux]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be dropped if gcc ends up being dropped above.

- libgit2
- fftw
- gmp 6.1.*
- mpfr 3.1.*
- openblas 0.2.19|0.2.19.*
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

...and again should add

    - blas 1.1 {{ variant }}

- blas 1.1 {{ variant }}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You probably need curl here

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

libcurl is also indirect at runtime (for proxies in libgit2 if it's built with that support), though curl or wget is used for downloads at build and test time

Copy link
Member

@tkelman tkelman Aug 17, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can keep it for the Base.download function though (edit: on non-windows platforms, since we use something else on windows)

- arpack
- suitesparse
- curl

test:
commands:
- julia -e 'Base.runtests("all"); Base.runtests("pkg libgit2-online")':
timeout: 1800s
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does this work?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's CircleCI.yml functionality. This file is for conda-build

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's CircleCI.yml functionality. This file is for conda-build

requires:
- perl

about:
home: http://julialang.org/
license: MIT
license_file: LICENSE.md
summary: A high-performance dynamic programming language for data science

extra:
recipe-maintainers:
- bgruening
- dfornika
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feel free to add me.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I'm going to add @acaprez too.

- acaprez