libcf #2334

dnadeau4 · 2017-02-02T02:43:00Z

Build libcf dependency for UV-CDAT.

conda-forge-linter · 2017-02-02T02:43:05Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipes/libcf) and found it was in an excellent condition.

ocefpaf · 2017-02-02T09:13:39Z

recipes/libcf/build.sh

@@ -0,0 +1,15 @@
+export CFLAGS="-Wall -g -m64 -pipe -O2  -fPIC"


Please change these to: export CFLAGS="-Wall -g -m64 -pipe -O2 -fPIC $CFLAGS"
So the original flags are preserved.

hum... In general I'm reluctant to picking up the user's flags... Does your script sets CFALGS in any special ways?

Don't worry about that in conda-forge. In a normal build there should be nothing in there. However, we want the scripts to be generic enough so other people with different build systems can inject whatever they need there. (We even have packages that do that in conda-forge to ensure some OS X specific variables for clang are used.)

So if a user is modifying that they should know what they are doing.

ocefpaf · 2017-02-02T09:15:07Z

recipes/libcf/build.sh

+export LFLAGS="-fPIC"
+
+CONDA_LST=`conda list`
+if [[ ${CONDA_LST}'y' == *'openmpi'* ]]; then


Ideally this should not be here. I believe @msarahan and @minrk are working on a way to track mpi variants with features and they can probably chime with the best practice for doing this.

For now I recommend you to choose one and avoid this if-clause.

@msarahan @minrk I would love to hear back from you on this. It is indeed a thorny point with conda and our system. If you want to take adavantage of mpi then some software needs to be compiled in a special fashion. Easiest example is hdf5. If I want to take advantage on hdf5 mpi's capabilities then I need hdf5 to be compiled against mpi. And flavors probably matter to. The way we solve this in our system is that my meta.yaml are actually pre-processed by a prep_for_build.py script (see: https://github.com/UV-CDAT/conda-recipes/blob/master/prep_for_build.py ) that make sure the dependencies are then set properly. It does smell a lot like conda's feature thing but it's not really. The only way to support different implementation is to add the "feature" name to the package for example using the same meta.yaml.in I generate either a meta.yaml that will be producing "mypackage" or "mypackage-openmpi" I implemented a {} system similar to conda's [] to support this. It is not ideal.
In the case of conda-forge I'm ok with offering two different recipes one for the "Standard" build one for the "package-openmpi" or "package-mpch" etc... But having a conda-proper way of doing things would be much better.

The discussion of how to express which mpi implementation is used is mainly here. Features should work for this, it's mainly a question of deciding precisely how those features should be defined / tracked.

If there does need to be logic in build.sh, I'm not sure that such things are passed down to the build environment. Are they?

ocefpaf · 2017-02-02T09:16:06Z

recipes/libcf/meta.yaml

+requirements:
+  build:
+    - hdf5
+    - libnetcdf 4.3.3.1


Do you need this strict pinning for libnetcdf? Please see conda-forge pinning docs at:
https://github.com/conda-forge/staged-recipes/wiki/Pinned-dependencies

historical no need to. I'll fix this.

ocefpaf

Fix the pinnings to be consistent with https://github.com/conda-forge/staged-recipes/wiki/Pinned-dependencies

dnadeau4 · 2017-02-07T17:17:57Z

@ocefpaf Appveyor hung here as well!

dnadeau4 · 2017-02-17T23:54:11Z

@ocefpaf Yeah!!!

ocefpaf · 2017-02-17T23:56:36Z

recipes/libcf/build.sh

+
+if [ "$(uname)" == "Darwin" ]; then
+    export CXXFLAGS="${CXXFLAGS} -fno-common"
+    export MACOSX_DEPLOYMENT_TARGET=$(sw_vers -productVersion | sed -E "s/([0-9]+\.[0-9]+).*/\1/")


This should be fixed in conda-forge to ensure compatibility with the rest of the packages we ship. Take a look at packages like the toolchain that defines this.

I hope your activate/deactivate works. This is a hard one to compile on OSX.
toolchain

ocefpaf · 2017-02-17T23:57:49Z

recipes/libcf/build.sh

+fi
+
+CONDA_LST=`conda list`
+if [[ ${CONDA_LST}'y' == *'openmpi'* ]]; then


This is not OK. We cannot rely on conda list. For conda-forge you need enforce openmpi and wait until the features for the other variants is resolved.

@ocefpaf @dnadeau4 let's take it out for now. We need to revisit the whole suite eventually to use conda features for this. As discussed in earlier conda-forge recipes.

I'll get rid of that!

Sounds like the best approach for now. (Please join the discussion for the features to help move this forward.)

ocefpaf · 2017-02-17T23:58:39Z

recipes/libcf/meta.yaml

+requirements:
+  build:
+    - hdf5
+    - libnetcdf 


Please check the pinning script and pin the version here.

See https://github.com/conda-forge/staged-recipes/wiki/Pinned-dependencies

@ocefpaf I'm not sure what you're asking for here. Do you want us to pin or just confirm it will work against this? Or run the pinning script? We currently build in house against cnda-forge versions of hdf5 and netcdf. Please let us know what's really wanted here. Thanks!

I need to pin the version using the wiki page info

pin according to the wiki I sent you. We need that to ensure ABI stability and package compatibility.

The pinnings are automatically updated when we update the dependencies.

ocefpaf · 2017-02-17T23:59:29Z

@dnadeau4 there are still a few points we need to address before merging this to make this package OK for conda-forge.

conda-forge-linter · 2017-02-18T00:42:14Z

Hi! This is the friendly automated conda-forge-linting service.

I wanted to let you know that I linted all conda-recipes in your PR (recipes/libcf) and found some lint.

Here's what I've got...

For recipes/libcf:

Selectors are suggested to take a <two spaces>#<one space>[<expression>] form.

conda-forge-linter · 2017-02-18T00:45:32Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipes/libcf) and found it was in an excellent condition.

ocefpaf · 2017-02-18T11:38:05Z

recipes/libcf/build.sh

+
+./configure --prefix=${PREFIX}
+${PYTHON} setup.py install
+if [ `uname` == Darwin ]; then install_name_tool -change /System/Library/Frameworks/Python.framework/Versions/2.7/Python @rpath/libpython2.7.dylib ${SP_DIR}/pycf/*.so ; fi


If this is a Python 2 only package we need a skip statement below.

ocefpaf · 2017-02-18T11:38:22Z

recipes/libcf/meta.yaml

+
+build:
+  number: 1
+  skip: True  # [win or py3k] 


Ah. We already have it!

ocefpaf · 2017-02-18T11:38:31Z

recipes/libcf/meta.yaml

+  sha256: e40d3826ca2b47538b075f69f716798e255fe396fd3ae68e8b051be57a951bb4
+
+build:
+  number: 1


Can you reset this to 0?

ocefpaf · 2017-02-18T11:39:20Z

recipes/libcf/meta.yaml

+    - clapack >=3.2.1
+    - curl
+    - ossuuid
+    - gcc


Do you need Fortran or libgmop for this package? If not you don't need the gcc package.

I need Fortran in this package.

ocefpaf · 2017-02-18T11:40:02Z

recipes/libcf/meta.yaml

+    - clapack >=3.2.1
+    - curl
+    - ossuuid
+    - libgcc


If you are using gcc/libgcc only for Fortran you can try using the libgfortan package instead of libgcc.

I'll change it for libgfortran.

dnadeau4 · 2017-02-26T08:34:25Z

@ocefpaf for some reason, travis did not build the osx and return green. I don't have libcf for osx in libcf-feedstock. I created an issue, but thought we could re-trigger here. What should be done?

ocefpaf · 2017-02-26T12:12:30Z

I'm in transit. I'll take a look at it soon. Try restarting Travis-CI for now.

…

On Feb 26, 2017 5:34 AM, "Denis Nadeau" ***@***.***> wrote: @ocefpaf <https://github.com/ocefpaf> for some reason, travis did not build the osx and return green. I don't have libcf for osx in libcf-feedstock. I created an issue, but thought we could re-trigger here. What should be done? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2334 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA6BL3RZT_srBwwPS2dgahNtb52XTFyuks5rgTkSgaJpZM4L0qFv> .

dnadeau4 · 2017-03-02T18:52:10Z

@ocefpaf If we can get travis.yml in feedstock to compile osx, that will make it possible for me to add cdms2. I have cdutil and genutil packages depending on cdms2 that is in my queue for conda-forge.

As I told @jakirkham I can also create a brand new PR in staged-recipe for this. conda-forge/conda-smithy#465

build libcf

d991d9f

ocefpaf reviewed Feb 2, 2017

View reviewed changes

ocefpaf requested changes Feb 2, 2017

View reviewed changes

doutriaux1 mentioned this pull request Feb 2, 2017

build ESMP #2333

Merged

fix skip windows and py3k section

3f20dcb

BoPeng mentioned this pull request Feb 7, 2017

Cluster job submission and monitoring features. vatlab/sos#406

Closed

dnadeau4 and others added 14 commits February 7, 2017 17:43

retrigger

b11987c

add gcc

7c7a79d

delete space

ff3ea0d

Merge branch 'master' into libcf

578494d

Fix Build and pinning libnetcdf [skip appveyor]

e25a640

add libgfortran

3e23342

reset number to 0 and remove libgfortran

baca045

cat config.log to see what is going on

4883676

Deploy for MAC OSX

84664dc

Try more env variables

6b58dc9

try again

8279229

Change python library Path to rpath

7766d46

change SP_DIR/libcf

0e2ff2f

change wrong directory libcf to pycf

34ec668

ocefpaf reviewed Feb 17, 2017

View reviewed changes

Add activate/deactivate like in toolchain example

9b6b3c8

let toolchain do its own magick in meta.yaml [skip appveyor]

aedaa32

add space before #[]

9851167

ocefpaf reviewed Feb 18, 2017

View reviewed changes

recipes/libcf/meta.yaml Outdated

build:

number: 1

skip: True # [win or py3k]

Copy link

Member

ocefpaf Feb 18, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah. We already have it!

ocefpaf reviewed Feb 18, 2017

View reviewed changes

change number to 1 and replace libgcc for libgfortran

b48bbdd

ocefpaf approved these changes Feb 18, 2017

View reviewed changes

ocefpaf merged commit ad5ed95 into conda-forge:master Feb 18, 2017

		@@ -0,0 +1,15 @@
		export CFLAGS="-Wall -g -m64 -pipe -O2 -fPIC"

libcf #2334

libcf #2334

Conversation

dnadeau4 commented Feb 2, 2017

conda-forge-linter commented Feb 2, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

doutriaux1 Feb 2, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ocefpaf left a comment

Choose a reason for hiding this comment

dnadeau4 commented Feb 7, 2017

dnadeau4 commented Feb 17, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ocefpaf Feb 18, 2017 • edited Loading

Choose a reason for hiding this comment

ocefpaf commented Feb 17, 2017

conda-forge-linter commented Feb 18, 2017

conda-forge-linter commented Feb 18, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dnadeau4 commented Feb 26, 2017

ocefpaf commented Feb 26, 2017 via email

dnadeau4 commented Mar 2, 2017

doutriaux1 Feb 2, 2017 •

edited

Loading

ocefpaf Feb 18, 2017 •

edited

Loading