Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[documentation] JuliaHSL was renamed libHSL #721

Closed
amontoison opened this issue Nov 8, 2023 · 9 comments
Closed

[documentation] JuliaHSL was renamed libHSL #721

amontoison opened this issue Nov 8, 2023 · 9 comments

Comments

@amontoison
Copy link
Contributor

amontoison commented Nov 8, 2023

@svigerske
We renamed JuliaHSL into libHSL.
The new link is https://licences.stfc.ac.uk/product/libhsl.
This new version contains a few fixes and improvements.
It also provides precompiled shared libraries for Windows and Mac (Intel or M1) with the dependencies that can be easily used outside Julia.

Note that the new release of Coin-HSL (https://licences.stfc.ac.uk/product/coin-hsl) also contains the precompiled binaries for Windows / Mac.

@svigerske
Copy link
Member

svigerske commented Nov 9, 2023

Do the precompiled libraries from Coin-HSL also include the CoinHslConfig.h header and coinhsl.pc pkg-config file, so they can be used to link Ipopt to HSL (at build time)?

And from the checksums and what you wrote, I take it that the precompiled binaries of Coin-HSL are not available for Linux.

But then also the checksums of libhsl and the https://licences.stfc.ac.uk/product/libhsl page look as if there are no precompiled libHSL available for Linux. Except for HSL_jll.jl, so that, in case of Linux, one would need to unpack the Julia package?

Finally, so far, we had in the description that one should get dependencies from OpenBLAS32, METIS, and
CompilerSupportLibraries. But I see some openblas_HSL_jll.jl* files at libhsl as well. There are the same thing?

@amontoison
Copy link
Contributor Author

amontoison commented Nov 9, 2023

Coin-HSL doesn't contain the CoinHslConfig.h header and coinhsl.pc pkg-config file but we can easily add it!
I was not aware that it was needed to link Ipopt to HSL.
I thought that the link flag -lcoinhsl or -lhsl was enough.

We didn't precompiled binaries for Coin-HSL or libHSL on Linux but we could add it if you think that it's relevant.
Linux users generally know how to compile code compared to Windows / Mac users.
If the user doesn't want to compile code, he can unpack HSL_jll.jl and download the dependencies but it will be a little bit annoying.

The archives libHSL_binaries-2023.11.7.aarch64-apple-darwin-libgfortran5.tar.gz, libHSL_binaries-2023.11.7.x86_64-apple-darwin-libgfortran5.tar.gz and libHSL_binaries-2023.11.7.x86_64-w64-mingw32-libgfortran5.zip contains the shared library of libhsl as well as a static library, Fortran module files and shared libraries of all dependencies.
We renamed JuliaHSL into libHSL because of that.

openblas_HSL_jll.jl* and lbt_HSL_jll.jl* only contains the shared library libhsl.$dlext and the library is linked with libopenblas.$dlext and libblastrampoline.$dlext, respectively.
The dependencies are downloaded automatically by Julia when we install the package HSL_jll.jl.

@svigerske
Copy link
Member

The CoinHslConfig.h tells Ipopt, which linear solvers are included in the HSL library. The coinhsl.pc tells the Ipopt buildsystem, what compiler and linker flags it should use for the Coin-HSL library, so one won't have to specify --with-hsl-cflags and --with-hsl-lflags manually. For that, coinhsl.pc needs to be relocatable (contain only relative paths).

It's fine that there is no Linux build (except for the Julia package). Just need to know what to write in the docu.

So openblas_HSL_jll.jl* and lbt_HSL_jll.jl* are different variants of libHSL. If libHSL now brings all dependencies, then that simplifies things.

svigerske added a commit that referenced this issue Nov 10, 2023
- at least libHSL already includes all dependencies
- Coin-HSL so far misses the header file to be used as buildtime, so
  it is essentially the same as libHSL (just smaller)
- both are only for macOS and Windows (for Linux, one would need to
  unpack a julia package; we skip mentioning that)
- see #721
@amontoison
Copy link
Contributor Author

amontoison commented Nov 10, 2023

@AndrewLister-STFC @jfowkes
I suggest to add CoinHslConfig.h and coinhsl.pc in Coin-HSL.

Yes, openblas_HSL_jll.jl* and lbt_HSL_jll.jl* are not linked with the same library for BLAS and LAPACK symbols.
The LBT version enables dynamic dispatch to BLAS/LAPACK symbols of BLIS, Intel MKL, Apple Accelerate, OpenBLAS, etc...

ldd openblas_HSL_jll.jl-2023.11.7/HSL_jll.jl-2023.11.7/override/lib/x86_64-linux-gnu-libgfortran5/libhsl.so

	linux-vdso.so.1 (0x00007fff51ff9000)
	libopenblas.so => /lib/x86_64-linux-gnu/libopenblas.so (0x00007fe4155b0000)
	libmetis.so => /lib/x86_64-linux-gnu/libmetis.so (0x00007fe418b16000)
	libmpifort.so.12 => not found
	libgfortran.so.5 => /lib/x86_64-linux-gnu/libgfortran.so.5 (0x00007fe415200000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fe418a2d000)
	libgomp.so.1 => /lib/x86_64-linux-gnu/libgomp.so.1 (0x00007fe415566000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fe415546000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fe414e00000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fe418b97000)
	libquadmath.so.0 => /lib/x86_64-linux-gnu/libquadmath.so.0 (0x00007fe4154fe000)
ldd lbt_HSL_jll.jl-2023.11.7/HSL_jll.jl-2023.11.7/override/lib/x86_64-linux-gnu-libgfortran5/libhsl.so

	linux-vdso.so.1 (0x00007fffa1186000)
	libblastrampoline.so.5 => not found
	libmetis.so => /lib/x86_64-linux-gnu/libmetis.so (0x00007f72efd96000)
	libmpifort.so.12 => not found
	libgfortran.so.5 => /lib/x86_64-linux-gnu/libgfortran.so.5 (0x00007f72efa00000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f72ef919000)
	libgomp.so.1 => /lib/x86_64-linux-gnu/libgomp.so.1 (0x00007f72f0e22000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f72efd76000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f72ef600000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f72f0e85000)
	libquadmath.so.0 => /lib/x86_64-linux-gnu/libquadmath.so.0 (0x00007f72efd2e000)

svigerske added a commit to coin-or-tools/ThirdParty-HSL that referenced this issue Nov 10, 2023
- STFC's Coin-HSL now has prebuild libs for macOS and Windows, see
  coin-or/Ipopt#721
- julia-hsl is now called libHSL, but that seems to be just a superset
  of Coin-HSL
@svigerske
Copy link
Member

OK, thank you for the clarification. The Ipopt docu has been updated accordingly.

@AndrewLister-STFC
Copy link

@AndrewLister-STFC @jfowkes I suggest to add CoinHslConfig.h and coinhsl.pc in Coin-HSL.

We can do this. For the header it's just a file with #defines right?

#define COINHSL_HAS_MA27S
#define COINHSL_HAS_MA27D
...

And for coinhsl.pc it looks like meson's pkg module will make it.

I'm still sorting out a coinhsl release so can include it in that when I upload next week (having some issues uploading zip files to our licence portal). On a related note, the libHSL windows files aren't available to download yet but we're working on it.

@svigerske
Copy link
Member

On ThirdParty-HSL, we don't have the D on the double-precision variants, otherwise yes.
https://github.com/coin-or-tools/ThirdParty-HSL/blob/stable/2.2/CoinHslConfig.h.in

The COINHSL_VERSION* can be omitted. Noone uses these, I believe.

@AndrewLister-STFC
Copy link

On ThirdParty-HSL, we don't have the D on the double-precision variants, otherwise yes. coin-or-tools/ThirdParty-HSL@stable/2.2/CoinHslConfig.h.in

The COINHSL_VERSION* can be omitted. Noone uses these, I believe.

Sorry to keep posting here... meson gave this pkgconfig file. I can make the description a bit more informative, but is this sufficient for IPOPT's needs?

$ cat lib/pkgconfig/coinhsl.pc
prefix=${pcfiledir}/../..
includedir=${prefix}/include
libdir=${prefix}/lib

Name: coinhsl
Description: coinhsl: coinhsl
Version: 2023.11.14
Libs: -L${libdir} -lcoinhsl
Libs.private: -lopenblas -lmetis
Cflags: -I${includedir}

For the header file I have this. Does COINHSL_HAS_METIS mean that the lib was linked to metis when built or that it contains metis (and so doesn't need linking)?

$ cat include/CoinHslConfig.h
/* config_coinhsl.h.in. */

#ifndef __CONFIG_COINHSL_H__
#define __CONFIG_COINHSL_H__

/* Define to 1 if MA27 is available. */
#define COINHSL_HAS_MA27 1
#define COINHSL_HAS_MA27S 1

/* Define to 1 if MA28 is available. */
#define COINHSL_HAS_MA28 1
#define COINHSL_HAS_MA28S 1

/* Define to 1 if MA57 is available. */
#define COINHSL_HAS_MA57 1
#define COINHSL_HAS_MA57S 1

/* Define to 1 if MA77 is available. */
#define COINHSL_HAS_MA77 1
#define COINHSL_HAS_MA77S 1

/* Define to 1 if MA86 is available. */
#define COINHSL_HAS_MA86 1
#define COINHSL_HAS_MA86S 1

/* Define to 1 if MA97 is available. */
#define COINHSL_HAS_MA97 1
#define COINHSL_HAS_MA97S 1

/* Define to 1 if MC19 is available. */
#define COINHSL_HAS_MC19 1
#define COINHSL_HAS_MC19S 1

/* Define to 1 if MC68 is available. */
#define COINHSL_HAS_MC68 1

/* Define to 1 if METIS is available */
#define COINHSL_HAS_METIS 1

#endif

@svigerske
Copy link
Member

I think so, thank you.

In case of HSL build as static library, it would be good to have the Libs.private in Libs, i.e.,

Libs: -L${libdir} -lcoinhsl -lopenblas -lmetis

(but I suppose you plan to distribute shared libs only)

COINHSL_HAS_METIS only means that the lib was linked to metis, that is, that it should work to select metis for ordering.
Whether metis is included or a library dependency is handled by the .so or .pc file.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants