Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sync with develop (20240802) #3401

Merged
merged 98 commits into from
Aug 11, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
98 commits
Select commit Hold shift + click to select a range
d5bde64
To enable the psmpi easyblock to activate CUDA when it is a dependency,
damianam Sep 19, 2022
f5d7c9b
Style
damianam Sep 19, 2022
ecbe191
Deambiguate variable name
damianam Sep 19, 2022
3db4405
make a project environment and manage LOAD_PATH for each JuliaPackage…
lexming Mar 1, 2024
eed5e76
move preparation of Julia environment to installation step of JuliaPa…
lexming Mar 1, 2024
f9ae2a0
replace explicit precompile command on JuliaPackage by enforcing auto…
lexming Mar 4, 2024
65dc786
explicitly set LOAD_PATH for installations of JuliaPackages
lexming Mar 7, 2024
637f2be
prepare installation environement of JuliaPackages in install step
lexming Mar 8, 2024
1f57204
make JuliaBundle a child of both Bundle and JuliaPackage
lexming Mar 8, 2024
ab84e00
remove unused imports from JuliaBundle
lexming Mar 8, 2024
a749219
expand comments about path manipulation in JuliaPackage
lexming Mar 8, 2024
c9032f3
reset julia environment before extension installation of JuliaPackage
lexming Mar 11, 2024
5467de5
Update tensorflow block for changes in 2.14 and 2.15
yqshao Apr 11, 2024
6d6bea9
Set build type for PyTorch explicitely
Flamefire May 21, 2024
8898865
Merge of the ConfigMake and CMakeCMake easyblocks
Crivella May 24, 2024
1d882b6
linting
Crivella May 24, 2024
19318a2
Another python2 compatibility fix
Crivella May 24, 2024
82eed22
lint
Crivella May 24, 2024
1d08f5b
Adding support to RISCV64 when using tagged_layout
julianmorillo May 24, 2024
af8d0a2
Adding RISCV64 support when using tagged_layout
julianmorillo May 24, 2024
7f9134f
Fix
Crivella May 24, 2024
65720d7
Update configure options for new versions of Extrae and add RISC-V clock
julianmorillo May 31, 2024
b704bbc
Merge branch 'easybuilders:develop' into fix-pytorch-build-type
Flamefire Jun 6, 2024
32b7445
Patch ctypes for Python installations when filtering LD_LIBRARY_PATH
ocaisa Jun 7, 2024
f8c2321
Add test step by default for recent Python installations
ocaisa Jun 7, 2024
a771002
Also skip a curses test
ocaisa Jun 7, 2024
9af5750
Add comment on why we skip some tests
ocaisa Jun 7, 2024
ac1ce4e
Class is inherited so restrict application of new test step
ocaisa Jun 7, 2024
d07f920
Adjust approach
ocaisa Jun 10, 2024
0cc765f
Split off the testing step to another PR
ocaisa Jun 10, 2024
8a4646d
Fix for possible race condition
Crivella Jun 11, 2024
0c6f93e
lint
Crivella Jun 11, 2024
4a1493e
Removed old comment
Crivella Jun 11, 2024
7cd6096
Fix appending option
Flamefire Jun 12, 2024
488992e
set ESMFMKFILE environment variable on ESMF modules
lexming Jun 19, 2024
53c13ea
support >= 2024a `for MCR`
branfosj Jun 19, 2024
4147d9e
Add support for Bazel 7+
Flamefire Jun 20, 2024
357e3ce
Merge pull request #3369 from branfosj/mcr
boegel Jun 24, 2024
1a297ea
Use build dir for building TensorFlow pip package
Flamefire Jun 20, 2024
819c0a1
make PythonPackage use the build dir for $XDG_CACHE_HOME
Flamefire Jun 27, 2024
0c44bb6
Move XDG_CACHE_HOME to init
Flamefire Jun 27, 2024
4c0d748
fix PythonPackage installation with $PIP_REQUIRE_VIRTUALENV
Flamefire Jun 27, 2024
2399cbd
Already change the temp dir during build
Flamefire Jun 27, 2024
e53a3e2
Also set $TMPDIR for the tests
Flamefire Jun 27, 2024
a60c47d
Don't consider lib/release for impi >= 2021.11
bartoldeman Jun 27, 2024
c5099ea
add RISC-V support to Boost easyblock
SebastianAchilles Jun 30, 2024
7a9782e
Merge pull request #3376 from SebastianAchilles/20240630102439_new_pr…
bedroge Jul 1, 2024
aee8a7b
Remove dummy license server info
Flamefire Jul 1, 2024
92f02fd
Merge branch 'develop' into develop
bedroge Jul 2, 2024
52e3c0d
Merge pull request #3370 from Flamefire/20240620181649_new_pr_bazel
branfosj Jul 2, 2024
11230bc
Merge pull request #3371 from Flamefire/tensorflow-tmp-dir
branfosj Jul 3, 2024
81b488e
Merge branch 'develop' of https://github.com/easybuilders/easybuild-e…
branfosj Jul 3, 2024
640021d
Merge pull request #3239 from lexming/julia-env-load-path
branfosj Jul 3, 2024
96e48ff
Merge pull request #3368 from lexming/esmfmkfile
boegel Jul 3, 2024
4fa3fd6
Merge pull request #3377 from Flamefire/20240701143255_new_pr_ansys
boegel Jul 3, 2024
8f4ca9d
Merge pull request #3375 from bartoldeman/impi-no-lib-release-subdir
boegel Jul 3, 2024
3d0a948
Merge pull request #3374 from Flamefire/pythonpackage
boegel Jul 3, 2024
4346da9
Add support for generating `.gem` files from `.gemspec` files and sup…
PetrKralCZ Jul 4, 2024
1009921
Fiix continuation line under-indent
PetrKralCZ Jul 4, 2024
2020b62
simplify `Tkinter` install step
branfosj Jul 4, 2024
7853074
fix typo
branfosj Jul 4, 2024
2c04af4
flake8
branfosj Jul 4, 2024
ef9335c
Merge branch 'develop' of https://github.com/easybuilders/easybuild-e…
SebastianAchilles Jul 4, 2024
473eaa0
Merge pull request #3303 from yqshao/tensorflow-214_215-updates
boegel Jul 4, 2024
36af558
Merge pull request #2787 from damianam/psmpi_cuda_and_check_static_libs
SebastianAchilles Jul 4, 2024
bb86f05
Merge pull request #3381 from PetrKralCZ/20240704125705_new_pr_rubygem
boegel Jul 5, 2024
a136909
To add MSA and specially PMIx support for ParaStationMPI
damianam Jul 5, 2024
d4076b0
Explicitly calling csh in wps.py easyblock
thanghn90 Jul 6, 2024
1e24ea4
Calling csh command for link_grib.csh
thanghn90 Jul 6, 2024
064e3e2
Merge pull request #3339 from julianmorillo/develop
ocaisa Jul 6, 2024
c8256a3
replace absolute paths to object files by relative ones
bedroge Jul 9, 2024
e275742
QE prioritizes intel BLAS/LAPACK vendor on x86_64 machines causing sy…
Crivella Jul 11, 2024
3454236
Improved defining of BLA_VENDOR
Crivella Jul 15, 2024
5138e8f
Workaround for framework issue 4543
Crivella Jul 16, 2024
5f95e61
lint
Crivella Jul 16, 2024
7d6ee36
Disable dynamic memory tracing, which causes MPI tests fail in some
julianmorillo Jul 16, 2024
769749d
Added the configure option to the easyconfig instead
julianmorillo Jul 16, 2024
073ba94
Solved formatting issue
julianmorillo Jul 16, 2024
0e6e2bb
Merge pull request #3332 from Flamefire/fix-pytorch-build-type
ocaisa Jul 17, 2024
32e45bd
Merge pull request #3338 from Crivella/feature-QE_merge_easyblocks
ocaisa Jul 17, 2024
325ea3c
Merge branch 'easybuilders:develop' into develop
julianmorillo Jul 17, 2024
c2d082a
Force --without-unwind for ARM and add --with-libz option for all arc…
julianmorillo Jul 17, 2024
ee8ca28
Solved formatting issues
julianmorillo Jul 17, 2024
3f1cb57
libunwind dependency managed now in the easyconfig
julianmorillo Jul 18, 2024
1ea2cf3
Removed unused AARCH64 import
julianmorillo Jul 18, 2024
c4951c7
Merge pull request #3393 from julianmorillo/develop
ocaisa Jul 18, 2024
38cea13
Merge pull request #3383 from damianam/psmpi_msa_pmix
SebastianAchilles Jul 24, 2024
4818e6e
Return full path when gcc or ld are used to find the library. That wa…
Jul 24, 2024
8480c21
If we're installing with custom sysroot, we should replace a hardcode…
Jul 24, 2024
1ee17c0
Update python.py
ocaisa Jul 26, 2024
f2242d3
Remove redundant backslashes in GEANT easyblock
ocaisa Jul 26, 2024
8f54a66
Update linting.yml
ocaisa Jul 26, 2024
2e2d6a3
Merge pull request #3394 from easybuilders/ocaisa-patch-1
branfosj Jul 26, 2024
0cb2cc9
Merge pull request #3388 from bedroge/openfoam_objects_relative_paths
ocaisa Jul 30, 2024
bb4cb6f
Merge pull request #3352 from ocaisa/ctypes_python
casparvl Jul 30, 2024
871d9be
Merge pull request #3384 from thanghn90/patch-2
boegel Jul 31, 2024
89ea2e4
Merge pull request #3382 from branfosj/20240704132553_new_pr_tkinter
boegel Jul 31, 2024
aa994b0
Merge branch 'develop' into 5.0.x
boegel Aug 2, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 9 additions & 8 deletions easybuild/easyblocks/a/ansys.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,16 +49,17 @@ def __init__(self, *args, **kwargs):

def install_step(self):
"""Custom install procedure for ANSYS."""
licserv = self.cfg['license_server']
if licserv is None:
licserv = os.getenv('EB_ANSYS_LICENSE_SERVER', 'license.example.com')
licport = self.cfg['license_server_port']
if licport is None:
licport = os.getenv('EB_ANSYS_LICENSE_SERVER_PORT', '2325:1055')

# Sources (e.g. iso files) may drop the execute permissions
adjust_permissions('INSTALL', stat.S_IXUSR)
cmd = "./INSTALL -silent -install_dir %s -licserverinfo %s:%s" % (self.installdir, licport, licserv)

cmd = "./INSTALL -silent -install_dir %s" % self.installdir
# E.g. license.example.com or license1.example.com,license2.example.com
licserv = self.cfg.get('license_server', os.getenv('EB_ANSYS_LICENSE_SERVER'))
# E.g. '2325:1055' or just ':' to use those defaults
licport = self.cfg.get('license_server_port', os.getenv('EB_ANSYS_LICENSE_SERVER_PORT'))
if licserv is not None and licport is not None:
cmd += ' -licserverinfo %s:%s' % (licport, licserv)

run_shell_cmd(cmd)

adjust_permissions(self.installdir, stat.S_IWOTH, add=False)
Expand Down
6 changes: 5 additions & 1 deletion easybuild/easyblocks/b/bazel.py
Original file line number Diff line number Diff line change
Expand Up @@ -164,7 +164,11 @@ def configure_step(self):
# We want to enforce it using the JDK we provided via modules
# This is required for Power where Bazel does not have a JDK, but requires it for building itself
# See https://github.com/bazelbuild/bazel/issues/10377
bazel_args += ' --host_javabase=@local_jdk//:jdk'
if LooseVersion(self.version) >= LooseVersion('7.0'):
# Option changed in Bazel 7.x, see https://github.com/bazelbuild/bazel/issues/22789
bazel_args += ' --tool_java_runtime_version=local_jdk'
else:
bazel_args += ' --host_javabase=@local_jdk//:jdk'

# Link C++ libs statically, see https://github.com/bazelbuild/bazel/issues/4137
static = self.cfg['static']
Expand Down
4 changes: 3 additions & 1 deletion easybuild/easyblocks/b/boost.py
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@
from easybuild.tools.filetools import apply_regex_substitutions, read_file, symlink, which, write_file
from easybuild.tools.modules import get_software_root, get_software_version
from easybuild.tools.run import run_shell_cmd
from easybuild.tools.systemtools import AARCH64, POWER, UNKNOWN
from easybuild.tools.systemtools import AARCH64, POWER, RISCV64, UNKNOWN
from easybuild.tools.systemtools import get_cpu_architecture, get_glibc_version, get_shared_lib_ext


Expand Down Expand Up @@ -326,6 +326,8 @@ def sanity_check_step(self):
lib_mt_suffix += '-a64'
elif get_cpu_architecture() == POWER:
lib_mt_suffix += '-p64'
elif get_cpu_architecture() == RISCV64:
lib_mt_suffix += '-r64'
else:
lib_mt_suffix += '-x64'

Expand Down
7 changes: 6 additions & 1 deletion easybuild/easyblocks/e/esmf.py
Original file line number Diff line number Diff line change
Expand Up @@ -151,6 +151,11 @@ def make_module_extra(self):
"""Add install path to PYTHONPATH or EBPYTHONPREFIXES"""
txt = super(EB_ESMF, self).make_module_extra()

# set environment variable ESMFMKFILE
# see section 9.9 in https://earthsystemmodeling.org/docs/release/latest/ESMF_usrdoc/node10.html
esmf_mkfile_path = os.path.join(self.installdir, "lib", "esmf.mk")
txt += self.module_generator.set_environment('ESMFMKFILE', esmf_mkfile_path)

if self.cfg['multi_deps'] and 'Python' in self.cfg['multi_deps']:
txt += self.module_generator.prepend_paths('EBPYTHONPREFIXES', '')
else:
Expand All @@ -172,7 +177,7 @@ def sanity_check_step(self):
binaries = ['ESMF_PrintInfo', 'ESMF_PrintInfoC', 'ESMF_Regrid', 'ESMF_RegridWeightGen',
'ESMF_Scrip2Unstruct', 'ESMF_WebServController']

libs = ['libesmf.a', 'libesmf.%s' % get_shared_lib_ext()]
libs = ['esmf.mk', 'libesmf.a', 'libesmf.%s' % get_shared_lib_ext()]
custom_paths = {
'files': [os.path.join('bin', x) for x in binaries] + [os.path.join('lib', x) for x in libs],
'dirs': ['include', 'mod'],
Expand Down
36 changes: 28 additions & 8 deletions easybuild/easyblocks/e/extrae.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,9 @@

from easybuild.easyblocks.generic.configuremake import ConfigureMake
from easybuild.tools.modules import get_software_root
from easybuild.tools import LooseVersion
from easybuild.tools.systemtools import RISCV64
from easybuild.tools.systemtools import get_cpu_architecture


class EB_Extrae(ConfigureMake):
Expand All @@ -42,14 +45,27 @@ def configure_step(self):
self.cfg.update('configopts', "--with-mpi=%s" % get_software_root(self.toolchain.MPI_MODULE_NAME[0]))

# Optional dependences
deps = {
'binutils': ('', '--with-binutils=%s', ''),
'Boost': ('', '--with-boost=%s', ''),
'libdwarf': ('', '--with-dwarf=%s', '--without-dwarf'),
'libunwind': ('', '--with-unwind=%s', ''),
'libxml2': (' --enable-xml --enable-merge-in-trace', '', ''),
'PAPI': ('--enable-sampling', '--with-papi=%s', '--without-papi'),
}
# Both --enable-xml and --with-dwarf options are no longer available from 4.1.0 version
# Instead, --with-xml is used
if LooseVersion(self.version) >= LooseVersion('4.1.0'):
deps = {
'binutils': ('', '--with-binutils=%s', ''),
'Boost': ('', '--with-boost=%s', ''),
'libunwind': ('', '--with-unwind=%s', '--without-unwind'),
'libxml2': ('--enable-merge-in-trace', '--with-xml=%s', ''),
'PAPI': ('--enable-sampling', '--with-papi=%s', '--without-papi'),
'zlib': ('', '--with-libz=%s', ''),
}
else:
deps = {
'binutils': ('', '--with-binutils=%s', ''),
'Boost': ('', '--with-boost=%s', ''),
'libdwarf': ('', '--with-dwarf=%s', '--without-dwarf'),
'libunwind': ('', '--with-unwind=%s', '--without-unwind'),
'libxml2': (' --enable-xml --enable-merge-in-trace', '', ''),
'PAPI': ('--enable-sampling', '--with-papi=%s', '--without-papi'),
}

for (dep_name, (with_opts, with_root_opt, without_opt)) in deps.items():
dep_root = get_software_root(dep_name)
if dep_root:
Expand All @@ -64,6 +80,10 @@ def configure_step(self):
# TODO: make this optional dependencies
self.cfg.update('configopts', "--without-dyninst")

# Needed to build in RISC-V architectures
if get_cpu_architecture() == RISCV64:
self.cfg.update('configopts', "--enable-posix-clock")

super(EB_Extrae, self).configure_step()

def sanity_check_step(self):
Expand Down
24 changes: 10 additions & 14 deletions easybuild/easyblocks/generic/juliabundle.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,11 +31,9 @@

from easybuild.easyblocks.generic.bundle import Bundle
from easybuild.easyblocks.generic.juliapackage import EXTS_FILTER_JULIA_PACKAGES, JuliaPackage
from easybuild.tools.build_log import EasyBuildError
from easybuild.tools.modules import get_software_root


class JuliaBundle(Bundle):
class JuliaBundle(Bundle, JuliaPackage):
"""
Bundle of JuliaPackages: install Julia packages as extensions in a bundle
Defines custom sanity checks and module environment
Expand Down Expand Up @@ -82,17 +80,10 @@ def prepare_step(self, *args, **kwargs):
"""Prepare for installing bundle of Julia packages."""
super(JuliaBundle, self).prepare_step(*args, **kwargs)

if get_software_root('Julia') is None:
raise EasyBuildError("Julia not included as dependency!")

def make_module_extra(self, *args, **kwargs):
"""
Module has to append installation directory to JULIA_DEPOT_PATH to keep
the user depot in the top entry. See issue easybuilders/easybuild-easyconfigs#17455
"""
txt = super(JuliaBundle, self).make_module_extra()
txt += self.module_generator.append_paths('JULIA_DEPOT_PATH', [''])
return txt
def install_step(self):
"""Prepare installation environment and dd all dependencies to project environment."""
self.prepare_julia_env()
self.include_pkg_dependencies()

def sanity_check_step(self, *args, **kwargs):
"""Custom sanity check for bundle of Julia packages"""
Expand All @@ -101,3 +92,8 @@ def sanity_check_step(self, *args, **kwargs):
'dirs': [os.path.join('packages', self.name)],
}
super(JuliaBundle, self).sanity_check_step(custom_paths=custom_paths)

def make_module_extra(self, *args, **kwargs):
"""Custom module environement from JuliaPackage"""
mod = super(JuliaBundle, self).make_module_extra(*args, **kwargs)
return mod
Loading
Loading