Skip to content

Conference call notes 20231206

Jump to bottom
Kenneth Hoste edited this page Dec 6, 2023 · 3 revisions

(back to Conference calls)

Notes on the 235th EasyBuild conference call, Wednesday 6 Dec 2023 (09:00 UTC)

Attendees

List of attendees (12):

  • Sebastian Achilles (JSC, Germany)
  • Simon Branford (Univ. of Birmingham, UK)
  • Alex Domingo (Vrije Universiteit Brussel, Belgium)
  • Jasper Grimm (University of York, UK)
  • Kenneth Hoste (HPC-UGent, Belgium)
  • Adam Huffman (Big Data Institute, Oxford, UK)
  • Sam Moors (Vrije Universiteit Brussel, Belgium)
  • Sébastien Moretti (SIB, Switzerland)
  • Mikael Öhman (Chalmers University of Technology, Sweden)
  • Jure Pečar (EMBL, Heidelberg, Germany)
  • Åke Sandgren (Umeå University, Sweden)
  • Alexandre Strube (JSC, Germany)

Agenda

  • overview of recent developments
  • 2023b update of common toolchains
  • Q&A

Recent developments

  • latest EasyBuild release: 4.8.2 (29 Oct 2023)
    • ETA for next EasyBuild release (v4.9.0): mid Dec'23 (ASAP)
    • ETA for EasyBuild 5.0 release: first release of 2024?
      • started doing short sprint meetings, each Monday at 10:00 CE(S)T to set next 5 goals to tackle that week
      • monthly sync meetings every 1st Wed of the month at 14:00 CE(S)T: next today Wed 6 Dec'23 at 14:00 CET
      • see also https://hackmd.io/tZtz6vMmRfGL6N9Rf__OEA
  • recent changes
    • docs (merged PRs)
      • update common toolchains overview: move 2020a and older to deprecated toolchains, fix LAPACK column for recent toolchains (PR #229)
      • restructure documentation on EasyBuild v5.0 (PR #230)
    • framework (merged PRs)
      • bug fixes
        • eliminate duplicate multideps from generated module files (PR #4386)
        • use source toolchain version when passing only --try-toolchain (PR #4395)
      • enhancements
        • ...
      • changes
        • ...
      • EasyBuild 5.0 (to 5.0.x branch)
        • bug fixes
          • ...
        • changes
          • ...
    • easyblocks (merged PRs)
      • bug fixes
        • update error detection for PyTorch tests (PR #3033)
        • disable LLVM build downloads from CI in Rust (PR #3038)
      • enhancements
        • ...
      • updates
        • update custom intel-compilers easyblock for versions >= 2024 (PR #3037)
        • update numpy easyblock for v1.26+ (PR #3041)
      • changes
        • ...
      • new easyblocks
        • ...
      • EasyBuild 5.0 (to 5.0.x branch)
        • ...
    • easyconfigs (merged PRs)
      • ~50 easyconfig PRs were merged since last conf call
      • bug fixes
        • patch setup.py for grpcio extension in TensorFlow 2.13.0 easyconfigs to take into account alternate sysroot (PR #19268)
        • fix failing tokenizers with CargoPythonPackage easyblock (PR #19333)
          • quite annoying to fix this after the fact
          • especially because Rust crates that won't actually be used also have to be listed (like windows_*)
          • you often also need to provide multiple versions of the same crate
      • enhancements
        • ...
      • (noteworthy) new software
        • ...
      • noteworthy software updates
        • ...
      • changes
        • ...
      • EasyBuild 5.0 (to 5.0.x branch)
        • ...
  • work-in-progress
    • docs (open PRs + issues)
    • framework (open PRs + issues)
      • reported bugs / bug fixes
        • fix support for Environment Modules as modules tool to pass unit tests with v4.2+ (PR #4369)
        • adapt module function check for Environment Modules v4+ (PR #4371)
        • fix LIBBLAS_MT for FlexiBLAS, ensure -lpthread is included (PR #4379)
        • earlier return to original directory after executing a command in run_cmd (via complete_cmd) (PR #4390)
        • make --job a bit smarter when uploading test reports (issue #4387)
      • enhancements
        • add support for --list-software --output-format=json (PR #4152)
        • allow --review-pr-filter to override the criteria in find_related_easyconfigs (PR #4388)
        • create an option for a developer mode (issue #4394)
      • changes
        • drop load storm safe guard for Environment Modules v4.2.4+ (PR #4373)
      • EasyBuild 5.0 (to 5.0.x branch)
        • deprecate failure to resolve a template value (PR #3285)
        • detect Fortran .mod files in GCCcore installations (PR #4389)
    • easyblocks (open PRs + issues)
      • bug reports/fixes
        • fix extension filter for Perl packages (PR #2699)
          • easyconfigs PR #18789 is a big step towards merging this, but there are other easyconfigs that needs to be tested first, like XML-LibXML, worker, GD, ...
        • add requirement for EULA acceptance to CUDA easyblock (PR #3045)
        • sanity check fails for Clang 16.x when building the nvptx target (issue #3040)
      • enhancements
        • add custom easyconfig parameter cmake_options to SuiteSparse easyblock (PR #3031)
        • update KOKKOS_CPU_MAPPING for AArch64 in LAMMPS (PR #3036)
      • updates
        • update custom easyblock for Intel MPI easyblock for v2021.11 (PR #3039)
        • update custom easyblock for Intel MKL for v2024.x (PR #3042)
      • new easyblocks
        • new custom easyblocks for Spparks and Stitch (PR #2948)
        • add generic CargoPythonBundle easyblock (PR #2964)
          • still stuck due to weird MRO problem
          • Mikael has asked Lars for input
          • similar issue in issue #2815
      • changes
        • Use $EB_COMSOL_LICENSE_FILE variable in custom easyblock for COMSOL (PR #3044)
      • EasyBuild 5.0 (to 5.0.x branch)
        • enable download_dep_fail, use_pip, sanity_pip_check by default in PythonPackage easyblock (PR #3022)
          • needs some tests to make sure nothing is horribly broken (but should be fine)
          • follow-up is needed in easyconfigs, to:
            • clean up easyconfigs in 5.0.x branch that now set use_pip & co to True (see easyconfigs PR #19265)
            • explicitly set use_pip & co to False in (non-archived) easyconfig that currently don't specify them at all (which should be a minimal set of easyconfigs)
        • update ConfigureMake easyblock to error out on unknown configure args (PR #3025)
        • CMakeMake could default CMAKE_BUILD_TYPE to RelWithDebInfo instead of Release (issue #3032)
        • inconsistent license variable naming (issue #3043)
    • easyconfigs (open PRs + issues)
      • bug fixes/reports
        • move cmake options to easyblock and support customizing cmake_options for SuiteSparse-7.1.0 (PR #19177)
        • add patch to fix [cz]asums in OpenBLAS 0.3.24 (PR #19331)
      • enhancements
        • add test for duplicate $PYTHONPATH in modextrapaths (PR #19061)
      • new software
        • ...
      • noteworthy software updates
        • PETSc 3.19.4 w/ foss/2023a (PR #18608)
        • PyTorch v1.13.1 w/ foss/2022b + CUDA 11.7.0 (PR #18853)
        • PyTorch v1.13.1 w/ foss/2022b + CUDA 12.0.0 (PR #18806)
        • PyTorch v2.0.1 w/ foss/2022b (PR #19067)
        • PYTorch v2.1.0 w/ foss/2022b (PR #19087)
        • R-bundle-CRAN v2023.11 w/ foss/2023a (PR #19170)
          • requires R v4.3.2 with gfbf/2023a (PR #19185)
          • R should ideally have a minimal set of extensions, maybe only devtools and what it requires
            • is shiny really required by devtools? probably not yes it is :man-facepalming:
        • Qt5 v5.15.11 w/ GCCcore/13.2.0 (for foss/2023b) (PR #19320)
      • changes
        • ...
      • EasyBuild 5.0 (to 5.0.x branch)
        • clean up easyconfigs that explicitly set use_pip, sanity_pip_check and download_dep_fail (PR #19265)
      • closed PRs
        • dead end for TensorFlow v2.13.0 w/ CUDA 12.1.1 w/ foss/2023a (PR #19182)

2023b update of common toolchains

  • 2023b toolchains should be included in EasyBuild 4.9.0 release
    • candidate toolchains are merged, ready for more extensive testing of "big" apps
  • most significant change is jump to GCC 13.x
  • foss/2023.09 (PR #18886) - candidate for foss/2023b
    • GCC 13.2.0 + binutils 2.40
    • OpenMPI 4.1.6 (+ UCX 1.15.0, PMIx 4.2.6, libfabric 1.19.0)
    • FlexiBLAS 3.3.1 (+ OpenBLAS 0.3.24)
    • FFTW 3.3.10
    • ScaLAPACK 2.1.0
  • intel/2023.07 (PR #18439) - candidate for intel/2023b
    • GCC 13.2.0 + binutils 2.40
    • intel-compilers 2023.2.1
    • impi 2021.10.0
    • imkl 2023.2.0
  • testing
    • OSU-Micro-Benchmarks (foss in PR #18893)
    • SciPy-bundle (numpy, scipy)
      • keep an eye out for need for CargoPythonBundle
    • GROMACS (C++)
    • OpenFOAM (C++)
      • requires ParaView, Qt5, etc.
      • should we keep building on top of ParaView (only needed for paraFoam utility)?
        • installing paraFoam stand-alone is a PITA
        • paraFoam isn't actually used when running OpenFOAM simulations
    • CP2K (Fortran)
    • check if Qt6 can be used
  • Mikael saw a strange compilation with GCC 13.2 for assimp (dep for Qt6)
  • latest NCCL does not support GCC 13.x yet, but next patch release should
  • CUDA 12.3.0 should support GCC 13.2.0
  • releasing 2023b implies deprecating 2020b
    • that means no more need for Intel license server to test PRs... \o/

Q&A / others

Clone this wiki locally