Conference call notes 20180523

(back to Conference calls)

Notes on the 102nd EasyBuild conference call, Wednesday May 23rd 2018 (5pm - 6pm CEST)

Attendees

Alphabetical list of attendees (6):

• Damian Alvarez (JSC, Germany)
• Fotis Georgatos (Illumina, UK)
• Kenneth Hoste (HPC-UGent, Belgium)
• Alan O'Cais (JSC, Germany)
• Åke Sangren (Umeå University, Sweden)
• Shahzeb Siddiqui (Pfizer)

Agenda

• outlook to EasyBuild v3.6.1
• progress towards making EasyBuild compatible with Python 3.x
• Q&A

EasyBuild v3.6.1

• ETA: this week (?)
• merged PRs for EB v3.6.1
  • https://github.com/easybuilders/easybuild-framework/milestone/56?closed=1
  • https://github.com/easybuilders/easybuild-easyblocks/milestone/48?closed=1
  • https://github.com/easybuilders/easybuild-easyconfigs/milestone/51?closed=1
• framework:
  • all PRs targeted for EasyBuild v3.6.1 have been merged
  • highlights:
    • fixed bug to ensure temporary module is picked up
      • see https://github.com/easybuilders/easybuild-framework/pull/2506
      • ml use with high priority + set default
      • same problem with user modules?
        • TODO: double check, update docs to mention priority
    • lib vs lib64 fallback in sanity check
      • see https://github.com/easybuilders/easybuild-framework/pull/2477
      • makes sanity check slightly more flexible
      • should help significantly on SuSE-based systems
    • no more blind overwrite of existing easyconfigs on --try-*
      • see https://github.com/easybuilders/easybuild-framework/pull/2504
• easyblocks:
  • all PRs targeted for EasyBuild v3.6.1 have been merged
  • highlights:
    • fixed sanity check for imkl (but no "damage" done, sanity check still passes)
      • cfr. https://github.com/easybuilders/easybuild-easyblocks/pull/1392
    • provide control over adding of root install dir to $PATH in Binary generic easyblock
      • https://github.com/easybuilders/easybuild-easyblocks/pull/1426
    • support for auto-detecting downloading of dependency pkgs in PythonPackage easyblock
      • https://github.com/easybuilders/easybuild-easyblocks/pull/1377
      • we should enforce a policy to enable download_dep_fail in new easyconfigs that use PythonPackage
• easyconfigs:
  • mostly software updates, minor bug fixes and new software (incl. dotNET!)
  • handful of PRs will get get merged (hopefully)
    • OpenBLAS 0.3.0 (just released): https://github.com/easybuilders/easybuild-easyconfigs/pull/6361
    • archive URLs for Bioconductor: https://github.com/easybuilders/easybuild-easyconfigs/pull/6311
    • GCC 8.1: https://github.com/easybuilders/easybuild-easyconfigs/pull/6296

Progress towards making EasyBuild compatible with Python 3.x

• Vanessa Sochat started looking into making vsc-base compatible with Python 3.x
  • https://github.com/hpcugent/vsc-base/pull/258
  • after making vsc-install Py3 compatible
    • see https://github.com/hpcugent/vsc-install/pull/83
• once vsc-base is compatible with Python 3, we can work towards making EasyBuild Python 3 compatible as well...
• fueled by interest in:
  • leveraging HPC Container Maker (https://github.com/NVIDIA/hpc-container-maker) in EasyBuild
    • was Python 3-only until very recently (cfr. https://github.com/NVIDIA/hpc-container-maker/pull/28)
  • Python API for Singularity: https://singularityhub.github.io/singularity-cli/commands

Q&A

• follow-up conf call w.r.t. container support tomorrow (Kenneth, Shahzeb, ...)
  • https://github.com/easybuilders/easybuild-framework/pull/2479 should go in first?
• Shahzeb:
  • better way to handle Bioconductor
    • support installing biocViews?
      • https://bioconductor.org/packages/release/bioc/html/biocViews.html
    • no version control?
    • could be supported as alternate way of installing, but requires enhanced easyblock?
  • can we support alternatives to CRAN?
    • miniCRAN:
    • MRAN: https://mran.microsoft.com/documents/rro/reproducibility (see checkpoint library)
      • https://mran.microsoft.com/documents/rro/reproducibility
    • pakrat:
    • could be useful for easy_update script (https://github.com/fizwit/easy_update)?
    • could existing R easyblock be enhanced to use local snapshot of source for packages?
  • too much of a pain to deal with R packages right now...
  • Fotis: could $HTTP_PROXY (or similar) be used to convince R to use sources from somewhere else