Skip to content

EasyBuild Sprint 2020a toolchains

Jump to bottom
Miguel Dias Costa edited this page Mar 18, 2020 · 25 revisions

What?

Starting to use a brand new toolchain is always a challenge, since the limited number of EasyConfigs mean you need to develop a lot yourself. This sprint synchronizes that effort amongst the community, so that together we can kickstart the development of GCCcore-9.3, foss-2020a, intel-2020a based EasyConfigs. Hopefully, this will result in a large number of EasyConfigs being available quickly after the release of the 2020a toolchains.

When?

The sprint will take place on Tuesday 17th and Thursday 19th of March 2020, 9:00-13:00h CET. During this time, a large number of maintainers will be available for reviewing and accepting PRs.

The idea of having two separate mornings is that straightforward EasyConfigs can go through the full review cycle on the 17th. For more difficult ones, you can get some feedback on the 17th, work on it in the meantime, and try to finalize the PR on the 19th.

How to participate?

  • Before the sprint: make sure to prepare your environment so you can participate (see the 'What to prepare before the sprint?' section)
  • Check out the wishlist and choose an a software package you want to work on (i.e. make an EasyConfig for). If you want to work on something that is not on the wishlist but that is relevant for your site, just ask one of the maintainers to add it to the wishlist.
  • Ask one of the maintainers to add your name to the software package that you will work on. If your software package has dependencies, please also ask the maintainers to add your name to these. This avoids having two people working on the same EasyConfig at the same time.
  • Don't start with the highest level packages until EasyConfigs for a lot of low level dependencies are available. E.g. looking at an older version of Qt5, I see it has many (indirect) dependencies:
$ eb -D Qt5-5.13.1-GCCcore-8.3.0.eb | wc -l
75

In such a case, it is probably better to start with one of Qt5's dependencies, such as $ eb -D JasPer-2.0.14-GCCcore-8.3.0.eb | wc -l 25 The first 20 deps here are part of binutils - which already exists for 2020a - meaning you would have to write ~5 easyconfigs. Typically, PRs with

  • Develop your EasyConfig(s)
  • Use --new-pr to submit a pull request. In addition, make sure it targets the 2020a branch by adding --pr-target-branch=2020a
  • Regularly update your local repository. Do this by changing directory to your local easybuild-easyconfigs checkout and running git pull github_easybuilders 2020a

How to keep in touch

  • Join the #merge-sprint channel on the EasyBuild Slack
  • In addition, we may set up an open Zoom-call. If so, we'll paste a link here.

What to prepare before the sprint?

It is advisable to set up an environment beforehand that

A particularly easy way to obtain such an EasyBuild installation is by using the easyconfig file in attachment (also available via https://users.ugent.be/~kehoste/EasyBuild-2020a.eb).

As the sprint progresses you'll need to reinstall this easyconfig file a couple of times, using the following command:

eb EasyBuild-2020a.eb --force --force-download --ignore-checksums

Configuring the EasyBuild installation

The above steps have created an installation of a development version of EasyBuild that you will use throughout the sprint. You may however want to mimic (part of) the configuration of your production installation of EasyBuild. One option is to manually modify the installed $INSTALL_PREFIX/modules/EasyBuild-develop modulefile to export desired EASYBUILD_* environment variables.

You can check your configuration using

eb --show-config

Configuring the GitHub integration

To set up the GitHub integration, please follow the official EasyBuild documentation on that. You can check it has been set up correctly by trying

eb --check-github

(if you haven't configured a default github user for your EasyBuild installation, don't forget to add --github-user=<username>)

Install the 2020a-based toolchains

Depending on what you want to participate in, you may want to install the foss-2020a or intel-2020a toolchains. As of March 17th, the final toolchain definitions have been merged in the 2020a branch (https://github.com/easybuilders/easybuild-easyconfigs/pull/10072).

Since there is no CUDA version that currently supports GCCcore-9.x, you do not need to install a newer fosscuda - development for fosscuda will likely not happen within this sprint.

Appendix

If you prefer to use the develop and 2020a branches directly from git repos instead of installing the EasyBuild-2020a.eb easyconfig mentioned above, please take the following steps:

# pick an installation prefix (adjust as you like)
INSTALL_PREFIX=$(mktemp -d $HOME/EasyBuild-XXXXXX)
# download script
curl -O https://raw.githubusercontent.com/easybuilders/easybuild-framework/develop/easybuild/scripts/install-EasyBuild-sprint.sh
# run downloaded script, specifying *your* GitHub username, the installation prefix and the target easyconfigs-surfsara branch
bash install-EasyBuild-sprint.sh GITHUB_USERNAME $INSTALL_PREFIX 2020a
# update $MODULEPATH via 'module use', and load the module
module use $INSTALL_PREFIX/modules
module load EasyBuild-develop
eb --version  ## This should ensure you have a reasonable instance of EasyBuild

In each of the three repositories in your $INSTALL_PREFIX, you can check that they are set up correctly:

  • git branch should return * 2020a (if not: run git checkout 2020a)
  • git remote -v should return something like
github_easybuilders     git@github.com:easybuilders/easybuild-easyconfigs.git (fetch)
github_easybuilders     git@github.com:easybuilders/easybuild-easyconfigs.git (push)
origin  git@github.com:${GITHUB_NAME}/easybuild-easyconfigs.git (fetch)
origin  git@github.com:${GITHUB_NAME}/easybuild-easyconfigs.git (push)
  • For each of the three repositories, do a git pull github_easybuilders to make sure all are up to date. You will probably want to do so repeatedly during the sprint (especially in the easybuild-easyconfigs repository, in order to pull in EasyConfigs contributed by others during the sprint)
Clone this wiki locally