Add a architecture env variable to the user subdir module path. #2330
Conversation
On systems with multiple architectures sharing a single home directory for the users there need to be a way to install architecture specific builds when using the subdir-user-modules extension.
|
Travis test report: 9/9 runs failed - see https://travis-ci.org/easybuilders/easybuild-framework/builds/290804221 Only showing partial log for 1st failed test suite run 1481.1;
(bleep, bloop, I'm just a bot, please talk to my owner @boegel if you notice you me acting stupid) |
|
Can't you just be clever in how you construct the user subdirectory path? Something like |
|
@ocaisa I think the problem is that you can't use @akesandgren I feel the solution here is a bit too specific. If the problem is indeed that you can't use environment variables in an EasyBuild configuration file, then we should try to solve that in a general way? |
|
Yes that might be better, but remember that it needs to work for both Tcl and Lua modules. The problem is that the env variable needs to be resolved when the user loads modules and not when EB creates the modules. |
|
In case the problem description is unclear. The user initiated EB build/install needs to have a separate path for each architecture at our site. The problem is that the system installed modules need to get the extra "module use" (actually prepend_path("MODULEPATH", "some-user-prefix") with Lua) correct and resolvable at module load time. I.e. it must look something like: where we have this in our site-wide easyconfig.cfg I.e. one easyconfig.cfg for all systems. |
@akesandgren OK, I overlooked that, that's indeed quite important, since that implies that Tcl or Lua needs to grab the value of the environment variable when the module is being processed, so this is orthogonal to supporting the use of environment variables in EasyBuild configuration files... |
|
So, with that in mind is this at least in an acceptable direction? |
|
@akesandgren I think the general direction is OK, but what you came up with for now is not flexible enough... You're enforcing that only one single variable can be used, and the order of the subdir and the variable is fixed. I think we can do better. I was thinking about coming up with a clear/safe syntax to express that the specified value for Transforming the provided string into the correct syntax for combining the static part(s) with the 'dynamic' part(s) (each determined by an env var to resolve) could be done by a new method in Am I making any sense? |
|
Yes, it makes sense. Not sure if you actually meant to put a " " in there but that's details... And then doing $SOME_VAR in the easyconfig could be dealt with separately? For when you want it resolved at eb runtime. |
|
@akesandgren I certainly did not mean to include a space in there (edited my comment above to remove it). And yes, supporting the use of environment variables in EasyBuild configuration files (not easyconfigs, that's something else ;)), is an issue entirely separate from this, since it doesn't help with your use case at all. |
|
the current module_generator.use code does quote the path argument, which won't work well with having a possible os.getenv("xx") in it though. So that part would then need to be moved to the caller(s), and there is probably more things to consider here. |
|
@akesandgren We could still do the quoting in BTW, we briefly discussed this during today's conf call, see https://github.com/easybuilders/easybuild/wiki/Conference-call-notes-20171025#support-use-of-environment-variables-in---subdir-user-modules . Rather than using Two downsides to that:
|
|
@akesandgren After discussing this with you:
which would be equivalent (as module load time) with using |
|
I hope we can limit this to {RUNTIME_ENV::SYSTEM_ARCH} being surrounded only by "/", i.e. os.path.sep characters? |
…(>=4.0) like EnvironmentModulesTcl (<=1.293)
…(>=4.0) like EnvironmentModulesTcl (<=1.293)
…s 4+ behaves similarly to ModulesC, so also append GCCcore/6.2.0 if we are an instance of EnvironmentModules
… (+ break up long comment)
Examples: 'GCCcore', 'icc', 'ifort'. They will be considered before
the compiler family ('Intel', etc).
…mentModules and EnvironmentModulesTcl modules tool
…ommand is modulecmd.tcl introduce a is_modulecmd_tcl_modulestcl def to make distinction between old Modules-Tcl and Modules 4+.
Change from using a specific subdir_arch_env to expanding
"{RUNTIME_ENV::SOME_ENV_VAR}" into module runtime equivalences of
os.getenv("SOME_ENV_VAR")
|
Crap.... made an incorrect pull at some stage... |
|
Closing this PR since i messed up. Opened PR #2395 instead. |
On systems with multiple architectures sharing a single home directory
for the users there need to be a way to install architecture specific
builds when using the subdir-user-modules extension.