-
Notifications
You must be signed in to change notification settings - Fork 124
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adds System sub-module for HPC Wisteria #176
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…hat is: submit jobs one by one and track individual job ids rather than submitting one array job"
…ity of inheriting conda env on compute node
…ored in root dir. instead this is defined once at the top of cluster script and inherited by all child classes. this also allows wisteria to override this and set custom run and submit scripts which allow for activating conda environment after submitted job to scheduler
… get around inability to pass command line arguments in pjsub command and inability to inherit conda env from login node
… for array jobs but not all systems run array style
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Relevant Issue: #161
This PR adds support for the U. Tokyo cluster, Wisteria, which is a Fujitsu brand cluster with it's own workload management software.
Changelog:
system.Cluster
submit
andrun
scripts a Class variable which can be overwritten.Fujitsu
sub-module to override this and point to a different script.system.Fujitsu
:Cluster
.Slurm
orPBS
system.Wisteria
:Fujitsu
to provide more specific arguments for the Wisteria system.Notes from System.Wisteria docstring:
submit
andrun
script which first load the correct modules, and then run the corresponding scriptsubmit
andrun
script, normally input like '--key value' interfere with the batch submission cmdpjsub
. So instead we use thepjsub
'-x' flag which allows us to set environment variables. We use these in place of command line arguments