-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Resolving a batchtools_slurm-related error, probably a configuration problem #11
Comments
Hi all, I get the above error when running this as well:
The jobs successfully appear in @wlandau-lilly , could you edit out the working directory above. |
Sure, I scrubbed it and inserted |
I made only minor changes to the config file:
|
Hmm... that > library("future.batchtools")
> future::plan(batchtools_local)
> future_lapply(1:2, identity)
[[1]]
[1] 1
[[2]]
[1] 2 |
Yep that one works for me. Same output as you. |
Next, check if it's unrelated to future.batchjobs. First, verify that you can run the following: library("batchtools")
cf <- makeClusterFunctionsSocket(2)
reg <- makeRegistry(NA)
reg$cluster.functions <- cf
batchMap(fun = identity, x = 1:2)
submitJobs()
waitForJobs()
reduceResultsList()
## [[1]]
## [1] 1
##
## [[2]]
## [1] 2 When that works, restart R and retry by using: stopifnot(file.exists("batchtools.slurm.tmpl"))
cf <- makeClusterFunctionsSlurm("slurm") ## for ./batchtools.slurm.tmpl |
OK, I solved it I believe. It was my fault - we have two architectures on our cluster and I had compiled my R packages on sandybridge but the job was getting sent to westmere. This was as simple as adding another configuration flag to the |
So glad to hear this! No time was wasted at all, thank you for sticking with it. I have big dreams for |
Good to hear. No time wasted; this thread adds to the searchable knowledge base. |
See ropensci/drake#115, particularly here. @kendonB and I have been trying to debug his
drake
/SLURM
powered project, and we are running into trouble. The following assumes developmentdrake
8a3558a3fa9269f2c19c98e1a6404603486d5c3a.On my Debian 9.2 VM with a toy installation of SLURM, this runs perfectly. But on his serious SLURM cluster, @kendonB gets the following.
I suspect a configuration error that could be resolved with the right template file. I would rather not have to implement a special slurm_apply backend, especially since
drake
already has two different ways to talk to SLURM already.The text was updated successfully, but these errors were encountered: