Skip to content

Commit

Permalink
Expose the template argument of clustermq functions
Browse files Browse the repository at this point in the history
  • Loading branch information
wlandau-lilly committed Aug 8, 2018
1 parent 82f3e79 commit 3288dc5
Show file tree
Hide file tree
Showing 5 changed files with 49 additions and 7 deletions.
17 changes: 15 additions & 2 deletions R/config.R
Original file line number Diff line number Diff line change
Expand Up @@ -360,6 +360,17 @@
#' @param garbage_collection logical, whether to call `gc()` each time
#' a target is built during [make()].
#'
#' @param template a named list of values to fill in the `{{ ... }}`
#' placeholders in template files (e.g. from [drake_hpc_template_file()]).
#' Same as the `template` argument of `clustermq::Q()` and
#' `clustermq::workers`.
#' Enabled for `clustermq` only (`make(parallelism = "clustermq_staged")`),
#' not `future` or `batchtools` so far.
#' For more information, see the `clustermq` package:
#' <https://github.com/mschubert/clustermq>.
#' Some template placeholders such as `{{ job_name }}` and `{{ n_jobs }}`
#' cannot be set this way.
#'
#' @examples
#' \dontrun{
#' test_with_dir("Quarantine side effects.", {
Expand Down Expand Up @@ -417,7 +428,8 @@ drake_config <- function(
makefile_path = "Makefile",
console_log_file = NULL,
ensure_workers = TRUE,
garbage_collection = FALSE
garbage_collection = FALSE,
template = list()
){
force(envir)
unlink(console_log_file)
Expand Down Expand Up @@ -522,7 +534,8 @@ drake_config <- function(
ensure_workers = ensure_workers,
all_targets = all_targets,
all_imports = all_imports,
garbage_collection = garbage_collection
garbage_collection = garbage_collection,
template = template
)
}

Expand Down
6 changes: 4 additions & 2 deletions R/make.R
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,8 @@ make <- function(
makefile_path = "Makefile",
console_log_file = NULL,
ensure_workers = TRUE,
garbage_collection = FALSE
garbage_collection = FALSE,
template = list()
){
force(envir)
if (!is.null(return_config)){
Expand Down Expand Up @@ -164,7 +165,8 @@ make <- function(
makefile_path = makefile_path,
console_log_file = console_log_file,
ensure_workers = ensure_workers,
garbage_collection = garbage_collection
garbage_collection = garbage_collection,
template = template
)
}
make_with_config(config = config)
Expand Down
5 changes: 4 additions & 1 deletion R/staged.R
Original file line number Diff line number Diff line change
Expand Up @@ -167,7 +167,10 @@ run_clustermq_staged <- function(config){
assert_pkgs("clustermq")
schedule <- config$schedule
withr::with_preserve_seed(
workers <- clustermq::workers(n_jobs = config$jobs)
workers <- clustermq::workers(
n_jobs = config$jobs,
template = config$template
)
)
on.exit(workers$finalize())
while (length(V(schedule)$name)){
Expand Down
14 changes: 13 additions & 1 deletion man/drake_config.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

14 changes: 13 additions & 1 deletion man/make.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

0 comments on commit 3288dc5

Please sign in to comment.