function dependencies are missing: drake_config() in a magrittr pipe

[idavydov]

Hi, thanks for a really amazing package!

I noticed that sometimes drake doesn't see function dependencies. Any suggestions why is this happening?

library(drake)
library(magrittr)

my.rnorm <- function(...) rnorm(...)

plan <- drake_plan(
  a = my.rnorm(1),
  b = my.rnorm(1),
  c = my.rnorm(10, a, b)
)


plan %>%
  drake_config %>%
  vis_drake_graph

sessionInfo()
R version 3.5.1 (2018-07-02)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: CentOS Linux 7 (Core)

Matrix products: default
BLAS/LAPACK: /pstore/apps/apps_dev/.testing/OpenBLAS/0.2.13-GCC-4.8.4-LAPACK-3.5.0/lib/libopenblas_prescottp-r0.2.13.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8       
 [4] LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C              
[10] LC_TELEPHONE=C             LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] magrittr_1.5     bindrcpp_0.2.2   drake_6.0.0.9000

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.19         pillar_1.3.0         BiocInstaller_1.30.0
 [4] compiler_3.5.1       formatR_1.5          bindr_0.1.1         
 [7] R.methodsS3_1.7.1    R.utils_2.7.0        tools_3.5.1         
[10] digest_0.6.17        lubridate_1.7.4      jsonlite_1.5        
[13] evaluate_0.11        tibble_1.4.2         pkgconfig_2.0.2     
[16] rlang_0.2.2          igraph_1.2.2         rstudioapi_0.7      
[19] yaml_2.2.0           parallel_3.5.1       withr_2.1.2         
[22] dplyr_0.7.6          stringr_1.3.1        storr_1.2.0         
[25] fs_1.2.6             htmlwidgets_1.2      globals_0.12.1      
[28] tidyselect_0.2.4     glue_1.3.0           listenv_0.7.0       
[31] R6_2.2.2             purrr_0.2.5          codetools_0.2-15    
[34] htmltools_0.3.6      assertthat_0.2.0     future_1.9.0        
[37] stringi_1.2.4        visNetwork_2.0.4     crayon_1.3.4        
[40] R.oo_1.22.0         

[wlandau]

Thanks for the report. This is an instance of #335 and tidyverse/magrittr#163. Your my.norm() function is in your global environment, and the environment in later stages of the pipe is different. Unfortunately, this edge case is unlikely to be fixed in future versions. Here are two alternative workarounds.

1. Explicitly supply an environment.

drake_config() and make() each have an envir argument, which is the environment with the all the functions (and data objects) you define. Depending on your project, envir could be the global environment, a new.env() that you create and modify manually, or some other environment.

envir <- environment() # Get the global environment with all your custom code.

plan %>%
  drake_config(envir = envir) %>% # Explicitly supply the environment.
  vis_drake_graph()

2. Or just call drake_config() first.

By default, the envir argument of make() and drake_config() is the calling environment (parent.frame()). So as long as the function that takes envir is the first command in the magrittr pipe chain, you should be fine.

drake_config(plan) %>%
  vis_drake_graph()

[idavydov]

Thanks for your response!

In my case it was necessary not only to change the code, by also to clean() (I used clean(destroy=TRUE). Otherwise it was always showing me the old graph without function dependencies.

[wlandau]

Ah yes, this is because of #504 and #507. The graph construction steps in the preprocessing are now memoized (via memo_expr()). get_cache()$clear(namespace = "memoize") should also work in addition to clean(destroy = TRUE).