Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fully embrace vctrs for dynamic branching #1105

Closed
2 tasks done
wlandau opened this issue Dec 10, 2019 · 4 comments
Closed
2 tasks done

Fully embrace vctrs for dynamic branching #1105

wlandau opened this issue Dec 10, 2019 · 4 comments

Comments

@wlandau
Copy link
Member

wlandau commented Dec 10, 2019

Prework

Description

After writing out #1087 (comment), I think we should fully embrace vctrs for dynamic branching. It will technically be a breaking change for workflows that use readd() on dynamic targets in plans, but dynamic branching is so new that I am willing to let it slide. In the end, we gain type more type stability as described at https://vctrs.r-lib.org/articles/stability.html.

We want readd() and loadd() to be fully type-stable. Current behavior:

library(drake)
plan <- drake_plan(
  x = seq_len(2),
  y = mtcars[x, ],
  z = target(y, dynamic = group(y, .by = x))
)
make(plan)
#> In drake, consider r_make() instead of make(). r_make() runs make() in a fresh R session for enhanced robustness and reproducibility.
#> target x
#> target y
#> dynamic z
#> subtarget z_25909f33
#> subtarget z_7fb9021c
#> aggregate z

readd(z)
#> [[1]]
#>           mpg cyl disp  hp drat   wt  qsec vs am gear carb
#> Mazda RX4  21   6  160 110  3.9 2.62 16.46  0  1    4    4
#> 
#> [[2]]
#>               mpg cyl disp  hp drat    wt  qsec vs am gear carb
#> Mazda RX4 Wag  21   6  160 110  3.9 2.875 17.02  0  1    4    4

Created on 2019-12-10 by the reprex package (v0.3.0)

Desired behavior:

library(drake)
plan <- drake_plan(
  x = seq_len(2),
  y = mtcars[x, ],
  z = target(y, dynamic = group(y, .by = x))
)
make(plan)
#> In drake, consider r_make() instead of make(). r_make() runs make() in a fresh R session for enhanced robustness and reproducibility.
#> target x
#> target y
#> dynamic z
#> subtarget z_25909f33
#> subtarget z_7fb9021c
#> aggregate z

readd(z)
#>               mpg cyl disp  hp drat    wt  qsec vs am gear carb
#> Mazda RX4      21   6  160 110  3.9 2.620 16.46  0  1    4    4
#> Mazda RX4 Wag  21   6  160 110  3.9 2.875 17.02  0  1    4    4

We should be able to achieve similar type-stable aggregation across all types if we use vctrs::vec_c() in loadd() and readd(). Similarly, dynamic_subvalue() should just call vctrs::vec_slice(). And as long as we depend on vctrs, we should set meta$size to vctrs::vec_size() instead of NROW().

@wlandau
Copy link
Member Author

wlandau commented Dec 10, 2019

cc @kendonB, @billdenney, @brendanf

@billdenney
Copy link
Contributor

I like it, and I think it'll make dynamic branching simpler to use.

@wlandau
Copy link
Member Author

wlandau commented Dec 10, 2019

Awesome!

Rethinking vec_size(). NROW() is faster, it agrees with vec_size() for the sensible cases, and does not error out when we have non-vectors.

@wlandau
Copy link
Member Author

wlandau commented Dec 10, 2019

Good news: with vec_slice() seems to be a lot faster than the existing dynamic_subvalue().

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants