-
Notifications
You must be signed in to change notification settings - Fork 66
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add pretty()
method for validation_set
class
#456
Comments
I attempted to transition to the new interface in this test but saw new warnings from library(tidymodels)
data(ames, package = "modeldata")
mod <- parsnip::decision_tree(cost_complexity = tune()) %>%
parsnip::set_mode("regression")
set.seed(1)
three_way_split <- rsample::initial_validation_split(ames, prop = c(.8, .1))
folds <- rsample::validation_set(three_way_split)
set.seed(1)
res <-
mod %>%
tune_bayes(
Sale_Price ~ Neighborhood + Gr_Liv_Area + Year_Built + Bldg_Type +
Latitude + Longitude,
resamples = folds,
initial = 3,
metrics = yardstick::metric_set(rsq),
param_info = parameters(dials::cost_complexity(c(-2, 0)))
)
#> Warning in `[.tbl_df`(x, is.finite(x <- as.numeric(x))): NAs introduced by
#> coercion
#> Warning in `[.tbl_df`(x, is.finite(x <- as.numeric(x))): NAs introduced by
#> coercion
#> Warning in `[.tbl_df`(x, is.finite(x <- as.numeric(x))): NAs introduced by
#> coercion
#> Warning in `[.tbl_df`(x, is.finite(x <- as.numeric(x))): NAs introduced by
#> coercion
#> Warning in `[.tbl_df`(x, is.finite(x <- as.numeric(x))): NAs introduced by
#> coercion
#> Warning in `[.tbl_df`(x, is.finite(x <- as.numeric(x))): NAs introduced by
#> coercion
#> → A | warning: A correlation computation is required, but `estimate` is constant and has 0 standard deviation, resulting in a divide by 0 error. `NA` will be returned.
#> ! For the rsq estimates, 1 missing value was found and removed before fitting
#> the Gaussian process model.
#> Warning in `[.tbl_df`(x, is.finite(x <- as.numeric(x))): NAs introduced by
#> coercion
#> ! For the rsq estimates, 2 missing values were found and removed before
#> fitting the Gaussian process model.
#> Warning in `[.tbl_df`(x, is.finite(x <- as.numeric(x))): NAs introduced by
#> coercion
#> ! For the rsq estimates, 3 missing values were found and removed before
#> fitting the Gaussian process model.
#> Warning in `[.tbl_df`(x, is.finite(x <- as.numeric(x))): NAs introduced by
#> coercion
#> ! For the rsq estimates, 4 missing values were found and removed before
#> fitting the Gaussian process model.
#> Warning in `[.tbl_df`(x, is.finite(x <- as.numeric(x))): NAs introduced by
#> coercion
#> ! For the rsq estimates, 5 missing values were found and removed before
#> fitting the Gaussian process model.
#> Warning in `[.tbl_df`(x, is.finite(x <- as.numeric(x))): NAs introduced by
#> coercion
#> ! For the rsq estimates, 6 missing values were found and removed before
#> fitting the Gaussian process model.
#> Warning in `[.tbl_df`(x, is.finite(x <- as.numeric(x))): NAs introduced by
#> coercion
#> There were issues with some computations A: x7
#> Created on 2023-09-27 with reprex v2.0.2 Reprex with old interfacelibrary(tidymodels)
data(ames, package = "modeldata")
mod <- parsnip::decision_tree(cost_complexity = tune()) %>%
parsnip::set_mode("regression")
set.seed(1)
folds <- rsample::validation_split(ames, .9)
#> Warning: `validation_split()` was deprecated in rsample 1.2.0.
#> ℹ Please use `initial_validation_split()` instead.
#> This warning is displayed once every 8 hours.
#> Call `lifecycle::last_lifecycle_warnings()` to see where this warning was
#> generated.
set.seed(1)
res <-
mod %>%
tune_bayes(
Sale_Price ~ Neighborhood + Gr_Liv_Area + Year_Built + Bldg_Type +
Latitude + Longitude,
resamples = folds,
initial = 3,
metrics = yardstick::metric_set(rsq),
param_info = parameters(dials::cost_complexity(c(-2, 0)))
)
#> → A | warning: A correlation computation is required, but `estimate` is constant and has 0 standard deviation, resulting in a divide by 0 error. `NA` will be returned.
#> ! For the rsq estimates, 1 missing value was found and removed before fitting
#> the Gaussian process model.
#> ! For the rsq estimates, 2 missing values were found and removed before
#> fitting the Gaussian process model.
#> ! For the rsq estimates, 3 missing values were found and removed before
#> fitting the Gaussian process model.
#> ! For the rsq estimates, 4 missing values were found and removed before
#> fitting the Gaussian process model.
#> ! For the rsq estimates, 5 missing values were found and removed before
#> fitting the Gaussian process model.
#> ! For the rsq estimates, 6 missing values were found and removed before
#> fitting the Gaussian process model.
#> ! For the rsq estimates, 7 missing values were found and removed before
#> fitting the Gaussian process model.
#> There were issues with some computations A: x8
#> Created on 2023-09-27 with reprex v2.0.2 Is this behavior expected? cc @hfrick—I think I may have missed that these were new in tidymodels/tune#697. |
hm no, this is not expected. I also don't see that warning with my versions installed. which versions are you using? (I also don't get that warning with the dev version of tune.)
|
Hmm, they look very similar. My dplyr is one version off from yours but I was able to downgrade and still see those warnings. library(tidymodels)
sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#> setting value
#> version R version 4.3.1 (2023-06-16)
#> os macOS Ventura 13.5.2
#> system aarch64, darwin20
#> ui X11
#> language (EN)
#> collate en_US.UTF-8
#> ctype en_US.UTF-8
#> tz America/Chicago
#> date 2023-09-28
#> pandoc 3.1.1 @ /Applications/RStudio.app/Contents/Resources/app/quarto/bin/tools/ (via rmarkdown)
#>
#> ─ Packages ───────────────────────────────────────────────────────────────────
#> package * version date (UTC) lib source
#> backports 1.4.1 2021-12-13 [1] CRAN (R 4.3.0)
#> broom * 1.0.5 2023-06-09 [1] CRAN (R 4.3.0)
#> class 7.3-22 2023-05-03 [2] CRAN (R 4.3.1)
#> cli 3.6.1 2023-03-23 [1] CRAN (R 4.3.0)
#> codetools 0.2-19 2023-02-01 [2] CRAN (R 4.3.1)
#> colorspace 2.1-0 2023-01-23 [1] CRAN (R 4.3.0)
#> data.table 1.14.8 2023-02-17 [1] CRAN (R 4.3.0)
#> dials * 1.2.0 2023-04-03 [1] CRAN (R 4.3.0)
#> DiceDesign 1.9 2021-02-13 [1] CRAN (R 4.3.0)
#> digest 0.6.33 2023-07-07 [1] CRAN (R 4.3.0)
#> dplyr * 1.1.3 2023-09-03 [1] CRAN (R 4.3.0)
#> evaluate 0.21 2023-05-05 [1] CRAN (R 4.3.0)
#> fansi 1.0.4 2023-01-22 [1] CRAN (R 4.3.0)
#> fastmap 1.1.1 2023-02-24 [1] CRAN (R 4.3.0)
#> foreach 1.5.2 2022-02-02 [1] CRAN (R 4.3.0)
#> fs 1.6.3 2023-07-20 [1] CRAN (R 4.3.0)
#> furrr 0.3.1 2022-08-15 [1] CRAN (R 4.3.0)
#> future 1.33.0 2023-07-01 [1] CRAN (R 4.3.0)
#> future.apply 1.11.0 2023-05-21 [1] CRAN (R 4.3.0)
#> generics 0.1.3 2022-07-05 [1] CRAN (R 4.3.0)
#> ggplot2 * 3.4.3 2023-08-14 [1] CRAN (R 4.3.0)
#> globals 0.16.2 2022-11-21 [1] CRAN (R 4.3.0)
#> glue 1.6.2 2022-02-24 [1] CRAN (R 4.3.0)
#> gower 1.0.1 2022-12-22 [1] CRAN (R 4.3.0)
#> GPfit 1.0-8 2019-02-08 [1] CRAN (R 4.3.0)
#> gtable 0.3.4 2023-08-21 [1] CRAN (R 4.3.0)
#> hardhat 1.3.0 2023-03-30 [1] CRAN (R 4.3.0)
#> htmltools 0.5.6 2023-08-10 [1] CRAN (R 4.3.0)
#> infer * 1.0.5 2023-09-06 [1] CRAN (R 4.3.0)
#> ipred 0.9-14 2023-03-09 [1] CRAN (R 4.3.0)
#> iterators 1.0.14 2022-02-05 [1] CRAN (R 4.3.0)
#> knitr 1.44 2023-09-11 [1] CRAN (R 4.3.0)
#> lattice 0.21-8 2023-04-05 [2] CRAN (R 4.3.1)
#> lava 1.7.2.1 2023-02-27 [1] CRAN (R 4.3.0)
#> lhs 1.1.6 2022-12-17 [1] CRAN (R 4.3.0)
#> lifecycle 1.0.3 2022-10-07 [1] CRAN (R 4.3.0)
#> listenv 0.9.0 2022-12-16 [1] CRAN (R 4.3.0)
#> lubridate 1.9.2 2023-02-10 [1] CRAN (R 4.3.0)
#> magrittr 2.0.3 2022-03-30 [1] CRAN (R 4.3.0)
#> MASS 7.3-60 2023-05-04 [2] CRAN (R 4.3.1)
#> Matrix 1.6-1.1 2023-09-18 [1] CRAN (R 4.3.1)
#> modeldata * 1.2.0 2023-08-09 [1] CRAN (R 4.3.0)
#> munsell 0.5.0 2018-06-12 [1] CRAN (R 4.3.0)
#> nnet 7.3-19 2023-05-03 [2] CRAN (R 4.3.1)
#> parallelly 1.36.0 2023-05-26 [1] CRAN (R 4.3.0)
#> parsnip * 1.1.1 2023-08-17 [1] CRAN (R 4.3.0)
#> pillar 1.9.0 2023-03-22 [1] CRAN (R 4.3.0)
#> pkgconfig 2.0.3 2019-09-22 [1] CRAN (R 4.3.0)
#> prodlim 2023.08.28 2023-08-28 [1] CRAN (R 4.3.0)
#> purrr * 1.0.2 2023-08-10 [1] CRAN (R 4.3.0)
#> R6 2.5.1 2021-08-19 [1] CRAN (R 4.3.0)
#> Rcpp 1.0.11 2023-07-06 [1] CRAN (R 4.3.0)
#> recipes * 1.0.8 2023-08-25 [1] CRAN (R 4.3.0)
#> reprex 2.0.2 2022-08-17 [1] CRAN (R 4.3.0)
#> rlang 1.1.1 2023-04-28 [1] CRAN (R 4.3.0)
#> rmarkdown 2.25 2023-09-18 [1] CRAN (R 4.3.1)
#> rpart 4.1.19 2022-10-21 [2] CRAN (R 4.3.1)
#> rsample * 1.2.0 2023-08-23 [1] CRAN (R 4.3.0)
#> rstudioapi 0.15.0 2023-07-07 [1] CRAN (R 4.3.0)
#> scales * 1.2.1 2022-08-20 [1] CRAN (R 4.3.0)
#> sessioninfo 1.2.2 2021-12-06 [1] CRAN (R 4.3.0)
#> survival 3.5-7 2023-08-14 [1] CRAN (R 4.3.0)
#> tibble * 3.2.1 2023-03-20 [1] CRAN (R 4.3.0)
#> tidymodels * 1.1.1 2023-08-24 [1] CRAN (R 4.3.0)
#> tidyr * 1.3.0 2023-01-24 [1] CRAN (R 4.3.0)
#> tidyselect 1.2.0 2022-10-10 [1] CRAN (R 4.3.0)
#> timechange 0.2.0 2023-01-11 [1] CRAN (R 4.3.0)
#> timeDate 4022.108 2023-01-07 [1] CRAN (R 4.3.0)
#> tune * 1.1.2 2023-08-23 [1] CRAN (R 4.3.0)
#> utf8 1.2.3 2023-01-31 [1] CRAN (R 4.3.0)
#> vctrs 0.6.3 2023-06-14 [1] CRAN (R 4.3.0)
#> withr 2.5.0 2022-03-03 [1] CRAN (R 4.3.0)
#> workflows * 1.1.3 2023-02-22 [1] CRAN (R 4.3.0)
#> workflowsets * 1.0.1 2023-04-06 [1] CRAN (R 4.3.0)
#> xfun 0.40 2023-08-09 [1] CRAN (R 4.3.0)
#> yaml 2.3.7 2023-01-23 [1] CRAN (R 4.3.0)
#> yardstick * 1.2.0 2023-04-21 [1] CRAN (R 4.3.0)
#>
#> [1] /Users/simoncouch/Library/R/arm64/4.3/library
#> [2] /Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library
#>
#> ────────────────────────────────────────────────────────────────────────────── Created on 2023-09-28 with reprex v2.0.2 |
Looks like they were present in the original issue tidymodels/tune#697 which gives us a slightly more minimal reprex: library(tidymodels)
val_split <- initial_validation_split(mtcars)
val_set <- validation_set(val_split)
# fit_resamples() and tune_*() work with rsets
res <- fit_resamples(linear_reg(), mpg ~ ., resamples = val_set)
#> Warning in `[.tbl_df`(x, is.finite(x <- as.numeric(x))): NAs introduced by
#> coercion Created on 2023-09-28 with reprex v2.0.2 |
Ah, okay. library(tidymodels)
val_split <- initial_validation_split(mtcars)
val_set <- validation_set(val_split)
pretty(val_set)
#> Warning in `[.tbl_df`(x, is.finite(x <- as.numeric(x))): NAs introduced by
#> coercion
#> # A tibble: 0 × 0
# previously...
val_set_old <- validation_split(mtcars)
#> Warning: `validation_split()` was deprecated in rsample 1.2.0.
#> ℹ Please use `initial_validation_split()` instead.
#> This warning is displayed once every 8 hours.
#> Call `lifecycle::last_lifecycle_warnings()` to see where this warning was
#> generated.
pretty(val_set_old)
#> [1] "Validation Set Split (0.75/0.25) " Created on 2023-09-28 with reprex v2.0.2 |
Thanks for finding this! So |
pretty()
method for validation_set
class
This issue has been automatically locked. If you believe you have found a related problem, please file a new issue (with a reprex: https://reprex.tidyverse.org) and link to this issue. |
The text was updated successfully, but these errors were encountered: