[R] Allow predictions on lgb.Dataset objects

[vspinu]

Surprisingly one has to pass an lgb.Dataset for lgb.train evaluation but cannot use predict method on it.

For instance I would expect something along the following to work:

> dtest <- lgb.Dataset(..)
> fit <- lgb.train(...,  valids = list(tst = dtest))
> fit$predict(dtest)
Error in predictor$predict(data, num_iteration, rawscore, predleaf, predcontrib,  : 
  predict: cannot predict on data of class ‘lgb.Dataset’‘R6’

---

─ Session info ───────────────────────────────────────────────────────────────
 setting  value
 version  R Under development (unstable) (2018-12-01 r75733)
 os       Ubuntu 18.04.1 LTS
 system   x86_64, linux-gnu
 ui       X11
 language
 collate  en_GB.UTF-8
 ctype    en_GB.UTF-8
 tz       Europe/Amsterdam
 date     2019-01-11

─ Packages ───────────────────────────────────────────────────────────────────
 package     * version    date       lib source
 assertthat    0.2.0      2017-04-11 [1] CRAN (R 3.5.0)
 backports     1.1.2      2017-12-13 [1] CRAN (R 3.5.0)
 callr         3.0.0.9002 2018-12-03 [1] Github (r-lib/callr@1cbce9d)
 cli           1.0.1      2018-09-25 [1] CRAN (R 3.6.0)
 crayon        1.3.4      2017-09-16 [1] CRAN (R 3.5.0)
 desc          1.2.0      2018-05-01 [1] CRAN (R 3.6.0)
 devtools      2.0.1.9000 2018-12-03 [1] Github (r-lib/devtools@a6bbe87)
 digest        0.6.18     2018-10-10 [1] CRAN (R 3.6.0)
 fs            1.2.6      2018-08-23 [1] CRAN (R 3.6.0)
 glue          1.3.0      2018-07-17 [1] CRAN (R 3.6.0)
 magrittr      1.5        2014-11-22 [1] CRAN (R 3.5.0)
 memoise       1.1.0      2017-04-21 [1] CRAN (R 3.5.0)
 pkgbuild      1.0.2.9000 2018-12-03 [1] Github (r-lib/pkgbuild@6e4ebdf)
 pkgload       1.0.1.9000 2018-12-03 [1] Github (r-lib/pkgload@0ef4f58)
 prettyunits   1.0.2      2015-07-13 [1] CRAN (R 3.5.0)
 processx      3.2.0.9001 2018-12-03 [1] Github (r-lib/processx@366475b)
 ps            1.2.1      2018-11-06 [1] CRAN (R 3.6.0)
 R6            2.3.0      2018-10-04 [1] CRAN (R 3.6.0)
 Rcpp          1.0.0      2018-11-07 [1] CRAN (R 3.6.0)
 remotes       2.0.2.9000 2018-12-03 [1] Github (r-lib/remotes@2e3bf35)
 rlang         0.3.0.9001 2018-12-18 [1] Github (r-lib/rlang@327740f)
 rprojroot     1.3-2      2018-01-03 [1] CRAN (R 3.5.0)
 sessioninfo   1.1.1      2018-11-05 [1] CRAN (R 3.6.0)
 testthat      2.0.0      2017-12-13 [1] CRAN (R 3.5.0)
 usethis       1.4.0      2018-08-14 [1] CRAN (R 3.6.0)
 withr         2.1.2.9000 2018-10-23 [1] Github (jimhester/withr@be57595)

[1] /home/vspinu/.lib/3.6.0
[2] /home/vspinu/bin/R-latest/library

[Laurae2]

Using lgb.Dataset to predict would not work because it is not the original unbinned data. The lgb.Dataset is a binary dataset optimized only for training.

[vspinu]

I see. How come that you can do evaluation on that dataset while traning. Don't you need to predict first?

[Laurae2]

If I remember, during the model training, the model is still using the bins and not the exact values, therefore the prediction uses the bins directly.

ping @guolinke for clarification

[Laurae2]

@vspinu Predictions during training are done using binned data. That's why while training you do not need the "out of LightGBM" predict.

However, for real predictions, passing LightGBM binary data to predict is not correct: LightGBM binary datasets hold the bins, not the exact values of each observation/feature combination. Therefore, reverse engineering the original values to predict from binary data is not possible (as one bin might fall in two splits at the same time).

[vspinu]

Thanks @Laurae2. Do you mean 'bins" like in histogram bins?

I am now worried about the following scenario:

dtrain <- lgb.Dataset(...)
dtest <- lgb.Dataset(...)
fit <- lgb.train(dtrain,  valids = list(tst = dtest))

If binary contains only bins, does this mean that the dtrain and dtest binary datasets can have different "bins" assignments? If so, then validation on the dtest dataset has no sense just like prediction on dtest.

[Laurae2]

Your interpretation should be correct: lgb.Dataset.create.valid should be used instead for the validation.

Bins do refer to histogram bins constructed by lgb.Dataset.

@guolinke How do you pass free_raw_data = FALSE to lgb.Dataset.create.valid, or should it not be used in conjunction with lgb.Dataset.create.valid?

Example:

require(lightgbm)
require(methods)

# We load in the agaricus dataset
# In this example, we are aiming to predict whether a mushroom is edible
data(agaricus.train, package = "lightgbm")
data(agaricus.test, package = "lightgbm")
train <- agaricus.train
test <- agaricus.test

dtrain <- lgb.Dataset(data = train$data, label = train$label, free_raw_data = FALSE)
dtest <- lgb.Dataset.create.valid(dtrain, data = test$data, label = test$label, free_raw_data = FALSE)

valids <- list(train = dtrain, test = dtest)

print("Train lightgbm using lgb.train with valids")
bst <- lgb.train(data = dtrain,
                 num_leaves = 4,
                 learning_rate = 1,
                 nrounds = 2,
                 valids = valids,
                 nthread = 2,
                 objective = "binary")

Leads to fatal error:

[LightGBM] [Warning] Starting from the 2.1.2 version, default value for the "boost_from_average" parameter in "binary" objective is true.
This may cause significantly different results comparing to the previous versions of LightGBM.
Try to set boost_from_average=false, if your old models produce bad results
[LightGBM] [Info] Number of positive: 3140, number of negative: 3373
[LightGBM] [Info] Total Bins 128
[LightGBM] [Info] Number of data: 6513, number of used features: 107
[LightGBM] [Fatal] [Used indices of subset]: does not tolerate element [#0 = -1] outside [0, 6512]
[LightGBM] [Fatal] Length of label is not same with #data
[LightGBM] [Info] [binary:BoostFromScore]: pavg=0.482113 -> initscore=-0.071580
[LightGBM] [Info] Start training from score -0.071580
[1]:	train's binary_logloss:0.198597	test's binary_logloss:0.198597 
[2]:	train's binary_logloss:0.111535	test's binary_logloss:0.111535 

[StrikerRUS]

@Laurae2 Can this be closed since demos were fixed?